Image analysis method for sensing a moving ball and sensing device using it
By defining the analysis target area and using the image analysis method based on the sphere's position, the complexity and low accuracy of sphere image analysis in existing technologies are solved, achieving faster processing speed and higher accuracy.
Patent Information
- Application Number
- CN202280050373.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-07-22
- Filing Date
- 2022-07-08
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-07-08
AI Technical Summary
Existing technologies for image analysis of a moving ball require a complex process to define the region of interest, and suffer from low sensing accuracy and slow processing speed.
By setting an analysis target area and using the detected ball position as a reference, image analysis is performed on each acquired image using the previously set analysis target area to calculate the physical quantities of the ball's movement.
It significantly improves the speed and accuracy of image processing and reduces the burden of analyzing the overall image.
Smart Images

Figure CN117716388B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a sensing device for sensing a ball that is moved by being hit by a player in a ball-based sport using a camera, and a method for analyzing the camera images used therein. Background Technology
[0002] Recently, virtual simulation systems for sports such as golf and baseball, where the experience of playing them live is quite limited, have emerged and become widely popular.
[0003] Furthermore, virtual sports simulation systems have emerged for net sports such as tennis, squash, and badminton, where two players receive serves across a net, creating a trend where users can easily enjoy diverse sports in popular cultural spaces.
[0004] In this virtual sports simulation, the game is basically played by players hitting a ball. In order to simulate the image of the ball being hit by the players, a sensing device that can effectively sense the movement of the ball is needed.
[0005] In the past, optical sensing methods such as infrared sensors or laser sensors were commonly used. In this case, it was necessary to set up structures with multiple optical sensors that could emit and receive light in the player's playing space. This restricted the player's playing and had limitations in achieving a realistic playing experience. Furthermore, the structure of the sensor was frequently damaged during the game. More importantly, there was a problem with very low sensing accuracy.
[0006] Due to the limitations and problems of this optical sensing method, recently, image sensing devices that acquire camera images of a moving ball and calculate the sensing information of the moving ball by analyzing the acquired images have been widely used.
[0007] This image sensing method obtains sensing information by analyzing images, thus having the advantage of obtaining very accurate sensing information. However, since it requires processing images one by one from tens to hundreds of frames per second, it presents the problem of needing to equip a camera device and a data processing device with a fairly high data processing capability.
[0008] As prior art in this regard, Korean Patent Publication No. 10-2018-0054279, Korean Patent Publication No. 10-2019-0085152, U.S. Patent Publication No. 7,497,780, and U.S. Patent Publication No. 7,324,663, among other prior art documents, have been published.
[0009] The aforementioned prior art all commonly discloses a method for designating the area surrounding the ball in an image including a ball captured by a camera as a region of interest, and for analyzing the image primarily based on this region of interest.
[0010] However, in the prior art as described above, in order to set the region of interest, the position of the ball is predicted and a predetermined area around it is specified based on the predicted position. There is a problem that setting the region of interest requires a complex process, such as calculating physical quantities about the ball's previous movement in order to predict the position of the ball, and performing another complex calculation in order to use the calculated physical quantities to predict the position of the ball.
[0011] However, even after such a complex process of setting the region of interest, the problem of not being able to detect the ball in the region of interest can still occur if the calculations used to predict the ball's movement position are insufficient or the predictions are incorrect.
[0012] Existing technical documents
[0013] Korean Patent Publication No. 10-2018-0054279
[0014] Korean Patent Publication No. 10-2019-0085152
[0015] US Patent Publication No. 7,497,780
[0016] US Patent Publication No. 7,324,663 Summary of the Invention
[0017] Technical issues
[0018] The present invention provides an image analysis method for sensing a moving ball and a sensing device thereof. In the analysis of an image for sensing a ball moving in motion, instead of analyzing the entire image, an analysis object region is set as a partial region, and the ball is detected within the set analysis object region. Physical quantities relating to the movement of the ball are calculated. By setting the analysis object region on the image being analyzed, based on the position of the detected ball, and including the position where the ball will appear in the next image to be analyzed, the previously set analysis object region is applied to the current image for each acquired image to perform image analysis, thereby aiming to significantly improve the speed of image processing.
[0019] Technical solution
[0020] An embodiment of the present invention provides an image analysis method for sensing a moving ball, comprising: acquiring an image from the perspective of a camera in the space in which the ball moves; detecting the position of the ball from the acquired image; setting an analysis object region based on the detected position of the ball, in a manner including the position where the ball will appear in the next image; and detecting the position of the ball within the set analysis object region in the next image of the image in which the set analysis object region is used.
[0021] Furthermore, preferably, the method is characterized by further comprising: a step of setting a new analysis object region based on the position of the ball detected in the set analysis object region, in a manner including the position where the ball will appear in the next image; and a step of detecting the position of the ball in the new analysis object region in the next image, thereby performing image analysis for sensing the ball only in the set analysis object region for each acquired image by resetting the analysis object region of the next image based on the position of the ball detected in the analysis object region of the current image.
[0022] Furthermore, preferably, the method is characterized in that the step of setting the analysis object region in a manner that includes the position where the ball will appear on the next image includes: calculating a three-dimensional region of the range in which the ball can move, using pre-set information, from the detected position of the ball as the limiting physical quantity in which the ball can move; and setting the region of the portion of the three-dimensional region including the range in which the ball can move as the analysis object region as the area of the surface observed by the camera.
[0023] Furthermore, preferably, the method is characterized in that the step of acquiring images from each of a plurality of interconnected stereoscopic cameras, and setting the analysis object region in a manner including the position where the ball will appear in the next image, includes: calculating a three-dimensional region of the range in which the ball can move, using pre-set information, from the detected position of the ball as the limiting physical quantity in which the ball can move; and setting each region of the three-dimensional region of the range in which the ball can move as a portion of the surface observed by each of the plurality of cameras as the analysis object region of the image of each of the cameras.
[0024] Furthermore, preferably, the method is characterized in that each acquired image reflects information about the change in the relative positional relationship between the moving ball and the camera to calculate a three-dimensional region of the range in which the ball can move, and the analysis object region is calculated and set from the calculated three-dimensional region.
[0025] Furthermore, preferably, the method is characterized by pre-calculating the change in the relative positional relationship between the moving ball and the camera, and pre-setting a defined three-dimensional region that includes the current ball position and the ball position in the next image at any position, and calculating and setting the analysis object region from the defined three-dimensional region based on the ball position for each acquired image.
[0026] Furthermore, preferably, the method is characterized in that a limit physical quantity of the ball's movement is pre-set for each type of golf club used to strike the ball, and in the step of calculating the three-dimensional region of the range of the ball's movement, the type of golf club used by the user to strike the ball is identified, and the limit physical quantity corresponding to the identified type of golf club is applied to calculate the three-dimensional region of the range of the ball's movement.
[0027] On the other hand, one embodiment of the present invention provides a sensing device that senses a ball by analyzing images of a ball that has been struck and moved. The sensing device includes: a camera device that acquires images from a perspective of the space in which the ball moves; and a sensing processing device that detects the position of the ball from the acquired images, and, based on the detected position of the ball, sets an analysis target area in a manner that includes the position where the ball will appear in the next image, and performs image analysis for sensing the ball only within the analysis target area set for each acquired image in a manner that detects the position of the ball within the set analysis target area in the next image of the image utilizing the set analysis target area.
[0028] Furthermore, preferably, the sensing device is characterized in that the sensing processing device is configured to calculate a three-dimensional region of the range of motion of the ball using pre-set information, based on the detected position of the ball as the limiting physical quantity of the ball's movement, and to set the region including the three-dimensional region of the range of motion of the ball as the area of the surface observed by the camera device.
[0029] Furthermore, preferably, the sensing device is characterized in that the sensing processing device is configured to calculate a three-dimensional region of the range in which the ball can move by reflecting information about the change in the relative positional relationship between the moving ball and the camera in each acquired image, thereby recalculating and setting the analysis object region for each position of the ball.
[0030] Furthermore, preferably, the sensing device is characterized in that the sensing processing device is configured to pre-calculate the change in the relative positional relationship between the moving ball and the camera, so as to pre-set an area of a size that can be used at any position within the field of view of the camera as the analysis object area, thereby uniformly applying the pre-set analysis object area to the image that becomes the object of analysis.
[0031] The effects of the invention
[0032] The image analysis method for sensing a moving ball and the sensing device using the same of the present invention have the following advantages: In the analysis of an image for sensing a ball moving in motion, instead of analyzing the entire image, an analysis object region is set as a partial region, and the ball is detected within the set analysis object region. Physical quantities related to the movement of the ball are calculated. By setting the analysis object region on the image being analyzed in the current image, based on the position of the detected ball, and including the position where the ball will appear in the next image to be analyzed, the previously set analysis object region is applied to the current image for image analysis for each acquired image, thereby significantly improving the speed of image processing. Attached Figure Description
[0033] Figure 1 This is a block diagram illustrating the configuration of a sensing device for a moving ball according to an embodiment of the present invention.
[0034] Figure 2 This is a flowchart illustrating a sensing method of a sensing device according to an embodiment of the present invention.
[0035] Figure 3 Is Figure 2 The flowchart shown in the figure illustrates more specifically a flowchart of an image analysis method according to an embodiment of the present invention.
[0036] Figure 4 This diagram illustrates an image analysis method for sensing a moving ball according to an embodiment of the present invention. It derives a three-dimensional region of the ball's movable range based on the ball's position using the limiting physical quantities of the ball's movable range, and projects this region onto the surfaces observed by each camera to calculate the analysis object region.
[0037] Figure 5 and Figure 6 This is a diagram illustrating an example of the process of detecting a sphere using an image analysis method according to an embodiment of the present invention, utilizing the analysis object region.
[0038] Figure 7 It is a diagram used to illustrate the change in the distance between the ball and the camera as the ball moves. Detailed Implementation
[0039] The specific details of the image analysis method for sensing a moving ball according to the present invention and the sensing device utilizing the same will be described in detail with reference to the accompanying drawings.
[0040] First, refer to Figure 1 The configuration of a moving ball sensing device according to an embodiment of the present invention will be described. Figure 1 This is a block diagram illustrating the configuration of a sensing device for a moving ball according to an embodiment of the present invention.
[0041] In the sensing device and image analysis method of the present invention, whether it is a single camera or a camera device configured in a stereoscopic manner by linking multiple cameras, the images captured by the camera of the movement of the ball are analyzed to calculate the position coordinate information of the ball in actual space. Based on this, various physical quantities related to the movement of the ball can be calculated, such as the ball's speed, the ball's direction angle, the ball's rotation, etc., and it is applicable to the movement of the ball, such as golf, baseball, tennis, squash, bowling, etc.
[0042] In this specification, although the sensing device used in golf in various sports of the ball described above is described as an example, it is obvious that the technical content applicable thereto can also be applied to other sports.
[0043] Figure 1 The diagram illustrates the configuration of a sensing device in so-called screen golf that senses when a user places the ball GB on the golf mat GM in the striking zone and strikes the ball GB with a golf club GC.
[0044] like Figure 1 As shown, an embodiment of the present invention provides a sensing device for sensing a ball, which can be a device that calculates the three-dimensional coordinate information of the ball moving in space by taking pictures of the ball as it moves as the user hits the ball GB with a golf club GC, analyzing the captured images, and calculating information about the ball's motion characteristics such as its initial velocity, direction angle, and altitude angle based on the three-dimensional coordinate information.
[0045] The sensing device can be applied to various fields such as ball-hitting analysis corresponding to a user's golf swing or virtual golf simulation based on virtual reality.
[0046] like Figure 1 As shown, such a sensing device according to an embodiment of the present invention may include a camera device 100 and a sensing processing unit 200.
[0047] The camera device 100 is configured to continuously acquire images from the perspective of observing a moving golf ball, preferably for calculating the position information of the moving golf ball in three-dimensional space. The camera device 100 comprises multiple cameras that acquire images of the same object from different perspectives, for example, such as... Figure 1 As shown, the first camera 110 and the second camera 120 are arranged in a stereoscopic manner in sync with each other.
[0048] By using a camera device with multiple cameras 110 and 120 arranged in a stereoscopic manner as described above, two-dimensional information of the golf ball extracted from each of the images obtained by the first camera 110 and the images obtained by the second camera 120 for the same object ball can be converted into three-dimensional information.
[0049] like Figure 1 As shown, the sensing processing unit 200 may include: an image receiving unit 210, which collects images from each of a plurality of cameras 110, 120; an image processing unit 220, which performs necessary image processing on the images collected by the image receiving unit 210 to detect a ball from the images; and a computing unit 230, which is responsible for calculating various operations for sensing, such as three-dimensional position information, from the two-dimensional position information of the ball detected from each image.
[0050] As described above, the sensing processing unit 200 of the sensing device of an embodiment of the present invention detects a moving ball from each of the images collected by each of the cameras 110, 120, calculates the position information of the ball, and transmits it to the client 300, so that the client 300 can perform the inherent functions of the client 300, such as calculating new information or calculating and analyzing information using the received position information of the ball.
[0051] For example, when the client 300 is implemented as a simulator for a screen golf system, it can receive position information of the golf ball and golf club from the sensing processing unit 200 and use it to simulate the trajectory of the virtual golf ball on a virtual golf course.
[0052] Furthermore, when the client 300 is implemented as a golf swing analysis device, it can receive position information of the golf ball and golf club from the sensing and processing unit 200 and use it to provide analysis information on the user's golf swing, diagnosis of swing problems, and experience information for solving them.
[0053] The image processing unit 220 can be configured to perform image processing to extract a differential operation image of a reference image of each of the images continuously acquired by the cameras 110 and 120, and the calculation unit 230 can be configured to calculate the position information of the moving ball from the images processed by the image processing unit.
[0054] In addition to the method of using difference operations on images as described above, another method for extracting a moving ball from an acquired image can be to use a pre-prepared template image of the ball and extract the part corresponding to the golf ball from each acquired image by the similarity to the template image of the ball.
[0055] On the other hand, the camera device 100 of the sensing device according to an embodiment of the present invention described above can be configured to acquire images from a perspective of the space in which the ball moves. The sensing processing unit 200 can be configured to detect the position of the ball from the acquired images, and based on the detected position of the ball, to set an analysis target area in a manner that includes the position where the ball will appear in the next image. Furthermore, the image analysis for sensing the ball is performed only within the analysis target area set for each acquired image in a manner that detects the position of the ball within the set analysis target area in the next image of the image using the set analysis target area.
[0056] As described above, when the sensing processing unit 200 analyzes the image acquired by the camera device, it does not analyze the entire acquired image, but only analyzes the analysis object area corresponding to a part of the overall image to calculate the position information of the ball, thus significantly improving the data processing speed.
[0057] On the other hand, it will refer to Figure 2 and Figure 3 An image analysis method for sensing a moving ball according to an embodiment of the present invention will be described.
[0058] Figure 2 This is a flowchart illustrating a sensing method of a sensing device according to an embodiment of the present invention. Figure 3 Is Figure 2 The flowchart shown in the figure illustrates more specifically a flowchart of an image analysis method according to an embodiment of the present invention.
[0059] like Figure 2 As shown, as the sensing device operates, the camera device acquires images from a perspective of the space in which the ball moves (S100).
[0060] The sensing processing unit can analyze a specific area on the acquired image (S110) to detect the position of the ball (S120).
[0061] For example, in golf, since the golf ball is always hit from the same predetermined position as the golf mat, if the position information of the golf mat is pre-set, the position of the golf mat can be determined on the image containing the golf mat when it is acquired. Furthermore, by investigating the area including the determined position of the golf mat, the initial position of the ball can be easily detected. Other sports such as baseball can also easily detect the initial position of the ball using the same method.
[0062] On the other hand, as the position of the ball is detected, the sensing device will be in a ball sensing preparation state (S130). By continuously acquiring images to sense the position of the ball, it can sense whether the ball has been hit and moved.
[0063] If the ball is sensed and movement occurs (S140), that is, the ball is hit and begins to move, the sensing processing unit can determine the time point at which the ball movement occurs, and collect images acquired before and after the time point as the object of analysis (S150).
[0064] For example, one can analyze how the golf club moves from images before and after the point in time when the ball moves, and analyze how the ball moves from images before and after the point in time when the ball moves.
[0065] On the other hand, the sensing processing unit can set an analysis target area for each image collected as described above, detect the position of the ball within the analysis target area of each image, and calculate the position coordinate information of the detected ball (S200).
[0066] The sensing processing unit can use the ball's position coordinate information calculated as described above to calculate various physical quantities of the moving ball (S310) and transmit them to the client, so that the client can perform its inherent functions (S320).
[0067] Here, the sensing device of one embodiment of the present invention does not analyze every whole image in the images collected in step S200, but analyzes the analysis target region as a part of it, so as to perform fast image processing. The specific process is described in [link to documentation]. Figure 3 As shown.
[0068] like Figure 3 As shown, the sensing processing unit can use information about the ball's limit physical quantity, i.e. the limit of the ball's mobility, to calculate the "three-dimensional region of the ball's movable range" (S210) based on the ball's position detected in step S120 above.
[0069] The three-dimensional region of the ball's movable range calculated above is projected onto the surface facing each camera, and in each camera, a two-dimensional region including the projected portion can be set as the analysis object region of the next image (i.e., the next image to be analyzed after the analysis of the image in the current analysis) (S220).
[0070] The sensing processing unit can acquire the next image of the image analyzed in the above process and search for the presence of a ball within the previously set analysis target area on that image (S230). That is, instead of searching the entire image, it only searches for the ball within the previously set analysis target area, thus significantly improving the data processing speed.
[0071] When a ball is detected within the analysis target area (S240), the sensing processing unit can calculate the position coordinate information of the detected ball (S250). That is, the position coordinates (two-dimensional coordinates) of the ball detected in each analysis target area by the camera in each image can be converted into position coordinate information in actual space (three-dimensional space) for calculation.
[0072] While repeating steps S210 to S250 in this manner until the sensing ends (S260), the analysis object area calculated and set based on the ball position during the analysis of the previous image can be applied to the current image analysis. This allows the ball's position coordinate information to be calculated by analyzing only the previously set analysis object area in the current image, thereby significantly improving the data processing speed.
[0073] After searching for a ball within the analysis object area, if no ball is detected (S240), the range of the "three-dimensional area of the ball's movable range" (S270) can be adjusted to recalculate, and the analysis object area can be recalculated and set based on this.
[0074] Thus, in this invention, when analyzing images of the ball's movement, the "three-dimensional region of the ball's movable range" derived from the ball's position during the analysis of previous images is projected onto a two-dimensional region using the ball's position as a reference. The analysis object region set based on this is then applied to the current analysis image, and only the previously set analysis object region is analyzed in the current analysis image, thereby improving processing speed.
[0075] Reference Figures 4 to 7 A more detailed description is given of the calculation and setting of the "analysis object area" as described above.
[0076] Figure 4This diagram illustrates an image analysis method for sensing a moving ball according to an embodiment of the present invention. Based on the position of the ball, it derives a three-dimensional region of the range of motion of the ball using the limit physical quantity of the ball's mobility and projects it onto the surface observed by each camera to calculate the analysis object region.
[0077] like Figure 4 As shown, the three-dimensional region RD of the ball's movable range can be derived using the position Po of the ball GB analyzed in the current image, i.e., its position in three-dimensional space, as a reference, and the limiting physical quantity of the ball's movable range.
[0078] The aforementioned "3D region RD of the ball's movable range" is calculated as a 3D region encompassing the range from the ball's position in the current image to its position in the next image.
[0079] Such a three-dimensional region can be calculated using the limiting physical quantities of a ball, such as the maximum speed of a ball that humans can achieve and the shooting speed of a camera.
[0080] For example, in the case of golf, the Guinness World Record for the highest ball speed achievable by a human is 97 m / s. If we set this to 100 m / s and use an image sensor that captures 500 images per second, the ball can move 0.2 m between frames. Based on the ball's position in the previous image, the ball in the next image must be within a sphere with a radius of 0.2 m.
[0081] The sphere, with its position as described above as a reference, can be effectively constrained by reflecting the characteristics of motion using a sensing device.
[0082] For example, in the case of golf, since the direction of the ball's flight is constant, the rear hemisphere of the ball can be excluded. Also, in the case of golf, since the ball rises upward at a predetermined launch angle when hit, there is no need to consider the lower side of the ball's equator, so this part can also be excluded (of course, in the case of a ball that can travel slightly downward above the tee, such as a tee shot, the portion corresponding to the predetermined angle range below the equator of the ball can be included).
[0083] In this way, the sphere, based on the sphere's position as described above, is explicitly limited to the area where the sphere will inevitably exist in the next image, thus allowing the derivation of the aforementioned "three-dimensional region RD of the sphere's movable range". Figure 4 The portion indicated by RD shown is an example of the aforementioned "three-dimensional region of the range in which the ball can move". Figure 4The three-dimensional region RD, representing the range of motion of the sphere, is shown as an example of a three-dimensional region corresponding to approximately 1 / 4 of the sphere, calculated based on the sphere's position Po.
[0084] When the "three-dimensional region RD of the range of movement of the ball" is calculated in space, the three-dimensional region RD can be projected onto the surfaces A1 and A2 observed by each camera 110 and 120 respectively to calculate the two-dimensional projection parts pr1 and pr2 respectively.
[0085] The two-dimensional region including the aforementioned two-dimensional projection parts pr1 and pr2 can be represented as follows: Figure 4 The settings shown are the analysis target areas Ra1 and Ra2 for each camera. When analyzing the next image, the ball can be detected by searching within the analysis target areas Ra1 and Ra2.
[0086] Reference Figure 5 and Figure 6 An example of the process of detecting a sphere using the analyzed object region in the manner described above is given.
[0087] Figure 5 Figures (a) to (d) illustrate the process by which the sensing processing unit of a sensing device according to an embodiment of the present invention acquires images including fr1 to fr4 and analyzes these images respectively. Figure 6 It is used for in Figure 5 The diagram shown illustrates the calculation of the three-dimensional region where the sphere used to define the analysis object area can move during the analysis process. Figure 5 Images (a) through (d) are images captured by one of the multiple cameras.
[0088] Figure 5 (a) is an image processed by the sensing processing unit. It is an image fr1 acquired by the camera when the ball is placed on the golf mat. Initially, a specific region of interest can be searched for the ball in the image.
[0089] That is, in golf, since the ball always starts from the golf mat, the position coordinates of the golf mat are stored in advance. When an image is acquired, the ball can be detected by searching for the region of interest in the image based on the pre-stored position of the golf mat.
[0090] like Figure 5 As shown in (a), since the part gm corresponding to the golf ball mat is the part whose location information is known in advance, a region of interest Rm of a predetermined size can be set based on this, and the ball b1 can be found in the region of interest Rm.
[0091] The sphere b1 found in the image in this way can be calculated as its coordinate information in three-dimensional space, and as... Figure 6 The diagram shows something existing in three-dimensional space. That is, in... Figure 5 (a) Find the sphere b1 in the image. Figure 6 The figure shown corresponds to sphere B1 in three-dimensional space.
[0092] exist Figure 6 In the three-dimensional space shown, the three-dimensional region RD1, the range of motion of the ball with respect to the position of ball B1, can be calculated by applying the limiting physical quantity of the ball's mobility as described above. This three-dimensional region RD1 is then projected onto the surfaces observed by cameras 110 and 120 to calculate... Figure 5 The analysis object region R1 is shown in (a).
[0093] Figure 5 The R1 region shown in (a) is calculated based on the projection of the three-dimensional region RD1, which is the range of the ball's movement relative to the position of the ball B1, and can be set as the analysis object region for the next image.
[0094] Figure 5 (b) shows the next image fr2 of the above-mentioned fr1 image. By applying the previously set analysis object area R1, the sensing processing unit can detect the ball b2 by searching inside the analysis object area R1 when analyzing the fr2 image. The remaining areas outside the analysis object area R1 are areas NR that have not undergone image processing or data processing.
[0095] Here, pb1 is the position of the ball detected in the previous image fr1. If the analysis object region R1 is calculated in the fr1 image, then the position of the reference ball pb1 and the position of the ball that will be detected in the next image will both exist within the analysis object region.
[0096] The sensing processing unit can be like Figure 5 As shown in (b), sphere b2 is detected by searching the interior of the analysis object region R1 on the fr2 image. The position coordinates of the detected sphere b2 correspond to... Figure 6 The position of sphere B2 in three-dimensional space is shown.
[0097] Depend on Figure 6 It can be seen that the position of ball B2 is within the three-dimensional region RD1 of the previously movable range of the ball. Similarly, the three-dimensional region RD2 of the movable range of the ball can be calculated based on the position of ball B2.
[0098] The following calculation can be performed by projecting the three-dimensional region RD2 of the ball's movable range onto the surfaces observed by cameras 110 and 120: Figure 5 The analysis object region R2 is shown in (b).
[0099] Figure 5 The R2 region shown in (b) is calculated based on the projection of the three-dimensional region RD2, which is the range of the ball's movement relative to the position of the ball B2, and can be set as the analysis object region for the next image.
[0100] Figure 5 (c) shows the next image fr3 of the fr2 image. By applying the previously set analysis target area R2, the sensing processing unit can detect the ball b3 by searching inside the analysis target area R2 when analyzing the fr3 image. The remaining area outside the analysis target area R2 is the area NR that has not undergone image processing or data processing.
[0101] Here, pb2 is the position of the ball detected in the previous image fr2. If the analysis object region R2 is calculated in the fr2 image, the position of the reference ball pb2 and the position of the ball that will be detected in the next image will both exist within the analysis object region.
[0102] The sensing processing unit can be like Figure 5 As shown in (c), sphere b3 is detected by searching the interior of the analysis object region R2 on the fr3 image, and the position coordinates of the detected sphere b3 correspond to... Figure 6 The position of sphere B3 in three-dimensional space is shown.
[0103] Depend on Figure 6 It can be seen that the position of ball B3 is within the three-dimensional region RD2 of the previously movable range of the ball. Similarly, the three-dimensional region RD3 of the movable range of the ball can be calculated based on the position of ball B3.
[0104] The calculation can be performed by projecting the three-dimensional region RD3 of the ball's movable range onto the surfaces observed by cameras 110 and 120. Figure 5 The analysis object region R3 is shown in (c).
[0105] Figure 5 The R3 region shown in (c) is calculated based on the projection of the three-dimensional region RD3, which is the range of the ball's movement relative to the position of the B3 ball, and can be set as the analysis object region for the next image.
[0106] Figure 5 (d) shows the next image fr4 of the above-mentioned fr3 image. By applying the previously set analysis target area R3, the sensing processing unit can detect the ball b4 by searching inside the analysis target area R3 when analyzing the fr4 image. The remaining areas outside the analysis target area R3 are areas NR that have not undergone image processing or data processing.
[0107] Here, pb3 is the position of the ball detected in the previous image fr3. If the analysis object region R3 is calculated in the fr3 image, the position of the reference ball pb3 and the position of the ball that will be detected in the next image will both exist in the analysis object region.
[0108] The sensing processing unit can be like Figure 5 As shown in (d), sphere b4 is detected by searching inside the analysis object region R3 on the fr4 image. The position coordinates of the detected sphere b4 can also be calculated as the position coordinates of the sphere in three-dimensional space.
[0109] In this way, the positions of balls such as b2, b3, and b4 can be detected in the images of each camera by using the analysis object area set in the previous images, and the position coordinate information of the balls in three-dimensional space can be quickly calculated.
[0110] Here, the analysis object area mentioned above can be either recalculated and set for each image under new conditions (flexible application), or it can be an analysis object area of a specified size applied to all images being analyzed (uniform application).
[0111] Under the aforementioned "flexible application" scenario, the analysis object area for each image can be reset to reflect the distance relationship between the ball and the camera when the ball moves in actual space.
[0112] Figure 7 This is a diagram used to illustrate the distance relationship between the ball and camera 110 as the ball moves, such as... Figure 7 As shown, when the ball moves from position B1 to position B8, the distance between the ball and the camera 110 that detects the ball continues to change.
[0113] like Figure 7 As shown, the distances from each of positions B1 to B8 to the camera 110 are d1 to d8, respectively. It can be seen that each of these distances will be different from the others.
[0114] As the distance between the camera and the ball gets closer, the size of the ball appears larger in the image captured by the camera, and the distance between balls within a frame appears farther; conversely, as the distance between the camera and the ball increases, the size of the ball appears smaller in the image captured by the camera, and the distance between balls within a frame appears closer.
[0115] Therefore, even if the same "limiting physical quantity of ball movement" is applied to each image, considering the distance relationship between the camera and the ball when the ball moves, the size of the analysis object area will inevitably be different for each image.
[0116] Therefore, the sensing processing unit of the sensing device in one embodiment of the present invention can calculate a three-dimensional region of the range in which the ball can move by reflecting information about the change in the relative positional relationship between the moving ball and the camera for each acquired image, and apply an analysis object region of different sizes to each image by calculating and setting the analysis object region from the calculated three-dimensional region.
[0117] On the other hand, under the aforementioned "uniform application" scenario, the change in distance between the ball and the camera when the ball moves in the actual space is calculated in advance, and a specified size analysis object area that can be uniformly applied regardless of the ball's position is defined accordingly, and will be uniformly applied to each image.
[0118] At this point, by placing the position of the ball detected for each image at a relatively identical position within a uniformly applicable analysis object region, and applying the analysis object region to each image, it is possible to achieve the following: Figure 5 As shown, the ball can always be detected within the area of the analysis object.
[0119] That is, the sensing processing unit of the sensing device in one embodiment of the present invention can pre-calculate the change in the relative positional relationship between the moving ball and the camera, and pre-set a defined three-dimensional region of a size that can include both the current ball position and the ball position in the next image at any position (meaning not that the three-dimensional region includes both the current "ball" and the "ball" in the next image, but has a three-dimensional region of a size or shape that can include both the current "ball position" and the "ball position" in the next image), and calculate and set the analysis object region from the defined three-dimensional region as described above based on the ball position for each acquired image.
[0120] On the other hand, as mentioned earlier, the analysis object area is calculated from the three-dimensional area of the ball's movable range based on the ball's position in the previous image, and it is calculated based on the limiting physical quantities of the ball's movable range. At this time, the limiting physical quantities of each type of motion can be different.
[0121] For example, the maximum speed of a golf ball hit by a golf club in golf is different from the maximum speed of a baseball in baseball, and the maximum speed of a ball in tennis is also different. That is, the maximum speed that a human can achieve with a ball is different for each sport, so the aforementioned limiting physical quantities of the ball can also be defined and used for each type of sport.
[0122] Furthermore, in golf, since the maximum ball speed achievable by each type of golf club is different, the limit physical quantities of the ball as described above can be defined differently for each golf club to calculate the three-dimensional region of the ball's range of motion and the region of analysis.
[0123] That is, by pre-setting the limit physical quantity of ball movement for each type of golf club hitting the ball, the sensing device identifies the type of golf club the user hits the ball and applies the limit physical quantity corresponding to the identified golf club type to calculate the three-dimensional area of the ball's movement range as described above, and accordingly calculates the limit physical quantity that is applied differently to the analysis object area for each golf club, which becomes the basis of the analysis object area.
[0124] Furthermore, since the range of launch angles of the ball hit by each type of golf club is different, different launch angle ranges can be selected for each type of golf club, and this can be reflected in the definition of the limiting physical quantity corresponding to the golf club to calculate the three-dimensional region and the region of analysis.
[0125] The data regarding the ball launch angle of each golf club can be arbitrarily set based on the generally known launch angle of each golf club, or the maximum launch angle range of each golf club can be selected from the cumulative recorded data of shots taken by the user through virtual golf simulators, golf practice devices, or golf analysis devices, or the average launch angle range of each golf club can be calculated and used in the calculation of the aforementioned limiting physical quantities.
[0126] As described above, the image analysis method for sensing a moving ball according to the present invention and the sensing device using the same method do not analyze the entire image in the analysis of images for sensing a ball moving in motion. Instead, an analysis object region is set as a partial region, and the ball is detected within the set analysis object region. Physical quantities related to the movement of the ball are calculated. By setting the analysis object region on the image being analyzed in the current image, based on the position of the detected ball, and including the position where the ball will appear on the next image to be analyzed, the previously set analysis object region is applied to the current image for each acquired image to perform image analysis, thereby significantly improving the speed of image processing.
[0127] Industrial utilization potential
[0128] The image analysis method for sensing a moving ball and the sensing device using it of the present invention can be applied to industries related to ball sports such as golf, and virtual simulation systems such as screen golf, where users practice or enjoy ball sports in a virtual environment.
Claims
1. An image analysis method for sensing a sphere, characterized in that, include: The steps of acquiring multiple frames of imagery from the perspective of a camera moving toward the space of the ball; The steps to detect the position of a ball from an acquired image frame; The step of defining the analysis object region based on the detected position of the ball, including the position where the ball will appear in the next image frame; as well as The step of detecting the position of the ball within the defined analysis object region in the next image frame after the defined image frame that utilizes the analysis object region. The steps for setting the analysis object area, including determining the position where the ball will appear in the next image frame, include: The step of calculating the three-dimensional region of the range that the ball can move, using pre-set information, from the detected position of the ball as the limiting physical quantity that the ball can move; as well as The step of defining the area of the three-dimensional region, including the range in which the ball can move, as the part of the surface observed by the camera as the analysis object region.
2. The image analysis method for sensing a sphere according to claim 1, characterized in that, Also includes: The step of setting a new analysis object region based on the position of the ball detected within the set analysis object region, including the position where the ball will appear in the next image frame; as well as The step of detecting the position of the ball within the new analysis object region in the next image frame. Thus, by resetting the analysis object area of the next image frame based on the position of the ball detected in the analysis object area of the current image frame, image analysis for sensing the ball is performed only within the set analysis object area for each acquired image.
3. An image analysis method for sensing a sphere, characterized in that, include: The steps of acquiring multiple frames of imagery from the perspective of a camera moving toward the space of the ball; The steps to detect the position of a ball from an acquired image frame; The step of defining the analysis object region based on the detected position of the ball, including the position where the ball will appear in the next image frame; as well as The step of detecting the position of the ball within the defined analysis object region in the next image frame after the defined image frame that utilizes the analysis object region. The step of acquiring multiple frames of imagery from the perspective of a camera moving toward the sphere includes: The process involves each of multiple interconnected, three-dimensional cameras acquiring images. The step of defining the analysis object region in a manner that includes the position where the ball will appear in the next image frame includes: The steps include: using the detected position of the ball as the limiting physical quantity of the ball's movement, and calculating a three-dimensional region of the range of motion that the ball can move using pre-defined information; and... The step of projecting a three-dimensional region of the range in which the ball can move onto each of the plurality of cameras and setting each region of the observed surface as the analysis object region of the image of each of the cameras.
4. The image analysis method for sensing a sphere according to claim 1 or 3, characterized in that, Each acquired image reflects information about the change in the relative positional relationship between the moving ball and the camera to calculate a three-dimensional region of the range in which the ball can move, and the analysis object region is calculated and set from the calculated three-dimensional region.
5. The image analysis method for sensing a sphere according to claim 1 or 3, characterized in that, The change in the relative positional relationship between the moving ball and the camera is calculated in advance, and a defined three-dimensional region is pre-set that includes the current ball position and the ball position in the next image at any position. For each acquired image, the analysis object region is calculated and set from the defined three-dimensional region with the ball position as the reference.
6. The image analysis method for sensing a sphere according to claim 1 or 3, characterized in that, For each type of ball, the golf club pre-sets the limit physical quantities that the ball can move, and In the step of calculating the three-dimensional region of the range in which the ball can move, the type of golf club used by the user to hit the ball is identified, and a limiting physical quantity corresponding to the identified type of golf club is applied to calculate the three-dimensional region of the range in which the ball can move.
7. A sensing device that senses a ball by analyzing images of a ball that has been struck and moved, the sensing device being characterized by comprising: A camera device that captures multiple frames of images from a perspective of the space in which the ball moves. as well as A sensing processing device detects the position of a ball from an acquired image frame, and, based on the detected position of the ball, defines an analysis target region in a manner that includes the position where the ball will appear in the next image frame. Furthermore, it performs image analysis for sensing the ball only within the defined analysis target region for each acquired image frame, in a manner that detects the position of the ball within the defined analysis target region in the next image frame following the image frame using the defined analysis target region. The sensing processing device is configured to calculate a three-dimensional region of the range of motion of the ball using pre-set information, based on the detected position of the ball as the limiting physical quantity of the ball's movement, and to define the region including the three-dimensional region of the range of motion of the ball as the area of the surface observed by the camera device as the analysis object region.
8. The sensing device according to claim 7, characterized in that, The sensing processing device is configured to calculate a three-dimensional region of the range in which the ball can move, based on information reflecting changes in the relative positional relationship between the moving ball and the camera in each acquired image, thereby recalculating and setting the analysis object region for each position of the ball.
9. The sensing device according to claim 8, characterized in that, The sensing processing device is configured to pre-calculate the change in the relative positional relationship between the moving ball and the camera, so as to pre-set an area of a size that can be used at any position within the field of view of the camera as the analysis object area, thereby uniformly applying the pre-set analysis object area to the image that becomes the object of analysis.
Citation Information
Patent Citations
Device and Method for measuring flight data of flying objects using high speed video camera and computer readable recording medium having program the same
KR1020180054279A
Device, system, and method for tracking an object using radar data and imager data
KR1020190085152A
Flight parameter measurement system
US7324663B2
Integrated golf ball launch monitor
US7497780B2
Device and method for sensing moving ball
CN107646125A
