Video analysis method for sensing a moving ball and a sensing device using the same

Abstract

The present invention provides an image analysis method for sensing a moving ball in ball sports, which does not analyze the entire image but sets an analysis target area which is a part of the image, detects the ball within the set analysis target area, calculates a physical quantity of the ball's movement based on the detected information, and sets the analysis target area to include the position where the ball will appear in the next image to be analyzed based on the position of the ball detected in the image currently being analyzed. The present invention provides an image analysis method for sensing a moving ball and a sensing device using the same, which significantly improves the speed of image processing by applying the analysis target area set immediately before to the current image for each acquired image and proceeding with image analysis.

Description

Technical Field

[0001] The present invention relates to a sensing device that senses a ball that is hit and moved by a player in the field of sports using a ball, using a camera, and a method for analyzing a camera image used therefor.

Background Art

[0002] Recently, virtual simulation systems for sports that are very restricted in direct enjoyment in the field, such as golf and baseball, have emerged and have been widely popularized.

[0003] Furthermore, virtual sports simulation systems for net sports in which players on both sides exchange balls across a net, such as tennis, squash, and badminton, have also emerged, and there is a tendency to enable users to easily enjoy various sports in a popular cultural space.

[0004] In such virtual sports simulations, basically, the game progresses while the player hits the ball. However, for the simulation of the ball hit by the player on the video, a sensing device that can effectively sense the moving ball is required.

[0005] In the past, optical sensing methods such as infrared sensors and laser sensors were often used. However, such optical sensing methods require a structure equipped with a large number of optical sensors that irradiate and receive light to be installed on the player's playing space. Therefore, there are limitations in restricting the player's play and realizing a realistic play, and there is a problem that the structure of the sensor is frequently damaged during the play process. Above all, there is a problem that the sensing accuracy is very low.

[0006] Due to the limitations and problems of such optical sensing methods, recently, sensing devices using an image sensing method that acquire camera images of a moving ball and calculate sensing information of the moving ball through analysis of the acquired images have been widely used.

[0007] Such an image sensing method has the advantage of being able to obtain very accurate sensing information because it analyzes images to obtain sensing information. However, since it is necessary to process each of the dozens to hundreds of frames per second of images, there is a problem that a camera device and a data processing device with very high data processing capabilities must be provided.

[0008] As related prior art, prior art documents such as Korean Patent Publication No. 10-2018-0054279, Korean Patent Publication No. 10-2019-0085152, US Patent Registration Publication US7,497,780, and US Patent Registration Publication US7,324,663 have been published.

[0009] The above-mentioned prior arts commonly disclose a method of designating a peripheral region where a ball exists as a region of interest (ROI) in an image including the ball photographed by a camera and analyzing the image centered on the region of interest.

[0010] However, the above-mentioned prior arts predict the position where the ball moves in order to set the region of interest, and designate a predetermined peripheral region based on the predicted position. However, in order to predict the position where the ball moves, a physical quantity for the movement of the ball up to that point is calculated, and another complex calculation for predicting the position where the ball moves using the calculated physical quantity must be performed. There was a problem that a complicated process had to be gone through for setting the region of interest.

[0011] However, even if the region of interest is set through such a complicated process, if the calculation for predicting the moving position of the ball is insufficient or the prediction is incorrect, there will also be a problem that the ball cannot be detected on the region of interest.

Prior Art Documents

Patent Documents

[0012]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0013] In the analysis of an image for sensing a ball moving in a sport using a ball, instead of analyzing the entire image, an analysis target area, which is a partial area, is set, and the ball is detected within the set analysis target area. Based on the detected information, a physical quantity related to the movement of the ball is calculated. Based on the position of the ball detected on the currently analyzed image, the analysis target area is set so as to include the position where the ball will appear on the next image to be analyzed. By applying the analysis target area set immediately before to the current image for each acquired image and proceeding with the image analysis, the present invention provides an image analysis method for sensing a moving ball and a sensing device using the same, which greatly improves the speed of image processing.

Means for Solving the Problems

[0014] A video analysis method for sensing a moving ball according to an embodiment of the present invention includes: a step of acquiring a video at an angle of view of a camera directed at a space in which the ball moves; a step of detecting the position of the ball from the acquired video; a step of setting an analysis target area so as to include a position where the ball appears on the next video based on the detected position of the ball; and a step of detecting the position of the ball within the set analysis target area on the next video of the video used for setting the analysis target area.

[0015] Also, preferably, a step of setting a new analysis target area so as to include a position where the ball appears on the next video based on the position of the ball detected within the set analysis target area, and a step of detecting the position of the ball within the new analysis target area on the next video of the video used for setting the new analysis target area are further included. In this way, video analysis for sensing the ball is performed only within the previously set analysis target area for each acquired video by newly setting the analysis target area of the next video based on the position of the ball detected within the analysis target area of the current video.

[0016] Also, preferably, the step of setting the analysis target area so as to include a position where the ball appears on the next video includes: a step of calculating a three-dimensional area within which the ball can move using information preset as a limit physical quantity within which the ball can move, at the detected position of the ball; and a step of setting, as the analysis target area, an area including a portion where the three-dimensional area within which the ball can move is projected onto the plane viewed by the camera.

[0017] Also, preferably, images are acquired by a plurality of cameras in a stereoscopic scope system that are interlocked with each other, and the step of setting an analysis target area so as to include the position where the ball appears on the next image includes, at the position of the detected ball, calculating a three-dimensional area of the range in which the ball can move by using information preset as the limit physical quantity in which the ball can move, and setting, as the analysis target area of each camera image, each area including the portion where the three-dimensional area of the range in which the ball can move is projected onto the plane viewed by each of the plurality of cameras.

[0018] Also, preferably, information regarding the change in the relative positional relationship between the moving ball and the camera is reflected for each acquired image, a three-dimensional area of the range in which the ball can move is calculated, and the analysis target area is calculated and set from the calculated three-dimensional area.

[0019] Also, preferably, the change in the relative positional relationship between the moving ball and the camera is calculated in advance, a constant three-dimensional area of a size that includes all the current ball position and the ball position in the next image at any position is preset, and the analysis target area is calculated and set from the constant three-dimensional area based on the position of the ball for each acquired image.

[0020] Also, preferably, the limit physical quantity in which the ball can move is preset for each type of golf club that strikes the ball, and the step of calculating the three-dimensional area of the range in which the ball can move includes identifying the type of golf club used by the user to strike the ball and calculating the three-dimensional area of the range in which the ball can move by applying the limit physical quantity according to the identified type of golf club.

[0021] On the one hand, a sensing device according to an embodiment of the present invention is a sensing device that senses a ball through video analysis of a ball that is struck and moves. It includes a camera device that acquires a video at an angle of view toward the space where the ball moves, and a sensing processing device that detects the position of the ball from the acquired video, sets an analysis target area so as to include the position where the ball appears on the next video based on the detected position of the ball, and detects the position of the ball within the set analysis target area on the next video of the video used for setting the analysis target area, and performs video analysis for sensing the ball only within the analysis target area set for each acquired video.

[0022] Also, preferably, the sensing processing device calculates a three-dimensional area within the range where the ball can move by using information preset as the limit physical quantity that the ball can move at the detected position of the ball, and is configured to set, as the analysis target area, an area including the portion where the three-dimensional area within the range where the ball can move is projected onto the surface viewed by the camera device.

[0023] Also, preferably, the sensing processing device reflects information regarding the change in the relative positional relationship between the moving ball and the camera for each acquired video, and calculates the three-dimensional area within the range where the ball can move, so that the analysis target area is newly calculated and set for each position of the ball.

[0024] Also, preferably, the sensing processing device calculates in advance the change in the relative positional relationship between the moving ball and the camera, and preset, as the analysis target area, an area of a size that can be used at any position within the camera angle of view, and the preset analysis target area is applied to the video to be analyzed in a batch manner.

Advantages of the Invention

[0025] The video analysis method for sensing a moving ball according to the present invention and the sensing device using the same set an analysis target area, which is a partial area, without analyzing the entire video in the analysis of a video for sensing a moving ball in a sport using a ball, detect the ball within the set analysis target area, calculate a physical quantity with respect to the movement of the ball, and set the analysis target area so as to include the position where the ball appears on the video to be analyzed next, based on the position of the ball detected on the currently analyzed video. By applying the analysis target area set immediately before to the current video for each acquired video and proceeding with video analysis, the speed of video processing can be greatly improved.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0027] Specific details of the video analysis method for sensing a moving ball according to the present invention and a sensing device using the same will be described in detail with reference to the drawings.

[0028] First, with reference to FIG. 1, the configuration of a sensing device for a moving ball according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing the configuration of a sensing device for a moving ball according to an embodiment of the present invention.

[0029] The sensing device according to the present invention and the video analysis method used therein can analyze a video captured by a camera of a ball moving, whether it is a single camera or a camera device configured in a stereoscopic manner with a plurality of cameras interlocked with each other, calculate the position coordinate information of the ball in the actual space, and based on this, calculate various physical quantities related to the movement of the ball, such as the speed of the ball, the direction angle of the ball, the spin of the ball, etc., and can be applied to all sports using the ball, such as golf, baseball, squash, bowling, etc.

[0030] In this specification, a sensing device used in the golf field among various sports using the ball as described above will be described as an example, but it goes without saying that the technical content applied thereto can be similarly applied to other sports.

[0031] FIG. 1 is a block diagram showing the configuration of a sensing device that senses when a user places a ball GB on a golf mat GM at a hitting station and hits the ball GB with a golf club GC in so-called screen golf.

[0032] As shown in FIG. 1, a sensing device for ball sensing according to an embodiment of the present invention photographs a moving ball when a user strikes a ball GB with a golf club GC, and analyzes the photographed video to obtain three-dimensional coordinate information of the ball moving in space and information regarding motion characteristics such as the initial velocity, direction angle, and height angle of the ball based on the three-dimensional coordinate information. It can be a device that calculates such information.

[0033] The above-described sensing device can be applied to various fields such as ball hitting analysis of a ball by a user's golf swing and virtual golf using a virtual reality-based simulation.

[0034] Such a sensing device according to an embodiment of the present invention can be configured to include a camera device 100 and a sensing processing unit 200 as shown in FIG. 1.

[0035] The above-described camera device 100 is configured to continuously acquire images within an angle of view looking at the moving golf ball. However, in order to calculate the position information in the three-dimensional space with respect to the moving golf ball, the above-described camera device 100 is a plurality of cameras that acquire images of the same object at different viewing angles, for example, as shown in FIG. 1, the first camera 110 and the second camera 120 are synchronized with each other and configured in a stereoscopic manner. It is desirable.

[0036] As described above, by configuring the camera device of the sensing device in a stereoscopic manner in which a plurality of cameras 110 and 120 are synchronized with each other, the two-dimensional information of the corresponding golf ball extracted from the images acquired through the first camera 110 and the images acquired through the second camera 120 for the same object (ball) can be converted into three-dimensional information.

[0037] As shown in FIG. 1, the sensing processing unit 200 can include a video receiving unit 210 that collects videos from a plurality of cameras 110 and 120 respectively, a video processing unit 220 that performs necessary video processing on the videos collected by the video receiving unit 210 to detect a ball from the videos, and an arithmetic unit 230 that is responsible for various calculations for sensing, such as calculating three-dimensional position information from the two-dimensional position information of the balls detected from each video.

[0038] As described above, the sensing processing unit 200 of the sensing device according to an embodiment of the present invention detects a moving ball from each video collected through each camera 110 and 120, calculates the position information of the ball, and transmits it to the client 300, so that the client 300 can perform client 300-specific functions such as calculating new information or calculating analysis information using the transmitted position information of the ball.

[0039] For example, when the client 300 is implemented as a simulator used in a screen golf system, the position information of the golf ball and the golf club is transmitted from the sensing processing unit 200, and using this, a simulation video of the trajectory of the virtual golf ball flying on the virtual golf course can be implemented.

[0040] Also, when the aforementioned client 300 is implemented as a golf swing analysis device, the position information of the golf ball and the golf club is transmitted from the sensing processing unit 200, and using this, it can be implemented to provide analysis information on the user's golf swing, diagnosis of swing problems, and lesson information for solving them.

[0041] The aforementioned video processing unit 220 is configured to perform video processing to extract the next calculation image of the reference image for each of the videos continuously acquired by the cameras 110 and 120, and the arithmetic unit 230 can be configured to calculate the position information of the moving ball from the videos processed by the video processing unit.

[0042] As a method for extracting a moving ball from the acquired video, in addition to the method using the next operation image as described above, a method may also be used in which a template image for the ball is prepared in advance, and a portion corresponding to the golf ball is extracted through the degree of similarity with the template image for the ball on each acquired video.

[0043] On the other hand, as described above, the camera device 100 of the sensing device according to an embodiment of the present invention is configured to acquire a video at an angle of view facing the space where the ball moves, and the sensing processing unit 200 detects the position of the ball from the acquired video, and based on the detected position of the ball, sets an analysis target area so as to include the position where the ball appears on the next video, and detects the position of the ball within the set analysis target area on the next video of the video used for setting the analysis target area, and can be configured to proceed with video analysis for sensing the ball only within the analysis target area set for each acquired video.

[0044] As described above, when the sensing processing unit 200 analyzes the video acquired by the camera device, instead of analyzing the entire acquired video, only the analysis target area corresponding to a partial area of the entire video is analyzed to calculate the position information of the ball, so that the data processing speed can be greatly improved.

[0045] On the other hand, with reference to FIGS. 2 and 3, a video analysis method for sensing a moving ball according to an embodiment of the present invention will be described.

[0046] FIG. 2 is a flowchart showing a sensing method of a sensing device according to an embodiment of the present invention, and FIG. 3 is a flowchart showing more specifically a video analysis method according to an embodiment of the present invention in the flowchart shown in FIG. 2.

[0047] As shown in FIG. 2, as the sensing device operates, the camera device acquires a video at an angle of view facing the space where the ball moves (S100).

[0048] The sensing processing unit can analyze a specific area on the acquired video (S110) and detect the position of the ball (S120).

[0049] For example, in the case of golf, since a golf ball is always struck at a fixed position such as a golf mat, if the position information of the golf mat is set in advance, when a video including the golf mat is obtained, the position of the golf mat on the video can be specified, and the initial ball position can be easily detected by investigating the area including the specified position of the golf mat. The initial positions of balls in other sports such as baseball can also be easily detected in the same way.

[0050] On the other hand, by detecting the position of the ball, the sensing device comes to be in a sensing ready state for the ball (S130), and it can sense whether the ball is struck and moves by sensing through the video continuously acquired of the position where the ball is placed.

[0051] If a sensing result for the ball indicates that ball movement has occurred (S140), that is, if the ball is struck and starts to move, the sensing processing unit can specify the time point when the above-mentioned ball movement occurs, and based on that time point, the videos acquired before and after that time can be collected as the videos to be analyzed (S150).

[0052] For example, it is possible to analyze how the golf club moves from the videos before and after the time point when the ball movement occurs, and it is possible to analyze how the ball moves from the time point when the ball movement occurs and the subsequent videos.

[0053] On the other hand, the sensing processing unit sets an analysis target area for each of the videos collected as described above, detects the ball position within the analysis target area of each video, and can calculate the position coordinate information of the detected ball (S200).

[0054] As described above, the sensing processing unit can calculate various physical quantities for the moving ball by using the position coordinate information of the ball calculated as described above (S310), and can transmit this to the client so that the client can implement its specific function (S320).

[0055] Here, the sensing device according to an embodiment of the present invention can enable fast video processing by analyzing an analysis target area, which is a partial area, for each of the videos collected in the above-described S200 step, without analyzing the entire video. The specific process for this is shown in FIG. 3.

[0056] As shown in FIG. 3, the sensing processing unit can calculate a three-dimensional area within which the ball can move by using the limit physical quantity of the ball, that is, information regarding the limit to which the ball can move, at the position of the ball detected in the above-described S120 step (S210).

[0057] The three-dimensional area within which the ball can move calculated as described above is projected onto the surfaces viewed by the respective cameras, and each camera can set a two-dimensional area including the above-described projected portion as the analysis target area for the next video (that is, the video to be analyzed next after analyzing the currently analyzed video) (S220).

[0058] The sensing processing unit can obtain the next video of the video analyzed in the above-described process and search for the ball within the analysis target area set previously on that video (S230). That is, since the ball is searched only within the previously set analysis target area instead of searching the entire video, the data processing speed can be significantly improved.

[0059] When detecting a ball within the analysis target area (S240), the sensing processing unit can calculate the position coordinate information of the detected ball (S250). That is, as a result of each camera analyzing its respective analysis target area in its respective video, it is possible to calculate and convert the position coordinates (two-dimensional coordinates) of the detected ball into position coordinate information in the actual space (three-dimensional space) for use.

[0060] Until sensing ends in this manner (S260), while repeating the process from step S210 to step S250, by applying the analysis target area calculated and set based on the ball position during the analysis of the previous video to the current video analysis and analyzing only the previously set analysis target area in the current video to calculate the position coordinate information of the ball, the data processing speed can be greatly improved.

[0061] If the ball is searched for within the analysis target area but not detected (S240), the range of the "three-dimensional area within which the ball can move" can be adjusted (S270) and recalculated, and based on this, the analysis target area can be recalculated and set again.

[0062] Thus, when analyzing a video related to ball movement, the present invention projects the "three-dimensional area within which the ball can move", derived by using the limiting physical quantity within which the ball can move based on the ball position during the analysis of the previous video, onto a two-dimensional area, and by applying the analysis target area set based on this to the current analysis video and analyzing only the previously set analysis target area in the current analysis video, the processing speed is improved.

[0063] More specific details regarding the calculation and setting of the "analysis target area" as described above will be explained with reference to FIGS. 4 to 7.

[0064] FIG. 4 is a drawing showing that a three-dimensional region of a range in which a ball can move is derived by using a limit physical quantity in which the ball can move, based on the ball position, by a video analysis method for sensing a moving ball according to an embodiment of the present invention, and this is projected onto the plane viewed by each camera to calculate an analysis target region.

[0065] As shown in FIG. 4, based on the position Po of the ball GB analyzed in the current video, that is, the position in the three-dimensional space, a three-dimensional region RD of a range in which the ball can move can be derived by using a limit physical quantity in which the ball can move.

[0066] The above-mentioned "three-dimensional region RD of a range in which the ball can move" is a region calculated as a three-dimensional region including all the ball positions on the next video from the ball position on the current video.

[0067] Such a three-dimensional region can be calculated by using the limit physical quantity of the ball. For example, the limit physical quantity described above can be calculated by using the maximum speed of a sports ball that a human can throw and the shooting speed of the camera.

[0068] For example, in the case of golf, the maximum speed of the ball that a human can throw recorded in the Guinness Book is 97 m / s, but assuming this is 100 m / s and the camera is an image sensor that shoots 500 frames per second, the distance that the ball can move during one frame of the camera video is 0.2 m. Based on the ball position in the previous video, the ball in the next video can only exist within a sphere with a radius of 0.2 m.

[0069] The sphere based on the ball position as described above can be effectively limited by reflecting the characteristics of the sport in which the sensing device is used.

[0070] For example, in the case of golf, since the direction in which the ball flies is constant, the rear hemisphere part of the sphere can be excluded. In the case of golf, since the ball rises upward with a predetermined launch angle by a shot, the lower side of the equator of the sphere does not need to be considered and can also be excluded (of course, in the case where the ball can move slightly downward on the tee as in a tee shot, the portion corresponding to a predetermined angle range below the equator of the sphere described above can be included).

[0071] The sphere based on the ball position as described above in such a manner can be clearly derived as the aforementioned "three-dimensional region RD of the range where the ball can move" by restricting it to the region where the ball must necessarily exist on the following video. The portion indicated by RD in FIG. 4 is exactly an example of the aforementioned "three-dimensional region of the range where the ball can move". The three-dimensional region RD of the range where the ball can move shown in FIG. 4 shows an example calculated as a three-dimensional region corresponding to approximately 1 / 4 of the sphere based on the position Po of the ball.

[0072] If the "three-dimensional region RD of the range where the ball can move" is calculated in space in this way, the three-dimensional region RD can be projected onto the surfaces (A1, A2) viewed by each camera 110, 120 respectively to calculate two-dimensional projection portions (pr1, pr2).

[0073] The two-dimensional region including the aforementioned two-dimensional projection portions (pr1, pr2) can be set as the analysis target regions (Ra1, Ra2) for each camera as shown in FIG. 4. When analyzing the following video, the ball can be detected by applying the aforementioned analysis target regions (Ra1, Ra2) and searching within those regions.

[0074] An example of the process of detecting the ball using the analysis target region in the manner described above will be described with reference to FIGS. 5 and 6.

[0075] Figures (a) to (d) of FIG. 5 are drawings showing the process of obtaining a video including fr1 to fr4 as the video analyzed by the sensing processing unit of the sensing device according to an embodiment of the present invention and analyzing each of these videos. FIG. 6 is for explaining the calculation of a three-dimensional area in which the ball for setting the analysis target area can move in the analysis process shown in FIG. 5. Figures (a) to (d) of FIG. 5 show the video acquired by one of the plurality of cameras.

[0076] Figure 5(a) is the video processed by the sensing processing unit and is the video (fr1) acquired by the camera with the ball placed in the golf format. Initially, a specific region of interest can be searched in the video to find the ball.

[0077] That is, in golf, since the ball always starts from above the golf format, by saving the position coordinates of the golf format in advance and searching for the ball within the region of interest based on the position of the golf format saved in advance on the video when the video is acquired, the ball can be detected.

[0078] Looking at Figure 5(a), the part gm corresponding to the golf format is the part whose position information is known in advance. Based on this, a region of interest Rm of a predetermined size is set, and the ball b1 can be searched for in the region of interest Rm.

[0079] The ball b1 found on the video in this way can be calculated with the coordinate information in the three-dimensional space and exists in the three-dimensional space as shown in FIG. 6. That is, the ball b1 found on the video in Figure 5(a) corresponds to the ball B1 in the three-dimensional space as shown in FIG. 6.

[0080] In the three-dimensional space shown in FIG. 6, by applying the limit physical quantity of the ball movement as described above, a three-dimensional area RD1 within which the ball can move with the position of the B1 ball as the reference can be calculated. The three-dimensional area RD1 is projected onto the surfaces viewed by each camera 110 and 120 to calculate the analysis target area R1 as shown in Figure 5(a).

[0081] The R1 region shown in Fig. 5(a) is calculated based on the projected part of the three-dimensional region RD1, which is the range where the ball can move with reference to the position of the B1 ball, and can be set as the analysis target region for the next video.

[0082] Fig. 5(b) shows the next video fr2 of the aforementioned fr1 video. When the sensing processing unit analyzes the fr2 video by applying the previously set analysis target region R1, it can detect the ball b2 by searching inside the R1 region of the analysis target region. The remaining region outside the analysis target region R1 is the non-processed region NR for video processing or data processing.

[0083] Here, pb1 is the position of the ball detected in the previous video fr1. If the analysis target region R1 is calculated in the fr1 video, the position pb1 of the reference ball and the position of the ball detected in the next video will all be within the analysis target region.

[0084] As shown in Fig. 5(b), the sensing processing unit can search inside the analysis target region R1 on the fr2 video to detect the ball b2, and the position coordinates of the detected ball b2 correspond to the position of the B2 ball in the three-dimensional space shown in Fig. 6.

[0085] As shown in Fig. 6, the position of the B2 ball is within the range of the three-dimensional region RD1, which is the range where the previous ball can move. In the same way, the three-dimensional region RD2, which is the range where the ball can move with reference to the position of the B2 ball, can be calculated.

[0086] The three-dimensional region RD2, which is the range where the aforementioned ball can move, is projected onto the surfaces viewed by the cameras 110 and 120, and the analysis target region R2 as shown in Fig. 5(b) can be calculated.

[0087] The R2 region shown in Fig. 5(b) is calculated based on the projected part of the three-dimensional region RD2, which is the range where the ball can move with reference to the position of the B2 ball, and can be set as the analysis target region for the next video.

[0088] Although (c) in FIG. 5 shows the next video fr3 of the aforementioned fr2 video, when the sensing processing unit analyzes the fr3 video by applying the previously set analysis target area R2, the ball b3 can be detected by searching inside the area of the analysis target area R2, and the remaining area other than the analysis target area R2 is an area NR where no video processing or data processing is performed.

[0089] Here, pb2 is the position of the ball detected in the previous video fr2. However, if the analysis target area R2 is calculated in the fr2 video, the position pb2 of the reference ball and the position of the ball detected in the next video will all exist within the analysis target area.

[0090] As shown in (c) of FIG. 5, the sensing processing unit can search inside the analysis target area R2 on the fr3 video to detect the ball b3, and the position coordinates of the detected ball b3 correspond to the position of the B3 ball in the three-dimensional space shown in FIG. 6.

[0091] As shown in FIG. 6, the position of the B3 ball is within the range of the three-dimensional area RD2 of the range where the previous ball can move, and the three-dimensional area RD3 of the range where the ball can move can be calculated based on the position of the B3 ball in the same way.

[0092] The three-dimensional area RD3 of the range where the aforementioned ball can move is projected onto the surfaces viewed by the cameras 110 and 120, and the analysis target area R3 as shown in (c) of FIG. 5 can be calculated.

[0093] The R3 area shown in (c) of FIG. 5 is calculated based on the projected part of the three-dimensional area RD3 of the range where the ball can move with the position of the B3 ball as the reference, and can be set as the analysis target area of the next video.

[0094] Figure 5(d) shows the next video fr4 of the aforementioned fr3 video. When the sensing processing unit analyzes the fr4 video by applying the previously set analysis target area R3, the ball b4 can be detected by searching inside the area of the analysis target area R3, and the remaining area outside the analysis target area R3 is the area NR where no video processing or data processing is performed.

[0095] Here, pb3 is the position of the ball detected in the previous video fr3. If the analysis target area R3 is calculated in the fr3 video, the position pb3 of the reference ball and the position of the ball detected in the next video will all be within the analysis target area.

[0096] As shown in Figure 5(d), the sensing processing unit can search inside the analysis target area R3 on the fr4 video to detect the ball b4, and the position coordinates of the detected ball b4 can also be calculated as the ball position coordinates in the three-dimensional space.

[0097] In such a manner, the ball positions such as b2, b3, and b4 can be detected from the videos of each camera by using the analysis target area set in the previous video on the video, and the position coordinate information of the ball in the three-dimensional space can be calculated quickly by using this.

[0098] Here, the aforementioned analysis target area can be newly calculated and set under new conditions for each video (fluid application), or an analysis target area of a certain size can be applied to all the videos to be analyzed (batch application).

[0099] In the case of the aforementioned "fluid application", when the ball in the actual space moves, a new analysis target area can be set and applied for each video by reflecting the distance relationship between the ball and the camera.

[0100] Figure 7 is a drawing for explaining the distance relationship with the camera 110 when the ball moves. As shown in Figure 7, when the ball moves from the B1 position to the B8 position, the distance between the ball and the camera 110 that senses the ball will continue to change.

[0101] As shown in FIG. 7, the distances from the B1 position to the B8 position to the camera 110 are d1 to d8 respectively, and it can be seen that the distances d1 to d8 are different from each other.

[0102] The closer the distance between the camera and the ball, the larger the size of the ball appears on the video captured by the camera, and the distance between the balls from frame to frame becomes farther and farther. The farther the distance between the camera and the ball, the smaller the size of the ball becomes on the video captured by the camera, and the distance between the balls from frame to frame becomes closer and closer.

[0103] Therefore, even if the same "limiting physical quantity of ball movement" is applied for each video, considering the distance relationship between the camera and the ball when the ball moves, the size of the analysis target area must change for each video.

[0104] Thus, the sensing processing unit of the sensing device according to an embodiment of the present invention calculates a three-dimensional area within which the ball can move by reflecting information on the change in the relative positional relationship between the moving ball and the camera for each acquired video, and calculates and sets the analysis target area from the calculated three-dimensional area, so that analysis target areas of different sizes can be applied for each video.

[0105] On the other hand, in the case of the aforementioned "batch application", when the ball in the actual space moves, the change in the distance between the ball and the camera is calculated in advance, and thereby a certain-sized analysis target area that can be applied uniformly regardless of the position of the ball is defined in advance, and this is applied uniformly for each video.

[0106] At this time, the position of the ball detected for each video is placed at a relatively same position on the batch application analysis target area, and by applying the analysis target area for each video, the ball can always be detected within the analysis target area as shown in FIG. 5.

[0107] That is, the sensing processing unit of the sensing device according to an embodiment of the present invention calculates in advance the change in the relative positional relationship between the moving ball and the camera, and at any position, it includes both the current ball position and the ball position in the next video (not including the current "ball" and the "ball" in the next video in a three-dimensional region, but a three-dimensional region having a size or shape that includes all of the current "ball position" and the ball position in the next video). A fixed-size three-dimensional region is set in advance, and for each acquired video, an analysis target region can be calculated and set from the fixed pre-dimension region as described above based on the ball position.

[0108] On the other hand, as described above, the analysis target region is calculated from a three-dimensional region within the range where the ball can move, with reference to the ball position in the previous video. This is calculated based on the limiting physical quantity of the ball's movement, but the limiting physical quantity at this time may vary depending on the type of sport.

[0109] For example, the maximum speed of a golf ball struck by a golf club in golf is different from the maximum speed of a baseball in baseball, and the maximum speed of a ball in tennis is also different.

[0110] That is, since the maximum speed of the ball that can be realized by humans differs for each sport, the above-described limiting physical quantity of the ball can also be defined and used for each type of sport.

[0111] Also, in golf, the maximum speed of the ball that can be hit differs depending on the type of golf club. Therefore, for each type of golf club, the above-described limiting physical quantity of the ball can be defined differently to calculate the three-dimensional region and the analysis target region within the range where the ball can move.

[0112] That is, the limit physical quantity that the ball can move can be preset according to the type of golf club that strikes the ball. The sensing device can identify the type of golf club that the user strikes the ball and apply the limit physical quantity according to the identified type of golf club to calculate the three-dimensional area within the range where the ball can move as described above, and thereby calculate the analysis target area, so that the limit physical quantity that is the basis of the analysis target area can be applied differently for each type of golf club.

[0113] In addition, since the launch angle range of the struck ball is different for each type of golf club, the launch angle range can be selected differently for each type of golf club, and the three-dimensional area and the analysis target area can be calculated by reflecting it in the definition of the limit physical quantity by the golf club described above.

[0114] Data on the launch angle of the ball for each type of golf club can also be arbitrarily set based on the generally known launch angle for each type of golf club. The maximum launch angle range for each type of golf club can be selected, or the average launch angle range for each type of golf club can be calculated from the accumulated recorded data of the user's shot data through a virtual golf simulation device, a golf practice device, a golf analysis device, etc., and used for the above-mentioned limit physical quantity calculation.

[0115] As described above, the "video analysis method for sensing a moving ball and a sensing device using the same" according to the present invention sets an analysis target area, which is a partial area, without analyzing the entire video in the analysis of the video for sensing a moving ball in a sport that uses a ball, detects the ball within the set analysis target area, calculates the physical quantity with respect to the movement of the ball, and sets the analysis target area so as to include the position where the ball appears on the video to be analyzed next, based on the position of the ball detected on the currently analyzed video. By applying the analysis target area set immediately before to the current video for each acquired video and proceeding with the video analysis, the speed of video processing can be greatly improved.

Industrial Applicability

[0116] The video analysis method for sensing a moving ball according to the present invention and the sensing device using the same can be used in industrial fields related to ball sports played using a ball such as golf, and industrial fields such as virtual simulation systems that allow users to practice or enjoy ball sports in a virtual environment like so-called screen golf where golf play can be enjoyed based on virtual golf simulation.

Claims

1. A step of acquiring an image at the angle of view of a camera facing the space where the ball moves; A step of detecting the position of the ball from the acquired image; A step of setting an analysis target area so as to include the position where the ball appears on the next image based on the detected position of the ball; A step of detecting the position of the ball within the set analysis target area on the next image of the image used for setting the analysis target area; including; The step of setting the analysis target area so as to include the position where the ball appears on the next image: A step of calculating a three-dimensional area within the range where the ball can move by using information preset as the limit physical quantity for the ball to move at the detected position of the ball; An image analysis method for sensing a ball, including a step of setting, as the analysis target area, an area including a portion where the three-dimensional area within the range where the ball can move is projected onto the plane viewed by the camera.

2. A step of setting a new analysis target area so as to include the position where the ball appears on the next image based on the position of the ball detected within the set analysis target area; and a step of detecting the position of the ball within the new analysis target area on the next image of the image used for setting the new analysis target area, further including; The image analysis method according to claim 1, wherein the analysis target area for the next image is newly set based on the position of the ball detected within the analysis target area of the current image, so that image analysis for sensing the ball is performed only within the previously set analysis target area for each acquired image.

3. A step of acquiring an image at the angle of view of a camera facing the space where the ball moves; A step of detecting the position of the ball from the acquired image; A step of setting an analysis target area so as to include the position where the ball appears on the next image based on the detected position of the ball; A step of detecting the position of the ball within the set analysis target area on the next image of the image used for setting the analysis target area, including; Images are acquired by a plurality of cameras in a stereoscopic system linked to each other; The step of setting the analysis target area so as to include the position where the ball appears on the next image: A step of calculating a three-dimensional area within the range where the ball can move by using information preset as the limit physical quantity for the ball to move at the detected position of the ball; A video analysis method for ball sensing, including the step of setting, as an analysis target area of the video acquired by each of the plurality of cameras, each area including a portion where a three-dimensional area of a range in which the ball can move is projected onto a surface viewed by each of the plurality of cameras.

4. The video analysis method according to claim 1 or 3, wherein, for each acquired video, information regarding a change in the relative positional relationship between the moving ball and the camera is reflected to calculate a three-dimensional area of a range in which the ball can move, and the analysis target area is calculated and set from the calculated three-dimensional area.

5. The video analysis method according to claim 1 or 3, wherein a change in the relative positional relationship between the moving ball and the camera is calculated in advance, a constant three-dimensional area of a size that includes all current ball positions and ball positions in the next video at any position is set in advance, and for each acquired video, the analysis target area is calculated and set from the constant three-dimensional area based on the position of the ball.

6. The limit physical quantity that the ball can move is preset for each type of golf club that strikes the ball. The step of calculating a three-dimensional area of a range in which the ball can move is The video analysis method according to claim 1 or 3, wherein the type of golf club used by the user to strike the ball is identified, and the limit physical quantity according to the identified type of golf club is applied to calculate a three-dimensional area of a range in which the ball can move.

7. A sensing device that senses a ball through video analysis of the ball moving after being struck, A camera device that acquires a video at an angle of view facing the space where the ball moves, A sensing processing device that detects the position of the ball from the acquired video, sets an analysis target area so as to include the position where the ball appears on the next video based on the detected position of the ball, and detects the position of the ball within the set analysis target area on the next video of the video used for setting the analysis target area, and performs video analysis for ball sensing only within the analysis target area set for each acquired video. including The sensing processing device is At the position of the detected ball, a three-dimensional region within which the ball can move is calculated using information preset as a limit physical quantity for the ball's movement, and a region including a portion where the three-dimensional region within which the ball can move is projected onto the surface viewed by the camera device is set as the analysis target region. A sensing device configured as such.

8. The sensing processing device By reflecting information regarding changes in the relative positional relationship between the moving ball and the camera device for each acquired video so that a three-dimensional region within which the ball can move is calculated, the analysis target region is newly calculated and set for each position of the ball. The sensing device according to claim 7, wherein the sensing device is configured to perform the above operation.

9. The sensing processing device Calculates in advance the change in the relative positional relationship between the moving ball and the camera device, and presets a region of a size that can be used at any position within the viewing angle of the camera device as the analysis target region, and applies the preset analysis target region to the video to be analyzed in a batch manner. The sensing device according to claim 7, wherein the sensing device is configured to perform the above operation.

Images