A method for visualizing the ball trajectory using a user's golf shot video, and a radar-based ball trajectory visualization device using the same.

Abstract

The present invention provides a method for visualizing a golf ball trajectory using a user's golf shot video, and a radar sensing-based ball trajectory visualization device using this method, which uses camera footage acquired to film the user taking a golf shot and provide it to the user, to sense the initial position of a golf ball placed on the bottom surface for the golf shot, and to visualize the trajectory of the golf ball calculated using radar sensing results for the golf ball moving due to the impact as an accurate ball trajectory starting from the initial position of the golf ball on the video.

Description

Technical Field

[0001] The present invention relates to a method for visualizing a ball trajectory on a captured video of a user's golf shot, which can visualize a ball trajectory starting from an accurate initial position and progressing using a captured video of a scene where the user makes a golf shot with a golf club and sensing results by a radar, and a ball trajectory visualization device based on radar sensing using the same.

Background Art

[0002] In the case of golf, when a user makes a golf shot with a golf club and strikes a golf ball, since the golf ball, which is very small in size, flies at a very high speed over a very long distance, it is very difficult to visually confirm the flight of the golf ball with the naked eye. <00**********11> Thus, since it is difficult to visually confirm the flight of the golf ball with the naked eye, it is also very difficult to confirm the trajectory along which the golf ball flies even in a video of a scene where the user makes a golf shot at a certain position.

[0004] [[ID=**********19]] In the case of a system such as screen golf, if a user makes a golf shot toward a front screen in a narrow space, a camera sensor senses the initial speed, direction angle, height angle, etc. of the golf ball struck by the golf club, and based on the initial movement characteristics of the golf ball, calculates the entire trajectory of the golf ball by physical calculation and shows it as a screen. However, this is only a trajectory estimated by calculation, not a measurement of the actual movement of the golf ball, so there is a difference from the actual trajectory of the golf ball.

[0005] ​As described above, camera sensors are sometimes used to sense the trajectory of a golf ball. However, as mentioned earlier, camera sensors have a very narrow sensing range and can only sense the initial movement characteristics of the golf ball. Therefore, they are not suitable for sensing the trajectory of a golf ball after a golf shot in a wide area like an actual golf course.

[0006] Radar sensors are widely used as typical sensing devices to detect the movement of a golf ball during a golf shot on an actual golf course.

[0007] A radar sensor is a device that analyzes the motion vector of a moving golf ball by receiving a radar signal that has been shifted due to the Doppler shift phenomenon of radar when a radar signal emitted towards the golf ball is reflected from the golf ball and then received by a radar antenna.

[0008] Using such radar sensors, it is possible to calculate the trajectory of a golf ball as it moves due to the user's golf shot.

[0009] However, since radar sensors use the Doppler shift phenomenon of radar signals, they can obtain phase change information in response to the positional change caused by the movement of the golf ball, and from there, they can calculate the movement trajectory using information about the positional change of the golf ball from the moment it starts moving. However, there is a problem in that it is difficult to confirm the position of the golf ball in the front-to-back direction, and the initial position of the golf ball when taking a golf shot, that is, the position of the golf ball when it is stationary on the bottom or on the tee, cannot be confirmed by the radar sensor.

[0010] For these reasons, even if a radar sensor detects the movement of a golf ball as it is struck and moves, and calculates its trajectory, it cannot accurately determine the golf ball's initial position. This results in a problem where the calculated trajectory of the golf ball starts from a position different from its actual initial position.

[0011] Since radar sensors cannot determine the initial position of a golf ball, a reference distance (distance from the radar sensor installed on the bottom) must be set in advance. Only when the golf ball is struck at a fixed position on that reference distance can a reasonably accurate trajectory starting from the golf ball's initial position be completed. However, if the user places the golf ball at an arbitrary position and takes a golf shot, the starting position of the golf ball's trajectory will differ considerably from the actual position, resulting in a significant error in the overall trajectory.

[0012] Attempts are being made to compensate for the shortcomings of both camera sensing and radar sensing methods by combining them to calculate the trajectory of a golf ball as it moves when struck by a golfer with a golf club. To sense the initial position of a golf ball placed at any given location, a stereoscopic camera system (a camera system in which two or more cameras placed at different locations sense the same subject and calculate the subject's position in three-dimensional space) can be used. However, if such a stereoscopic camera system and radar sensing system are implemented as a single piece of equipment, the cost of the equipment will increase significantly. Therefore, there is a need to develop equipment that can accurately visualize the trajectory of a golf ball using radar sensing without using a stereoscopic camera system with two or more cameras. [Prior art documents] [Patent Documents]

[0013] [Patent Document 1] Korean Patent Publication No. 10-1845503

[0014] [Patent Document 2] Korean Patent Publication No. 10-2335944

[0015] [Patent Document 3] Korean Patent Publication No. 10-2353481

[0016] [Patent Document 4] Korean Patent Publication No. 10-2033703 [Overview of the project] [Problems that the invention aims to solve]

[0017] The present invention provides a method for visualizing a golf ball trajectory using a user's golf shot video, and a radar sensing-based ball trajectory visualization device using this method, which uses camera footage acquired to film the user taking a golf shot and provide it to the user, to sense the initial position of a golf ball placed on the bottom surface for the golf shot, and to visualize the trajectory of the golf ball calculated using radar sensing results for the golf ball moving due to the impact as an accurate ball trajectory starting from the initial position of the golf ball on the video. [Means for solving the problem]

[0018] A method for visualizing a ball trajectory using radar sensing results from camera footage captured by a user taking a golf shot includes the steps of: setting a ball sensing area for golf ball sensing using the camera footage on the bottom surface where the user takes the golf shot; detecting a ball object corresponding to the golf ball within the ball sensing area on the camera footage and calculating the initial position coordinates of the ball, and calculating the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball; collecting radar sensing data for the golf ball moving due to the user's golf shot based on the actual initial position coordinates of the golf ball; and using the collected radar sensing data for the golf ball to display the ball trajectory starting from the initial position coordinates of the ball or corrected coordinates of the initial position coordinates of the ball on the camera footage.

[0019] Preferably, the step of setting the ball sensing area includes the step of setting the ball sensing area as the ball sensing area, where the camera is a single-view single camera, matching the coordinates of an element that can identify an area of ​​a predetermined size on the image acquired by the single camera with coordinates in real space, storing coordinate matching information, and setting the area of ​​a predetermined size on the image as the ball sensing area.

[0020] Preferably, the step of setting the ball sensing area includes the step of setting the ball sensing area by matching the coordinates of elements that can identify the specifications of a structure that appears on the image captured by the single camera, which is a single-view camera, with coordinates in real space, and storing coordinate matching information, and setting the area corresponding to the structure that appears on the image as the ball sensing area.

[0021] Preferably, the step of calculating the actual initial position coordinates of the golf ball includes the steps of: detecting a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera; calculating the position coordinates of the ball object as the initial position coordinates of the ball using coordinate information for the ball sensing area; and converting the initial position coordinates of the ball into the actual initial position coordinates of the golf ball, which are the coordinates of the golf ball in actual space, using coordinate matching information.

[0022] Preferably, the steps of pre-setting the height of the tee used by the user when making a golf shot and calculating the position coordinates of the ball object as the initial ball position coordinates include the steps of recognizing the pre-set tee height and applying the height information on the video corresponding to the recognized tee height to the position coordinates of the ball object detected within the ball sensing area to calculate the initial ball position coordinates.

[0023] Also, preferably, the step of collecting radar sensing data for the moving golf ball includes a step of collecting golf ball orientation information including the vertical height angle and the left-right direction angle of the golf ball in the actual space at the time of receiving a radar signal reflected from the golf ball, and the step of displaying the ball trajectory includes a step of converting and connecting the collected respective golf ball orientation information into coordinate information on the camera-acquired image based on the ball initial position coordinates, thereby displaying a ball trajectory starting from the ball initial position coordinates.

[0024] Also, preferably, the step of displaying the ball trajectory includes a step of verifying the ball initial position coordinates on the camera-acquired image using the radar sensing data at the time when the golf ball is struck, a step of correcting the ball initial position coordinates according to the verification result, and a step of converting the radar sensing data collected for the golf ball moving from the time of being struck into coordinate information on the camera-acquired image and displaying a ball trajectory starting from the corrected ball initial position coordinates on the image.

[0025] Also, preferably, the step of collecting radar sensing data for the moving golf ball includes a step of generating a trigger signal when the golf ball is struck, and a step of collecting golf ball orientation information including an up-down height angle and a left-right direction angle in the actual space of the golf ball at the time of receiving a radar signal reflected from the moving golf ball from the time the trigger signal is generated. The step of displaying the ball trajectory includes a step of converting the golf ball orientation information at the time the trigger signal is generated into coordinate information on the camera-acquired image based on the ball initial position coordinates to generate trigger-time ball coordinate information, a step of correcting the ball initial position coordinates using the trigger-time ball coordinates, and a step of converting and connecting the golf ball orientation information collected for the moving golf ball from the trigger time into coordinate information on the camera-acquired image based on the corrected ball initial position coordinates, thereby displaying a ball trajectory starting from the corrected ball initial position coordinates.

[0026] Also, preferably, the step of collecting radar sensing data for the moving golf ball includes a step of generating a trigger signal when the golf ball is struck, and a step of collecting golf ball orientation information including an up-down height angle and a left-right direction angle in the actual space of the golf ball at the time of receiving a radar signal reflected from the moving golf ball from the time the trigger signal is generated. The step of displaying the ball trajectory includes a step of calculating three-dimensional position information of the golf ball at the time the trigger signal is generated using the golf ball orientation information and the actual initial position coordinates of the golf ball at the time the trigger signal is generated, a step of calculating corrected coordinate information of the ball initial position coordinates by converting the calculated three-dimensional position information of the golf ball into coordinate information on the camera-acquired image, and a step of converting and connecting the golf ball orientation information collected for the moving golf ball from the trigger time into coordinate information on the camera-acquired image based on the corrected coordinates, thereby displaying a ball trajectory starting from the corrected coordinates.

[0027] On the other hand, a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention includes: a single-view camera that acquires images of the scene in which the user takes a golf shot on the bottom surface behind the bottom surface in which the user takes a golf shot; a sensing processing unit that sets a ball sensing area for golf ball sensing using the images acquired by the single camera, detects a ball object corresponding to the golf ball within the ball sensing area on the acquired images, calculates the initial position coordinates of the ball, and calculates the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball; a radar sensing unit that collects radar sensing data for the golf ball moving due to the user's golf shot with reference to the actual initial position coordinates of the golf ball; and a trajectory display processing unit that uses the collected radar sensing data for the golf ball to display the ball trajectory starting from the initial position coordinates of the ball or corrected coordinates of the initial position coordinates of the ball on the images acquired by the camera.

[0028] Preferably, the sensing processing unit is configured to store coordinate matching information by matching the coordinates of elements that can identify the specifications of a structure that appears on an image captured by the single camera in a field of view that includes a structure of known specifications placed on the bottom surface behind where the golf shot is taken, with coordinates in actual space, and setting the area corresponding to the structure that appears on the image as the ball sensing area, and to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object as the initial ball position coordinates using the coordinate information for the ball sensing area, and convert the initial ball position coordinates to the actual initial position coordinates of the golf ball, which are the coordinates in actual space of the golf ball, using the coordinate matching information.

[0029] Preferably, the sensing processing unit is configured to set in advance the height of the tee used by the user when taking a golf shot, and to match the coordinates of elements that can identify the specifications of a structure that appears on an image captured by the single camera in a field of view that includes a structure of known specifications placed on the bottom surface behind where the golf shot is taken, with coordinates in actual space, and store coordinate matching information, and to set the area corresponding to the structure that appears on the image as the ball sensing area, and to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object using the coordinate information for the ball sensing area, recognize the preset tee height, calculate the initial ball position coordinates by applying the height information on the image corresponding to the recognized tee height to the position coordinates of the ball object, and to convert the initial ball position coordinates to the actual initial position coordinates of the golf ball, which are the actual space coordinates of the golf ball, using the coordinate matching information.

[0030] Preferably, the radar sensing unit is configured to collect golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space at the time of receiving the radar signal reflected from the golf ball, and the trajectory display processing unit is configured to display the ball trajectory starting from the initial ball position coordinates by converting each of the golf ball orientation information collected by the radar sensing unit into coordinate information on the camera acquired image based on the initial ball position coordinates and concatenating them.

[0031] Preferably, the trajectory display processing unit is configured to verify the initial position coordinates of the ball on the camera-acquired image using radar sensing data at the time the golf ball is struck, correct the initial position coordinates of the ball based on the verification result, convert the radar sensing data collected by the radar sensing unit for the golf ball moving from the time of impact into coordinate information on the camera-acquired image, and display the ball trajectory starting from the corrected initial position coordinates of the ball on the image. [Effects of the Invention]

[0032] The present invention provides a method for visualizing a golf ball trajectory using a user's golf shot video, and a radar-sensing-based ball trajectory visualization device using the same. This method uses camera footage acquired to film the user taking a golf shot and provide it to the user. It senses the initial position of the golf ball placed on the bottom surface for the golf shot, and uses radar sensing results for the golf ball moving due to the impact to visualize the golf ball's movement as an accurate ball trajectory starting from the initial position of the golf ball in the video. [Brief explanation of the drawing]

[0033] [Figure 1] This figure shows a scenario in which a user takes a golf shot on a golf course or a similar field with a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention installed.

[0034] [Figure 2] This is a block diagram showing the configuration of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention.

[0035] [Figure 3] Figure 2 shows a more specific configuration of the radar sensing unit and the radar signal receiving unit of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention.

[0036] [Figure 4] This figure shows an example of calculating the position information of a golf ball using a radar sensing unit, as shown in Figure 3.

[0037] [Figure 5] Figure 4(c) shows an example of video footage captured by a single camera of a ball trajectory visualization device behind the user when they take a golf shot.

[0038] [Figure 6] This figure shows that a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention sets the ball sensing area using a structure whose specifications are known in advance, where (a) shows the state in actual space and (b) shows the image taken with a single camera in the state shown in (a).

[0039] [Figure 7] This figure shows that, with a user placing a golf ball on the bottom surface for a golf shot, a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention calculates the initial position coordinates of the ball using the ball sensing area. (a) shows a golf shot scene of user U in actual space, and (b) shows an image captured by a single camera in the state shown in (a).

[0040] [Figure 8] (a) is a diagram showing the movement of a golf ball when a user hits it in the state shown in Figure 7(a), and (b) shows the ball trajectory calculated using radar sensing results from video footage of the actual scene shown in Figure 8(a) captured by a single camera.

[0041] [Figure 9](a) shows images of a ball trajectory visualization device according to one embodiment of the present invention, when a golf ball is placed on the bottom surface and when a tee is inserted into the bottom surface and a golf ball is placed on the tee, and (b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0042] [Figure 10] This figure illustrates a ball trajectory visualization device according to one embodiment of the present invention, specifically the possibility of errors in the initial ball position coordinates calculated from video footage, and a method for correcting these errors.

[0043] [Figure 11] Figure 10 shows a video captured by a single camera of a golf ball flying when a user hits it during a golf shot. The video shows the ball's trajectory displayed based on the initially calculated initial ball position coordinates, and the ball's trajectory displayed after correcting the initial ball position coordinates to a more accurate position.

[0044] [Figure 12] This flowchart shows the process of visualizing the ball trajectory using a ball trajectory visualization device according to one embodiment of the present invention, based on a video of a user's golf shot. [Figure 13] This flowchart shows the process of visualizing the ball trajectory using a ball trajectory visualization device according to one embodiment of the present invention, based on a video of a user's golf shot. [Modes for carrying out the invention]

[0045] A method for visualizing the ball trajectory using a user's golf shot video according to one embodiment of the present invention, and a radar sensing-based ball trajectory visualization device using the same, will be described in detail with reference to the attached drawings.

[0046] A method for visualizing the ball trajectory in a user's golf shot video according to one embodiment of the present invention, and a radar-based ball trajectory visualization device using the same, are a device and method that can, when filming a scene in which a user takes a golf shot with a golf club in an outdoor location such as a golf course and providing the filmed video to the user, sense the golf ball that is hit and flies by the golf shot using radar, and visualize the ball trajectory based on the sensing result on the filmed video.

[0047] The configuration of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention will be described with reference to Figures 1 and 2.

[0048] Figure 1 shows a scene in which a user takes a golf shot on a golf course or a similar field with a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention installed, and Figure 2 is a block diagram showing the configuration of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention.

[0049] A radar sensing-based ball trajectory visualization device 100 according to one embodiment of the present invention may include a single-view single camera 200, a sensing processing unit 250 that calculates position information of a golf ball from the image acquired by the single camera, a radar sensing unit 300 that collects radar sensing data for a golf ball that is struck and moving, and a trajectory display processing unit 350 that displays the ball trajectory on the image acquired by the single camera using the radar sensing data collected by the radar sensing unit.

[0050] As shown in Figure 1, a ball trajectory visualization device 100 based on radar sensing according to one embodiment of the present invention can be used by positioning it so as to face from the rear to the front of the bottom surface BT where the user U holds the golf club 20 and strikes the golf ball 10 (the direction in which the golf ball flies after being struck).

[0051] The single camera 200 of the ball trajectory visualization device 100 can acquire video footage of the scene in which user U takes a golf shot, from behind the bottom surface BT where user U takes the golf shot, towards the front, with a single field of view.

[0052] The video footage of the user's golf shots is a simple two-dimensional image from a single camera, and not video acquired through multi-camera imaging using a stereoscopic system.

[0053] If images of a golf ball are acquired from multiple stereoscopic cameras, the golf ball's position in three-dimensional space can be calculated by analyzing each image. This allows for obtaining information that cannot be obtained from radar sensing data. However, radar sensors themselves are quite expensive, and stereoscopic camera systems are also quite expensive. Therefore, implementing both systems as a single piece of equipment would drastically increase costs, making commercialization extremely difficult.

[0054] A ball trajectory visualization device based on radar sensing according to one embodiment of the present invention has the features and advantages of being able to calculate information necessary for ball trajectory visualization at a level equivalent to or higher than the information that can be obtained using a stereoscope-type camera system, using a single camera which is basically necessary to provide video of the user's golf shot scene, and using this together with the radar sensing results to accurately display the ball trajectory on the video.

[0055] The information necessary for visualizing the ball trajectory using the single camera can be calculated by the sensing processing unit 250 from the video footage of the user's golf shot acquired by the single camera 200.

[0056] In other words, the sensing processing unit 250 of the radar sensing-based ball trajectory visualization device according to one embodiment of the present invention can perform the following functions: set a ball sensing area for golf ball sensing using the video acquired by the single camera 200, detect a ball object corresponding to a golf ball within the ball sensing area on the acquired video, calculate the initial position coordinates of the ball, calculate the actual initial position coordinates of the golf ball corresponding to those initial position coordinates, and transmit them to the radar sensing unit 300.

[0057] As described above, the radar sensing unit 300 of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention collects radar sensing data for the golf ball moving due to the user's golf shot, based on the actual initial position coordinates of the golf ball transmitted from the sensing processing unit 250. The trajectory display processing unit 350 can use the radar sensing data for the golf ball collected by the radar sensing unit 300 to display the ball trajectory starting from the initial position coordinates of the ball on the camera-acquired image.

[0058] The overall operation of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention, as described above, will be explained later. First, with reference to Figures 3 and 4, the process of obtaining radar sensing data from a moving golf ball using the radar sensing unit 300 of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention will be explained.

[0059] Figure 3 shows a more specific configuration of the radar sensing unit and the radar signal receiving unit of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention shown in Figure 2, and Figure 4 shows an example of calculating the position information of a golf ball using the radar sensing unit as shown in Figure 3.

[0060] The radar sensing unit 300 can calculate the phase change and position information of a golf ball at each measurement point while it is being struck and moving, basically using the Doppler effect of radar.

[0061] As shown in Figure 3(a), the radar sensing unit 300 may include a signal transmission unit 310, a signal receiving unit 320, and a signal analysis unit 330.

[0062] The radar sensing unit 300 may be configured to track the golf ball that has been struck and is moving by transmitting a radar signal of a specific frequency in the direction of movement of the golf ball that is being struck, at a position a predetermined distance behind the golf ball struck by the user, and receiving and analyzing the reflected waves reflected from the moving golf ball.

[0063] The signal transmission unit 310 is configured to transmit a specific radar signal St in the aiming direction and may include a transmitting antenna for transmitting radar signals, although this is not shown in the drawings.

[0064] The signal receiving unit 320 is configured to receive the reflected wave signal Sr that is reflected off the moving golf ball 10 from the radar signal St transmitted by the signal transmitting unit 310. The reflected wave signal Sr transmitted from the signal transmitting unit 310 and reflected off the golf ball 10 due to the Doppler effect is a signal in which the frequency of the signal transmitted by the signal transmitting unit 310 has been changed and a Doppler shift has occurred. In other words, the signal receiving unit 320 receives a signal in which a Doppler shift has occurred.

[0065] The signal receiving unit 320 is equipped with multiple receiving antennas that receive reflected wave signals, and can calculate the position information of the golf ball using the phase difference of the received signals from each receiving antenna.

[0066] Figure 3(b) shows a simplified example of the configuration of the signal receiving unit 320. As shown in the figure, the signal receiving unit 320 may be equipped with three or more appropriately arranged receiving antennas, including RA1, RA2, and RA3, and the trajectory (height angle) and direction angle of the moving golf ball 10 can be calculated using the phase difference of the signals between each receiving antenna.

[0067] For example, in the signal receiving unit 320 shown in Figure 3(b), the vertical height angle of the moving golf ball 10 can be calculated using the phase difference of the signals received by RA1 and RA2, and the horizontal direction angle of the moving golf ball 10 can be calculated using the phase difference of the signals received by RA1 and RA3.

[0068] The signal analysis unit 330 can analyze radar signals reflected by the golf ball at predetermined time intervals while the golf ball is moving, calculate phase information, and may be configured to collect azimuth information (including vertical height angle and horizontal direction angle information) of the moving golf ball as radar sensing data at predetermined time intervals.

[0069] Figure 4(a) shows that the radar sensing unit of the ball trajectory visualization device 100 based on radar sensing according to one embodiment of the present invention measures the left-right direction angles d1, d2, d3, ... at each measurement position B1, B2, B3, ... of the moving golf ball, and Figure 4(b) shows that the radar sensing unit measures the up-down height angles z1, z2, z3, ... at each measurement position B1, B2, B3, ... of the moving golf ball.

[0070] As shown in Figures 4(a) and 4(b), the radar sensing unit of the ball trajectory visualization device 100 based on radar sensing according to one embodiment of the present invention can calculate and collect information on the time at which it receives the reflected wave signal of the radar signal from the moving golf ball, as well as the left-right direction angle and up-down height angle at each measurement position.

[0071] In other words, as shown in the example in Figure 4(c), when user U takes a golf shot and the golf ball moves, the radar sensing unit can collect golf ball orientation information (information on the left-right direction angle and the up-down height angle) as radar sensing data, such as d1 and z1 at position B1, d2 and z2 at position B2, and d3 and z3 at position B3.

[0072] However, the golf ball orientation information collected by the radar sensing unit for a moving golf ball consists of angle information in the left-right direction (see Figure 4(a)) and angle information in the up-down direction (see Figure 4(b)) of the golf ball in actual three-dimensional space, and it is difficult to confirm information in the front-to-back direction of the golf ball from the radar sensing data.

[0073] By using the information on the lateral direction angle and vertical height angle of the golf ball obtained by radar as described above, it is possible to convert the golf ball orientation information at each measurement point into coordinate information on the video, and then connect these converted coordinates on the golf shot video, which is a 2D video captured by a single camera, to derive the ball's trajectory itself.

[0074] However, as mentioned above, radar sensing makes it difficult to measure the forward and backward position of the golf ball, for example, where the golf ball is located in front of the device. Therefore, radar sensing does not make it easy to determine where the ball's trajectory begins for a moving golf ball.

[0075] Figure 5 shows an example of video IM captured by a single camera of the ball trajectory visualization device 100 behind the point where user U takes a golf shot, as shown in Figure 4(c). In Figure 5, the drawing reference numeral iU represents the part of the video corresponding to the user, and iBi represents the object part corresponding to the golf ball placed in its initial position.

[0076] In Figure 5, when user iU hits ball iBi at its initial position, if the ball trajectory is accurately calculated and displayed, it can be shown as the dotted iTrr trajectory. However, if radar sensing data is collected for a moving golf ball and the ball trajectory is calculated, and the exact initial position of the ball is unknown, it can be displayed as the iTra trajectory or iTrb trajectory shown in Figure 5. This shows that the iTra trajectory and iTrb trajectory differ considerably from the actual ball trajectory iTrr trajectory.

[0077] A method for visualizing the ball trajectory using a user's golf shot video, and a radar-sensing-based ball trajectory visualization device using the same, according to one embodiment of the present invention, is characterized by accurately recognizing the initial position of the ball from video captured by a single camera, and accurately displaying the ball trajectory calculated using radar sensing data, showing that it starts from the recognized initial position of the ball and progresses accordingly. These features of the present invention will be explained in more detail with reference to Figures 6 to 8.

[0078] The sensing processing unit of a radar sensing-based ball trajectory visualization device according to one embodiment of the present invention can set a ball sensing area in order to identify the position coordinates of a golf ball placed at any position on the bottom surface where the user takes a golf shot, based on the image acquired by a single camera.

[0079] The sensing processing unit can identify an area of ​​a predetermined size on the image acquired by a single camera, match the coordinates of an element with coordinates in actual space, store the coordinate matching information, and set that area of ​​a predetermined size on the image as the ball sensing area.

[0080] As an example of a method for obtaining the "coordinates of an element that can identify an area of ​​a predetermined size," one can use a "structure whose specifications are known in advance," for example, a quadrilateral structure whose four side lengths are precisely known can be used.

[0081] Figures 6(a) and 6(b) show how a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention sets the ball sensing area using a structure whose specifications are known in advance. Figure 6(a) shows the state in actual space, and Figure 6(b) shows the image captured by a single camera in the state shown in (a).

[0082] Figure 6(a) shows the rectangular structure 120, which is described as a "structure whose specifications are known in advance," positioned within the camera's field of view.

[0083] The quadrilateral structure 120 shown in Figure 6(a) is a structure whose four side lengths, namely L1, L2, L3, and L4, are known in advance. As shown in Figure 6(b), the respective side lengths L1, L2, L3, and L4 correspond to the four side lengths l1, l2, l3, and l4 of the quadrilateral i120 on the video IM.

[0084] Furthermore, the elements that can identify the specifications of the quadrilateral structure 120 shown in Figure 6(a) are the four vertices E1, E2, E3, and E4, which correspond to the four vertices e1, e2, e3, and e4 that can identify the specifications of the quadrilateral i120 in the image shown in Figure 6(b).

[0085] If we define an xy coordinate system with position e1 as the origin on the video IM in Figure 6(b), we can determine the position coordinates of each vertex e1, e2, e3, and e4 in the xy coordinate system. That is, based on the xy coordinates, we can determine the coordinates of e1 as (0,0), e2 as (0,l2), e3 as (l3(=l1), l4(=l2)), and e4 as (l1,0).

[0086] As mentioned above, the position coordinates of the identified vertices e1, e2, e3, and e4 can be matched with the coordinates of the vertices of the structure 120 in real space, as shown in Figure 6(a).

[0087] In other words, in an XY coordinate system with position E1 as the origin in real space as shown in Figure 6(a), the coordinates of E1 can be identified as (0,0), the coordinates of E2 as (0,L2), the coordinates of E3 as (L3(=L1), L4(=L2)), and the coordinates of E4 as (L1,0). By matching these respective position coordinates of E1, E2, E3, and E4 with the respective position coordinates of e1, e2, e3, and e4 on the video, coordinate matching information can be calculated.

[0088] As mentioned above, coordinate matching information, which matches the coordinates of elements identified in the video with their corresponding coordinates in real space, can be saved and used in an important way in the process of visualizing the ball trajectory of a user's golf shot.

[0089] The region identified by e1, e2, e3, and e4 on the video IM is an xy coordinate plane, and any position within that coordinate plane can be identified by its xy coordinates. These identified xy coordinates can then be converted into coordinate information in real space using the coordinate matching information.

[0090] As described above, the sensing processing unit of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention can set the area identified by e1, e2, e3, and e4 on the video IM as the "ball sensing area".

[0091] As mentioned above, once the settings for the "ball sensing area" are complete, the structure can be removed, and the process of visualizing the ball trajectory from the user's golf shot can begin.

[0092] Figures 7(a) and 7(b) show that, with the user placing a golf ball on the bottom surface for a golf shot, a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention calculates the initial position coordinates of the ball using the ball sensing area. Figure 7(a) shows the user U taking a golf shot in real space, and Figure 7(b) shows the image captured by a single camera in the state shown in (a).

[0093] As shown in Figure 7(a), when the ball trajectory visualization device 100 according to the present invention is installed, and user U places a golf ball Bi on the bottom surface BT for a golf shot, as shown in Figure 7(b), in the video footage of that scene, as described above, the ball sensing region SA, which is specified by e1, e2, e3, and e4, is pre-set, so an object corresponding to the golf ball, i.e., the ball object iBi, can be detected within the ball sensing region SA, and the position coordinate information of the ball object iBi on the ball sensing region SA can be calculated.

[0094] Thus, the position coordinates of the ball object iBi within the ball sensing region SA on the video are called the "initial ball position coordinates".

[0095] By calculating the position coordinates of the ball object iBi, i.e., the initial position coordinates of the ball, using coordinate information for the pre-set ball sensing region SA, it is possible to convert the initial position coordinates of the ball in the video to the actual initial position coordinates of the golf ball Bi, which are the actual spatial coordinates of the golf ball, using the coordinate matching information described above.

[0096] As described above, the ball trajectory visualization device according to one embodiment of the present invention utilizes the initial ball position coordinates calculated from the video acquired by a single camera as a reference point for future ball trajectory display. The actual initial position coordinates of the golf ball, obtained by transforming the initial ball position coordinates using coordinate matching information, are used as the reference position to collect sensing data from the radar sensing unit and can be used to calculate the ball trajectory.

[0097] Figure 8(a) shows the movement of a golf ball as a user hits it in the state shown in Figure 7(a), and Figure 8(b) shows the ball trajectory calculated using radar sensing results from video footage of the actual scene shown in Figure 8(a) captured by a single camera.

[0098] In Figure 8(a), when user U takes a golf shot and hits the golf ball, the ball trajectory visualization device 100 according to one embodiment of the present invention generates a trigger signal at the time of impact. Upon generation of the trigger signal, the radar sensing unit can continuously collect radar sensing data for the golf ball as it moves from the trigger point onward.

[0099] Here, the aforementioned "radar sensing data" refers to the orientation information (vertical height angle and horizontal direction angle information) of the moving golf ball at each measurement point, as explained with reference to Figures 3 and 4 above.

[0100] The radar sensing unit can then collect azimuth information of the golf ball using the actual initial position coordinates Pbi of the golf ball, which are obtained by transforming the initial position coordinates of the ball, determined by the ball sensing area in the video, using coordinate matching information, as the reference position.

[0101] In other words, as shown in Figure 8(a), the radar sensing unit can collect orientation information (horizontal direction angle, vertical height angle), such as (d1, z1), (d2, z2), (d3, z3), etc., of the moving golf ball at each radar measurement point. Here, the golf ball orientation information can be measured based on the actual initial position coordinates Pbi of the golf ball.

[0102] As described above, the trajectory display processing unit of the ball trajectory visualization device according to one embodiment of the present invention can convert the orientation information of each golf ball collected by the radar sensing unit into coordinate information on the video image, based on the initial position coordinates pbi of the ball on the video image.

[0103] In other words, the golf ball orientation information (d1, z1), (d2, z2), (d3, z3), etc., as shown in Figure 8(a), can be converted into coordinate information p1, p2, p3, etc. on the video IM based on the ball's initial position coordinate pbi, as shown in Figure 8(b). For example, (d1, z1) can be converted to the p1 coordinate on the video, (d2, z2) to the p2 coordinate on the video, and (d3, z3) to the p3 coordinate on the video.

[0104] As described above, the trajectory display processing unit can accurately display the ball trajectory iTri, starting from the ball's initial position coordinates, on the video IM by concatenating the respective ball position coordinates p1, p2, p3, ..., which are calculated on the video IM based on the ball's initial position coordinates pbi.

[0105] As described above, the ball trajectory visualization device according to one embodiment of the present invention sets a ball sensing area using video footage from a single camera and saves coordinate matching information, which is matching information with coordinates in real space. The ball sensing area detects a ball object corresponding to a golf ball in the video footage when the user takes a golf shot and calculates the initial position coordinates of the ball. When the user hits the golf ball, a trigger signal is generated. From the trigger point onward, the radar sensing unit collects golf ball orientation information using radar sensing data for the moving golf ball. The trajectory display processing unit converts the collected golf ball orientation information into coordinate information on the video and displays the ball trajectory starting from the initial position coordinates of the ball, thereby enabling the visualization of the ball trajectory at an accurate position in the video.

[0106] However, when a user detects a ball-like object (equivalent to a golf ball) and calculates its initial position coordinates before taking a golf shot, the calculated initial position coordinates may differ from the actual position where the golf ball was initially placed.

[0107] This is because the video acquired by a single camera with a single field of view is a two-dimensional image, making it difficult to determine the actual position in three-dimensional space. Furthermore, the ball sensing area can only calculate the position on a plane, so for example, if a golf ball is placed on a tee in real space, the initial position coordinates of the ball calculated from the video may differ from the actual position.

[0108] Figure 9(a) shows images of the case where a golf ball is placed on the bottom surface, and when a tee is inserted into the bottom surface and the golf ball is placed on the tee. Figure 9(b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0109] As shown in Figure 9(a), the position of the ball object iB1 detected within the ball sensing region SA in the video IM acquired by a single camera corresponds to the same position Pb1 as the actual golf ball B1, as shown in Figure 9(b). Therefore, the position coordinates of the ball object iB1 as determined by the ball sensing region SA on the video IM, i.e., the initial position coordinates of the ball, are coordinates that match the initial position coordinates Pb1 of the actual golf ball B1.

[0110] In this case, the initial position coordinates of the ball object iB1 are matched to the actual position coordinates, so by displaying the ball trajectory obtained using radar sensing data as a trajectory starting from the initial position coordinates of the ball on the video, it is possible to visualize the accurate ball trajectory.

[0111] However, in the video IM acquired by a single camera as shown in Figure 9(a), the position of the ball object iB2 detected within the ball sensing region SA should actually correspond to the position of the golf ball B2b placed on the tee, as shown in Figure 9(b). However, the golf ball B2b in the line of sight of the single camera 200 is recognized as the position of the golf ball B2a placed on a plane.

[0112] Therefore, the initial ball position coordinates obtained by the ball sensing region SA for the ball object iB2 in the video are not based on the actual position of golf ball B2b, which is Pb2b, but rather on the position of golf ball B2a, which is Pb2a, which is a different position from the actual golf ball. Thus, it can be considered that an error occurred in the initial ball position coordinates.

[0113] To address such situations, the present invention calculates corrected initial ball position coordinates through a process for correcting the initial ball position coordinates, and displays the ball trajectory based on these corrected coordinates. This will be explained with reference to Figures 10 and 11.

[0114] Figure 10(a) shows a portion of the video footage taken when a golf ball is placed on a tee. As shown in Figure 10(b), the ball trajectory visualization device according to one embodiment of the present invention recognizes the ball object iB2 on the video IM at the position B2a on the bottom surface. Therefore, the initial ball position coordinates calculated therefrom are based on the position of the golf ball placed at B2a, which is different from the actual position of the golf ball.

[0115] A ball trajectory visualization device according to one embodiment of the present invention can verify whether the initial position coordinates of the ball on the image are correct using radar sensing data at the time the golf ball is struck, and can perform a process of correcting the initial position coordinates of the ball based on the verification result.

[0116] If a trigger signal is generated at the moment the golf ball is struck, the radar sensing unit can collect golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space, from the moment the trigger signal is generated until the moment the radar signal reflected from the golf ball is received.

[0117] In other words, as shown in Figure 10(b), golf ball orientation information such as P1a, P2a, and P3a is collected based on the position of golf ball B2a.

[0118] Here, the trajectory display processing unit verifies the ball's initial position coordinates or the actual initial position coordinates of the golf ball based on them, using the golf ball orientation information calculated by the radar at the trigger point or the ball coordinates at the trigger point obtained by converting that orientation information into coordinates on the video. The unit then determines whether the ball's initial position coordinates are accurate and can correct them based on the verification results.

[0119] During the verification process described above, if, for example, the coordinates based on the golf ball's orientation information at the trigger point are determined to be far from the ball's initial position coordinates, then it can be determined that there is an error in the ball's initial position coordinates.

[0120] The trigger point is the moment when the golf ball is struck by the golf club while stationary and begins to move. Therefore, the position obtained by converting the golf ball's orientation information at the trigger point into coordinates on the video must be almost the same as, or very close to, the position of the ball's initial position coordinates. For example, as shown in Figure 10(b), if the position P1a of the golf ball at the trigger point, as sensed by the radar, is more than a certain distance away from the position B2a of the ball's initial position coordinates (the position coordinates of the ball object calculated by the ball sensing area on the video), it is difficult to consider that the golf ball started from the initially determined position B2a of the ball's initial position coordinates.

[0121] Therefore, the initial ball position coordinates obtained first can be corrected to more accurate position coordinates using the position of the golf ball at the trigger point, which is sensed by the radar.

[0122] For example, the height h of the single camera 200 of the ball trajectory visualization device 100, and the angle a of the line of sight in which the single camera 200 views the golf ball at that height, are known values. Using these, the line AL in the line of sight in which the single camera 200 views the golf ball at height h can be determined. The actual golf ball lies on this line AL in the line of sight.

[0123] Here, considering the height of the golf ball's position P1a at the trigger point, the actual position of the golf ball can be found within a certain range RB on the line AL in the line of sight direction, below the height of the golf ball's position P1a.

[0124] Since no Doppler shift occurs in the radar when the golf ball is stationary, its position cannot be determined by radar sensing data. The golf ball's position can only be determined when it starts moving. Therefore, the golf ball's position when it is stationary will have a slight difference (hereinafter referred to as "position difference") from its position at the trigger point. If this position difference is saved as an average value or other statistical value, it can be used to correct the actual position of the golf ball to its accurate position.

[0125] For example, in the case of Figure 10(c), within a certain range of section RB on the line AL in the line of sight, it is possible to identify the position corresponding to the average value of the position difference described above as the actual initial position of the golf ball, based on the position P1a of the golf ball at the trigger point.

[0126] Therefore, by converting the actual position of the golf ball, as described above, into coordinates on the video, the converted coordinates can be calculated as the corrected coordinates of the ball's initial position, which were initially determined.

[0127] Figure 11 shows the ball trajectory displayed based on the initial ball position coordinates initially calculated from video IM (Information Memory) footage taken with a single camera of the golf ball flying as the user hits the golf ball during a golf shot, as shown in Figure 10, and the ball trajectory displayed after correcting the initial ball position coordinates to a more accurate position.

[0128] In the video shown in Figure 11, iB2 represents the ball object that was initially placed.

[0129] In the video IM shown in Figure 11, the point displayed by pb2a corresponds to the initial ball position coordinates calculated using the ball sensing area, and the trajectory shown by iTr1 is the ball trajectory calculated from the radar sensing results of the golf ball based on this point.

[0130] As shown in Figure 11, the initial ball position coordinate pb2a, which was initially determined, was found to be an error when verified using radar sensing data at the trigger point, as explained earlier in Figure 10. Therefore, the iTr1 ball trajectory starting from the initial ball position coordinate pb2a in the video IM differs slightly from the actual trajectory.

[0131] As described above, this invention verifies the initial ball position coordinates using radar sensing data, and if an error is detected, it corrects the error to calculate the accurate initial ball position coordinates, and displays the ball trajectory based on these corrected position coordinates.

[0132] In Figure 11, the initial ball position coordinates pb2a, which were initially determined, are corrected to a more accurate position using radar sensing data at the trigger point, and the position indicated by the diagram symbol pb2b is the ball trajectory iTr2, which starts from the pb2b position.

[0133] The present invention has the features and advantages of being able to obtain more accurate initial ball position coordinates by verifying and correcting the initial ball position coordinates initially obtained from the video of the user's golf shot acquired by a single camera using the method described above, using radar sensing data, and visualizing the accurate ball trajectory starting from these corrected position coordinates on the video.

[0134] The above describes a configuration and method by which a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention can obtain more accurate initial ball position coordinates by verifying and correcting the initial ball position coordinates obtained initially using radar sensing data. In connection with this, a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention can provide a configuration and method that can obtain accurate initial ball position coordinates from the beginning using information set in advance before sensing is performed.

[0135] As mentioned above, the present invention calculates the initial position coordinates of the ball using a single camera and is not based on the position coordinates of the golf ball in three-dimensional space. For example, when a user actually places a golf ball on a tee and takes a golf shot, analyzing the video captured by the single camera to determine the initial position coordinates of the ball will result in an error equal to the height of the tee. However, the present invention corrects this error by verifying the initially determined initial position coordinates of the ball using radar sensing data.

[0136] Furthermore, as mentioned above, in the case of a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention, when a tee is used, the height of the tee is set in advance, and the initial position coordinates of the ball are calculated by considering the height on the image corresponding to the set tee height, thereby providing a function that can calculate the initial position coordinates of the ball more accurately from the beginning.

[0137] Specifically, a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention may have a configuration that allows it to sense the current location information by incorporating GPS or by using other sensors, and may also be configured to pre-store or pre-recognize topographic information or map information of the golf course where the user is currently playing golf.

[0138] With this configuration, a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention can recognize the location of the hole on the golf course where the golf game is currently being played, and can also determine whether the hole currently being played is a PAR3 hole, a PAR4 hole, or a PAR5 hole.

[0139] Prior to starting a golf game, the user can pre-set the tee height to be used for each hole of the golf course using a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention (the user can also download and recognize tee height information previously saved by the user via a server, or the user can directly input and pre-set the height into the device).

[0140] For example, the user can pre-set the tee height to be 1.5 cm for the 2nd hole (PAR 3 hole), 4.5 cm for the 3rd hole (PAR 4 hole), and 4.5 cm for the 4th hole (PAR 5 hole).

[0141] Therefore, a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention can recognize the location of the hole on the golf course where the golf game is currently being played, recognize the tee height used by the user on the current hole, and when calculating the initial ball position coordinates from the video captured by a single camera, it can calculate accurate initial ball position coordinates by taking into account the height on the video relative to the preset and recognized tee height.

[0142] Thus, the initial ball position coordinates calculated considering the tee height can eliminate or significantly reduce the errors mentioned above. Furthermore, as previously stated, even when calculating the initial ball position coordinates considering the tee height, it is possible to calculate more accurate initial ball position coordinates by further correction through verification using radar sensing data.

[0143] If multiple users play a round of golf together and all of them use the ball trajectory visualization device according to the present invention, the present invention can store the tee height information set by each of the multiple users. When a specific user takes a golf shot, the caddy or another user sets the user currently taking the shot on a golf terminal, and by receiving the user's shot sequence information from the golf terminal, the tee height information that the user has set in advance can be applied. In this manner, the tee height set individually for each of the multiple users can be applied to the visualization of individual golf shots.

[0144] On the other hand, with reference to the flowcharts shown in Figures 12 and 13, respectively, a method for visualizing the ball trajectory in a user's golf shot video according to one embodiment of the present invention will be explained.

[0145] Since we have already described in detail how to visualize the ball trajectory using a radar sensing-based ball trajectory visualization device according to one embodiment of the present invention with a user's golf shot video, we will now summarize and explain the ball trajectory visualization method again based on the flowcharts shown in Figures 12 and 13.

[0146] The flowchart shown in Figure 12 is linked to the flowchart shown in Figure 13 via reference symbol A, and together they represent a single process.

[0147] First, a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention is placed at a predetermined position behind the bottom surface where the user takes a golf shot (S100).

[0148] The sensing processing unit of the ball trajectory visualization device matches the coordinates of an element that can identify a predetermined area on the video acquired by a single camera with coordinates in real space and stores the coordinate matching information, and can set the area identified by the coordinates of the element that can identify a predetermined area on the video as the ball sensing area (S110).

[0149] The user positions the golf ball at any position on its bottom surface within the field of view of the single camera and prepares for a golf shot. At this time, the single camera acquires video footage, and the sensing processing unit searches for a pre-set ball sensing area using the video footage acquired by the single camera (S120).

[0150] Here, if the ball sensing area search results and a ball object corresponding to a golf ball placed on the bottom surface is detected (S130), the sensing processing unit can use the coordinate information of the ball sensing area to calculate the position coordinates of the detected ball object as the ball's initial position coordinates (S140).

[0151] The sensing processing unit can use coordinate matching information to convert the ball's initial position coordinates into the actual initial position coordinates of the golf ball, which are the coordinates of the golf ball in real space, and transmit this to the radar sensing unit (S150).

[0152] Subsequently, when the user hits the golf ball with the golf club (S160), the ball trajectory visualization device first generates a trigger signal (S170), and the radar sensing unit operates in response to that trigger signal.

[0153] The radar sensing unit transmits a radar signal towards the golf ball as it moves due to impact, receives the reflected wave signal from the golf ball, and can collect golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball at the time of measurement (S180).

[0154] The trajectory display processing unit of the ball trajectory visualization device can verify the initial position coordinates of the ball or the actual initial position coordinates of the golf ball based on this, using the golf ball orientation information calculated by radar at the trigger point or the ball coordinates at the trigger point obtained by converting that orientation information into coordinates on the video (S200). For example, it can determine whether the initial position coordinates of the ball calculated earlier in step S140 are accurate.

[0155] If, as described above, the verification results in step S140 indicate that there are no errors in the initial ball position coordinates and no correction is necessary (S210), the trajectory display processing unit converts the golf ball orientation information collected by the radar sensing unit at each measurement point into coordinates on the image based on the initial ball position coordinates and concatenates them, thereby displaying the ball trajectory starting from the initial ball position coordinates on the image (S220).

[0156] However, if the verification results described above indicate an error in the ball's initial position coordinates calculated in 140 steps and it is determined that correction of the ball's initial position coordinates is necessary (S210), the trajectory display processing unit can correct the ball's initial position coordinates calculated in step S140 to accurate position coordinates based on the verification results (S230).

[0157] The trajectory display processing unit then converts the golf ball orientation information collected by the radar sensing unit at each measurement point into coordinates on the video based on the ball's initial position coordinates corrected in step S230, and connects them, thereby displaying the ball's trajectory on the video, starting from the ball's initial position coordinates corrected to a more accurate position (S240).

[0158] As shown in the flowchart in the drawing, the sensing processing unit of the present invention, as described above, pre-sets the tee height used by the user, and when calculating the initial ball position coordinates in step S140, it calculates the initial ball position coordinates by applying the pre-set tee height information, thereby enabling the calculation of accurate information.

[0159] As described above, the method for visualizing a ball trajectory using a user's golf shot video and the radar sensing-based ball trajectory visualization device using the same according to the present invention have the following features and advantages: they use camera footage acquired to film the user taking a golf shot and provide it to the user; they sense the initial position of the golf ball placed on the bottom surface for the golf shot; they can visualize the movement trajectory of the golf ball calculated using radar sensing results for the golf ball moving due to the impact as an accurate ball trajectory starting from the initial position of the golf ball on the video; and they can verify whether there is an error in the initial ball position coordinates calculated on the video using radar sensing data, correct it to a more accurate initial ball position coordinate if there is an error, and visualize it as an accurate ball trajectory starting from that corrected position. [Industrial applicability]

[0160] The method for visualizing the ball trajectory using a user's golf shot video according to the present invention, and the ball trajectory visualization device based on radar sensing using the same, can be used in golf-related industries such as sensing technology for golf balls, golf practice, and screen golf systems.

Claims

1. A method for visualizing the ball trajectory using radar sensing results from video footage captured by a camera that films a user taking a golf shot, The steps include setting a ball sensing area on the bottom surface where the user takes a golf shot, for sensing the golf ball using the image acquired by the camera, The steps include detecting a ball object corresponding to the golf ball within the ball sensing area on the image acquired by the camera, calculating the initial position coordinates of the ball, and calculating the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball, The steps include: collecting radar sensing data for the golf ball as it moves due to the user's golf shot, based on the actual initial position coordinates of the golf ball; The steps include: using the radar sensing data of the golf ball collected, displaying the ball's trajectory starting from the ball's initial position coordinates or corrected coordinates of the ball's initial position coordinates in the image acquired by the camera; A method for visualizing the ball trajectory using video footage of the user's golf shots.

2. The step of setting the ball sensing area is as follows: The aforementioned camera is a single-view camera, A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the steps of matching the coordinates of an element that can identify an area of ​​a predetermined size on the video acquired by the single camera with coordinates in real space and storing coordinate matching information, and setting the area of ​​a predetermined size on the video as the ball sensing area.

3. The step of setting the ball sensing area is as follows: The aforementioned camera is a single-view camera, A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the steps of: matching the coordinates of elements that can identify the specifications of a structure appearing in a video taken by a single camera placed on the bottom surface where the golf shot is taken at the rear, with coordinates in real space, storing coordinate matching information; and setting the area corresponding to the structure appearing in the video as the ball sensing area.

4. The step of calculating the actual initial position coordinates of the golf ball is as follows: The steps include detecting a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, Using the coordinate information for the ball sensing region, the step of calculating the position coordinates of the ball object as the initial position coordinates of the ball, A method for visualizing a ball trajectory in a user's golf shot video according to claim 2 or 3, comprising the step of converting the initial ball position coordinates using the coordinate matching information to the actual initial position coordinates of the golf ball, which are the coordinates of the golf ball in actual space.

5. The height of the tee used by the user when taking a golf shot is set in advance. The step of calculating the position coordinates of the ball object as the initial position coordinates of the ball is as follows: The step of recognizing the aforementioned preset tee height, A method for visualizing a ball trajectory in a user's golf shot video according to claim 4, comprising the step of applying height information on the video corresponding to the recognized tee height to the position coordinates of a ball object detected within the ball sensing area to calculate the initial position coordinates of the ball.

6. The step of collecting radar sensing data for the moving golf ball is as follows: The step includes collecting golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space at the time of receiving the radar signal reflected from the golf ball. The step of displaying the ball trajectory is: A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the step of displaying a ball trajectory starting from the initial position coordinates of the ball by converting the collected golf ball orientation information into coordinate information on the video acquired by the camera based on the initial position coordinates of the ball and concatenating them.

7. The step of displaying the ball trajectory is: The steps include verifying the initial position coordinates of the ball on the image acquired by the camera using radar sensing data at the time the golf ball is struck, The steps include correcting the initial position coordinates of the ball based on the results of the verification, A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the steps of: converting radar sensing data collected on a golf ball moving from the point of impact into coordinate information on the video acquired by the camera; and displaying the ball trajectory starting from the corrected initial ball position coordinates on the video acquired by the camera.

8. The step of collecting radar sensing data for the moving golf ball is as follows: A step in which a trigger signal is generated at the moment the golf ball is struck, The step includes collecting golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space, from the time the trigger signal is generated to the time the radar signal reflected from the moving golf ball is received, The step of displaying the ball trajectory is: The steps include: generating ball coordinate information at the time the trigger signal is generated by converting the golf ball orientation information at the time the trigger signal is generated into coordinate information on the camera's acquired image based on the ball's initial position coordinates; The steps include correcting the initial ball position coordinates using the ball coordinates at the time the trigger signal is generated, A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the steps of: converting the golf ball orientation information collected for a golf ball moving from the time the trigger signal is generated into coordinate information on the video acquired by the camera based on the corrected initial ball position coordinates and concatenating them, thereby displaying the ball trajectory starting from the corrected initial ball position coordinates.

9. The step of collecting radar sensing data for the moving golf ball is as follows: A step in which a trigger signal is generated at the moment the golf ball is struck, The step includes collecting golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space, from the time the trigger signal is generated to the time the radar signal reflected from the moving golf ball is received, The step of displaying the ball trajectory is: A step of calculating the three-dimensional position information of the golf ball at the time the trigger signal is generated, using the golf ball orientation information and the actual initial position coordinates of the golf ball at the time the trigger signal is generated. The steps include: converting the calculated three-dimensional position information of the golf ball into coordinate information on the image acquired by the camera to calculate the corrected coordinate information of the ball's initial position; A method for visualizing a ball trajectory in a user's golf shot video according to claim 1, comprising the steps of: converting the golf ball orientation information collected for a golf ball moving from the time the trigger signal is generated into coordinate information on the video acquired by the camera based on the corrected coordinates and concatenating them, thereby displaying the ball trajectory starting from the corrected coordinates.

10. A single-view camera located behind the bottom surface where the user takes a golf shot, and which acquires video footage of the user taking a golf shot on the bottom surface. A sensing processing unit sets a ball sensing region for sensing a golf ball using the video acquired by the single camera, detects a ball object corresponding to the golf ball within the ball sensing region on the acquired video, calculates the initial position coordinates of the ball, and calculates the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball. A radar sensing unit collects radar sensing data for a golf ball that moves due to the user's golf shot, based on the actual initial position coordinates of the golf ball. A trajectory display processing unit that uses the radar sensing data of the golf ball collected above to display the ball's trajectory starting from the ball's initial position coordinates or corrected coordinates of the ball's initial position coordinates in the video acquired by the single camera, A ball trajectory visualization device based on radar sensing, including [specific feature].

11. The sensing processing unit is, The single camera is positioned behind the golf shot, on the bottom surface, and is configured to capture images with a field of view that includes a structure whose specifications are known in advance. The system matches the coordinates of elements that can identify the specifications of the structure appearing in the image with coordinates in real space, stores the coordinate matching information, and sets the area corresponding to the structure appearing in the image as the ball sensing area. A ball trajectory visualization device based on radar sensing according to claim 10, configured to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object as the initial ball position coordinates using coordinate information for the ball sensing area, and convert the initial ball position coordinates to the actual initial position coordinates of the golf ball, which are the actual spatial coordinates of the golf ball, using coordinate matching information.

12. The sensing processing unit is, The system is configured to allow the user to pre-set the height of the tee they use when taking a golf shot. The single camera is positioned behind the golf shot, on the bottom surface, and is configured to capture images with a field of view that includes a structure whose specifications are known in advance. The system matches the coordinates of elements that can identify the specifications of the structure appearing in the image with coordinates in real space, stores the coordinate matching information, and sets the area corresponding to the structure appearing in the image as the ball sensing area. A ball trajectory visualization device based on radar sensing according to claim 10, configured to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object using coordinate information for the ball sensing area, recognize the preset tee height, apply the height information on the image corresponding to the recognized tee height to the position coordinates of the ball object to calculate the initial position coordinates of the ball, and convert the initial position coordinates of the ball to the actual initial position coordinates of the golf ball, which are the actual spatial coordinates of the golf ball, using the coordinate matching information.

13. The radar sensing unit is The system is configured to collect golf ball orientation information, including the vertical height angle and horizontal direction angle of the golf ball in actual space at the time of receiving the radar signal reflected from the golf ball. The aforementioned trajectory display processing unit, A ball trajectory visualization device based on radar sensing according to claim 10, configured to display the ball trajectory starting from the initial position coordinates of the ball by converting the orientation information of each golf ball collected by the radar sensing unit into coordinate information on the image acquired by the single camera based on the initial position coordinates of the ball and concatenating them.

14. The aforementioned trajectory display processing unit, The initial position coordinates of the ball on the video acquired by the single camera are verified using radar sensing data at the time the golf ball is struck, and the initial position coordinates of the ball are corrected based on the verification result. A ball trajectory visualization device based on radar sensing according to claim 10, configured to convert radar sensing data collected by the radar sensing unit on a golf ball moving from the time of impact into coordinate information on an image acquired by the single camera, and to display the ball trajectory starting from the corrected initial position coordinates of the ball on the image.

Images