Method for visualizing ball trajectory in golf shot image of user, and apparatus for visualizing ball trajectory on basis of radar sensing by using same

The method combines camera and radar sensing to accurately determine the initial position and trajectory of a golf ball, addressing the limitations of existing systems and providing precise golf shot analysis without the need for expensive stereoscopic cameras.

WO2024232644A1PCT designated stage expired Publication Date: 2024-11-14GOLFZON CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2024/006148
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing systems struggle to accurately visualize the ball trajectory in golf shots due to limitations in camera sensors' narrow sensing range and radar sensors' inability to determine the initial position of the golf ball, leading to errors in trajectory calculation, especially in large spaces like golf courses.

Method used

A method and device that uses a single camera to detect the initial position of the golf ball and combines this with radar sensing data to accurately calculate and display the ball's trajectory, correcting for errors by verifying and adjusting the initial position using radar sensing data.

Benefits of technology

Enables accurate visualization of the golf ball's trajectory from its initial position, reducing errors and providing a cost-effective solution without the need for a stereoscopic camera system, thus improving the precision of golf shot analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024006148_14112024_PF_FP_ABST
    Figure KR2024006148_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The objective of the present invention is to provide a method for visualizing a ball trajectory in a golf shot image of a user, and an apparatus for visualizing a ball trajectory on the basis of radar sensing by using same, the method and the apparatus being capable of: photographing a golf shot of a user so as to use a camera acquisition image to be provided to the user, thereby sensing an initial position of a golf ball positioned on the ground for the golf shot by the user; and visualizing, as an accurate ball trajectory starting from the initial position of the golf ball on the image, the golf ball movement trajectory calculated using a radar sensing result for the golf ball, which moves by being hit.
Need to check novelty before this filing date? Find Prior Art

Description

Method for visualizing ball trajectory in a user's golf shot video and a ball trajectory visualization device based on radar sensing using the same

[0001] The present invention relates to a method for visualizing a ball trajectory in a video of a user's golf shot, which can visualize a ball trajectory starting from an exact initial position and progressing on a video of a scene in which a user takes a golf shot with a golf club by using the video and the sensing results by radar, and a ball trajectory visualization device based on radar sensing using the method.

[0002] In the case of golf, when a user hits a golf ball with a golf club by hitting a golf shot, it is very difficult to see the golf ball flying with the naked eye because the golf ball is very small and flies a very long distance at a very high speed.

[0003] Just as it is difficult to see the flight of a golf ball with the naked eye, it is also very difficult to see the trajectory of the golf ball even in a video taken from a certain location of a user taking a golf shot.

[0004] In the case of a system such as screen golf, when a user makes a golf shot toward the screen in front in a narrow space, the camera sensor senses the initial speed, direction angle, and height angle of the golf ball hit by the golf club as it flies, and based on the initial movement characteristics of the golf ball, the trajectory of the entire golf ball is calculated through physical calculations and displayed on the screen. However, this is an estimated trajectory through calculations, not a measurement of the actual movement of the golf ball, so there is a difference from the trajectory of the actual golf ball.

[0005] In this way, a camera sensor is used to sense the trajectory of a golf ball, but as mentioned above, the sensing range of the camera sensor is very narrow, so it can only sense the initial movement characteristics of the golf ball. Therefore, it is not suitable for sensing the movement trajectory of a golf ball according to a golf shot in a wide area such as an actual golf course.

[0006] Radar sensors are widely used as a representative sensing device to detect the movement of a golf ball according to a golf shot on an actual golf course.

[0007] A radar sensor is a device that analyzes the movement vector of a golf ball by receiving a radar signal shifted by the Doppler shift phenomenon of the radar when a radar signal radiated toward a moving golf ball is reflected from the golf ball, through a radar antenna.

[0008] Using a radar sensor like this, the trajectory of a golf ball moving according to the user's golf shot can be calculated.

[0009] However, since the radar sensor utilizes the Doppler shift phenomenon of the radar signal, it can obtain phase change information about the change in position according to the movement of the golf ball, and from there, it can calculate the movement trajectory using information about the change in position of the golf ball from the time of movement, but it is difficult to confirm the position of the golf ball in the front-back direction, and there is a problem that the initial position of the golf ball when a golf shot is made, that is, the position of the golf ball in a stationary state placed on the floor or tee, cannot be confirmed by the radar sensor.

[0010] For this reason, even if the radar sensor detects the movement of the golf ball and calculates the movement trajectory as the golf ball is struck and moves, there is a problem in that the golf ball movement trajectory is calculated to start at a different location from the actual initial location of the golf ball because the initial location of the golf ball cannot be accurately determined.

[0011] Since the radar sensor cannot determine the initial position of the golf ball, a reference distance (distance from the radar sensor installed on the floor) is set in advance, and only when the golf ball is hit at a fixed location within that reference distance can the movement trajectory starting from the initial position of the golf ball be completed with a certain degree of accuracy. However, when the user places the golf ball at an arbitrary location and takes a golf shot, the starting position of the golf ball movement trajectory shows a significant difference from the actual location, which causes a significant error in the overall trajectory.

[0012] In order to calculate the trajectory of a ball moving as a user hits the ball with a golf club, attempts are being made to complement the shortcomings of each by combining camera sensing and radar sensing. However, a stereoscopic camera system (a camera system in which two or more cameras installed in different locations detect the same subject and calculate the position of the subject in three-dimensional space) that can sense the position of an object in three-dimensional space can be used to detect the initial position of a golf ball placed at an arbitrary location. However, if such a stereoscopic camera system and a radar sensing system are implemented as a single device, the cost of the device increases significantly. Therefore, there is a need to develop a device that can accurately visualize the trajectory of a golf ball by radar sensing without using a stereoscopic camera system using two or more cameras.

[0013] [Prior patent literature]

[0014] Korean Patent Publication No. 10-1845503

[0015] Korean Patent Publication No. 10-2335944

[0016] Korean Patent Publication No. 10-2353481

[0017] Korean Patent Publication No. 10-2033703

[0018] The present invention provides a method for visualizing a ball trajectory in a video of a user's golf shot, which detects the initial position of a golf ball placed on the ground by a user for a golf shot using a camera-captured video for filming the user taking a golf shot and providing the same to the user, and visualizes the movement trajectory of the golf ball calculated using the radar sensing result for the golf ball moving by the hit as an accurate ball trajectory starting from the initial position of the golf ball in the video, and a ball trajectory visualization device based on radar sensing using the same.

[0019] A method for visualizing a ball trajectory using radar sensing results in a camera-captured image capturing a user taking a golf shot includes the steps of: setting a ball sensing area for sensing a golf ball using the camera-captured image on a floor where the user is to take a golf shot; detecting a ball object corresponding to the golf ball within the ball sensing area on the camera-captured image to calculate ball initial position coordinates and calculating actual initial position coordinates of the golf ball corresponding to the ball initial position coordinates; collecting radar sensing data for a golf ball moving according to the user's golf shot based on the actual initial position coordinates of the golf ball; and displaying a ball trajectory starting from the ball initial position coordinates or corrected coordinates of the ball initial position coordinates in the camera-captured image using the collected radar sensing data for the golf ball.

[0020] Also preferably, the step of setting the ball sensing area is characterized in that the camera is a single camera with a single view, and the step of matching coordinates of elements capable of specifying an area of ​​a predetermined size on an image acquired by the single camera with coordinates in real space to store coordinate matching information, and setting the area of ​​the predetermined size on the image as the ball sensing area is included.

[0021] Also preferably, the step of setting the ball sensing area is characterized in that the camera is a single camera with a single view, and the single camera captures an image with an angle of view that includes a structure whose specifications are known in advance on the ground where the golf shot is to be made, and stores coordinate matching information by matching coordinates of elements that can specify the specifications of the structure that appear on the image with coordinates in real space, and sets the area corresponding to the structure that appears on the image as the ball sensing area.

[0022] Also preferably, the step of calculating the actual initial position coordinates of the golf ball is characterized by including the steps of: detecting a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera; calculating the position coordinates of the ball object as the ball initial position coordinates using coordinate information about the ball sensing area; and converting the ball initial position coordinates into the actual initial position coordinates of the golf ball, which are actual spatial coordinates of the golf ball, using the coordinate matching information.

[0023] Also preferably, the step of presetting the height of the tee to be used when the user makes a golf shot, and calculating the position coordinates of the ball object as the ball initial position coordinates, is characterized by including the step of recognizing the preset tee height, and the step of applying height information on the image corresponding to the recognized tee height to the position coordinates of the ball object detected within the ball sensing area to calculate the ball initial position coordinates.

[0024] Also preferably, the step of collecting radar sensing data for the moving golf ball includes a step of collecting golf ball orientation information including an up-down height angle and a left-right direction angle of the golf ball in real space at the time of receiving a radar signal reflected from the golf ball, and the step of displaying the ball trajectory includes a step of converting each of the collected golf ball orientation information into coordinate information on the camera-captured image based on the ball initial position coordinate and connecting the information to display a ball trajectory starting from the ball initial position coordinate.

[0025] Also preferably, the step of displaying the ball trajectory is characterized by including the steps of verifying the ball initial position coordinates on the camera-captured image using radar sensing data at the time when the golf ball is hit, correcting the ball initial position coordinates according to the verification result, and converting radar sensing data collected for the golf ball moving from the time when the ball is hit into coordinate information on the camera-captured image to display the ball trajectory starting from the corrected ball initial position coordinates on the image.

[0026] Also preferably, the step of collecting radar sensing data for the moving golf ball includes a step of generating a trigger signal at the time when the golf ball is hit, and a step of collecting golf ball bearing information including an up-down height angle and a left-right direction angle of the golf ball in real space from the time when the trigger signal is generated to the time when the radar signal reflected from the moving golf ball is received, and the step of displaying the ball trajectory includes a step of converting the golf ball bearing information at the time when the trigger signal is generated into coordinate information on the camera-captured image based on the ball initial position coordinates to generate trigger point-of-time ball coordinate information, a step of correcting the ball initial position coordinates using the trigger point-of-time ball coordinates, and a step of displaying a ball trajectory starting from the corrected ball initial position coordinates by converting and connecting the golf ball bearing information collected for the moving golf ball from the trigger point-of-time into coordinate information on the camera-captured image based on the corrected ball initial position coordinates.

[0027] Also preferably, the step of collecting radar sensing data for the moving golf ball includes a step of generating a trigger signal at the time the golf ball is hit, and a step of collecting golf ball orientation information including an up-down height angle and a left-right direction angle of the golf ball in real space from the time of generating the trigger signal to the time of receiving a radar signal reflected from the moving golf ball, and the step of displaying the ball trajectory includes a step of calculating three-dimensional position information of the golf ball at the time of generating the trigger signal using the golf ball orientation information at the time of generating the trigger signal and the actual initial position coordinates of the golf ball, a step of converting the calculated three-dimensional position information of the golf ball into coordinate information on the camera-captured image, and a step of displaying a ball trajectory starting from the corrected coordinates by converting and connecting the golf ball orientation information collected for the moving golf ball from the trigger time into coordinate information on the camera-captured image based on the corrected coordinates.

[0028] Meanwhile, a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention includes: a single camera with a single viewpoint that acquires an image of a scene in which a user makes a golf shot on a floor from behind the floor on which the user makes a golf shot; a sensing processing unit that sets a ball sensing area for sensing a golf ball using the image acquired by the single camera, detects a ball object corresponding to the golf ball within the ball sensing area on the image acquired, calculates ball initial position coordinates, and calculates actual initial position coordinates of the golf ball corresponding to the ball initial position coordinates; a radar sensing unit that collects radar sensing data on a golf ball moving according to the user's golf shot based on the actual initial position coordinates of the golf ball; and a trajectory display processing unit that displays a ball trajectory starting from the ball initial position coordinates or corrected coordinates of the ball initial position coordinates in the image acquired by the camera using the collected radar sensing data on the golf ball.

[0029] Also preferably, the sensing processing unit is configured to match coordinates of elements that can specify the specifications of the structure that appears in an image captured by the single camera with an angle of view that includes a structure whose specifications are known in advance on the ground where the golf shot is to be made, with coordinates in real space, and store coordinate matching information, and set an area corresponding to the structure that appears in the image as the ball sensing area, and detect a ball object corresponding to the golf ball within the set ball sensing area in the image acquired by the single camera, and calculate the position coordinates of the ball object as the ball initial position coordinates using the coordinate information about the ball sensing area, and convert the ball initial position coordinates into the actual initial position coordinates of the golf ball, which are the actual spatial coordinates of the golf ball, using the coordinate matching information.

[0030] Also preferably, the sensing processing unit is configured to preset the height of a tee to be used when the user makes a golf shot, and is configured to match coordinates of elements that can specify the specifications of a structure that appears in an image captured by the single camera from the rear with an angle of view that includes a structure whose specifications are known in advance and placed on the ground on which the golf shot is to be made, with coordinates in real space, and store coordinate matching information, and set an area corresponding to the structure that appears in the image as the ball sensing area, and detect a ball object corresponding to the golf ball within the preset ball sensing area on the image acquired by the single camera, and calculate the position coordinates of the ball object using the coordinate information regarding the ball sensing area, and recognize the preset tee height and apply height information on the image corresponding to the recognized tee height to the position coordinates of the ball object to calculate the ball initial position coordinates, and convert the ball initial position coordinates into actual initial position coordinates of the golf ball, which are coordinates in real space, using the coordinate matching information.

[0031] Also preferably, the radar sensing unit is configured to collect golf ball orientation information including the vertical height angle and the left-right direction angle of the golf ball in the actual space at the time of receiving the radar signal reflected from the golf ball, and the trajectory display processing unit is configured to display the ball trajectory starting from the ball initial position coordinate by converting each piece of golf ball orientation information collected by the radar sensing unit into coordinate information on the camera acquired image based on the ball initial position coordinate and connecting the converted information.

[0032] Also preferably, the trajectory display processing unit verifies the ball initial position coordinates on the camera-captured image using radar sensing data at the time the golf ball is hit, corrects the ball initial position coordinates according to the verification result, and converts radar sensing data collected by the radar sensing unit for the golf ball moving from the time of the hit into coordinate information on the camera-captured image to display the ball trajectory starting from the corrected ball initial position coordinates on the image.

[0033] A method for visualizing a ball trajectory in a video of a user's golf shot according to the present invention and a ball trajectory visualization device based on radar sensing using the same detect the initial position of a golf ball placed on the ground by a user for a golf shot using a camera-captured image for filming the user taking a golf shot and providing the image to the user, and visualize the movement trajectory of the golf ball calculated using the radar sensing result for the golf ball moving by the hit as an accurate ball trajectory starting from the initial position of the golf ball in the video.

[0034] FIG. 1 is a drawing showing a scene in which a user takes a golf shot while a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention is installed on a golf course or a field equivalent thereto.

[0035] FIG. 2 is a block diagram showing the configuration of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention.

[0036] FIG. 3 is a drawing showing a more specific configuration of a radar sensing unit and a configuration of a radar signal receiving unit of a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention illustrated in FIG. 2.

[0037] FIG. 4 is a drawing showing an example of calculating location information of a golf ball by a radar sensing unit as illustrated in FIG. 3.

[0038] FIG. 5 is a drawing showing an example of an image captured by a single camera of a ball trajectory visualization device from behind a user making a golf shot, as shown in (c) of FIG. 4.

[0039] FIG. 6 is a drawing showing a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention that sets a ball sensing area using a structure whose specifications are known in advance. FIG. 6 (a) shows an appearance in actual space, and FIG. 6 (b) shows an image captured by a single camera in the state shown in (a).

[0040] FIG. 7 is a drawing showing how a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention calculates ball initial position coordinates using a ball sensing area in a state where a user places a golf ball on the ground for a golf shot. FIG. 7 (a) shows a user (U)'s golf shot in real space, and FIG. 7 (b) shows an image taken with a single camera in the state shown in (a).

[0041] Fig. 8 (a) shows a golf ball moving when a user hits the golf ball in the state shown in Fig. 7 (a), and Fig. 8 (b) shows a ball trajectory calculated using radar sensing results from an image taken with a single camera of the actual scene shown in Fig. 8 (a).

[0042] FIG. 9 (a) shows an image of a ball trajectory visualization device according to one embodiment of the present invention, in which a golf ball is placed on the floor and a tee is placed on the floor and the golf ball is placed on the tee, and FIG. 9 (b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0043] FIG. 10 is a drawing for explaining a case in which there may be an error in the ball initial position coordinates calculated through an image in a ball trajectory visualization device according to one embodiment of the present invention and a method for correcting the same.

[0044] FIG. 11 is a drawing showing a ball trajectory displayed based on the ball initial position coordinates initially calculated from a video taken with a single camera of a scene in which a golf ball flies as a user hits a golf shot in FIG. 10, and a ball trajectory displayed by correcting the initial ball initial position coordinates to a more accurate position.

[0045] FIGS. 12 and 13 are flowcharts illustrating a process for visualizing a ball trajectory in a user's golf shot video using a ball trajectory visualization device according to one embodiment of the present invention.

[0046] A method for visualizing a ball trajectory in a user's golf shot video according to one embodiment of the present invention and a ball trajectory visualization device based on radar sensing using the same are described in detail with reference to drawings.

[0047] A method for visualizing a ball trajectory in a video of a user's golf shot according to one embodiment of the present invention and a device for visualizing a ball trajectory based on radar sensing using the same are related to a device and method for sensing a golf ball flying after being hit by a golf shot using radar and visualizing a ball trajectory according to the sensing result on the video when filming a scene in which a user makes a golf shot with a golf club in an outdoor area such as a golf course and providing the filmed video to the user.

[0048] The configuration of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention will be described with reference to FIGS. 1 and 2.

[0049] FIG. 1 illustrates a scene where a user takes a golf shot while a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention is installed on a golf course or a field equivalent thereto, and FIG. 2 is a block diagram illustrating the configuration of a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention.

[0050] A ball trajectory visualization device (100) based on radar sensing according to one embodiment of the present invention may be configured to include a single camera (200) having a single viewpoint, a sensing processing unit (250) that calculates location information of a golf ball from an image acquired by the single camera, a radar sensing unit (300) that collects radar sensing data on a golf ball that has been hit and is moving, and a trajectory display processing unit (350) that displays a ball trajectory on an image acquired by the single camera using the radar sensing data collected by the radar sensing unit.

[0051] As illustrated in FIG. 1, a ball trajectory visualization device (100) based on radar sensing according to one embodiment of the present invention can be used by placing it so that it faces forward from the rear of the floor (BT) where a user (U) holds a golf club (20) and hits a golf ball (10) (the direction in which the golf ball flies upon hitting).

[0052] The single camera (200) of the ball trajectory visualization device (100) described above can acquire an image of a scene in which a user (U) makes a golf shot from the rear to the front of the floor (BT) where the user (U) makes a golf shot with a single view.

[0053] The video of the user's golf shot scene above is a simple two-dimensional video from a single camera, not a video acquired from multiple perspectives using multiple cameras in a stereoscopic manner.

[0054] If you acquire images of a golf ball using multiple stereoscopic cameras, you can derive information about the golf ball's location in three-dimensional space through analysis of each image, so you can obtain information that cannot be obtained from radar sensing data. However, the radar sensor itself is quite expensive, and the stereoscopic camera system is also quite expensive, so if these two systems are implemented as a single device, the cost increase is very large, making it very difficult to commercialize.

[0055] A ball trajectory visualization device based on radar sensing according to one embodiment of the present invention has the special advantage of being able to produce information necessary for ball trajectory visualization that is equivalent to or higher than information that can be acquired using a stereoscopic camera system by using a single camera that is basically necessary to provide an image of a user's golf shot scene, and to accurately display the ball trajectory on the image by using this together with the radar sensing result.

[0056] Information required for visualizing the ball trajectory calculated using a single camera as described above can be calculated by the sensing processing unit (250) through a video of the user's golf shot scene captured by the single camera (200).

[0057] That is, the sensing processing unit (250) of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention can perform the function of setting a ball sensing area for sensing a golf ball using an image acquired by a single camera (200), detecting a ball object corresponding to a golf ball within the ball sensing area on the acquired image to calculate the ball initial position coordinates, and calculating the actual initial position coordinates of the golf ball corresponding to the ball initial position coordinates and transmitting the calculated coordinates to the radar sensing unit (300).

[0058] The radar sensing unit (300) of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention collects radar sensing data for a golf ball moving according to a user's golf shot based on the actual initial position coordinates of the golf ball received from the sensing processing unit (250) as described above, and the trajectory display processing unit (350) can perform a function of displaying a ball trajectory starting from the ball initial position coordinates in a camera-captured image on the corresponding image using the radar sensing data for the golf ball collected by the radar sensing unit (300).

[0059] The overall operation of the ball trajectory visualization device based on radar sensing according to an embodiment of the present invention as described above will be described later, and first, with reference to FIGS. 3 and 4, a process of obtaining radar sensing data from a moving golf ball by the radar sensing unit (300) of the ball trajectory visualization device based on radar sensing according to an embodiment of the present invention will be described.

[0060] FIG. 3 is a drawing showing a more specific configuration of a radar sensing unit and a configuration of a radar signal receiving unit of a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention shown in FIG. 2, and FIG. 4 is a drawing showing an example of calculating location information of a golf ball by a radar sensing unit as shown in FIG. 3.

[0061] The radar sensing unit (300) can basically use the Doppler effect of radar to calculate the phase change and position information at each measurement point of a golf ball that is hit and moving.

[0062] The radar sensing unit (300) may be configured to include a signal transmitting unit (310), a signal receiving unit (320), and a signal analyzing unit (330), as shown in (a) of FIG. 3.

[0063] The radar sensing unit (300) can be configured to transmit a radar signal of a specific frequency from a position a predetermined distance behind a golf ball to be hit by a user toward the direction of movement of the golf ball that will move due to the hit, and to receive and analyze a reflected wave reflected from the moving golf ball to track the golf ball that has been hit and is moving.

[0064] The signal transmitter (310) is configured to transmit a specific radar signal (St) in a targeted direction and may be configured to include a transmitting antenna for transmitting a radar signal, although not shown in the drawing.

[0065] The signal receiving unit (320) is configured to receive a reflected wave signal (Sr) that is reflected back from a moving golf ball (10) by a radar signal (St) transmitted by the signal transmitting unit (310). The reflected wave signal (Sr) that is transmitted by the signal transmitting unit (310) and reflected from the golf ball (10) by the Doppler effect is a signal in which a Doppler shift is generated due to a change in the frequency of the signal transmitted by the signal transmitting unit (310). That is, the signal receiving unit (320) receives a signal in which a Doppler shift is generated.

[0066] The signal receiving unit (320) has a plurality of receiving antennas that receive reflected wave signals, and can calculate the position information of the golf ball by using the phase difference of the reception signal of each receiving antenna.

[0067] FIG. 3 (b) briefly illustrates an example of the configuration of the signal receiving unit (320) described above. As illustrated, the signal receiving unit (320) may be provided with three or more receiving antennas including RA1, RA2, and RA3 appropriately arranged, and the phase difference of the signal between each receiving antenna may be used to calculate the trajectory (height angle) and direction angle of the moving golf ball (10).

[0068] For example, in the signal receiving unit (320) illustrated in (b) of FIG. 3, the vertical height angle of the moving golf ball (10) can be calculated using the phase difference of the signals received by RA1 and RA2, and the left and right direction angle of the moving golf ball (10) can be calculated using the phase difference of the signals received by RA1 and RA3, respectively.

[0069] The signal analysis unit (330) can analyze the radar signal reflected by the golf ball at predetermined time intervals while the golf ball is moving to derive phase information, and can be configured to collect the direction information (including the up-down height angle and the left-right direction angle information) of the moving golf ball as radar sensing data at predetermined time intervals.

[0070] FIG. 4 (a) shows that the radar sensing unit of the ball trajectory visualization device (100) based on radar sensing according to one embodiment of the present invention measures left-right direction angles (d1, d2, d3...) at each measurement location (B1, B2, B3...) of a moving golf ball, and FIG. 4 (b) shows that the radar sensing unit measures up-down height angles (z1, z2, z3...) at each measurement location (B1, B2, B3...) of a moving golf ball.

[0071] As shown in (a) and (b) of FIG. 4, the radar sensing unit of the ball trajectory visualization device (100) based on radar sensing according to one embodiment of the present invention can calculate and collect left and right direction angles and up and down height angles for each measurement position at the time of receiving a reflected wave signal of a radar signal from a moving golf ball.

[0072] That is, as in the example shown in (c) of Fig. 4, when a golf ball hit by a user (U) moves after a golf shot, the radar sensing unit can collect golf ball azimuth information (left and right direction angles and up and down height angle information) such as (d1, z1) at position B1, (d2, z2) at position B2, and (d3, z3) at position B3 as radar sensing data.

[0073] However, the golf ball orientation information collected by the radar sensing unit for a moving golf ball is angle information in the left-right direction of the golf ball in an actual three-dimensional space (see (a) of FIG. 4) and angle information in the up-down direction of the golf ball (see (b) of FIG. 4), and it is difficult to confirm information about the front-back direction of the golf ball through radar sensing data.

[0074] By using the left-right directional angle and up-down height angle information of the golf ball by radar as described above, it is possible to convert the golf ball azimuth information at each measurement point into coordinate information on the image, and to derive the ball trajectory itself by connecting each converted coordinate on the golf shot image, which is a two-dimensional image captured by a single camera.

[0075] However, as described above, it is difficult to measure the forward and backward position information of a golf ball by radar sensing, for example, where in front of the device the golf ball is, so it is difficult to know where the ball trajectory of a golf ball moving by radar sensing begins.

[0076] FIG. 5 shows an example of an image (IM) captured by a single camera of a ball trajectory visualization device (100) from behind a user (U) making a golf shot, as shown in (c) of FIG. 4. In FIG. 5, the drawing symbol iU represents a portion corresponding to the user in the image, and iBi represents an object portion corresponding to a golf ball placed at an initial position.

[0077] In Fig. 5, when a user (iU) hits a ball (iBi) at an initial position, if the ball trajectory is calculated and displayed accurately, it can be represented by the iTrr trajectory indicated by a dotted line. However, if the ball trajectory is calculated by collecting radar sensing data for a moving golf ball, it can be represented by the iTra trajectory or iTrb trajectory shown in Fig. 5 when the exact initial position of the ball is not known. It can be seen that the iTra trajectory or iTrb trajectory has a significant difference from the actual ball trajectory iTrr trajectory.

[0078] A method for visualizing a ball trajectory in a user's golf shot video according to one embodiment of the present invention and a ball trajectory visualization device based on radar sensing using the same are characterized in that the initial position of a ball can be accurately recognized from an image captured by a single camera and the ball trajectory calculated using radar sensing data can be accurately displayed to start from the initial position of the ball recognized as described above and proceed. The features of the present invention will be described in more detail with reference to FIGS. 6 to 8.

[0079] A sensing processing unit of a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention can set a ball sensing area to specify the position coordinates of a golf ball to be placed at any location on the floor where a user will make a golf shot, through an image acquired by a single camera.

[0080] The sensing processing unit can store coordinate matching information by matching the coordinates of elements that can specify an area of ​​a predetermined size on an image acquired by a single camera with coordinates in real space, and set the area of ​​the predetermined size on the image as the ball sensing area.

[0081] As an example of a method for obtaining the above-mentioned 'coordinates of elements that can specify an area of ​​a certain size', a 'structure whose specifications are known in advance' can be used, for example, a rectangular structure whose four side lengths are precisely known can be used.

[0082] FIG. 6 (a) and (b) illustrate a ball trajectory visualization device based on radar sensing according to one embodiment of the present invention that sets a ball sensing area using a structure whose specifications are known in advance. FIG. 6 (a) illustrates an image in an actual space, and FIG. 6 (b) illustrates an image captured by a single camera in the state illustrated in (a).

[0083] In Fig. 6 (a), the above-mentioned ‘structure whose specifications are known in advance’ is shown as a square structure (120) placed within the camera’s field of view.

[0084] The square structure (120) illustrated in (a) of FIG. 6 is a structure whose four side lengths, i.e., L1, L2, L3, and L4, are known in advance, and as illustrated in (b) of FIG. 6, the lengths of each of the sides (L1, L2, L3, and L4) correspond to the four side lengths, l1, l2, l3, and l4, of the square (i120) on the image (IM), respectively.

[0085] In addition, the elements that can specify the specifications of the square structure (120) illustrated in (a) of FIG. 6 are four vertices E1, E2, E3, and E4, which correspond to the four vertices e1, e2, e3, and e4, which are elements that can specify the specifications of the square (i120) on the image illustrated in (b) of FIG. 6, respectively.

[0086] If an xy coordinate system with the location e1 as the origin is defined on the image (IM) of Fig. 6 (b), the location coordinates of each vertex of e1, e2, e3, and e4 can be specified in the xy coordinate system. That is, based on the xy coordinates, the coordinates of e1 can be specified as (0,0), the coordinates of e2 as (0,l2), the coordinates of e3 as (l3(=l1), l4(=l2)), and the coordinates of e4 as (l1,0).

[0087] As described above, the position coordinates of each vertex of the specified e1, e2, e3, and e4 can be matched with the coordinates of each vertex of the structure (120) in the actual space as shown in (a) of FIG. 6.

[0088] That is, in the XY coordinate system with the E1 position as the origin in the actual space as shown in (a) of Fig. 6, the coordinates of E1 can be specified as (0,0), the coordinates of E2 as (0,L2), the coordinates of E3 as (L3(=L1), L4(=L2)), and the coordinates of E4 as (L1,0), and coordinate matching information can be produced by matching the position coordinates of each of E1, E2, E3, and E4 with the position coordinates of each of e1, e2, e3, and e4 on the image.

[0089] As mentioned above, coordinate matching information that matches the coordinates of elements specified in the image with their coordinates in real space can be stored and used importantly in the process of visualizing the ball trajectory according to the user's golf shot.

[0090] The area specified by e1, e2, e3, and e4 on the image (IM) is an xy coordinate plane, and any location within the coordinate plane can be specified by xy coordinates, and the xy coordinates specified in this way can be converted into coordinate information in real space using the above-mentioned coordinate matching information.

[0091] The sensing processing unit of the ball trajectory visualization device based on radar sensing according to one embodiment of the present invention can set the area specified by e1, e2, e3, and e4 on the image (IM) as a 'ball sensing area' as described above.

[0092] After the 'ball sensing area' setting is completed as described above, the structure can be removed and the process of visualizing the ball trajectory according to the user's golf shot can begin.

[0093] FIG. 7 (a) and (b) illustrate how a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention calculates ball initial position coordinates using a ball sensing area in a state where a user places a golf ball on the ground for a golf shot. FIG. 7 (a) illustrates a user (U)'s golf shot in real space, and FIG. 7 (b) illustrates an image captured by a single camera in the state illustrated in (a).

[0094] As shown in (a) of FIG. 7, when a user (U) places a golf ball (Bi) on the floor (BT) for a golf shot while a ball trajectory visualization device (100) according to the present invention is installed, as shown in (b) of FIG. 7, in the video taken of the scene, as described above, a ball sensing area (SA) specified by e1, e2, e3, and e4 is preset, so that an object corresponding to a golf ball, i.e., a ball object (iBi), can be detected within the ball sensing area (SA) to calculate position coordinate information of the ball object (iBi) on the ball sensing area (SA).

[0095] In this way, the position coordinates of the ball object (iBi) within the ball sensing area (SA) on the image are referred to as “ball initial position coordinates.”

[0096] By calculating the position coordinates of the ball object (iBi), i.e., the ball initial position coordinates, using the coordinate matching information regarding the preset ball sensing area (SA), the ball initial position coordinates in the image can be converted into the actual initial position coordinates of the golf ball (Bi), which are the actual spatial coordinates of the golf ball.

[0097] A ball trajectory visualization device according to one embodiment of the present invention utilizes the ball initial position coordinates calculated from an image acquired by a single camera as described above as a reference point for displaying the ball trajectory later, and collects sensing data of a radar sensing unit and calculates a ball trajectory using the actual initial position coordinates of a golf ball converted using coordinate matching information as a reference position.

[0098] Fig. 8 (a) shows a golf ball moving when a user hits the golf ball in the state shown in Fig. 7 (a), and Fig. 8 (b) shows a ball trajectory calculated using radar sensing results from an image taken with a single camera of the actual scene shown in Fig. 8 (a).

[0099] In (a) of FIG. 8, when a user (U) hits a golf ball by making a golf shot, the ball trajectory visualization device (100) according to one embodiment of the present invention generates a trigger signal at the time of the hit, and according to the generation of the trigger signal, the radar sensing unit can continuously collect radar sensing data on the moving golf ball from the trigger time.

[0100] The 'radar sensing data' mentioned here is the azimuth information (up-down height angle and left-right direction angle information) of the golf ball at each measurement point of the moving golf ball, as explained above through FIGS. 3 and 4.

[0101] And the radar sensing unit can collect the azimuth information of the golf ball by using the actual initial position coordinates (Pbi) of the golf ball, which are converted using coordinate matching information from the initial position coordinates of the ball obtained by the ball sensing area in the image, as a reference position.

[0102] That is, as illustrated in (a) of Fig. 8, for a moving golf ball, the radar sensing unit can collect azimuth information (left and right direction angles, up and down height angles) of the golf ball at each radar measurement point, for example, (d1, z1), (d2, z2), (d3, z3), etc. At this time, the azimuth information of the golf ball can be measured based on the actual initial position coordinates (Pbi) of the golf ball.

[0103] The trajectory display processing unit of the ball trajectory visualization device according to one embodiment of the present invention can convert each golf ball bearing information collected by the radar sensing unit into coordinate information on the image (IM) based on the ball initial position coordinate (pbi) on the image (IM).

[0104] That is, each golf ball orientation information ((d1, z1), (d2, z2), (d3, z3), etc.) as shown in (a) of Fig. 8 can be converted into coordinate information (p1, p2, p3, etc.) on the image (IM) based on the ball initial position coordinate (pbi) as shown in (b) of Fig. 8. For example, (d1, z1) can be converted into the p1 coordinate on the image, (d2, z2) can be converted into the p2 coordinate on the image, and (d3, z3) can be converted into the p3 coordinate on the image.

[0105] As described above, the trajectory display processing unit can accurately display the ball trajectory (iTri) starting from the ball initial position coordinates on the image (IM) by connecting each ball position coordinate (p1, p2, p3...) calculated based on the ball initial position coordinates (pbi) on the image (IM).

[0106] As described above, the ball trajectory visualization device according to one embodiment of the present invention sets a ball sensing area in an image captured by a single camera and stores coordinate matching information, which is matching information with coordinates in an actual space, detects a ball object corresponding to a golf ball through the ball sensing area in an image when a user makes a golf shot, calculates the ball initial position coordinates, generates a trigger signal when the user hits the golf ball, and collects golf ball direction information using radar sensing data for a golf ball moving from the trigger point, and the trajectory display processing unit converts the collected golf ball direction information into coordinate information on the image and connects and displays a ball trajectory starting from the ball initial position coordinates, thereby enabling ball trajectory visualization at an accurate position in the image.

[0107] However, when a user detects a ball object corresponding to a golf ball before making a golf shot and calculates the ball's initial position coordinates, the calculated ball's initial position coordinates may be different from the actual position where the golf ball is initially placed.

[0108] Because the image acquired by a single camera with a single view is a two-dimensional image, there is a limitation in that it is difficult to know the position information in an actual three-dimensional space, and also because the ball sensing area can only calculate the position on a plane, for example, when a golf ball is placed on a tee in an actual space, the initial position coordinates of the ball obtained from the image may differ from the actual one.

[0109] Figure 9 (a) shows an image of a case where a golf ball is placed on the floor and a case where a tee is placed on the floor and a golf ball is placed on the tee, and Figure 9 (b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0110] As shown in (a) of Fig. 9, the position of the ball object iB1 detected within the ball sensing area (SA) in the image (IM) acquired by the single camera corresponds identically to the position (Pb1) of the actual golf ball B1 as shown in (b) of Fig. 9, so the position coordinates of the ball object iB1 identified by the ball sensing area (SA) on the image (IM), i.e., the ball initial position coordinates, are coordinates that match the initial position coordinates (Pb1) of the actual golf ball (B1).

[0111] In this case, since the ball initial position coordinates of the ball object iB1 are matched identically to the actual ball, the ball trajectory obtained using radar sensing data can be displayed as a trajectory starting from the ball initial position coordinates on the image, thereby visualizing an accurate ball trajectory.

[0112] However, the position of the ball object iB2 detected within the ball sensing area (SA) in the image (IM) acquired by the single camera as shown in (a) of FIG. 9 corresponds to the position of the golf ball B2b placed on the tee as shown in (b) of FIG. 9, but the golf ball B2b in the line of sight of the single camera (200) is recognized as the position of the golf ball B2a placed on a plane.

[0113] Therefore, the ball initial position coordinates obtained through the ball sensing area (SA) for the ball object iB2 in the image are not based on Pb2b, which is the actual position of the golf ball B2b, but on Pb2a, which is the position of the golf ball B2a, which is located at a different position from the actual position. Therefore, it can be seen that an error occurred in the ball initial position coordinates.

[0114] To prepare for such cases, the present invention calculates corrected ball initial position coordinates through a process of correcting ball initial position coordinates and displays the ball trajectory based on these coordinates. This will be described with reference to FIGS. 10 and 11.

[0115] FIG. 10 (a) is a part of an image taken when a golf ball is placed on a tee, and the ball trajectory visualization device according to an embodiment of the present invention recognizes the ball object iB2 on the image (IM) as the position of B2a placed on the floor as shown in FIG. 10 (b), and therefore the ball initial position coordinates calculated accordingly are also based on the position of the golf ball placed at position B2a, which is different from the actual position of the golf ball.

[0116] A ball trajectory visualization device according to one embodiment of the present invention can verify whether the initial position coordinates of the ball in the image are correct using radar sensing data at the time when the golf ball is hit, and can perform a process of correcting the initial position coordinates of the ball based on the verification result.

[0117] When a trigger signal is generated at the time the golf ball is hit, the radar sensing unit can collect golf ball orientation information including the vertical height angle and the left-right direction angle of the golf ball in real space from the time the trigger signal is generated to the time the radar signal reflected from the golf ball is received.

[0118] That is, as shown in (b) of Fig. 10, golf ball orientation information such as P1a, P2a, P3a, etc. is collected based on the position of the B2a golf ball.

[0119] At this time, the trajectory display processing unit verifies the ball initial position coordinates or the actual initial position coordinates of the golf ball based thereon using the golf ball orientation information calculated by the radar at the trigger point or the ball coordinates at the trigger point converted from the orientation information into coordinates on the image, thereby determining whether the ball initial position coordinates are the coordinates of the accurate ball position, and can correct the ball initial position coordinates according to the verification result.

[0120] In the verification process described above, if it is determined that the coordinates based on the golf ball bearing information at the trigger point are too far from the ball initial position coordinates, it can be determined that there is an error in the ball initial position coordinates.

[0121] Since the trigger point is the point in time when the golf ball starts to move after being struck by a golf club while still, the position of the golf ball bearing information at the trigger point converted into coordinates on the image must be at almost the same position as the position of the ball's initial position coordinates or at a very close distance. However, if the position (P1a) of the golf ball at the trigger point sensed by the radar as shown in (b) of FIG. 10 is a certain distance away from the position (B2a) of the ball's initial position coordinates (the position coordinates of the ball object calculated through the ball sensing area on the image), it is difficult to view the golf ball as having started from the position (B2a) of the ball's initial position coordinates that were initially obtained.

[0122] Therefore, the initial ball position coordinates obtained initially can be corrected to more accurate position coordinates by using the position of the golf ball at the trigger point sensed by the radar.

[0123] For example, since the height (h) information of the single camera (200) of the ball trajectory visualization device (100) and the angle (a) information of the line of sight from that height at which the single camera (200) views the golf ball are known values ​​in advance, the line (AL) of the line of sight from which the single camera (200) views the golf ball from the height h can be obtained using these, and the actual golf ball is located on the line (AL) of the line of sight.

[0124] Considering the height of the position (P1a) of the golf ball at the trigger point here, the actual position of the golf ball can be found in a certain range (RB) on the line (AL) in the line of sight below the height of the position (P1a) of the golf ball.

[0125] Since the Doppler shift of the radar does not occur when the golf ball is stationary, its position cannot be determined with radar sensing data, and the position of the golf ball can be determined only when the golf ball starts moving. Therefore, the position of the golf ball at the time of stationary golf ball has a slight difference (hereinafter referred to as “position difference”) from the position of the golf ball at the time of triggering. If the position difference is saved as an average value or other statistical value, it is possible to use this to correct the actual position of the golf ball to an accurate position.

[0126] For example, in the case of (c) of Fig. 10, it is possible to specify the position corresponding to the average of the position difference as described above from the position (P1a) of the golf ball at the trigger point within a certain range (RB) on the line (AL) of the above-mentioned line of sight direction as the actual initial position of the golf ball.

[0127] Therefore, by converting the actual position of the golf ball obtained as described above into coordinates on the image, the converted coordinates can be calculated as corrected coordinates of the initial position coordinates of the ball that were initially obtained.

[0128] FIG. 11 shows, in the case illustrated in FIG. 10, a ball trajectory displayed based on the ball initial position coordinates initially calculated from an image (IM) taken with a single camera of a golf ball flying as the user hits the golf ball with a golf shot, and a ball trajectory displayed by correcting the initial ball initial position coordinates to a more accurate position.

[0129] In the image shown in Fig. 11, iB2 represents the ball object that was initially placed.

[0130] The point indicated as pb2a in the image (IM) shown in Fig. 11 corresponds to the initial ball position coordinates calculated using the ball sensing area, and the ball trajectory calculated based on the radar sensing results of the golf ball based on this is the trajectory indicated by iTr1.

[0131] As shown in Fig. 11, the initial ball position coordinates pb2a, which were initially obtained, were erroneous when verified through radar sensing data at the trigger point as explained in Fig. 10, so the iTr1 ball trajectory starting from the initial ball position coordinates pb2a in the image (IM) is somewhat different from the actual trajectory.

[0132] As described above, the present invention verifies the initial ball position coordinates using radar sensing data, and if it determines that there is an error, it corrects the error to calculate accurate ball initial position coordinates and displays the ball trajectory based on the corrected position coordinates.

[0133] In Fig. 11, the position where the ball initial position coordinates pb2a, which were initially obtained, are corrected to a more accurate position using radar sensing data at the trigger point is the position indicated by the drawing symbol pb2b, and it can be seen that the ball trajectory iTr2 starting from the pb2b position is the accurate ball trajectory.

[0134] The present invention has a special advantage in that it obtains more accurate initial ball position coordinates by verifying and correcting the initial ball position coordinates initially obtained from an image of a user's golf shot acquired by a single camera using radar sensing data, and visualizes an accurate ball trajectory starting from the corrected position coordinates obtained in this way on an image.

[0135] Previously, a configuration and method for obtaining more accurate ball initial position coordinates by verifying and correcting the ball initial position coordinates initially obtained by a radar sensing-based ball trajectory visualization device according to an embodiment of the present invention using radar sensing data were described. In relation to this, a radar sensing-based ball trajectory visualization device according to another embodiment of the present invention can provide a configuration and method for obtaining accurate ball initial position coordinates from the beginning by using information set in advance before sensing.

[0136] As explained above, since the present invention calculates the initial ball position coordinates using a single camera, it is not based on the position coordinates of the golf ball in a three-dimensional space. Therefore, for example, when a user actually places a golf ball on a tee and takes a golf shot, if the initial ball position coordinates are calculated by analyzing the image taken by the single camera, an error equal to the height of the tee occurs. However, the present invention corrects the error by verifying the ball initial position coordinates that were initially calculated using radar sensing data.

[0137] Furthermore, a radar sensing-based ball trajectory visualization device according to another embodiment of the present invention can provide a function capable of calculating more accurate ball initial position coordinates from the beginning by calculating ball initial position coordinates by considering a height on an image corresponding to the preset tee height when using a tee as described above.

[0138] Specifically, a radar sensing-based ball trajectory visualization device according to another embodiment of the present invention may have a configuration capable of detecting current location information by incorporating GPS or detecting current location information using another sensor, and may also be configured to store terrain information or map information of a golf course where a user is currently playing golf in advance or recognize it in advance through communication with a server.

[0139] By this configuration, a radar sensing-based ball trajectory visualization device according to another embodiment of the present invention can recognize the location of a hole in a golf course where a golf game is currently in progress, and can also determine whether the hole currently located is a PAR 3 hole, a PAR 4 hole, or a PAR 5 hole.

[0140] Before playing a game of golf, a user can preset the height of the tee to be used in each hole of the golf course using a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention (the tee height information saved in advance by the user can be downloaded and recognized through a server, or the user can directly input it into the device and preset it).

[0141] For example, the user can preset the tee height to 1.5 cm for hole 2 (PAR 3 hole), 4.5 cm for hole 3 (PAR 4 hole), 4.5 cm for hole 4 (PAR 5 hole), etc.

[0142] Accordingly, a ball trajectory visualization device based on radar sensing according to another embodiment of the present invention can recognize the position of a hole in a golf course where a golf game is currently in progress and recognize the tee height of a user to be used in the current hole, and when calculating the ball initial position coordinates through an image captured by a single camera, it can calculate an accurate ball initial position coordinate by considering the height on the image for the preset and recognized tee height.

[0143] The ball initial position coordinates calculated by considering the tee height in this way can eliminate or significantly reduce the errors described above, and even when the ball initial position coordinates are calculated by considering the tee height in this way, it is possible to calculate more accurate ball initial position coordinates by performing additional corrections through verification using radar sensing data.

[0144] When multiple users play a round of golf together and the multiple users commonly use the ball trajectory visualization device according to the present invention, the present invention can store the tee height information set by each of the multiple users, and when a specific user takes a golf shot, when a caddy or another user sets the user to take the current shot on a golf terminal, the shot order information of the user can be received from the golf terminal and the tee height information set in advance by the user can be applied. In this way, it is possible to apply the tee height individually set for each of the multiple users to the visualization of each individual golf shot image.

[0145] Meanwhile, a method for visualizing a ball trajectory in a user's golf shot video according to one embodiment of the present invention will be described with reference to the flowcharts illustrated in FIGS. 12 and 13, respectively.

[0146] Although a method for visualizing a ball trajectory in a user's golf shot video using a radar sensing-based ball trajectory visualization device according to an embodiment of the present invention has already been described in detail with reference to FIGS. 1 to 11, the method for visualizing a ball trajectory will be summarized and described once again with reference to the flowcharts shown in FIGS. 12 and 13.

[0147] The flowchart illustrated in Fig. 12 is connected to the flowchart illustrated in Fig. 13 via reference symbol A, and represents one process as a whole.

[0148] First, a ball trajectory visualization device based on radar sensing according to an embodiment of the present invention is placed and positioned at a predetermined rear position on the floor where the user will make a golf shot (S100).

[0149] The sensing processing unit of the ball trajectory visualization device matches coordinates of elements capable of specifying an area of ​​a predetermined size on an image acquired by a single camera with coordinates in real space to store coordinate matching information, and sets an area specified by the coordinates of elements capable of specifying an area of ​​a predetermined size on the image as a ball sensing area (S110).

[0150] The user places a golf ball at an arbitrary location on the floor within the field of view of the single camera and prepares for a golf shot. At this time, the single camera acquires an image, and the sensing processing unit searches for a pre-set ball sensing area in the image acquired by the single camera (S120).

[0151] At this time, if a ball object corresponding to a golf ball placed on the floor is detected as a result of searching the ball sensing area (S130), the sensing processing unit can use coordinate information about the ball sensing area to calculate the position coordinates of the detected ball object as the ball initial position coordinates (S140).

[0152] The sensing processing unit can use coordinate matching information to convert the ball's initial position coordinates into the golf ball's actual spatial coordinates, which are the golf ball's actual initial position coordinates, and transmit them to the radar sensing unit (S150).

[0153] Afterwards, when the user hits the golf ball with a golf club (S160), the ball trajectory visualization device first generates a trigger signal (S170), and the radar sensing unit operates according to the trigger signal.

[0154] The radar sensing unit transmits a radar signal toward a golf ball moving by a hit and receives a reflected wave signal reflected from the golf ball, thereby collecting golf ball orientation information including information on the vertical height angle of the golf ball and the left and right direction angle of the golf ball at the time of measurement (S180).

[0155] The trajectory display processing unit of the ball trajectory visualization device can verify the ball initial position coordinates or the actual initial position coordinates of the golf ball based thereon using the golf ball orientation information calculated by the radar at the trigger point or the ball coordinates at the trigger point converted from the orientation information into coordinates on the image (S200). For example, it can determine whether the ball initial position coordinates calculated in step S140 are accurate position coordinates.

[0156] If it is determined that there is no error in the ball initial position coordinates calculated in step S140 as a result of the verification as described above and that no correction is necessary (S210), the trajectory display processing unit converts the golf ball bearing information collected by the radar sensing unit at each measurement point into coordinates on the image based on the ball initial position coordinates and connects them, thereby displaying the ball trajectory starting from the ball initial position coordinates on the image (S220).

[0157] However, if it is determined that there is an error in the ball initial position coordinates calculated in step 140 as a result of the verification as described above and that correction of the ball initial position coordinates is necessary (S210), the trajectory display processing unit can correct the ball initial position coordinates calculated in step S140 to accurate position coordinates according to the verification result as described above (S230).

[0158] In addition, the trajectory display processing unit converts the golf ball bearing information collected by the radar sensing unit at each measurement point into coordinates on the image based on the ball initial position coordinates corrected in step S230 and connects them, thereby displaying the ball trajectory starting from the ball initial position coordinates corrected to a more accurate position on the image (S240).

[0159] Although not shown in the flowchart in the drawing, the sensing processing unit of the present invention can calculate accurate information by applying the preset tee height information when calculating the ball initial position coordinates in step S140 by presetting the tee height to be used by the user as described above.

[0160] As described above, the method for visualizing a ball trajectory in a video of a user's golf shot according to the present invention and the ball trajectory visualization device based on radar sensing using the same detect the initial position of a golf ball that the user places on the floor for a golf shot by using a camera-captured video for filming the user making a golf shot and providing the video to the user, and visualize the movement trajectory of the golf ball calculated using the radar sensing result for the golf ball moving by the hit as an accurate ball trajectory starting from the initial position of the golf ball in the video, and furthermore, verify whether there is an error in the coordinates of the initial ball position calculated in the video using radar sensing data, and if there is an error, correct it to a more accurate coordinates of the initial ball position, and visualize the accurate ball trajectory starting from the corrected position.

[0161] The method for visualizing a ball trajectory in a user's golf shot video according to the present invention and the ball trajectory visualization device based on radar sensing using the same can be used in golf-related industries such as sensing technology for golf balls, golf practice, and screen golf systems.

Claims

1. A method for visualizing a ball trajectory using radar sensing results from a camera-captured video of a user taking a golf shot. A step of setting a ball sensing area for sensing a golf ball using the camera-captured image on the floor where the user will make a golf shot; A step of detecting a ball object corresponding to the golf ball within the ball sensing area on the camera-captured image to calculate the ball initial position coordinates and calculate the actual initial position coordinates of the golf ball corresponding to the ball initial position coordinates; A step of collecting radar sensing data on a golf ball moving according to the user's golf shot based on the actual initial position coordinates of the golf ball; and A step of displaying a ball trajectory starting from the ball initial position coordinates or corrected coordinates of the ball initial position coordinates in the camera-captured image using the radar sensing data for the golf ball collected above; A method for visualizing ball trajectory in a user's golf shot video including:

2. In the first paragraph, the step of setting the ball sensing area is: The above camera is a single camera with a single view. A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of matching coordinates of elements capable of specifying a predetermined size area on an image acquired by the single camera with coordinates in real space to store coordinate matching information, and setting the predetermined size area on the image as the ball sensing area.

3. In the first paragraph, the step of setting the ball sensing area is: The above camera is a single camera with a single view. A method for visualizing a ball trajectory in a user's golf shot video, characterized by comprising the steps of: matching coordinates of elements that can specify the specifications of a structure that appears in an image captured from the rear by the single camera with coordinates in real space, including a structure whose specifications are known in advance on the ground where the golf shot will be made, and storing coordinate matching information; and setting an area corresponding to the structure that appears in the image as the ball sensing area.

4. In the second or third paragraph, the step of calculating the actual initial position coordinates of the golf ball is: A step of detecting a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera; A step of calculating the position coordinates of the ball object as the ball initial position coordinates using coordinate information about the ball sensing area, A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of converting the ball initial position coordinates into the actual initial position coordinates of the golf ball, which are the actual spatial coordinates of the golf ball, using the coordinate matching information.

5. In paragraph 4, The above user presets the height of the tee to be used when taking a golf shot. The step of calculating the position coordinates of the ball object as the initial position coordinates of the ball is as follows: A step of recognizing the preset tee height, A method for visualizing a ball trajectory in a user's golf shot video, characterized by comprising a step of calculating the ball initial position coordinates by applying height information corresponding to the recognized tee height on the video to the position coordinates of a ball object detected within the ball sensing area.

6. In paragraph 1, The step of collecting radar sensing data for the above moving golf ball is: A step of collecting golf ball orientation information including the vertical height angle and the left and right direction angle of the golf ball in real space at the time of receiving the radar signal reflected from the golf ball, The step of displaying the above ball trajectory is: A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of converting each of the collected golf ball bearing information into coordinate information on the camera-captured image based on the ball initial position coordinates and connecting them to display a ball trajectory starting from the ball initial position coordinates.

7. In the first paragraph, the step of displaying the ball trajectory is: A step of verifying the initial position coordinates of the ball on the camera-captured image using radar sensing data at the time the golf ball is hit, A step of correcting the initial position coordinates of the ball according to the above verification results, A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of converting radar sensing data collected for a golf ball moving from the point of impact into coordinate information on the camera-captured image and displaying a ball trajectory starting from the corrected ball initial position coordinates on the image.

8. In the first paragraph, the step of collecting radar sensing data for the moving golf ball is: A step in which a trigger signal is generated at the time the golf ball is hit, A step of collecting golf ball orientation information including the vertical height angle and the left-right direction angle of the golf ball in real space from the time of generation of the trigger signal to the time of reception of the radar signal reflected from the moving golf ball, The step of displaying the above ball trajectory is: A step of converting the golf ball bearing information at the time of occurrence of the trigger signal into coordinate information on the camera-captured image based on the ball initial position coordinates to generate trigger point ball coordinate information; A step of correcting the initial position coordinates of the ball using the ball coordinates at the trigger point, A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of converting the collected golf ball bearing information about the golf ball moving from the trigger point into coordinate information on the camera-captured image based on the corrected ball initial position coordinates and connecting them to display a ball trajectory starting from the corrected ball initial position coordinates.

9. In the first paragraph, the step of collecting radar sensing data for the moving golf ball comprises: A step in which a trigger signal is generated at the time the golf ball is hit, A step of collecting golf ball orientation information including the vertical height angle and the left-right direction angle of the golf ball in real space from the time of generation of the trigger signal to the time of reception of the radar signal reflected from the moving golf ball, The step of displaying the above ball trajectory is: A step of calculating 3D position information of the golf ball at the time of trigger signal generation using the golf ball orientation information at the time of trigger signal generation and the actual initial position coordinates of the golf ball, A step of calculating corrected coordinate information of the ball's initial position coordinates by converting the three-dimensional position information of the golf ball calculated above into coordinate information on the camera-captured image, A method for visualizing a ball trajectory in a user's golf shot video, characterized by including a step of converting the collected golf ball bearing information about the golf ball moving from the trigger point into coordinate information on the camera-captured image based on the corrected coordinates and connecting them to display a ball trajectory starting from the corrected coordinates.

10. A single camera with a single viewpoint that acquires a video of a scene in which a user makes a golf shot on the floor from behind the floor on which the user makes a golf shot; A sensing processing unit that sets a ball sensing area for sensing a golf ball using an image acquired by the single camera, detects a ball object corresponding to the golf ball within the ball sensing area on the image acquired, calculates ball initial position coordinates, and calculates actual initial position coordinates of the golf ball corresponding to the ball initial position coordinates; A radar sensing unit that collects radar sensing data on a golf ball moving according to the user's golf shot based on the actual initial position coordinates of the golf ball; and A trajectory display processing unit that displays a ball trajectory starting from the ball initial position coordinates or corrected coordinates of the ball initial position coordinates in the camera-captured image using the radar sensing data for the golf ball collected above; A ball trajectory visualization device based on radar sensing including:

11. In the 10th paragraph, the sensing processing unit, The single camera is configured to store coordinate matching information by matching the coordinates of elements that can specify the specifications of the structure that appears in the image captured from the rear with an angle of view that includes a structure whose specifications are known in advance on the ground where the golf shot will be made, with coordinates in real space, and to set the area corresponding to the structure that appears in the image as the ball sensing area. A ball trajectory visualization device based on radar sensing, characterized in that it is configured to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object as the ball initial position coordinates using coordinate information about the ball sensing area, and convert the ball initial position coordinates into the actual initial position coordinates of the golf ball, which are the actual spatial coordinates of the golf ball, using the coordinate matching information.

12. In the 10th paragraph, the sensing processing unit, The above user is configured to preset the height of the tee to be used when making a golf shot, The single camera is configured to store coordinate matching information by matching the coordinates of elements that can specify the specifications of the structure that appears in the image captured from the rear with an angle of view that includes a structure whose specifications are known in advance on the ground where the golf shot will be made, with coordinates in real space, and to set the area corresponding to the structure that appears in the image as the ball sensing area. A ball trajectory visualization device based on radar sensing, characterized in that it is configured to detect a ball object corresponding to the golf ball within the set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object using coordinate information about the ball sensing area, recognize the preset tee height, apply height information on the image corresponding to the recognized tee height to the position coordinates of the ball object to calculate the ball initial position coordinates, and convert the ball initial position coordinates into actual initial position coordinates of the golf ball, which are actual spatial coordinates of the golf ball, using the coordinate matching information.

13. In paragraph 10, The above radar sensing unit, It is configured to collect golf ball orientation information including the vertical height angle and the left and right direction angle of the golf ball in the actual space at the time of receiving the radar signal reflected from the golf ball. The above trajectory display processing unit, A ball trajectory visualization device based on radar sensing, characterized in that each golf ball bearing information collected by the radar sensing unit is converted into coordinate information on the camera-captured image based on the ball initial position coordinates and connected to display the ball trajectory starting from the ball initial position coordinates.

14. In the 10th paragraph, the trajectory display processing unit, The initial position coordinates of the ball are verified on the camera-acquired image using radar sensing data at the time the golf ball is hit, and the initial position coordinates of the ball are corrected according to the verification result. A ball trajectory visualization device based on radar sensing, characterized in that the device is configured to convert radar sensing data collected for a golf ball moving from the point of impact by the radar sensing unit into coordinate information on the camera-captured image and display the ball trajectory starting from the corrected ball initial position coordinates on the image.

Citation Information

Patent Citations

  • Ball tracking device and ball tracking method

    JP6763559B1

  • Apparatus for virtual golf simulation, and sensing device and method used to the same

    KR101048090B1

  • Indoor video golf system using video camera and flash and method of operating the system

    KR1019950005045A

  • Architectural exploration and compression method of vision transformer and electronic device therefor

    KR1020240054158A

  • System and Method for Three Dimensional Object Tracking Using Combination of Radar and Image Data

    US20200391077A1