Method for visualizing ball trajectory in golf ball hitting image of user and apparatus for visualizing ball trajectory based on radar sensing using same

Abstract

The purpose of the present invention is to provide a method for visualizing a ball trajectory in a golf ball hitting image of a user, and a radar sensing-based ball trajectory visualization device using the same, the method being capable of obtaining an image by using a camera for capturing a golf ball hitting by the user and providing the golf ball hitting to the user. An initial position of a golf ball placed on a floor by a user for hitting a golf ball is sensed, and a movement trajectory of the golf ball calculated using a radar sensing result for the golf ball moved by hitting is visualized into an accurate ball trajectory starting from the initial position of the golf ball on an image.

Description

Technical Field

[0001] This invention relates to a method for visualizing the trajectory of a ball in a user's golf shot video and a radar-sensing-based ball trajectory visualization device using the same. The method can visualize the trajectory of a ball moving in parallel from an accurate initial position in a video of a user hitting a golf club, using the video and sensing results obtained through radar. Background Technology

[0002] In golf, when a user hits a golf ball with a golf club, the ball's flight is difficult to discern with the naked eye because of its extremely small size and the considerable distance it travels at very high speeds.

[0003] Because it is difficult to confirm the flight of a golf ball with the naked eye, even in footage of a user hitting a golf ball from a designated location, it is still difficult to determine the trajectory of the ball.

[0004] In systems like screen golf, when a user hits a golf ball in a confined space towards the screen in front of them, the camera sensors detect the initial speed, direction angle, and altitude angle of the golf ball as it is hit by the golf club. Based on the initial movement characteristics of the golf ball, the system calculates the overall trajectory of the golf ball through physical calculations and displays it on the screen. However, this is not a measurement of the actual movement of the golf ball, but rather a calculated and estimated trajectory, which therefore differs from the actual trajectory of the golf ball.

[0005] While camera sensors can also be used to sense the trajectory of a golf ball, as mentioned above, their sensing range is very narrow, only able to detect the initial movement characteristics of the golf ball. Therefore, they are not suitable for sensing the trajectory of a golf ball after a golf shot in a large venue like an actual golf course.

[0006] On actual golf courses, radar sensors are widely used as a representative sensing device for detecting the movement of the golf ball caused by a golf shot.

[0007] A radar sensor is a device that, when a radar signal emitted toward a moving golf ball is reflected from the golf ball, receives the radar signal deflected due to the Doppler shift phenomenon of the radar and analyzes the movement vector relative to the movement of the golf ball.

[0008] Using this radar sensor, the trajectory of a golf ball as it moves with a user's golf shot can be calculated.

[0009] However, since radar sensors operate using the Doppler shift phenomenon of radar signals, although they can obtain information on the phase changes related to the position changes caused by the movement of the golf ball and calculate the movement trajectory using information on the position changes of the golf ball from the start of its movement, there are the following problems: it is difficult to determine the position of the golf ball in the front-back direction, and it is impossible to determine the position of the golf ball at the initial moment of the golf ball being hit by the radar sensor, that is, the position of the golf ball that is stationary on the floor or tee.

[0010] For this reason, even though the radar sensor detects the movement of the golf ball after it is hit and calculates its trajectory, it cannot accurately determine the initial position of the golf ball. As a result, the calculated trajectory of the golf ball does not start from the actual initial position of the golf ball, but from a different position.

[0011] Because radar sensors cannot determine the initial position of a golf ball, a reference distance (the distance from the radar sensor set on the floor) is preset. The ball can only complete a movement trajectory from its relatively accurate initial position when it is hit at a position fixed at this reference distance. However, if the user places the golf ball in any position to hit the ball, the starting position of the golf ball's movement trajectory will differ significantly from the actual position, resulting in a considerable overall trajectory error.

[0012] To calculate the trajectory of a ball as it moves with a golf club, attempts are underway to combine camera sensing with radar sensing to compensate for their respective shortcomings. To sense the initial position of a golf ball placed arbitrarily, a stereoscopic camera system capable of sensing the position of objects in three-dimensional space (a system that uses two or more cameras positioned at different locations to sense the same subject and calculate its position in three-dimensional space) can be used. However, integrating this stereoscopic camera system with a radar sensing system into a single device significantly increases the price. Therefore, there is an urgent need to develop a device that can accurately visualize the trajectory of a golf ball using radar sensing without requiring a stereoscopic camera system consisting of two or more cameras.

[0013] [Existing Technical Documents]

[0014] Korean Patent Publication No. 10-1845503

[0015] Korean Patent Publication No. 10-2335944

[0016] Korean Patent Publication No. 10-2353481

[0017] Korean Patent Publication No. 10-2033703 Summary of the Invention

[0018] Technical issues

[0019] The present invention aims to provide a method for visualizing the trajectory of a ball in a user's golf shot video and a radar-sensing-based ball trajectory visualization device using the same. The method can acquire images using a camera used to capture and provide images to the user while the user is shooting a golf shot, sense the initial position of the golf ball placed on the floor by the user for the purpose of the golf shot, and visualize the movement trajectory of the golf ball calculated using radar sensing results of the golf ball moving due to being hit as an accurate ball trajectory starting from the initial position of the golf ball in the image.

[0020] Technical solution

[0021] A method for visualizing the trajectory of a ball using radar sensing results in camera footage of a user hitting a golf ball includes the following steps: setting a ball sensing area on the floor where the user will hit the golf ball for sensing the golf ball in the camera footage; detecting a ball object corresponding to the golf ball in the ball sensing area in the camera footage, calculating the ball's initial position coordinates, and calculating the actual initial position coordinates of the golf ball corresponding to the initial position coordinates; collecting radar sensing data of the golf ball moving as the user hits the golf ball, based on the actual initial position coordinates; and displaying the ball trajectory in the camera footage, originating from the initial position coordinates of the ball or a corrected coordinate of the initial position coordinates of the ball, using the collected radar sensing data of the golf ball.

[0022] Furthermore, preferably, the method is characterized in that, in the step of setting the ball sensing area: the camera is a single-view single camera, including: matching the coordinates of the elements that can determine the area of ​​a predetermined size on the image acquired by the single camera with the coordinates in the actual space and storing the coordinate matching information, and setting the area of ​​the predetermined size on the image as the ball sensing area.

[0023] Furthermore, preferably, the method is characterized in that, in the step of setting the ball sensing area: the camera is a single-view single camera, including: matching the coordinates of elements of the structure's specifications that appear in an image taken by the single camera from behind from a viewpoint including a structure of known specifications placed on the floor where the golf shot will be taken with coordinates in actual space and storing the coordinate matching information, and setting the area corresponding to the structure appearing in the image as the ball sensing area.

[0024] Furthermore, preferably, the method is characterized in that the step of calculating the actual initial position coordinates of the golf ball includes: detecting a ball object corresponding to the golf ball within a set ball sensing area on the image acquired by the single camera; calculating the position coordinates of the ball object as the initial position coordinates of the ball using coordinate information related to the ball sensing area; and converting the initial position coordinates of the ball into the actual initial position coordinates of the golf ball as coordinates of the golf ball in actual space using the coordinate matching information.

[0025] Furthermore, preferably, the method is characterized in that the step of pre-setting the height of the tee that the user will use when making a golf shot, and calculating the position coordinates of the ball as the initial position coordinates of the ball, includes: identifying the pre-set height of the tee; and applying the height information to the position coordinates of the ball detected in the ball sensing area on the image corresponding to the identified height of the tee, in order to calculate the initial position coordinates of the ball.

[0026] Furthermore, preferably, the method is characterized in that the step of collecting radar sensing data for the moving golf ball includes: collecting golf ball orientation information including the vertical elevation angle and horizontal direction angle of the golf ball in actual space at the time of receiving radar signals reflected from the golf ball; and the step of displaying the ball trajectory includes: converting each collected golf ball orientation information into coordinate information on the image acquired by the camera based on the initial position coordinates of the ball, and connecting them, thereby displaying the ball trajectory starting from the initial position coordinates of the ball.

[0027] Furthermore, preferably, the method is characterized in that the step of displaying the ball trajectory includes: verifying the initial position coordinates of the ball on the image acquired by the camera using radar sensing data at the time the golf ball was hit; correcting the initial position coordinates of the ball based on the verification result; converting the radar sensing data collected from the time the golf ball was hit for the moving golf ball into coordinate information on the image acquired by the camera, and displaying the ball trajectory starting from the corrected initial position coordinates of the ball on the image.

[0028] Furthermore, preferably, the method is characterized in that the step of collecting radar sensing data for the moving golf ball includes: the step of generating a trigger signal at the time the golf ball is hit; and the step of collecting golf ball orientation information from the time the trigger signal is generated, including the vertical elevation angle and horizontal direction angle of the golf ball in actual space at the time the radar signal reflected from the moving golf ball is received; and the step of displaying the ball trajectory includes: converting the golf ball orientation information at the time the trigger signal is generated into coordinate information on the image acquired by the camera based on the ball's initial position coordinates, to generate ball coordinate information at the trigger time point; correcting the ball's initial position coordinates using the ball coordinates at the trigger time point; and converting the golf ball orientation information collected from the moving golf ball at the trigger time point into coordinate information on the image acquired by the camera based on the corrected ball's initial position coordinates, and connecting them, thereby displaying the ball trajectory starting from the corrected ball's initial position coordinates.

[0029] Furthermore, preferably, the method is characterized in that the step of collecting radar sensing data for the moving golf ball includes: the step of generating a trigger signal at the time the golf ball is hit; and the step of collecting golf ball orientation information from the time the trigger signal is generated, including the vertical elevation angle and horizontal direction angle of the golf ball in actual space at the time the radar signal reflected from the moving golf ball is received; the step of displaying the ball trajectory includes: the step of calculating the three-dimensional position information of the golf ball at the time the trigger signal is generated using the golf ball orientation information at the time the trigger signal is generated and the actual initial position coordinates of the golf ball; the step of converting the calculated three-dimensional position information of the golf ball into coordinate information on the image acquired by the camera, thereby calculating the corrected coordinate information of the initial position coordinates of the ball; and the step of converting the golf ball orientation information collected from the moving golf ball at the trigger time point into coordinate information on the image acquired by the camera based on the corrected coordinates, thereby displaying the ball trajectory starting from the corrected coordinates.

[0030] On the other hand, an embodiment of the radar-sensing ball trajectory visualization device of the present invention includes: a single-view camera that acquires an image of a scene of the user hitting a golf ball on the floor from behind the floor where the user is about to hit the ball; a sensing processing unit that sets a ball sensing area for sensing the golf ball using the image acquired by the single camera, detects a ball object corresponding to the golf ball in the ball sensing area on the acquired image, calculates the initial position coordinates of the ball, and calculates the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball; a radar sensing unit that collects radar sensing data of the golf ball moving with the user's golf shot, based on the actual initial position coordinates of the golf ball; and a trajectory display processing unit that uses the collected radar sensing data of the golf ball to display the ball trajectory originating from the initial position coordinates of the ball in the image acquired by the camera or the corrected coordinates of the initial position coordinates of the ball.

[0031] Furthermore, preferably, the device is characterized in that the sensing processing unit is configured to match the coordinates of elements of the structure's dimensions in an image captured by the single camera from behind, from a perspective including a structure of known dimensions placed on the floor where the golf ball will be hit, with coordinates in actual space and store coordinate matching information, and set the area corresponding to the structure appearing in the image as the ball sensing area; and is configured to detect a ball object corresponding to the golf ball in the set ball sensing area in the image acquired by the single camera, calculate the position coordinates of the ball object as the initial position coordinates of the ball using coordinate information related to the ball sensing area, and convert the initial position coordinates of the ball into the actual initial position coordinates of the golf ball as coordinates of the golf ball in actual space using the coordinate matching information.

[0032] Furthermore, preferably, the device is characterized in that the sensing processing unit is configured to pre-set the height of the tee that the user will use when making a golf shot; configured to match the coordinates of the elements of the structure's specifications in an image captured by the single camera from behind from a perspective including a structure of known specifications placed on the floor where the golf shot will be made with coordinates in actual space and store coordinate matching information, and set the area corresponding to the structure appearing in the image as the ball sensing area; and configured to detect a ball object corresponding to the golf ball in the set ball sensing area in the image acquired by the single camera, calculate the position coordinates of the ball object using coordinate information related to the ball sensing area, identify the pre-set height of the tee, apply the height information to the position coordinates of the ball object in the image corresponding to the identified height of the tee to calculate the initial position coordinates of the ball, and use the coordinate matching information to convert the initial position coordinates of the ball into the actual initial position coordinates of the golf ball as coordinates in actual space.

[0033] Furthermore, preferably, the device is characterized in that the radar sensing unit is configured to collect golf ball orientation information, including the vertical elevation angle and horizontal direction angle of the golf ball in actual space at the time of receiving radar signals reflected from the golf ball; the trajectory display processing unit is configured to convert each golf ball orientation information collected by the radar sensing unit into coordinate information on the image acquired by the camera based on the initial position coordinates of the ball, and connect them to display the trajectory of the ball starting from the initial position coordinates of the ball.

[0034] Furthermore, preferably, the device is characterized in that the trajectory display processing unit is configured to: use radar sensing data at the time the golf ball is hit to verify the initial position coordinates of the ball on the image acquired by the camera; based on the verification result, correct the initial position coordinates of the ball; convert the radar sensing data collected by the radar sensing unit from the time the golf ball is hit for the moving golf ball into coordinate information on the image acquired by the camera; and display the trajectory of the ball starting from the corrected initial position coordinates of the ball on the image.

[0035] The effects of the invention

[0036] The method for visualizing the ball trajectory in a user's golf shot image and the radar-sensing-based ball trajectory visualization device of the present invention have the following effects: it can acquire images using a camera used to capture and provide images to the user while the user is shooting a golf shot, sense the initial position of the golf ball placed on the floor by the user for the purpose of the golf shot, and visualize the movement trajectory of the golf ball calculated using radar sensing results for the golf ball moving due to being hit as an accurate ball trajectory starting from the initial position of the golf ball in the image. Attached Figure Description

[0037] Figure 1 This diagram illustrates a scenario where a user is hitting a golf ball on a golf course or equivalent, with a radar-sensing ball trajectory visualization device according to an embodiment of the present invention in place.

[0038] Figure 2 This is a block diagram illustrating the structure of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention.

[0039] Figure 3 It is shown Figure 2 The diagram shows a more specific structure of the radar sensing unit and the structure of the radar signal receiving unit of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention.

[0040] Figure 4 It is shown as follows Figure 3 The diagram shows an example of how a radar sensor calculates the position information of a golf ball.

[0041] Figure 5 It is shown as follows Figure 4 Figure (c) shows an example of an image captured by a single camera of a ball trajectory visualization device behind a user making a golf shot.

[0042] Figure 6 This diagram illustrates an embodiment of the radar-sensing-based ball trajectory visualization device of the present invention, which uses a structure of known specifications to define the ball sensing area. Figure 6 (a) shows the state in actual space. Figure 6 (b) shows an image taken with a single camera in the state shown in (a).

[0043] Figure 7 This diagram illustrates how, in an embodiment of the present invention, a radar-sensing-based ball trajectory visualization device calculates the initial position coordinates of a ball using a ball-sensing area, with the ball placed on the floor by the user for a golf shot. Figure 7 (a) shows the state of user U making a golf shot in actual space. Figure 7(b) shows an image taken with a single camera in the state shown in (a).

[0044] Figure 8 (a) shows in Figure 7 In the state shown in (a), the golf ball moves after the user hits it. Figure 8 (b) shows the result captured using a single camera. Figure 8 The image of the actual scene shown in (a) displays the trajectory of the ball calculated using radar sensing results.

[0045] Figure 9 Image (a) shows an image related to a golf ball placed on the floor and a tee inserted on the floor, and the golf ball placed on the tee, in a ball trajectory visualization device according to an embodiment of the present invention. Figure 9 (b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0046] Figure 10 This is a diagram illustrating a situation where the initial position coordinates of a ball calculated from an image may be incorrect in a ball trajectory visualization device according to an embodiment of the present invention, and a method for correcting such errors.

[0047] Figure 11 It is shown Figure 10 The image, captured by a single camera as the user hits the golf ball, shows the ball's trajectory based on the initially calculated coordinates of its initial position, and the trajectory after the initial coordinates have been corrected to a more accurate position.

[0048] Figure 12 and Figure 13 This is a flowchart illustrating a method for visualizing the trajectory of a ball in a user's golf shot image using a ball trajectory visualization device according to an embodiment of the present invention. Detailed Implementation

[0049] The following detailed description, with reference to the accompanying drawings, details a method for visualizing the trajectory of a ball in a user's golf shot image according to an embodiment of the present invention, and a radar-sensing-based ball trajectory visualization device utilizing the same.

[0050] An embodiment of the present invention relates to a method for visualizing the trajectory of a ball in a user's golf shot image and a radar-sensing-based ball trajectory visualization device thereof, which involves the following apparatus and method: capturing a scene of a user hitting a golf ball outdoors, such as on a golf course, and providing the captured image to the user, wherein the ball is able to be sensed by radar after being hit by the golf ball and the ball trajectory based on the sensing result is visualized on the captured image.

[0051] Reference Figure 1 and Figure 2 The structure of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention will be described.

[0052] Figure 1 This illustration depicts a scenario where a user is hitting a golf ball on a golf course or equivalent, equipped with a radar-sensing ball trajectory visualization device according to an embodiment of the present invention. Figure 2 This is a block diagram illustrating the structure of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention.

[0053] An embodiment of the radar-sensing ball trajectory visualization device 100 of the present invention may include: a single-view single camera 200; a sensing processing unit 250, which calculates the position information of a golf ball from the image acquired by the single camera; a radar sensing unit 300, which collects radar sensing data for a golf ball that moves due to being hit; and a trajectory display processing unit 350, which displays the ball trajectory on the image acquired by the single camera using the radar sensing data collected by the radar sensing unit.

[0054] like Figure 1 As shown, an embodiment of the radar-sensing ball trajectory visualization device 100 of the present invention can be used by placing it in a forward-facing direction (the direction in which the golf ball flies after being hit) behind the floor BT where the user U holds the golf club 20 to hit the golf ball 10.

[0055] The single camera 200 of the aforementioned ball trajectory visualization device 100 can acquire images of the scene of user U hitting a golf ball from behind the floor BT where user U is about to hit the ball, looking forward from a single perspective.

[0056] The image of the user's golf shot is a simple two-dimensional image from a single camera, rather than an image acquired from multiple perspectives using multiple cameras in a stereoscopic vision manner.

[0057] If multiple cameras using stereo vision acquire images of a golf ball, the position of the golf ball in three-dimensional space can be calculated through the analysis of each image. This allows us to obtain information that cannot be obtained from radar sensing data. However, since radar sensors are expensive and stereo vision camera systems are also costly, combining these two systems into one device would significantly increase costs, making productization extremely difficult.

[0058] The advantage of the radar-sensing-based ball trajectory visualization device of one embodiment of the present invention is that, using a single camera which is essentially required to provide images related to the user's golf shot scene, it is able to calculate the information required for ball trajectory visualization at a level equivalent to or higher than the information that can be obtained using a stereo vision camera system, and by using this information together with the radar sensing results, it is able to accurately display the ball trajectory on the image.

[0059] As described above, the information required for visualizing the ball trajectory using a single camera can be calculated by the sensor processing unit 250 using images of the user's golf shot scene captured by the single camera 200.

[0060] That is, the sensing processing unit 250 of the radar-sensing ball trajectory visualization device according to an embodiment of the present invention can perform the following functions: setting a ball sensing area for sensing a golf ball using an image acquired by a camera 200; detecting a ball object corresponding to a golf ball within the ball sensing area on the acquired image to calculate the initial position coordinates of the ball; calculating the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball, and transmitting them to the radar sensing unit 300.

[0061] In one embodiment of the present invention, the radar sensing unit 300 of the radar-sensing ball trajectory visualization device collects radar sensing data of the golf ball as it moves with the user's golf shot, based on the actual initial position coordinates of the golf ball received from the sensing processing unit 250. The trajectory display processing unit 350 can display the ball trajectory from the initial position coordinates of the ball on the image captured by the camera using the radar sensing data of the golf ball collected by the radar sensing unit 300.

[0062] The overall operation of the radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention described above will be described later; firstly, in conjunction with Figure 3 and Figure 4 The process by which the radar sensing unit 300 of a radar-sensing ball trajectory visualization device based on a radar sensing according to an embodiment of the present invention obtains radar sensing data from a moving golf ball will be described.

[0063] Figure 3 It is shown Figure 2 The diagram shows a more detailed structure of the radar sensing unit and the radar signal receiving unit of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention. Figure 4 It is shown as follows Figure 3 The diagram shows an example of how a radar sensor calculates the position information of a golf ball.

[0064] The radar sensor unit 300 can basically use the Doppler effect of radar to calculate the phase change and position information of a golf ball that has moved after being hit at various measurement points in time.

[0065] like Figure 3 As shown in (a), the radar sensing unit 300 may include a signal transmitting unit 310, a signal receiving unit 320, and a signal analysis unit 330.

[0066] The radar sensor 300 can be configured to: transmit a radar signal of a predetermined frequency at a position at a predetermined distance behind the golf ball that the user is about to hit in the direction of the golf ball's movement due to the hit; receive the reflected wave reflected from the moving golf ball; and track the golf ball that has been hit and moved during the analysis of the reflected wave.

[0067] The signal transmitting unit 310 is configured to transmit a specified radar signal St in the aiming direction. Although not shown in the figure, it may be configured to include a transmitting antenna for transmitting radar signals.

[0068] The signal receiving unit 320 is configured to receive the reflected wave signal Sr from the moving golf ball 10, which is a radar signal St transmitted by the signal transmitting unit 310. Due to the Doppler effect, the reflected wave signal Sr transmitted by the signal transmitting unit 310 and reflected from the golf ball 10 is a signal whose frequency has changed, resulting in Doppler shift. That is, the signal receiving unit 320 will receive a signal that has undergone Doppler shift.

[0069] The signal receiving unit 320 is equipped with multiple receiving antennas for receiving reflected wave signals, and the position information of the golf ball can be calculated by using the phase difference of the received signals from each receiving antenna.

[0070] Figure 3 (b) briefly illustrates an example of the structure of the signal receiving unit 320 described above. As shown in the figure, the signal receiving unit 320 can be appropriately configured to have three or more receiving antennas, including RA1, RA2 and RA3, and the trajectory (elevation angle) and direction angle of the moving golf ball 10 can be calculated by utilizing the phase difference of the signals between each receiving antenna.

[0071] For example, in Figure 3 In the signal receiving unit 320 shown in (b), the vertical height angle of the moving golf ball 10 can be calculated by using the phase difference between the signals received by RA1 and RA2 respectively; and the horizontal direction angle of the moving golf ball 10 can be calculated by using the phase difference between the signals received by RA1 and RA3 respectively.

[0072] The signal analysis unit 330 can analyze the radar signal reflected by the golf ball according to a predetermined time interval of the golf ball's movement to calculate phase information, and can be configured to collect the azimuth information (including vertical elevation angle and horizontal direction angle information) of the moving golf ball as radar sensing data at predetermined time intervals.

[0073] Figure 4 (a) shows an embodiment of the present invention where the radar sensing unit of a radar-sensing ball trajectory visualization device 100 measures the left and right directional angles d1, d2, d3... of a moving golf ball at each measurement position B1, B2, B3...; Figure 4 (b) shows the radar sensor measuring the vertical elevation angles z1, z2, z3... of a moving golf ball at various measurement positions B1, B2, B3...

[0074] like Figure 4 As shown in (a) and (b), the radar sensing unit of the radar-sensing ball trajectory visualization device 100 according to an embodiment of the present invention can calculate and collect information on the left-right direction angle and the up-down elevation angle for each measurement position at the time point when the reflected wave signal of the radar signal is received from the moving golf ball.

[0075] That is, such as Figure 4 As shown in example (c), when user U moves by a golf ball being hit, the radar sensor can collect information on the golf ball's orientation (left and right azimuth angles and vertical elevation angles) at positions B1, B2, and B3, respectively, as radar sensing data.

[0076] However, the radar sensor collects the golf ball's orientation information for a moving golf ball as the angle information of the golf ball in the left-right direction in actual three-dimensional space (see reference). Figure 4 (a) and the angle information of the golf ball in the vertical direction (refer to) Figure 4 (b)), while the forward and backward orientation of a golf ball is difficult to confirm using radar sensor data.

[0077] Using the left-right azimuth angle and up-down elevation angle information of the golf ball obtained by radar as described above, the golf ball's orientation information at each measurement time point can be converted into coordinate information on the image. The converted coordinates are then connected on the golf shot image, which is a two-dimensional image captured by a single camera, to deduce the ball's trajectory itself.

[0078] However, as mentioned above, it is difficult to measure the position information of a golf ball in the forward and backward directions using radar sensing, such as where the golf ball is located in front of the device. Therefore, it is difficult to determine from which position the trajectory of a moving golf ball begins using radar sensing.

[0079] Figure 5 It shows that in such Figure 4 (c) shows an example of an image IM captured by a single camera of the ball trajectory visualization device 100 after user U makes a golf shot. Figure 5 In the accompanying diagram, iU represents the portion of the image corresponding to the user, and iBi represents the portion of the object corresponding to the golf ball placed in the initial position.

[0080] exist Figure 5 In the context of a game, when user iU hits the ball iBi at its initial position, if the ball's trajectory can be accurately calculated and displayed, it can be presented as the dotted iTrr trajectory. However, if radar sensor data of a moving golf ball is collected and the trajectory is calculated, it may be displayed as a different trajectory if the exact initial position of the ball cannot be determined. Figure 5 The iTra or iTrb trajectory shown demonstrates a significant difference between the iTra or iTrb trajectory and the actual ball trajectory iTrr.

[0081] An embodiment of the present invention describes a method for visualizing the trajectory of a ball in a user's golf shot image, and a radar-sensor-based ball trajectory visualization device thereof. The method is characterized by its ability to accurately identify the initial position of the ball from an image captured by a single camera, and to accurately display the ball trajectory calculated using radar sensor data, as described above, starting from and proceeding parallel to the identified initial position. This feature of the present invention will be further described with reference to... Figures 6 to 8 To provide a more specific explanation.

[0082] In one embodiment of the present invention, the sensing processing unit of the radar-sensing ball trajectory visualization device can set the ball sensing area using images acquired by a single camera to determine the position coordinates of a golf ball to be placed at any position on the floor where the user will make a golf shot.

[0083] The sensing processing unit can match the coordinates of elements in a region of a predetermined size on the image acquired by a single camera with the coordinates in the actual space and store the coordinate matching information, and set the region of the predetermined size on the image as the aforementioned sphere sensing region.

[0084] As an example of the method for obtaining the coordinates of the elements of the area whose size can be determined, a structure with known specifications can be used, such as a quadrilateral structure whose lengths of the four sides are known in detail.

[0085] Figure 6 (a) and (b) illustrate an embodiment of the present invention of a radar-sensing ball trajectory visualization device that uses a structure of known specifications to set the ball sensing area; Figure 6 (a) shows the state in actual space. Figure 6 (b) shows an image captured by a single camera in the state shown in (a).

[0086] Figure 6 (a) shows the state in which the quadrilateral structure 120, which is the aforementioned “structure with known specifications”, is positioned within the field of view of the camera.

[0087] Figure 6 The quadrilateral structure 120 shown in (a) is a structure in which the lengths of its four sides, namely L1, L2, L3, and L4, are known in advance. Figure 6 As shown in (b), the lengths L1, L2, L3, and L of the above sides correspond to the lengths l1, l2, l3, and l4 of the sides of quadrilateral i120 on image IM, respectively.

[0088] also, Figure 6 In the quadrilateral structure 120 shown in (a), the definite dimensional elements are four vertices E1, E2, E3, and E4, which are respectively consistent with... Figure 6 The elements defining the dimensions of quadrilateral i120 can be determined on the image shown in (b), namely, the four vertices e1, e2, e3, and e4.

[0089] If in Figure 6 If an xy coordinate system is defined on the image IM of (b) with position e1 as the origin, then the position coordinates of each vertex of e1, e2, e3, and e4 can be determined in the xy coordinate system. That is, based on the xy coordinates, the coordinates of e1 can be determined as (0,0), the coordinates of e2 can be determined as (0, L2), the coordinates of e3 can be determined as (l3 (=l1), l4 (=l2)), and the coordinates of e4 can be determined as (l1, 0).

[0090] The position coordinates of each vertex, e1, e2, e3, and e4, as determined above, can be compared with... Figure 6 The coordinates of each vertex of the structure 120 in the actual space shown in (a) are matched.

[0091] That is, in such Figure 6In the actual space shown in (a), in the XY coordinate system with E1 as the origin, the coordinates of E1 can be determined as (0, 0), the coordinates of E2 as (0, L2), the coordinates of E3 as (L3=L1, L4=L2), and the coordinates of E4 as (L1, 0). The coordinate matching information can be calculated by matching the position coordinates of E1, E2, E3, and E4 with the position coordinates of e1, e2, e3, and e4 on the image.

[0092] As mentioned above, the coordinate matching information, which matches the coordinates of the elements identified in the image with their corresponding coordinates in actual space, can be stored for later use and then used in the process of visualizing the trajectory of a user's golf shot.

[0093] The area defined by e1, e2, e3, and e4 on the image IM is the xy coordinate plane. Within this coordinate plane, regardless of location, the xy coordinates can be used to determine the position. Furthermore, the xy coordinates determined in this way can be converted into coordinate information in actual space using the aforementioned coordinate matching information.

[0094] In one embodiment of the present invention, the sensing processing unit of the radar-sensing ball trajectory visualization device can, as described above, set the area defined by e1, e2, e3, and e4 on the image IM as the "ball sensing area".

[0095] After setting the "ball sensing area" as described above, the structure can be removed, and the process of visualizing the ball trajectory corresponding to the user's golf shot can be initiated.

[0096] Figure 7 (a) and (b) show an embodiment of the present invention of a radar-sensing ball trajectory visualization device that calculates the initial position coordinates of the ball using a ball sensing area when the ball is placed on the floor by the user for making a golf shot. Figure 7 (a) shows the state of user U making a golf shot in actual space. Figure 7 (b) shows an image taken with a single camera in the state shown in (a).

[0097] like Figure 7 As shown in (a), when the ball trajectory visualization device 100 of the present invention is set up, and the user U places the golf ball Bi on the floor BT in order to perform a golf shot, Figure 7 As shown in (b), in the image of the scene, the ball sensing area SA determined by e1, e2, e3, and e4 as mentioned above has been pre-set. Therefore, the object corresponding to the golf ball, i.e., the ball object iBi, can be detected in the ball sensing area SA, and the position coordinate information of the ball object iBi on the ball sensing area SA can be calculated.

[0098] Thus, let the position coordinates of the sphere object iBi within the sphere sensing area SA on the image be the "initial position coordinates of the sphere".

[0099] By using the coordinate information related to the pre-set ball sensing area SA, the position coordinates of the ball object iBi, i.e. the initial position coordinates of the ball, can be calculated. Then, the coordinate matching information described above can be used to convert the initial position coordinates of the ball in the image into the coordinates of the golf ball in the actual space, i.e. the actual initial position coordinates of the golf ball Bi.

[0100] An embodiment of the ball trajectory visualization device of the present invention can use the initial position coordinates of the ball calculated from the image acquired by a single camera as described above as the reference point for subsequent ball trajectory display; and collect the sensing data of the radar sensor unit by using the actual initial position coordinates of the golf ball, which are transformed by using coordinate matching information, as the reference position, and use it to calculate the ball trajectory.

[0101] Figure 8 (a) shows in Figure 7 In the state shown in (a), the golf ball moves after the user hits it; Figure 8 (b) shows the footage taken with a single camera. Figure 8 The trajectory of the ball, calculated using radar sensing results, is displayed in the image of the actual scene shown in (a).

[0102] Figure 8 In (a), when user U hits a golf ball, at the point of impact, the ball trajectory visualization device 100 of an embodiment of the present invention generates a trigger signal; as the trigger signal is generated, the radar sensing unit can continuously collect radar sensing data for the moving golf ball from the point of impact.

[0103] Here, the aforementioned "radar sensor data" is as described above... Figure 3 and Figure 4 The description refers to the positional information (vertical elevation angle and left-right direction angle) of the golf ball at various measurement points during its movement.

[0104] In addition, the radar sensing unit can use the actual initial position coordinates Pbi of the golf ball, obtained by converting the initial position coordinates of the ball obtained through the ball sensing area in the image using coordinate matching information, as a reference position to collect the azimuth information of the golf ball.

[0105] That is, such as Figure 8As shown in (a), for a moving golf ball, the radar sensor can collect the golf ball's azimuth information (left-right azimuth angle, vertical elevation angle) at various radar measurement time points, such as (d1, z1), (d2, z2), (d3, z3), etc. In this case, the golf ball's azimuth information can be measured based on the golf ball's actual initial position coordinates Pbi.

[0106] In one embodiment of the present invention, the trajectory display processing unit of the ball trajectory visualization device can convert the orientation information of each golf ball collected by the radar sensing unit into coordinate information on the image IM, based on the ball's initial position coordinates pbi on the image IM.

[0107] That is, it can be like Figure 8 The orientation information of each golf ball shown in (a) ((d1, z1), (d2, z2), (d3, z3), etc.) is as follows: Figure 8 As shown in (b), the initial position coordinates pbi of the sphere are used as a reference to convert the coordinate information p1, p2, p3, etc. on the image IM. For example, (d1, z1) can be converted to the p1 coordinate on the image, (d2, z2) can be converted to the p2 coordinate on the image, and (d3, z3) can be converted to the p3 coordinate on the image.

[0108] The trajectory display processing unit connects the various ball position coordinates p1, p2, p3... calculated based on the ball's initial position coordinate pbi on the image IM, as described above, and can accurately display the ball trajectory iTri starting from the ball's initial position coordinate on the image IM.

[0109] By means of the method described above, a ball trajectory visualization device according to an embodiment of the present invention can achieve the following: during the process of setting the ball sensing area in the image captured by a single camera, it stores matching information with coordinates in the actual space, i.e., coordinate matching information; in the image of a user hitting a golf ball, it detects the ball object corresponding to the golf ball through the ball sensing area and calculates the initial position coordinates of the ball; as the user hits the golf ball, a trigger signal is generated, and from the trigger time point, for the moving golf ball, the radar sensing unit collects the golf ball's orientation information using radar sensing data; the trajectory display processing unit converts the collected golf ball's orientation information into coordinate information on the image, connects and displays the ball trajectory starting from the initial position coordinates of the ball, thereby realizing the visualization of the ball trajectory at an accurate position in the image.

[0110] However, when detecting the object corresponding to the golf ball and calculating the ball's initial position coordinates before the user makes a golf shot, the calculated initial position coordinates of the ball may differ from the actual initial position of the golf ball.

[0111] This is because a single camera captures a two-dimensional image from a single perspective, which limits its ability to know the actual position information in three-dimensional space. Furthermore, the ball sensing area can only calculate the position on a plane. Therefore, for example, when a golf ball is placed on a tee in actual space, the initial position coordinates of the ball calculated from the image may differ from the actual position.

[0112] Figure 9 Image (a) shows a golf ball placed on the floor, and related images of a tee inserted into the floor, with the golf ball placed on the tee. Figure 9 (b) shows the relationship between the line of sight of a single camera viewed from the side and the position of the golf ball.

[0113] like Figure 9 As shown in (a), in the image IM acquired by a single camera, the position of the sphere object iB1 detected within the sphere sensing area SA is as follows: Figure 9 As shown in (b), the position Pb1 of the actual golf ball B1 corresponds to the position of the ball object iB1. Therefore, the position coordinates of the ball object iB1 obtained by the ball sensing area SA on the image IM, that is, the initial position coordinates of the ball, are the coordinates that match the initial position coordinates Pb1 of the actual golf ball B1.

[0114] In this case, since the initial position coordinates of the sphere object iB1 match the actual position coordinates, the sphere trajectory calculated using radar sensing data is displayed as the trajectory starting from the initial position coordinates of the sphere in the image, thus achieving accurate visualization of the sphere trajectory.

[0115] However, as Figure 9 In the image IM acquired by the single camera shown in (a), the position of the sphere object iB2 detected within the sphere sensing area SA is as follows: Figure 9 As shown in (b), the position of the golf ball B2b placed on the tee should actually correspond to the position of the golf ball B2b in the line of sight observed by the single camera 200, but the golf ball B2b is identified as the position of the golf ball B2a placed on the plane.

[0116] Therefore, the initial position coordinates of the ball obtained by the ball sensing area SA for the ball object iB2 in the image are not based on the actual position of the golf ball B2b, i.e., Pb2b, but on the position of the golf ball B2a, which is located in a different position than the actual one, i.e., Pb2a. Therefore, it can be determined that there is an error in the initial position coordinates of the ball.

[0117] To address this situation, the present invention calculates the corrected initial position coordinates of the ball using flow calculation, and displays the ball's trajectory based on these corrected initial position coordinates. This will be discussed in more detail below. Figure 10 and Figure 11 Please provide an explanation.

[0118] Figure 10 (a) is a portion of an image taken with a golf ball placed on a tee. For the ball object iB2 on the image IM, an embodiment of the present invention provides a ball trajectory visualization device as follows: Figure 10 As shown in (b), the position B2a is identified as being placed on the floor. Therefore, the calculated initial position coordinates of the ball are also based on the position of the golf ball placed at position B2a, which is different from the actual position of the golf ball.

[0119] An embodiment of the ball trajectory visualization device of the present invention can use radar sensing data of the time point when the golf ball is hit to verify whether the initial position coordinates of the ball on the image are correct, and perform a process of correcting the initial position coordinates of the ball based on the verification result.

[0120] When a trigger signal is generated at the moment the golf ball is hit, the radar sensor can collect information about the golf ball's position, including its vertical elevation angle and horizontal azimuth angle in the actual space at the moment the trigger signal is generated, at the moment the radar signal reflected from the golf ball is received.

[0121] That is, such as Figure 10 As shown in (b), the location information of golf balls P1a, P2a, P3a, etc., is collected based on the position of golf ball B2a.

[0122] At this time, the trajectory display processing unit can use the golf ball's orientation information calculated by radar at the trigger time point or the ball coordinates at the trigger time point converted into coordinates on the image to verify the ball's initial position coordinates or the actual initial position coordinates of the golf ball based on them, so as to determine whether the ball's initial position coordinates are accurate ball position coordinates, and correct the ball's initial position coordinates according to the verification result.

[0123] In the verification process described above, for example, if it is determined that the coordinates of the golf ball's orientation information based on the trigger time point deviate too much from the ball's initial position coordinates, then it can be determined that the ball's initial position coordinates are incorrect.

[0124] Since the trigger point is the moment the golf ball begins to move after being struck by the golf club from a stationary state, the coordinates of the golf ball's position at the trigger point, converted into image coordinates, should be almost identical to or very close to the ball's initial position coordinates. However, if... Figure 10When the distance between the position P1a of the golf ball at the trigger time point sensed by radar and the position B2a of the ball's initial position coordinates (the position coordinates of the ball object calculated by the ball sensing area on the image) is more than a specified distance, it is difficult to assume that the golf ball started from the position B2a of the initially obtained ball position coordinates.

[0125] Therefore, the initial position coordinates of the ball can be corrected to more accurate position coordinates by using the position of the golf ball at the trigger time point sensed by radar.

[0126] For example, the height h of the single camera 200 of the ball trajectory visualization device 100 and the angle a of the line of sight from the single camera 200 to the golf ball at that height are known in advance. Therefore, these values ​​can be used to calculate the line AL of the line of sight from the single camera 200 to the golf ball at a height h, and the actual golf ball is on the line AL of the line of sight.

[0127] Here, considering the height of the golf ball's position P1a at the trigger time, the actual position of the golf ball can be found within a specified range RB on the line of sight AL below the height of the golf ball's position P1a.

[0128] When a golf ball is stationary, there is no Doppler shift in the radar, so its position cannot be determined by radar sensing data. The position of the golf ball can only be determined when the golf ball moves. Therefore, there is a certain difference between the position of the golf ball at the point when it is stationary and the position of the golf ball at the point of triggering (hereinafter referred to as "position difference"). By pre-storing the average value of this position difference or other statistical values, the actual position of the golf ball can be corrected to the accurate position.

[0129] For example, in Figure 10 In the case shown in (c), within the interval RB of the specified range on the line of sight AL, the position of the golf ball at the trigger time point P1a can be used to determine the position corresponding to the average position difference as described above as the accurate actual initial position of the golf ball.

[0130] Therefore, by converting the actual position of the golf ball obtained as described above into coordinates on the image, the converted coordinates can be calculated as the corrected coordinates of the ball's initial position coordinates.

[0131] Figure 11 It shows in Figure 10In the scenario shown, in an IM video captured by a single camera of a golf ball in flight as the user hits the ball, the trajectory of the ball is displayed based on the initially calculated initial position coordinates of the ball, and the trajectory of the ball is displayed after correcting the initial initial position coordinates of the ball to a more accurate position.

[0132] exist Figure 11 In the image shown, iB2 represents the initially placed spherical object.

[0133] exist Figure 11 In the image IM shown, the point marked pb2a corresponds to the initial position coordinates of the ball calculated using the ball sensing area, while the trajectory indicated by iTr1 is the ball trajectory calculated based on the radar sensing results of the golf ball using these coordinates as a reference.

[0134] like Figure 11 As shown, the initial coordinates of the sphere, i.e., the position pb2a, were obtained as described above. Figure 10 The description states that there are errors in the location when verifying using radar sensor data at the trigger time point. Therefore, the trajectory of the iTr1 ball in the image IM, starting from the initial ball position coordinates pb2a, differs slightly from the actual trajectory.

[0135] In this invention, the initial position coordinates of the ball are verified using radar sensing data as described above. If an error is detected, the error is corrected to calculate the accurate initial position coordinates of the ball, and the ball trajectory is displayed based on the corrected position coordinates.

[0136] exist Figure 11 In the process, the initial ball position coordinates pb2a obtained by using radar sensing data at the trigger time point are corrected to a more accurate position. The resulting position is the position indicated by the attached label pb2b. Therefore, the ball trajectory iTr2 starting from position pb2b is the accurate ball trajectory.

[0137] The present invention has the following features and advantages: By means of the method described above, in the golf shot-related images of a user acquired by a single camera, radar sensing data is used to verify and correct the initially obtained initial position coordinates of the ball, thereby obtaining more accurate initial position coordinates of the ball, and the accurate ball trajectory starting from the corrected position coordinates obtained in this way can be visualized on the image.

[0138] The preceding text described the structure and method of a radar-sensing-based ball trajectory visualization device according to an embodiment of the present invention, which uses radar sensing data to verify and correct the initially obtained initial position coordinates of the ball, thereby enabling the determination of more accurate initial position coordinates of the ball. In connection with this, another embodiment of the present invention provides a structure and method for a radar-sensing-based ball trajectory visualization device that can determine accurate initial position coordinates of the ball from the initial stage using pre-set information before sensing.

[0139] As mentioned earlier, since this invention uses a single camera to calculate the initial position coordinates of the ball, rather than based on the position coordinates of the golf ball in three-dimensional space, if the initial position coordinates of the ball are obtained by analyzing the image captured by the single camera when the user actually places the golf ball on the tee to hit the ball, an error equivalent to the height of the tee will occur. This invention corrects the error by using radar sensing data to verify the initially obtained initial position coordinates of the ball.

[0140] Furthermore, according to another embodiment of the present invention, the radar-sensing-based ball trajectory visualization device, when using a ball seat tee as described above, can provide the following functions: pre-setting the height of the ball seat, and calculating the initial position coordinates of the ball by considering the height on the image corresponding to the pre-set ball seat height, thereby enabling a more accurate calculation of the initial position coordinates of the ball from the initial stage.

[0141] Specifically, another embodiment of the radar-sensing ball trajectory visualization device of the present invention may have the following structure: built-in GPS to sense current location information, or using other sensors to sense current location information; and may be configured to: pre-store terrain information or map information of the golf course where the current user is playing golf, or pre-identify it by communicating with a server.

[0142] With this structure, another embodiment of the radar-sensing ball trajectory visualization device of the present invention can identify the location of the hole on a golf course where a golf game is currently in progress, and can also know whether the hole is a PAR3, PAR4 or PAR5 hole.

[0143] Before playing a golf game, users can use the radar-sensing ball trajectory visualization device of another embodiment of the present invention to pre-select the height of the tees to be used on each hole of the golf course (either by downloading the tee height information pre-stored by the user from the server for identification, or by the user directly inputting and pre-setting it on the device).

[0144] For example, users can preset the tee height for hole 2 (PAR3) to 1.5cm, hole 3 (PAR4) to 4.5cm, hole 4 (PAR5) to 4.5cm, and so on.

[0145] Therefore, the radar-sensing-based ball trajectory visualization device of another embodiment of the present invention can identify the position of the hole on a golf course where a golf game is currently in progress, identify the height of the tee to be used by the user at the current hole, and calculate the accurate initial position coordinates of the ball by taking into account the height on the image relative to the aforementioned preset and identified tee height when calculating the initial position coordinates of the ball from the image captured by a single camera.

[0146] The initial position coordinates of the ball calculated by considering the height of the ball seat can eliminate or significantly reduce the errors mentioned above. Even if the initial position coordinates of the ball are calculated by considering the height of the ball seat as described above, they can be further corrected by using radar sensor data for verification, thereby calculating a more accurate initial position coordinates of the ball.

[0147] When multiple users participate in a golf round together and share the ball trajectory visualization device of this invention, the invention can store the tee height information set by each user. When a user is scheduled to hit a golf ball, as the caddie or other user sets the user to hit the ball on a golf terminal, the invention can receive the user's hitting order information from the golf terminal and apply the user's pre-set tee height information. In this way, tee heights individually set for multiple users can be applied to the visualization of each individual golf shot image.

[0148] On the other hand, refer to Figure 12 and Figure 13 The flowcharts shown illustrate a method for visualizing the trajectory of a ball in a user's golf shot image according to an embodiment of the present invention.

[0149] Although the preceding text passed Figures 1 to 11 An embodiment of the present invention, utilizing a radar-sensing-based ball trajectory visualization device, describes in detail a method for visualizing the ball trajectory in a user's golf shot image. This will be explained in detail below. Figure 12 and Figure 13 The flowchart shown further organizes and explains the method for visualizing the ball trajectory.

[0150] Figure 12 The flowchart shown is connected to reference symbol A. Figure 13 The flowchart shown represents a process as a whole.

[0151] First, the radar-sensing ball trajectory visualization device of one embodiment of the present invention is configured and located at a predetermined position behind the floor where the user will make a golf shot (S100).

[0152] The sensing processing unit of the ball trajectory visualization device can match the coordinates of the elements in the image acquired by a single camera with the coordinates in the actual space and store the coordinate matching information, and set the area determined by the coordinates of the elements in the image with a predetermined size as the ball sensing area (S110).

[0153] The user places the golf ball anywhere on the floor within the field of view of the single camera and prepares to hit the ball. At this time, the single camera acquires an image, and the sensor processing unit searches for the pre-set ball sensing area in the image acquired by the single camera (S120).

[0154] At this time, if a ball object corresponding to a golf ball placed on the floor is detected based on the search results of the ball sensing area (S130), the sensing processing unit can use the coordinate information related to the sensing area to calculate the position coordinates of the detected ball object as the initial position coordinates of the ball (S140).

[0155] The sensor processing unit can use coordinate matching information to convert the initial position coordinates of the ball into the actual spatial coordinates of the golf ball, i.e., the actual initial position coordinates of the golf ball, and transmit them to the radar sensor unit (S150).

[0156] Then, when the user hits the golf ball with a golf club (S160), the ball trajectory visualization device first generates a trigger signal (S170), and the radar sensing unit operates according to the trigger signal.

[0157] The radar sensor sends radar signals to the golf ball that is moving due to being hit, and receives reflected wave signals reflected from the golf ball to collect golf ball orientation information including the vertical elevation angle and the horizontal direction angle of the golf ball at the measurement time point (S180).

[0158] The trajectory display processing unit of the ball trajectory visualization device can use the golf ball's orientation information calculated by radar at the trigger time point, or the ball coordinates at the trigger time point converted into coordinates on the image, to verify the ball's initial position coordinates or the actual initial position coordinates of the golf ball based on them (S200). For example, it can determine whether the initial position coordinates of the ball calculated in step S140 are accurate position coordinates.

[0159] When it is determined that the initial position coordinates of the ball calculated in the verification result step S140 as described above are correct and no correction is required (S210), the trajectory display processing unit can convert the golf ball orientation information collected by the radar sensor at each measurement time point into coordinates on the image and connect them based on the initial position coordinates of the ball, and display the trajectory of the ball starting from the initial position coordinates of the ball on the image (S220).

[0160] However, when it is determined that the initial position coordinates of the ball calculated in the verification result step S140 as described above are incorrect and need to be corrected (S210), the trajectory display processing unit can correct the initial position coordinates of the ball calculated in step S140 to accurate position coordinates based on the above verification result (S230).

[0161] Then, the trajectory display processing unit can convert the golf ball orientation information collected by the radar sensor at each measurement time point into coordinates on the image and connect them, based on the ball's initial position coordinates corrected in step S230 above, and display the ball trajectory starting from the ball's initial position coordinates corrected to a more accurate position on the image (S240).

[0162] Here, although not shown in the flowchart, the sensing processing unit of the present invention can pre-set the height of the ball seat that the user will use as described above. When calculating the initial position coordinates of the ball in the above step S140, the initial position coordinates of the ball can be calculated by applying the pre-set ball seat height information, thereby achieving accurate information calculation.

[0163] As described above, the method for visualizing the ball trajectory in a user's golf shot image and the radar-sensing-based ball trajectory visualization device of the present invention have the following features and advantages: Using a camera used to capture and provide images of a user's golf shot, the initial position of the golf ball placed on the floor by the user for the purpose of the shot is sensed, and the movement trajectory of the golf ball calculated using radar sensing results of the golf ball moving due to being hit is visualized as an accurate ball trajectory originating from the initial position of the golf ball in the image; furthermore, the radar sensing data can be used to verify whether the initial ball position coordinates calculated in the image are incorrect. If incorrect, they are corrected to more accurate initial ball position coordinates, and visualized as an accurate ball trajectory originating from the corrected position.

[0164] [Industry Applicability]

[0165] The method for visualizing the ball trajectory in a user's golf shot video and the radar-sensing-based ball trajectory visualization device of the present invention can be applied to golf-related industries such as sensing technologies for golf balls, golf practice, or screen golf systems.

Claims

1. A method for visualizing the trajectory of a ball in a user's golf shot video, wherein the ball trajectory is visualized using radar sensing results from images acquired by a camera capturing a user's golf shot, the method being characterized by comprising: The step of setting a ball sensing area for a golf ball sensor to acquire images using the camera on the floor where the user will make a golf shot; The steps include detecting a ball object corresponding to the golf ball in the ball sensing area on the image acquired by the camera, calculating the initial position coordinates of the ball, and calculating the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball; The step of collecting radar sensing data of the golf ball as it moves with the user's golf shot, based on the actual initial position coordinates of the golf ball; as well as The step of displaying the ball trajectory in the image acquired by the camera, starting from the initial position coordinates of the ball or the corrected coordinates of the initial position coordinates of the ball, using the collected radar sensing data for the golf ball.

2. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, In the step of setting the ball sensing area: The camera is a single-view camera. The steps include: matching the coordinates of elements that can determine a region of a predetermined size on the image acquired by the single camera with coordinates in actual space and storing the coordinate matching information, and setting the region of the predetermined size on the image as the ball sensing region.

3. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, In the step of setting the ball sensing area: The camera is a single-view camera. The method includes: matching the coordinates of elements of the structure's dimensions in an image taken from behind by the single camera from a perspective that includes a structure of known dimensions placed on the floor where the golf shot will be taken with coordinates in actual space and storing the coordinate matching information; and setting the area corresponding to the structure appearing in the image as the ball sensing area.

4. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 2 or 3, characterized in that, The steps for calculating the actual initial position coordinates of the golf ball include: The step of detecting a ball object corresponding to the golf ball within a defined ball sensing area on the image acquired by the single camera; The steps of calculating the position coordinates of the sphere object as the initial position coordinates of the sphere using coordinate information related to the sphere sensing area; and The step of converting the initial position coordinates of the ball into the actual initial position coordinates of the golf ball, which are the coordinates of the golf ball in actual space, using the coordinate matching information.

5. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 4, characterized in that, The height of the tee that the user will use when hitting a golf ball is preset. The steps for calculating the position coordinates of the spherical object as the initial position coordinates of the sphere include: The step of identifying the pre-set height of the ball seat; and The step of applying height information to the position coordinates of the ball object detected within the ball sensing area on the image corresponding to the height of the identified ball seat to calculate the initial position coordinates of the ball.

6. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, The steps for collecting radar sensing data on the moving golf ball include: The step of collecting golf ball orientation information, including the vertical elevation angle and horizontal azimuth angle of the golf ball in actual space at the time of receiving radar signals reflected from the golf ball, is as follows: The steps for displaying the ball's trajectory include: The steps involve converting the collected orientation information of each golf ball into coordinate information on the image acquired by the camera, based on the initial position coordinates of the ball, and connecting them to display the ball trajectory starting from the initial position coordinates of the ball.

7. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, The steps for displaying the ball's trajectory include: The step of verifying the initial position coordinates of the ball on the image obtained by the camera is to use radar sensing data at the time the golf ball is hit. Based on the verification results, the step of correcting the initial position coordinates of the ball; The steps include converting radar sensor data collected from a moving golf ball at the time of impact into coordinate information on an image acquired by the camera, and displaying the ball's trajectory on the image from the corrected initial position coordinates of the ball.

8. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, The steps for collecting radar sensing data on the moving golf ball include: The step of generating a trigger signal at the moment the golf ball is hit; and The step of collecting golf ball orientation information, including the vertical elevation angle and horizontal azimuth angle of the golf ball in actual space, from the time point of generation of the trigger signal, including the time point of reception of radar signals reflected from the moving golf ball. The steps for displaying the ball's trajectory include: The step of converting the golf ball's orientation information at the time of trigger signal generation into coordinate information on the image acquired by the camera, based on the ball's initial position coordinates, to generate ball coordinate information at the trigger time point; The step of correcting the initial position coordinates of the ball using the ball coordinates at the trigger time point; and The steps involve converting the golf ball's orientation information collected from the moving golf ball at the trigger time point into coordinate information on the camera's image based on the corrected initial position coordinates of the ball, and then connecting these coordinates to display the ball's trajectory starting from the corrected initial position coordinates.

9. The method for visualizing the trajectory of a ball in a user's golf shot image according to claim 1, characterized in that, The steps for collecting radar sensing data on the moving golf ball include: The step of generating a trigger signal at the moment the golf ball is hit; and The step of collecting golf ball orientation information, including the vertical elevation angle and horizontal azimuth angle of the golf ball in actual space, from the time point of generation of the trigger signal, including the time point of reception of radar signals reflected from the moving golf ball. The steps for displaying the ball's trajectory include: The step of calculating the three-dimensional position information of the golf ball at the time the trigger signal was generated, using the golf ball's orientation information at the time the trigger signal was generated and the actual initial position coordinates of the golf ball; The steps include: converting the calculated three-dimensional position information of the golf ball into coordinate information on the image acquired by the camera, and thereby calculating the corrected coordinate information of the ball's initial position coordinates; and... The steps involve converting the golf ball's orientation information collected from the moving golf ball at the trigger time point into coordinate information on the camera image based on the corrected coordinates, thereby displaying the ball's trajectory starting from the corrected coordinates.

10. A ball trajectory visualization device based on radar sensing, characterized in that, include: A single-view camera that captures images of the scene of the user hitting a golf ball on the floor from behind the floor where the user will be hitting the ball; The sensing processing unit is configured with a ball sensing area for sensing a golf ball using the image acquired by the single camera. It detects a ball object corresponding to the golf ball within the ball sensing area on the acquired image, calculates the initial position coordinates of the ball, and calculates the actual initial position coordinates of the golf ball corresponding to the initial position coordinates of the ball. The radar sensing unit collects radar sensing data of the golf ball as it moves with the user's swing, based on the actual initial position coordinates of the golf ball. as well as The trajectory display processing unit uses the collected radar sensing data for the golf ball to display the trajectory of the ball starting from the initial position coordinates of the ball in the image acquired from the camera or the corrected coordinates of the initial position coordinates of the ball.

11. The radar-sensing-based ball trajectory visualization device according to claim 10, characterized in that, The sensing processing unit is configured to match the coordinates of the elements of the structure's dimensions in an image taken from the rear by the single camera from a perspective that includes a structure of known dimensions placed on the floor where the golf shot will be taken with coordinates in actual space and store the coordinate matching information, and set the area corresponding to the structure appearing in the image as the ball sensing area. Furthermore, it is configured to detect a ball object corresponding to the golf ball within a set ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object using coordinate information related to the ball sensing area as the initial position coordinates of the ball, and use the coordinate matching information to convert the initial position coordinates of the ball into the actual initial position coordinates of the golf ball as the coordinates of the golf ball in actual space.

12. The radar-sensing-based ball trajectory visualization device according to claim 10, characterized in that, The sensing processing unit is configured to pre-set the height of the tee that the user will use when hitting a golf ball. The configuration is to match the coordinates of the elements of the structure's dimensions in an image taken from behind by the single camera from a perspective that includes a structure of known dimensions placed on the floor where the golf shot will be taken with coordinates in actual space and store the coordinate matching information, and to set the area corresponding to the structure appearing in the image as the ball sensing area. Furthermore, the system is configured to detect a ball object corresponding to the golf ball within a predetermined ball sensing area on the image acquired by the single camera, calculate the position coordinates of the ball object using coordinate information related to the ball sensing area, identify a predetermined tee height, apply the height information to the position coordinates of the ball object on the image corresponding to the identified tee height to calculate the initial position coordinates of the ball, and use the coordinate matching information to convert the initial position coordinates of the ball into the actual initial position coordinates of the golf ball as coordinates of the golf ball in actual space.

13. The radar-sensing-based ball trajectory visualization device according to claim 10, characterized in that, The radar sensor is configured to collect information on the golf ball's orientation in actual space, including the vertical elevation angle and horizontal azimuth angle of the golf ball at the time of receiving radar signals reflected from the golf ball. The trajectory display processing unit is configured to convert the orientation information of each golf ball collected by the radar sensor into coordinate information on the image acquired by the camera, based on the initial position coordinates of the ball, and then connect them to display the trajectory of the ball starting from the initial position coordinates of the ball.

14. The radar-sensing-based ball trajectory visualization device according to claim 10, characterized in that, The trajectory display processing unit is configured to: use radar sensing data at the time the golf ball is hit to verify the initial position coordinates of the ball on the image acquired by the camera; based on the verification result, correct the initial position coordinates of the ball; convert the radar sensing data collected by the radar sensing unit from the time the golf ball is hit for the moving golf ball into coordinate information on the image acquired by the camera; and display the trajectory of the ball starting from the corrected initial position coordinates of the ball on the image.

Images