Ar device and method for golf swing guidance

Abstract

The application discloses an AR device and method for golf guidance, which comprises an AR terminal display, a position sensor, a posture sensor, an image acquisition device, a processor and a golf course information database. The position sensor and the posture sensor have a relatively fixed position relationship and spatial relationship with the image acquisition device respectively, and acquire position information and posture information of the image acquisition device respectively. The image acquisition device acquires real-time image information. The processor is used for receiving data acquired by the position sensor, the posture sensor and the image acquisition device, obtaining the position of the golf ball, calling information in the golf course information database to obtain terrain data and the position of a target hole, generating initial movement conditions and a reference movement track of the golf ball, and then superimposing the above information on a real golf course and presenting the information on the AR terminal display. The application overcomes the problem that the hitting track of a player deviates from a target curve, and provides movement guidance in the training and daily movement of the golf ball.

Description

TECHNICAL FIELD

[0001] The present application relates to a sports guidance system in golf.

[0002] More specifically, the present application relates to presenting information for sports guidance and prediction of golf to players on an AR device through real-time augmented technology in golf. BACKGROUND

[0003] Golf is a healthy sport, playing golf can not only entertain the body and mind, but also enjoy the sunshine and fresh air, so golf has become a popular sport worldwide. The intelligent devices provided in the prior art for golf add convenience and fun to sports. The patent with application number 202111534187.X introduces a sports mode processing method and wearable device, which can display the movement path of the golf after being hit, which is beneficial for users to review the golf movement to improve their own stability. However, the swing action of golf seems simple, but it requires the coordination of various parts of the body. Within less than a second from swinging to hitting, the movement is very important to the movement trajectory of the golf ball. The hitting point and the direction of the hit directly affect the movement trajectory of the golf ball, and it is difficult for beginners to grasp these two points, so they often hit the ball away from the target line. SUMMARY

[0004] In view of the problems mentioned in the background art, the purpose of the present application is to provide an AR device and method for golf sports guidance to overcome the problem of deviation of the golfer's hitting trajectory from the target curve, and to provide sports guidance in golf training and daily sports.

[0005] The present application provides an AR device for golf sports guidance, comprising: an AR terminal display, a position sensor, a posture sensor, an image acquisition device, a processor, and a golf course information database.

[0006] The AR terminal display is used to present sports guidance information to the golfer.

[0007] The processor is used to receive the data obtained by the position sensor, the posture sensor and the image acquisition device, to retrieve the information in the golf course information database, and to further calculate the information required by the user.

[0008] The image acquisition device has a relatively fixed positional relationship with the position sensor, and has a relatively fixed spatial relationship with the attitude sensor; the position sensor acquires position information of the image acquisition device, the attitude sensor acquires attitude information of the image acquisition device, the image acquisition device acquires real-time image information, and a processor obtains a golf ball position after processing the acquired information; then a golf ball dynamics model is established, a numerical optimization problem is constructed with the initial motion condition of the golf ball as an independent variable and the distance deviation between the motion trajectory of the golf ball and a target hole as a target function, and the initial motion condition of the golf ball motion trajectory passing through the target point is obtained by solving, and a reference motion trajectory is generated.

[0009] The golf course information database collects the terrain of the golf course, and further, the terrain includes height information of the golf course, positions of the holes, positions and shapes of the obstacles and the putting green, and the height information is represented by a function z=h(x,y); further, the historical data of the golfer's hitting can also be included.

[0010] The application provides a method for guiding golf movement, specifically comprising:

[0011] In the first step, the method for obtaining the position of the golf ball is as follows:

[0012] a. establishing a three-dimensional golf course coordinate system with any position of the golf course as an origin, establishing a three-dimensional camera coordinate system with the optical center of the image acquisition device as an origin, and establishing a two-dimensional image coordinate system on the image obtained by the image acquisition device;

[0013] b. the position sensor acquires position data of the image acquisition device, and the position coordinates in the golf course coordinate system are marked as P cam ; the attitude sensor acquires attitude data of the image acquisition device; and the image acquisition device acquires an image;

[0014] c. obtaining two-dimensional coordinates of the golf ball in the image coordinate system based on a target detection method, and calculating a standard equation of a ray from the optical center of the image acquisition device to the golf ball in the golf course coordinate system;

[0015] d. calculating the position of the golf ball based on a ray method or a depth estimation method.

[0016] Preferably, in the step c, the image acquisition device transmits the captured real-time image to the processor for target detection, and when the golf ball is detected in the field of view of the image acquisition device, the next calculation is performed; the position of the golf ball in the image coordinate system is recorded as two-dimensional coordinates (u b ,v b );

[0017] Preferably, in step c, the transformation matrix from the image coordinate system to the camera coordinate system is obtained based on the intrinsic parameter matrix K of the image acquisition device; the transformation matrix from the camera coordinate system to the golf course coordinate system is obtained based on the pose data obtained in step b, denoted as R. Then, the direction vector from the optical center of the image acquisition device to the golf ball in the golf course coordinate system is... The standard equation for the ray from the optical center of the image acquisition device to the golf ball is:

[0018] Preferably, the object detection method is an object detection algorithm based on traditional computer vision or an object detection method based on deep learning; more preferably, the object detection method based on deep learning is a CNN model or a deep learning model based on Transformer; and even more preferably, the CNN model is a YOLO series model.

[0019] Preferably, in step d, the step of calculating the position of the golf ball using the ray method is as follows:

[0020] 1. Obtain the golf course terrain function z = h(x,y) from the golf course information database;

[0021] 2. Combine the topographic function z = h(x,y) and the ray equation. Solving the equation for δ, we obtain the intersection point of the ray and the ground, which is the position P of the golf ball. b .

[0022] Preferably, in step d, the step of calculating the position of the golf ball using the depth estimation method is as follows:

[0023] I. Using a depth estimation model, generate the depth map Depth(u,v) of the image;

[0024] II. Extract target coordinates (u b ,v b The depth value d at point (u) = Depth(u) b ,v b );

[0025] III. Substitute the depth value d into the standard equation of the ray to calculate the three-dimensional coordinates P of the golf ball in the course coordinate system. b .

[0026] The second step is to calculate the initial motion conditions and reference trajectory of the golf ball: Establish a golf ball dynamics model, constructing a numerical optimization problem with the initial motion conditions as the independent variable and minimizing the distance deviation between the golf ball's trajectory and the target hole as the objective function. Solve this problem to obtain the initial motion conditions that allow the golf ball's trajectory to pass through the target point, and generate the reference trajectory. The specific steps are as follows:

[0027] I. Constructing a numerical optimization problem:

[0028] When the golf ball is hit, its movement process includes flying until landing, and after landing, judging whether to bounce and roll according to its movement state and the terrain of the landing position. The dynamic constraint of the numerical optimization problem includes a flight dynamics model, a bounce dynamics model, and a rolling dynamics model. The independent variable is the initial movement condition. The path constraint is that the trajectory of the golf ball has no intersection with the obstacles. The objective function is to minimize the distance between the stopping point of the golf ball movement and the target hole.

[0029] Preferably, the initial movement condition includes the initial speed, launch angle, azimuth angle, and rotational angular velocity of the golf ball.

[0030] When the golf ball is hit and rolls, the dynamic constraint is the rolling dynamics model. The independent variable is the initial movement condition. The objective function is to minimize the distance between the movement trajectory and the target hole, and also needs to satisfy that the speed when the movement trajectory and the target hole take the minimum distance is less than a preset threshold.

[0031] Preferably, the preset threshold is a value in the interval of 0.01-0.2 m / s, and further preferably the preset threshold is 0.02 m / s, 0.03 m / s, 0.04 m / s, 0.05 m / s, 0.06 m / s, 0.07 m / s, 0.08 m / s, 0.09 m / s, 0.1 m / s.

[0032] Preferably, the golf ball dynamics model is a classical physics model: based on Newton's second law, considering the force conditions such as gravity, aerodynamic force, and Magnus force, a common differential dynamics equation is established to describe the flight of the golf ball in the air; considering the movement state of the golf ball when it falls on different terrains such as sand or grass, a bounce dynamics equation is established to describe the change of the velocity and angular velocity of the golf ball at the moment of landing bounce; considering the influence of gravity, green slope, and grass friction, a common differential dynamics equation is established to describe the rolling of the golf ball on the grass.

[0033] II. Solving the numerical optimization problem: the numerical optimization problem is solved by a numerical optimization algorithm. When the problem has a solution, the reference movement trajectory of the golf ball and its initial condition that satisfy the constraint condition are obtained; when the problem cannot converge, it means that the golf ball cannot be single-holed, then a way point is added to replace the target hole in the objective function, and the problem is solved again until the problem converges.

[0034] Preferably, the system recommends a way point according to the golfer's preference, and the golfer can also manually set the way point.

[0035] Thirdly, the AR terminal display displays the motion guidance information: the processor converts the reference motion trajectory into the camera coordinate system and superimposes it on the real golf course to present in the AR terminal display, and the AR terminal display also displays the initial motion conditions of the golf ball.

[0036] Preferably, the AR terminal display also displays the target hole distance, the golf course information, the hitting history information, the current golf course best score and other information.

[0037] Compared with the prior art, the application mainly has the following beneficial effects:

[0038] 1. Compared with the prior art of recording and displaying the motion trajectory of the golf ball, the application realizes the guidance of the action before the golfer hits the golf ball, calculates the best initial motion conditions of the golf ball, and generates the reference trajectory of the golf ball motion, thereby providing the golfer with real-time and interactive reference information, real-time dynamic visualization and interactive feedback, breaking through the static limitation of the traditional tool, helping the golfer to accurately master the state of the golf ball in the golf course and the golf course information near the golf ball.

[0039] 2. The AR device for golf motion guidance of the application can obtain the positioning of the golf ball in the golf course without installing a positioning device on the golf ball, and display the information of the golf ball and the golf course on the AR terminal display, and count the past hitting information. The device is simple and convenient to use. The AR device superimposes the golf ball trajectory prediction result calculated based on the dynamic equation into the real field in real time through the augmented reality technology, forms a dynamic visual reference route, and the athlete can directly see the hitting effect of different initial speeds and angles, realizes the training closed loop of "what you see is what you practice", and significantly improves the training efficiency.

[0040] 3. The golf ball positioning method provided by the application ingeniously obtains the real-time position information of the golf ball by using the image acquisition device and the pose sensor information, and has the advantages of simple device, low cost, high real-time performance and the like. Compared with the traditional positioning method, the real-time position information of the golf ball can be obtained without installing a positioning sensor in the golf ball, the efficient target detection algorithm and the pose of the image acquisition device are effectively combined based on the visual information, the pose information and the terrain information, and the need for modifying the golf ball is avoided; in the application scenario, the golf ball and the grass have distinct color contrast, the target detection algorithm has high accuracy and high calculation efficiency. The position information of the golf ball obtained by the method has real-time performance and accuracy, and the calculation speed and accuracy are not inferior to those of the traditional positioner.

[0041] 4. Existing golf training aids (e.g. swing trainers, putting mirrors, alignment sticks) mainly provide feedback through physical devices or offline data analysis, for example, putting aids only focus on the starting line or posture alignment, and cannot dynamically adjust the strategy during the shot. The AR glasses of the present application superimpose the golf ball trajectory prediction results calculated based on the dynamic equation into the real field of view in real time through augmented reality technology, forming a dynamic visual reference line, for example, displaying the trajectory correction curve under the influence of the slope of the fairway and wind speed. The player can directly see the effect of the shot with different initial speed and angle. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0043] Figure 1 The appearance of the AR glasses provided in Embodiment 1 of the present application;

[0044] Figure 2 The golf game guide method provided in Embodiment 2 of the present application;

[0045] Figure 3 The golf position calculation method provided in Embodiment 2 of the present application;

[0046] Figure 4 The numerical optimization problem solving process provided in Embodiment 2 of the present application;

[0047] Figure 5 The force condition and motion trajectory diagram of the golf ball in flight provided in Embodiment 2 of the present application;

[0048] Figure 6 When the numerical optimization problem provided in Embodiment 2 of the present application cannot converge, the schematic diagram of reconstructing the optimization problem by adding a way point;

[0049] Figure 7 The AR interactive display effect provided in Embodiment 2 of the present application. DETAILED DESCRIPTION

[0050] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application, and they do not constitute any limitation to the present application, but only serve as examples. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0051] Embodiment 1: Reference Figure 1 , AR glasses for golf game guidance, comprising:

[0052] a) frame 101: the frame serves as a main structure for mounting and fixing other components.

[0053] b) temple 102: a pair of temples are arranged on both sides of the frame for support and fixation when worn.

[0054] c) lens 103: two lenses are mounted on the frame for optical imaging and display functions.

[0055] d) camera 104: the camera is arranged at the connecting position between the two lenses for collecting image data.

[0056] e) processor 105 connected with the temple 102 through a data line to complete data interaction and calculation. The processor 105 can read the golf course terrain database from the cloud or a local memory card.

[0057] f) RTK GNSS positioning module:

[0058] 1) The RTK GNSS positioning module is a GNSS receiver moving in a dynamic environment, which receives satellite signals and differential data from a reference station, and calculates its coordinates in real time through a relative positioning algorithm. It is installed on the frame and has the characteristics of light weight and high precision;

[0059] 2) The positioning accuracy of the RTK GNSS positioning module in the horizontal and vertical directions is better than ±2cm;

[0060] 3) The positioning module is installed in the frame or the temple, and has a relative fixed position relationship with the camera, which is used to provide real-time high-precision position information.

[0061] g) 9-axis inertial sensor unit:

[0062] 1) The 9-axis inertial sensor unit is installed in the frame or the temple, and has a relative fixed spatial relationship with the camera, including the following sub-modules: a three-axis accelerometer for measuring acceleration data; a three-axis gyroscope for measuring angular velocity data; a three-axis magnetometer for measuring the direction of the geomagnetic field;

[0063] 2) Three-axis gyroscope and three-axis magnetometer work together to output high-precision attitude data of the camera, including pitch angle, roll angle and yaw angle, and quaternion;

[0064] 3) The data of three-axis accelerometer and RTK GNSS positioning module are processed through fusion algorithm, which is used to realize higher-precision dynamic positioning.

[0065] Example 2: Basic process of using AR glasses for golf game guidance Figure 2 The specific implementation steps are as follows:

[0066] Step 1, establish three coordinate systems: establish a right-handed course coordinate system O-XYZ with a certain tee point on the golf course as the origin, east as the X-axis, north as the Y-axis, and up as the Z-axis; establish a camera coordinate system o-xyz with the camera optical center on the AR glasses as the origin, the right direction as the positive direction of the x-axis, the forward direction as the positive direction of the y-axis, and the upward direction as the positive direction of the z-axis; establish an image coordinate system with the image center point as the origin, right as the positive direction of the u-axis, and up as the positive direction of the v-axis.

[0067] Step 2, obtain data: the camera obtains the image in front of the golfer's field of view, the RTK GNSS module obtains the position data, and the position data of the camera is obtained according to the relative position relationship between them and the accelerometer data in the 9-axis inertial measurement unit is fused to obtain higher positioning accuracy; the three-axis gyroscope and three-axis magnetometer in the 9-axis inertial measurement unit work together to obtain high-precision attitude information, and the 9-axis inertial measurement unit and the camera have a relatively fixed spatial attitude relationship when the golfer wears it, thereby obtaining high-precision attitude data of the camera. The foregoing obtained image, position data and attitude data are transmitted to the processor through the data line.

[0068] Step 3, target detection: the processor uses the improved YOLOv8 algorithm suitable for small target detection to process the image target detection, determines whether there is a golf ball in the image, and outputs the bounding box of the golf ball. The center of the rectangular bounding box is taken as the feature position of the golf ball in the image, and its two-dimensional coordinates in the image coordinate system are denoted as (u b ,v b ). The improved YOLOv8 algorithm can achieve a detection speed of 30 frames / s, and the detection accuracy of the golf ball small target reaches 82.5% AP, which can meet the real-time and accuracy requirements.

[0069] Step 4, calculate the ray equation from the camera optical center to the golf ball:

[0070] The camera intrinsic matrix can be obtained by the chessboard calibration method, and its expression is as follows:

[0071]

[0072] where f u ,f v are the equivalent focal length in u and v axis direction respectively (in pixel unit), reflecting the imaging scale of the camera; u0,v0are the principal point coordinates (projection of the optical center on the image plane), usually close to the image center; s is the skew parameter of the coordinate axis, ideally 0, indicating that the pixel rows and columns are strictly orthogonal.

[0073] The conversion matrix from image coordinate system to camera coordinate system is obtained according to the intrinsic matrix K of the camera, and the conversion matrix from the camera coordinate system to the ball court coordinate system is obtained according to the camera pose data, denoted as R, then the expression of the normalized direction vector from the camera optical center to the golf ball in the ball court coordinate system is The standard equation of the ray from the camera optical center to the golf ball is which represents an infinitely extended ray in the direction from the camera center point to the golf ball (refer to Figure 3 ). Where t is a time variable, P cam is the camera position, and δ is a variable representing the length of the ray.

[0074] Step 5, solving the golf ball position: the processor obtains the ball court terrain function z = h(x, y) from the ball court information database, which is a curved surface in three-dimensional space represented by X, Y coordinates; the terrain function z = h(x, y) and the ray equation are solved to obtain a one-variable equation about δ, and the intersection of the ray and the ground is obtained by solving, which is the three-dimensional position P b of the golf ball, with coordinate values represented by (X, Y, Z).

[0075] Based on the RTK GNSS positioning module, the camera position accuracy is in centimeter level, considering the attitude data error of the attitude sensor and the target detection error, the golf ball positioning accuracy can reach below 5 cm.

[0076] Step 6, establish a dynamic model: this embodiment gives a physical model, including a flight dynamics model, a bounce dynamics model and a rolling dynamics model.

[0077] (1) Flight dynamics model

[0078] Referring to Figure 5 , after the ball is hit and flies in the air, it is mainly affected by gravity air resistance and Magnus force due to rotation The flight dynamics model of the golf ball is established, specifically the position and angular velocity ordinary differential equation set:

[0079]

[0080] where, is the position of the golf ball, m is the mass of the golf ball, is the gravity vector, I is the moment of inertia of the golf ball, τ is the net torque experienced by the golf ball, ω is the angular velocity of the golf ball, t is the time variable. The magnitudes of the aerodynamic drag and the Magnus force are:

[0081]

[0082] A is the reference area of the golf ball, r is the radius of the golf ball, usually A = πr 2 , C d and C M are the drag coefficient and the Magnus force coefficient, respectively, which are related to air viscosity, ball surface interface, air density, flight speed and rotation speed, etc., and can be obtained by experiments; ρ is the atmospheric density, is the flight velocity vector.

[0083] (2) Bouncing dynamics model

[0084] The golf ball landing collision involves energy dissipation and momentum transfer. A feasible dynamics model is as follows: it is assumed that the tangential velocity component does not change after collision, and the change amplitude of the normal velocity component is described by the normal collision restitution coefficient, which is defined as follows:

[0085]

[0086] v n,before ,v n,after are the normal velocity magnitudes before and after collision, respectively, v t,before is the tangential velocity component before collision; a tangential friction model is defined, and the friction force causes the change in rotation rate to be

[0087]

[0088] μ is the grass friction coefficient. The normal collision restitution coefficient and the friction coefficient can be obtained by statistics. The dynamics equations of the velocity and rotation speed after collision are as follows:

[0089]

[0090] This dynamics model can predict the rolling distance and the second bounce trajectory of the golf ball after landing.

[0091] (3) Rolling dynamics model

[0092] A local coordinate system of the golf ball is defined, in which the x-axis points to the main slope direction (the maximum inclination direction) of the green, the y-axis points to the lateral slope direction, and the z-axis is perpendicular to the green surface and points upward.

[0093] The slope parameters of the green are:

[0094] • Main slope angle a: angle between x-axis and horizontal plane

[0095] • Transverse slope angle b: angle between y-axis and horizontal plane

[0096] • Compound slope vector:

[0097] Gravity decomposition:

[0098]

[0099] The motion of the ball is described by both translation and rotation. The coupled equations are established as follows:

[0100] The rolling resistance, sliding friction and slope effects are considered:

[0101]

[0102] Where:

[0103] • m: mass of the ball (standard ball 45.93 g)

[0104] • Rolling resistance F roll = m roll g z cos a cos b

[0105] • Sliding friction force v t is the tangential velocity component

[0106] • Rolling friction coefficient m roll (grass ≈ 0.05-0.15, affected by grass density and humidity)

[0107] • Moment of inertia of the ball Angular acceleration:

[0108] F fric,x ,F fric,y are the components of the sliding friction force in x, y directions, respectively x ,v y are the components of the velocity in x, y directions, respectively. When the pure rolling condition v = r w is satisfied, the rotation and translation are coupled.

[0109] The table below gives typical values in the dynamics equations, which can also be statistically obtained according to the situation of the green.

[0110] Table 1 Typical parameter values

[0111] Parameter Dry Hard Fruit Ridge Moist Soft Fruit Ridge Rolling friction coefficient μ roll ]] 0.06 0.12 coefficient of kinetic friction μ kinetic ]]> 0.18 0.25 Normal restitution coefficient e n ]]> 0.75 0.55

[0112] Step 7, construct a numerical optimization problem:

[0113] The state variable x(t) of the golf ball includes position p(t), velocity v(t) and angular velocity ω(t), i.e. x(t) = [p(t), v(t), ω(t)], where t represents the time variable, t = 0 is the time when the ball is hit, and t is the time when the golf ball stops moving f , then the value range of t is t ∈ [0, t f ]. p(t) is a curve in space, describing the trajectory of the golf ball.

[0114] The numerical optimization problem is to minimize the objective function

[0115] min J

[0116] subject to:

[0117] Dynamics constraints:

[0118] Path constraints: p(t) ∈ Ω

[0119] Initial position constraints: X(0) = X b , Y(0) = Y b , Z(0) = Z b

[0120] where Ω is the feasible spatial position of the golf ball, usually all areas within the golf course boundary except obstacles; f(x) represents an ordinary differential equation, g(x(t + ), x(t - )) = 0 is the dynamics constraint describing the state of the golf ball at the time points before and after landing and bouncing.

[0121] Determine whether the current position of the golf ball can reach the target hole only by rolling. If yes, construct the numerical optimization problem a; otherwise, construct the numerical optimization problem b.

[0122] Specifically, the numerical optimization problem a corresponds to the case when the golf ball is on the green and can reach the target hole by simply relying on rolling through a push. At this time, the objective function is to minimize the shortest distance between the trajectory and the target hole

[0123] J = ||p(t) - P h ||

[0124] The independent variables include the initial velocity and azimuth angle, and the dynamics constraint is the rolling dynamics model given in step 6.

[0125] For the numerical optimization problem a: the initial position is known, plus the independent variables to form the initial state, and the rolling dynamics model is integrated by using the Runge-Kutta 4th order integration until the velocity becomes 0, to obtain the discrete state variable sequence x(t = t i ) = [Pi V i ,ω i ], where i = 0, ..., N represents the discrete time point index, t f =t N P i Pi is the discrete-time representation of p(t). The continuous curve p(t) is obtained by cubic spline interpolation of Pi. The minimum distance between p(t) and the hole is the objective function.

[0126] Numerical optimization problem b, the objective function is to minimize the distance between the stopping point of the motion and the target hole.

[0127] J = ||p(t) f )-P h ||

[0128] Among them, P h These are the three-axis coordinates of the target hole, which are constants and therefore represented by uppercase letters, p(t). f ) is the time when the motion terminates, t = t f The position is a variable and is therefore represented by a lowercase letter; the independent variables include initial velocity, azimuth angle, launch angle and angular velocity. Depending on the type of club, there are also certain constraints on the initial velocity and angular velocity.

[0129] For numerical optimization problem b: the initial position is known, and the initial state is formed by adding the independent variable. The rolling dynamics model is integrated using the Runge-Kutta fourth-order integral until the ball lands. The landing criterion can be simplified to P. i and P i+1 They are located on opposite sides of the terrain surface z = h(x,y). After landing, the terrain at the landing point is assessed. If the landing site is in bushes, forests, a pond, a sandpit, etc., integration stops. If the landing site is on grass, the velocity and angular velocity changes after the bounce are calculated based on the bounce dynamics model. If the bounce speed drops below a certain threshold, the system transitions to rolling motion, and integration continues based on the rolling dynamics model until the velocity reaches zero. This yields a discrete sequence of state variables x(t = t). i )=[P i V i ,ω i ], through P i A continuous curve p(t) is obtained by performing cubic spline interpolation.

[0130] Step 8: Solve the numerical optimization problem.

[0131] A genetic algorithm is used to solve the numerical optimization problem. First, the objective function of the numerical optimization problem is transformed into a fitness function. Numerical solutions in the solution space are then mapped to chromosomes (individuals) in the genetic algorithm through encoding. The population consists of multiple individuals, initially generated randomly and covering multiple regions of the solution space.

[0132] During the iteration process of the algorithm, the population is gradually optimized through selection, crossover, and mutation operations. The selection operation filters out individuals with higher fitness according to the fitness value, making them have a higher probability of entering the next generation; the crossover operation generates new individuals by exchanging gene fragments of individuals, exploring the diversity of the solution space; the mutation operation avoids premature convergence to local optimal solutions by randomly changing the values of certain gene positions.

[0133] Through continuous iteration, the genetic algorithm gradually improves the average fitness of the population and finally converges to the global optimal solution or a region close to the global optimal solution. For this embodiment, a set of typical genetic algorithm parameter settings are as follows: the initial population size is 100, the maximum number of iterations is set to 50, the crossover probability is 0.6, the mutation probability is 0.01, and the convergence criterion is J < 0.1 or the fitness change rate is less than 1e-6.

[0134] Step 9, Figure 7 The reference trajectory and velocity of the push rod and their initial condition indication methods are illustrated.

[0135] The reference trajectory p(t) and initial condition are expressed in the court coordinate system. Through coordinate system conversion, the indication information such as the trajectory and initial condition can be converted to the camera coordinate system, and the AR glasses can render the trajectory and initial condition in real time into the field of view and superimposed on the real court. The coordinate conversion matrix is determined by the position and pose of the AR glasses, so the indication information can be transformed in the field of view as the golfer moves or turns, and the text data such as the hole distance and court information moves with the head rotation, but the position of the reference trajectory and the arrow representing the initial velocity in space remains unchanged. Even if the golfer turns his head and the golf ball moves out of the field of view, the trajectory curve can still be seen.

[0136] Embodiment 3:

[0137] Step 5 in Embodiment 2 can also use the depth estimation method to solve the golf ball position:

[0138] The position coordinates of the camera in the court coordinate system obtained by the RTK GNSS positioning module are denoted as P cam = (X c , Y c , Z c ), and the ray equation from the camera coordinate system origin (camera optical center) to the golf ball is The distance from the camera optical center to the golf ball obtained based on the depth estimation method is denoted as d. Let t = d, and the position coordinates of the golf ball can be obtained as

[0139]

Claims

1. A method for golf instruction, characterized by, The method comprises the following steps: S1. Obtain the position information of the golf ball; S2. Calculate the initial motion condition and the reference motion trajectory of the golf ball; S3. The AR terminal display displays the motion guidance information; Step S1 comprises the following steps: a. Establish a three-dimensional course coordinate system with any position on the golf course as the origin, a three-dimensional camera coordinate system with the optical center of the image acquisition device as the origin, and a two-dimensional image coordinate system with any point on the image obtained by the image acquisition device as the origin; b. The position sensor acquires position data of the image acquisition device, and the position coordinates in the court coordinate system are denoted as ; the attitude sensor acquires attitude data of the image acquisition device; and the image acquisition device acquires real-time image information; c. Obtain the two-dimensional coordinates of the golf ball in the image coordinate system based on the target detection method, and calculate the standard equation of the ray from the optical center of the image acquisition device to the golf ball in the course coordinate system; d. Calculate the position of the golf ball based on the ray method or the depth estimation method; Step S2 comprises the following steps: Establish a golf ball dynamics model, take the initial motion condition of the golf ball as the independent variable, construct a numerical optimization problem with the distance deviation of the golf ball motion trajectory from the target hole as the objective function, and solve the initial motion condition that makes the golf ball motion trajectory pass through the target point, and generate a reference motion trajectory; In step c, the image acquisition device transmits the captured real-time image to the processor for target detection, and when a golf ball is detected in the field of view of the image acquisition device, the next step is calculated; the position of the golf ball in the image coordinate system is recorded as a two-dimensional coordinate ​ In step c, a conversion matrix from image coordinate system coordinates to camera coordinate system is obtained according to the intrinsic matrix K of the image acquisition device; a conversion matrix from the camera coordinate system to the ball field coordinate system is obtained according to the attitude data obtained in step b, denoted as R, and the direction vector from the optical center of the image acquisition device to the golf ball in the ball field coordinate system is The standard equation of the ray from the optical center of the image acquisition device to the golf ball is Wherein is the length of the ray.

2. A method for golfing guidance according to claim 1, wherein, In step d, the step of calculating the position of the golf ball by the ray method is: 1) Obtain the course terrain function from the course information database , is the three-axis coordinate of a point on the course, represents the height function; 2) Incorporate the terrain function and the ray equation Solve the equation for to find the intersection of the ray with the ground, which is the location of the golf ball .

3. A method for golfing guidance according to claim 1, wherein, In step d, the step of calculating the position of the golf ball by the depth estimation method is: I. Generating a depth map of an image using a depth estimation model ; II. Extracting target coordinates depth value at the location ; III. The depth value is substituted into the standard equation of the ray to calculate the three-dimensional coordinates of the golf ball in the course coordinate system .​ 4. A method for golfing guidance according to any one of claims 1-3, characterized in that, The target detection method is a traditional computer vision-based target detection algorithm or a deep learning-based target detection method; The initial motion condition in step S2 includes initial velocity, launch angle, azimuth angle, and rotational angular velocity; In step S3, the processor converts the reference motion trajectory to the camera coordinate system and superimposes it on the real course to present it in the AR terminal display, and the AR terminal display also displays the initial motion condition of the golf ball; The AR terminal display also displays the target hole distance, course information, actual golf ball motion trajectory, hitting history information, and current course best score.

5. A method for golfing guidance according to claim 4, wherein, The deep learning-based target detection method is a CNN model or a deep learning model based on Transformer.

6. A method for golfing according to claim 5, wherein, The CNN model is a YOLO series model.

7. An AR device for golf instruction using the method of any of claims 1-6. It comprises an AR terminal display, a position sensor, an attitude sensor, an image acquisition device, a processor, and a course information database; The AR terminal display is used to present motion guidance information to the golfer; The course information database collects the terrain of the golf course; The processor is used for receiving data acquired by the position sensor, the attitude sensor and the image acquisition device, calling information in the golf course information database, and calculating information required by the user; the image acquisition device has a relatively fixed positional relationship with the position sensor, and has a relatively fixed spatial relationship with the attitude sensor; the position sensor acquires position information of the image acquisition device, the attitude sensor acquires attitude information of the image acquisition device, the image acquisition device acquires real-time image information, the processor obtains the golf ball position after processing the acquired information, and establishes a golf ball dynamics model, so as to construct a numerical optimization problem with the initial motion condition of the golf ball as an independent variable and the distance deviation between the motion trajectory of the golf ball and the target hole as a target function, and to obtain the initial motion condition of the golf ball motion trajectory passing through the target point by solving, and to generate a reference motion trajectory. 8.The AR device for golf guidance according to claim 7, wherein, The terrain of the golf course includes height information of the golf course, positions of the ball hole, positions and shapes of the obstacles and the green, the height information is represented by a function , is a three-axis coordinate of a point on the course, represents the height function.

9. The AR device for golf guidance according to any one of claims 7-8, wherein, The golf course information database further includes historical data of a golfer hitting a ball.

Images