Ball game trajectory tracking and prediction system based on multi-sensor perception

Abstract

The application discloses a ball trajectory tracking and prediction system based on multi-sensing, relates to the technical field of sports data analysis, and comprises a data acquisition module, a data synchronization module, a fusion tracking module, a trajectory prediction module and a visual output module; ball movement and environment data are collected through a multi-sensor, the three-dimensional position, speed and angular velocity of the ball are calculated after coordinate unification and time synchronization, a continuous tracking trajectory is obtained through combination and verification, the future ball movement trajectory is obtained based on the deduction of a dynamic equation and in combination with the boundary constraint of a field, and finally, three-dimensional visual rendering is performed; the application realizes complete tracking of the movement state of the ball through multi-sensing, performs trajectory deduction in combination with environment data and a dynamic equation, and improves the accuracy of prediction.

Description

Technical Field

[0001] This invention relates to the field of sports data analysis technology, specifically a ball movement trajectory tracking and prediction system based on multi-sensor perception. Background Technology

[0002] The technology of tracking and predicting ball motion trajectories has important application value in sports science and technology. Accurately obtaining the three-dimensional position, speed and rotation state of the ball during the motion process, and reliably predicting the future flight trajectory, is the key to improving the level of sports performance analysis.

[0003] Existing ball motion trajectory tracking technologies mostly use single-type sensors, which can only acquire partial motion information of the ball and cannot simultaneously obtain three-dimensional position, velocity, and rotation status, resulting in an incomplete description of the ball's motion state. In terms of trajectory prediction, existing methods do not fully utilize environmental factors, leading to deviations between prediction results and actual motion. How to achieve complete tracking of the ball's motion state through multi-sensor methods and improve the accuracy of trajectory prediction by combining environmental factors is a problem we need to solve. To this end, we now provide a ball motion trajectory tracking and prediction system based on multi-sensor methods. Summary of the Invention

[0004] The purpose of this invention is to provide a ball motion trajectory tracking and prediction system based on multi-sensor perception.

[0005] The objective of this invention can be achieved through the following technical solution: a ball motion trajectory tracking and prediction system based on multi-sensor perception, comprising:

[0006] The data acquisition module deploys multiple sensors to collect data on the sphere's motion and the environment. It then timestamps the collected data to obtain multi-source data with timestamps.

[0007] The data synchronization module converts multi-source data with timestamps to the same target area coordinate system and performs time synchronization to obtain standardized multi-source data.

[0008] The fusion tracking module calculates the sphere's three-dimensional position based on standardized multi-source data, obtains the sphere's three-dimensional velocity vector and three-dimensional angular velocity vector, verifies them with environmental data, and combines them into a continuous tracking trajectory.

[0009] The trajectory prediction module is used to extract the ball's motion state and environmental data, perform iterative deduction based on the dynamic equation, and combine the field boundary constraints to obtain the future trajectory of the ball.

[0010] The visualization output module is used to perform three-dimensional visualization rendering of the continuous tracking trajectory and the future trajectory of the sphere, annotate key events, and output physical quantity values.

[0011] Preferably, the data acquisition module deploys multiple sensors to collect sphere motion data and environmental data, and the process of unifying the timestamps of the collected data includes:

[0012] The range of the ball's motion is taken as the target area. Multiple sensors are deployed, a sampling period is set, and the multiple sensors are triggered so that they start collecting ball motion data and environmental data at the same starting moment.

[0013] Each piece of collected data is appended with a corresponding timestamp to obtain multi-source data with timestamps.

[0014] Preferably, the data synchronization module converts multi-source data with timestamps to the same target area coordinate system and performs time synchronization, including the following process:

[0015] Each sensor is calibrated, and the measurement results are uniformly converted to the target area coordinate system;

[0016] By setting a unified calculation time sequence, the data from each sensor at different sampling times are interpolated and aligned so that all sensors have corresponding data at each calculation time, thus obtaining standardized multi-source data.

[0017] Preferably, the process by which the fusion tracking module calculates the sphere's three-dimensional position and obtains the sphere's three-dimensional velocity vector and three-dimensional angular velocity vector includes:

[0018] Sphere target detection is performed on images captured by the camera, and the three-dimensional position of the sphere in the target area coordinate system is calculated by triangulation based on the multi-view detection results.

[0019] The echo signal data acquired by the Doppler radar is processed to extract the Doppler frequency shift and calculate the radial velocity. The three-dimensional velocity vector is obtained by combining the three-dimensional position change.

[0020] Micro-Doppler feature analysis was performed on the echo signal data to extract the sideband frequency shift and obtain the three-dimensional angular velocity vector.

[0021] Preferably, the process of obtaining a continuous tracking trajectory includes:

[0022] The three-dimensional position, three-dimensional velocity vector, and three-dimensional angular velocity vector at the same moment are combined to form the state vector at that moment;

[0023] Obtain the wind speed vector and air density at the current moment, calculate the difference between the theoretical resultant acceleration and the actual acceleration, and correct the combined result when the difference exceeds a preset threshold;

[0024] By combining and verifying the data in chronological order, a continuous tracking trajectory can be obtained.

[0025] Preferably, the trajectory prediction module extracts the current sphere motion state and environmental data, and performs iterative deduction based on the dynamic equations, including:

[0026] The current three-dimensional position, three-dimensional velocity vector, and three-dimensional angular velocity vector are obtained from the continuously tracked trajectory and used as the initial state for the deduction.

[0027] Obtain the wind speed vector and air density at the current moment, set the interval between adjacent moments, and perform a time-by-time deduction starting from the initial state. At each moment, calculate the resultant acceleration based on the forces acting on the sphere, and calculate the three-dimensional velocity vector and three-dimensional position at the next moment from the resultant acceleration, three-dimensional velocity vector, and three-dimensional position.

[0028] Preferably, the process by which the trajectory prediction module, in conjunction with site boundary constraints, obtains the future trajectory of the sphere includes:

[0029] Obtain the spatial location of the site boundary using the target area coordinate system;

[0030] Based on the predicted three-dimensional position of the sphere, the position is compared with the boundary of the field. When the boundary is touched, the corresponding event is determined and the iterative deduction is terminated. If the boundary is not touched, the next calculation is continued.

[0031] The generated position points are connected in chronological order to form the future trajectory of the sphere.

[0032] Preferably, the visualization output module performs 3D visualization rendering of the continuously tracked trajectory and the future trajectory of the sphere, annotates key events, and outputs physical quantity values, including:

[0033] Acquire the continuous tracking trajectory and the future trajectory of the sphere, construct a virtual 3D scene in the target area coordinate system, render and display the two trajectories with different colors, and annotate key events and display the corresponding physical quantity values.

[0034] Compared with existing technologies, the beneficial effects of this invention are as follows: by collecting ball motion data and environmental data through multiple sensors, and calculating the three-dimensional position, velocity, and angular velocity of the ball after coordinate unification and time synchronization, the continuous tracking trajectory is obtained through combination verification, realizing complete tracking of the ball's motion state and making up for the lack of information dimension of a single sensor; on this basis, environmental data and dynamic equations are introduced into the trajectory extrapolation process, and the future motion trajectory of the ball is predicted by combining the field boundary constraints, making the predicted trajectory closer to the real motion situation and improving the accuracy of the prediction. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0036] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0037] like Figure 1 As shown, a ball motion trajectory tracking and prediction system based on multi-sensor perception includes:

[0038] The data acquisition module deploys multiple sensors to collect data on the sphere's motion and the environment. It then timestamps the collected data to obtain multi-source data with timestamps.

[0039] The data synchronization module converts multi-source data with timestamps to the same target area coordinate system and performs time synchronization to obtain standardized multi-source data.

[0040] The fusion tracking module calculates the sphere's three-dimensional position based on standardized multi-source data, obtains the sphere's three-dimensional velocity vector and three-dimensional angular velocity vector, verifies them with environmental data, and combines them into a continuous tracking trajectory.

[0041] The trajectory prediction module is used to extract the ball's motion state and environmental data, perform iterative deduction based on the dynamic equation, and combine the field boundary constraints to obtain the future trajectory of the ball.

[0042] The visualization output module is used to perform three-dimensional visualization rendering of the continuous tracking trajectory and the future trajectory of the sphere, annotate key events, and output physical quantity values.

[0043] Furthermore, the process of deploying multiple sensors to collect data on the sphere's motion and the environment includes:

[0044] The area of ​​the ball game field is taken as the target area. Multi-sensor sensors are deployed at pre-marked locations in the target area, a uniform sampling period is set, and the multi-sensor sensors are triggered so that all sensors start collecting ball motion data and environmental data at the same start time.

[0045] It should be further explained that the multi-sensor system includes cameras, Doppler radar, anemometers, barometers, thermometers, and hygrometers; the sphere motion data includes multi-view image sequences collected by multiple cameras and electromagnetic wave echo signals collected by Doppler radar; the environmental data includes wind speed and direction collected by anemometers, air pressure values ​​collected by barometers, temperature values ​​collected by thermometers, and humidity values ​​collected by hygrometers.

[0046] The cameras are numbered, denoted as i, where i = 1, 2, ..., N, and N is the number of cameras, N ≥ 2; the image captured by the i-th camera at time t is denoted as... The echo signal collected by the Doppler radar at time t is denoted as The values ​​collected by the anemometer, barometer, thermometer, and hygrometer at time t are respectively , , , ;

[0047] The pre-marked locations are used to deploy multiple sensors, including cameras that are evenly distributed around the target area with each camera lens facing the center of the target area; a Doppler radar is deployed on one side of the target area with the radar emitting surface facing the center of the target area, covering the entire target area; and anemometers, barometers, thermometers and hygrometers are concentrated in an open area without obstructions on one side of the target area.

[0048] Furthermore, the process of unifying the timestamps of the collected data includes:

[0049] Each piece of collected data is appended with a corresponding timestamp, which contains information about the time of collection.

[0050] The data acquired by the data acquisition module at time t, after being marked with a timestamp, is denoted as [data type 1]. , is represented as:

[0051] ,

[0052] Obtain multi-source data with timestamps.

[0053] Furthermore, the process by which the data synchronization module uniformly transforms timestamped multi-source data to the same target region coordinate system includes:

[0054] Perform intrinsic parameter calibration for each camera to obtain the intrinsic parameter matrix of the i-th camera. Perform extrinsic parameter calibration on each camera to obtain the rotation matrix of the i-th camera relative to the target area coordinate system. Translation vector The image captured by the i-th camera at time t pixel coordinates Transform to three-dimensional coordinates in the target region coordinate system The transformation relationship is as follows:

[0055] ,

[0056] in, The distance from the camera to the sphere;

[0057] External parameter calibration of the Doppler radar is performed to obtain the transformation relationship between the radar coordinate system and the target area coordinate system, and the radar echo signal data is then processed. The corresponding measurement direction is converted to the target area coordinate system; the anemometer is calibrated for external parameters, and its wind direction measurement direction is converted to the target area coordinate system;

[0058] Furthermore, the time synchronization process of the data synchronization module includes:

[0059] Get data with timestamps Set a unified calculation time sequence , , , ...;

[0060] For each calculation time From each sensor The sensor's performance is calculated by linear interpolation of the data values ​​collected at two consecutive actual sampling times. The equivalent data value at each moment allows all sensors to perform calculations at every moment. Each has corresponding data, resulting in standardized multi-source data.

[0061] Furthermore, the process by which the fusion tracking module calculates the sphere's three-dimensional position includes:

[0062] The standardized multi-source data output by the data synchronization module is used to obtain images captured by each camera at time t. Perform sphere target detection and obtain the pixel coordinates of the sphere in the i-th camera image. Select the detection results of the sphere from at least two cameras at the same time, and use the intrinsic parameter matrix of each camera. Rotation matrix Translation vector The three-dimensional position of the sphere in the target area coordinate system is calculated using the principle of triangulation. , is represented as:

[0063] ,

[0064] Furthermore, the process of obtaining the sphere's three-dimensional velocity vector and three-dimensional angular velocity vector includes:

[0065] Echo signal data acquired by Doppler radar at time t Perform Fourier transform to extract the main Doppler frequency shift. Calculate the radial velocity of the sphere. :

[0066] ,

[0067] Where c is the speed of electromagnetic wave propagation. The radar transmission frequency; combined with the three-dimensional position of the sphere at multiple consecutive moments. The change in velocity vector of the sphere in the target region coordinate system is calculated. , is represented as:

[0068] ,

[0069] For echo signal data Micro-Doppler feature analysis was performed to extract the sideband frequency shift caused by the sphere's rotation, and the three-dimensional angular velocity vector of the sphere in the target region coordinate system was calculated. , is represented as:

[0070] ,

[0071] Furthermore, the process of obtaining a continuous tracking trajectory includes:

[0072] The three-dimensional position at the same moment Three-dimensional velocity vector And the three-dimensional angular velocity vector, combined to form the state vector at that moment. , is represented as:

[0073] ,

[0074] The combined results are verified by combining environmental data. The wind speed vector and air density at the current moment are obtained. The theoretical resultant acceleration of the sphere at the current moment is calculated from the wind speed vector, air density, current three-dimensional velocity vector, and three-dimensional angular velocity vector. The actual acceleration is obtained from the difference between the three-dimensional velocity vectors of two adjacent moments and the time interval. The theoretical resultant acceleration is compared with the actual acceleration. If the deviation between the two exceeds a preset threshold, the three-dimensional velocity vector and three-dimensional position at the current moment are corrected with the theoretical resultant acceleration. The combination and verification are performed step by step in time sequence. The corrected state vectors at each moment together constitute the continuous tracking trajectory of the sphere.

[0075] It should be further explained that, taking a certain sphere as an example, in At time t, the sphere's three-dimensional velocity vector is (20, 3, 1), with units of meters per second. The radar measured a three-dimensional velocity vector of (19.2, 3.1, 0.8). The actual acceleration was calculated to be 8.5 m / s² based on the velocity changes between adjacent moments. According to the current wind speed vector and air density, combined with the sphere's three-dimensional velocity vector, three-dimensional angular velocity vector, air drag coefficient, and Magnus force coefficient, the theoretical resultant acceleration was calculated to be 9.3 m / s². The deviation between these two values ​​was 0.8 m / s², exceeding the preset threshold of 0.5 m / s². Therefore, the three-dimensional velocity vector at moment t2 was recalculated using the theoretical resultant acceleration, replacing the radar measurement, thus correcting the current state vector.

[0076] Furthermore, the process of extracting the sphere's motion state and environmental data, and performing iterative deduction based on the dynamic equations, includes:

[0077] Obtain the current moment from the continuously tracked trajectory. The state vector of the sphere Extract the three-dimensional position from it. Three-dimensional velocity vector and three-dimensional angular velocity vector , as the initial state for the deduction;

[0078] The system acquires environmental data, extracts the wind speed and direction collected by the anemometer at the current moment, obtains the wind speed vector after coordinate transformation, and calculates the air density based on the values ​​collected by the barometer, thermometer and hygrometer.

[0079] Set the interval between adjacent time steps in the iterative simulation From the initial moment We begin by tracing the events in chronological order.

[0080] The trajectory of the sphere is affected by the resultant acceleration, which is calculated from the sphere's gravity, air resistance, Magnus force, wind force, and the sphere's mass. At time k, the resultant acceleration at that time is... Calculate the three-dimensional velocity vector and three-dimensional position at time k+1.

[0081] ,

[0082] ,

[0083] After each calculation is completed, the three-dimensional position and three-dimensional velocity vectors obtained at that time are used as the starting state for the calculation at the next time, and the calculation at the next time continues.

[0084] Furthermore, the process of obtaining the future trajectory of the sphere by combining the constraints of the site boundaries includes:

[0085] Using the established target area coordinate system, obtain the longitudinal coordinates of the ground plane, the spatial position of the net plane, and the spatial position of the target area's edge lines; thus obtaining the three-dimensional position of the sphere. back,

[0086] Compare this location with the site boundary constraints;

[0087] When the vertical coordinate of the position is lower than or equal to the vertical coordinate of the ground plane, the ball is determined to have landed, the current position is taken as the landing point, and the iterative deduction is terminated; when the position crosses the net plane and is within the net area, the ball is determined to have touched the net, and the iterative deduction is terminated; when the horizontal projection of the position exceeds the boundary of the target area, the ball is determined to have gone out of bounds, and the iterative deduction is terminated; if no boundary interference occurs, the calculation of the next moment continues.

[0088] Connect all the position points generated from the initial moment to the final moment during the iterative simulation in chronological order to form the future trajectory of the sphere.

[0089] Furthermore, the process of visualizing the tracking trajectory and the predicted trajectory includes:

[0090] The system acquires the continuous tracking trajectory output by the fusion tracking module and the future sphere motion trajectory output by the trajectory prediction module; it constructs a virtual 3D scene in the target area coordinate system, renders the continuous tracking trajectory and the future sphere motion trajectory with different colors, and displays them overlaid; it marks the landing point, touch point, and out-of-bounds point on the future sphere motion trajectory, and displays the magnitude of the 3D velocity vector and the 3D angular velocity vector at each moment on the trajectory in numerical form.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications or equivalent substitutions made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A ball motion trajectory tracking and prediction system based on multi-sensor perception, characterized in that, include: The data acquisition module deploys multiple sensors to collect data on the sphere's motion and the environment. It then timestamps the collected data to obtain multi-source data with timestamps. The data synchronization module converts multi-source data with timestamps to the same target area coordinate system and performs time synchronization to obtain standardized multi-source data. The fusion tracking module calculates the sphere's three-dimensional position based on standardized multi-source data, obtains the sphere's three-dimensional velocity vector and three-dimensional angular velocity vector, verifies them with environmental data, and combines them into a continuous tracking trajectory. The trajectory prediction module is used to extract the ball's motion state and environmental data, perform iterative deduction based on the dynamic equation, and combine the field boundary constraints to obtain the future trajectory of the ball. The visualization output module is used to perform three-dimensional visualization rendering of the continuous tracking trajectory and the future trajectory of the sphere, annotate key events, and output physical quantity values.

2. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 1, characterized in that, The data acquisition module deploys multiple sensors to collect data on the sphere's motion and the environment, and the process of unifying the timestamps of the collected data includes: The range of the ball's motion is taken as the target area. Multiple sensors are deployed, a sampling period is set, and the multiple sensors are triggered so that they start collecting ball motion data and environmental data at the same starting moment. Each piece of collected data is appended with a corresponding timestamp to obtain multi-source data with timestamps.

3. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 2, characterized in that, The data synchronization module transforms multi-source data with timestamps into the same target area coordinate system and performs time synchronization. The process includes: Each sensor is calibrated, and the measurement results are uniformly converted to the target area coordinate system; By setting a unified calculation time sequence, the data from each sensor at different sampling times are interpolated and aligned so that all sensors have corresponding data at each calculation time, thus obtaining standardized multi-source data.

4. The ball motion trajectory tracking and prediction system based on multiple sensing according to claim 3, characterized in that, The process by which the fusion tracking module calculates the sphere's three-dimensional position and obtains its three-dimensional velocity vector and three-dimensional angular velocity vector includes: Sphere target detection is performed on images captured by the camera, and the three-dimensional position of the sphere in the target area coordinate system is calculated by triangulation based on the multi-view detection results. The echo signal data acquired by the Doppler radar is processed to extract the Doppler frequency shift and calculate the radial velocity. The three-dimensional velocity vector is obtained by combining the three-dimensional position change. Micro-Doppler feature analysis was performed on the echo signal data to extract the sideband frequency shift and obtain the three-dimensional angular velocity vector.

5. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 4, characterized in that, The process of obtaining a continuous tracking trajectory includes: The three-dimensional position, three-dimensional velocity vector, and three-dimensional angular velocity vector at the same moment are combined to form the state vector at that moment; Obtain the wind speed vector and air density at the current moment, calculate the difference between the theoretical resultant acceleration and the actual acceleration, and correct the combined result when the difference exceeds a preset threshold; By combining and verifying the data in chronological order, a continuous tracking trajectory can be obtained.

6. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 5, characterized in that, The trajectory prediction module extracts the current sphere motion state and environmental data, and the iterative deduction process based on the dynamic equations includes: The current three-dimensional position, three-dimensional velocity vector, and three-dimensional angular velocity vector are obtained from the continuously tracked trajectory and used as the initial state for the deduction. Obtain the wind speed vector and air density at the current moment, set the interval between adjacent moments, and perform a time-by-time deduction starting from the initial state. At each moment, calculate the resultant acceleration based on the forces acting on the sphere, and calculate the three-dimensional velocity vector and three-dimensional position at the next moment from the resultant acceleration, three-dimensional velocity vector, and three-dimensional position.

7. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 6, characterized in that, The trajectory prediction module, in conjunction with site boundary constraints, obtains the future trajectory of the sphere through the following process: Obtain the spatial location of the site boundary using the target area coordinate system; Based on the predicted three-dimensional position of the sphere, the position is compared with the boundary of the field. When the boundary is touched, the corresponding event is determined and the iterative deduction is terminated. If the boundary is not touched, the next calculation is continued. The generated position points are connected in chronological order to form the future trajectory of the sphere.

8. The ball motion trajectory tracking and prediction system based on multi-sensoring according to claim 7, characterized in that, The visualization output module performs 3D visualization rendering of the continuously tracked trajectory and the future trajectory of the sphere, annotates key events, and outputs physical quantity values. The process includes: Acquire the continuous tracking trajectory and the future trajectory of the sphere, construct a virtual 3D scene in the target area coordinate system, render and display the two trajectories with different colors, and annotate key events and display the corresponding physical quantity values.

Images