Tennis serve trajectory prediction and correction method and system based on deep learning and binocular vision, terminal and storage medium

Abstract

The present application relates to the technical field of tennis trajectory deviation correction, and discloses a tennis serving trajectory prediction and correction method, system, terminal and storage medium based on deep learning and binocular vision, which comprises the following steps: collecting tennis serving images by means of high-speed synchronous binocular cameras, positioning the pixel of the ball center in the left and right views by means of an improved deep learning target detection network, obtaining the three-dimensional coordinate sequence of the ball by means of stereo matching in combination with the camera calibration parameters, quantitatively calculating the serving parameter deviation amount such as the landing point deviation, the speed deviation and the flight arc deviation, and correcting the trajectory based on the pre-established mapping relationship table of the serving parameters and the trajectory deviation. The present application realizes the automatic perception, quantitative evaluation and closed-loop correction of the tennis serving quality, significantly reduces the dependence on artificial experience, and improves the training consistency, efficiency and precision.

Description

Technical Field

[0001] This invention relates to the field of tennis trajectory deviation correction technology, and in particular to a method, system, terminal, and computer-readable storage medium for predicting and correcting tennis serve trajectories based on deep learning and binocular vision. Background Technology

[0002] In tennis training, the quality of the serve directly determines the outcome of a match. Traditional serve training relies heavily on the coach's visual observation and experience, requiring students to practice repeatedly over a long period to develop stable motor memory. With the development of intelligent sports equipment, ball machines have been widely introduced into training venues to simulate serves with different spins, speeds, and landing points.

[0003] However, the parameters of mainstream ball machines (spin wheel speed, launch angle, sidespin, etc.) still mainly rely on manual presets or manual adjustments, and cannot provide closed-loop feedback and automatic correction based on the user's actual hitting performance, resulting in low training efficiency and difficulty in adaptive parameter drift.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, terminal, and computer-readable storage medium for predicting and correcting tennis serve trajectories based on deep learning and binocular vision. This aims to solve the problems in the prior art where tennis serve parameter adjustment relies on human experience, the accuracy of three-dimensional trajectory reconstruction is low, and automatic closed-loop correction of serve parameters cannot be achieved.

[0006] To achieve the above objectives, this invention provides a method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision. The method includes the following steps: Acquire tennis ball flight images captured by binocular high-speed cameras, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively; Based on all the detection confidence scores, multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images are selected from all the left views and all the right views respectively. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, the three-dimensional coordinate representation of the target tennis ball in different frames is obtained. Extract the target's three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and input all the target's three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; If the complete trajectory is determined to have a deviation based on the deviation parameter, a mapping relationship between the serving parameters and the deviation parameter is established to correct the complete trajectory, and the serving trajectory of the ball machine is detected again until the serving trajectory is accurate.

[0007] Optionally, the method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, wherein acquiring tennis flight images captured by a binocular high-speed camera, inputting the tennis flight images into a target detection network for matching, and outputting the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, specifically includes: When the ball machine launches the target tennis ball, a binocular high-speed camera is used to capture the left and right views of the target tennis ball in each frame; For each frame, the left view and the right view are input into the constructed target detection network, and an attention module is introduced into the target detection network to perform edge feature enhancement processing on the left view and the right view; Small target detection is performed on the processed left and right views using a neck network. The left and right dual-stream branches are input separately for parallel inference to obtain the left view sphere center pixel coordinates, the right view sphere center pixel coordinates and the detection confidence for each frame in different frames.

[0008] Optionally, the method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, wherein the step of selecting multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left views and all the right views according to all the detection confidence scores, and aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images to obtain the three-dimensional coordinate representation of the target tennis ball in different frames, specifically includes: Left-eye short-focus 4K images and right-eye long-focus HD images with a detection confidence greater than a first confidence threshold are selected from all the left views and all the right views, respectively. Based on the intrinsic and extrinsic parameters of the binocular high-speed camera, the left-eye short-focus 4K image and the right-eye long-focus HD image are time-stamp aligned. Define the world coordinate system for the left-eye short-focus 4K image, and define the left eye of the binocular high-speed camera as the origin; The left view sphere center pixel coordinates and the right view sphere center pixel coordinates are mapped onto the world coordinate system to fuse the left eye short-focus 4K image and the right eye long-focus HD image to obtain a three-dimensional coordinate system; The three-dimensional coordinate representation of the target tennis ball in different frames is determined on the three-dimensional coordinate system.

[0009] Optionally, the method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, wherein the step of extracting the target 3D coordinates corresponding to all keyframes from all the 3D coordinate representations, and inputting all the target 3D coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball, specifically includes: After performing ID association on the three-dimensional coordinate representation of each frame, multiple missed detection frames are obtained; A constant-rate Kalman filter is established for all the three-dimensional coordinate representations to fill in each of the missed frames using the predicted values, thereby obtaining the three-dimensional coordinate representation of the target tennis ball throughout the entire time series; Taking the start of the ball-serving machine as the starting frame and the landing or netting of the target tennis ball as the ending frame, target frames with a velocity change rate greater than a velocity threshold, a detection confidence level greater than a second confidence threshold, and a sudden change in trajectory curvature are selected from all frames, and the three-dimensional coordinates corresponding to all the target frames are defined as target three-dimensional coordinates; All the target's three-dimensional coordinates are input into the constructed spatiotemporal sequence prediction network for trajectory prediction and deviation calculation, and the complete trajectory and deviation parameters of the target tennis ball are output.

[0010] Optionally, the method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, wherein the step of inputting all the target's three-dimensional coordinates into a pre-constructed spatiotemporal sequence prediction network for trajectory prediction and deviation calculation, and outputting the complete trajectory and deviation parameters of the target tennis ball, specifically includes: An initial prediction network is constructed, a joint training set is built based on historical real serve data and physical synthetic data, and the initial prediction network is trained using the joint training set to obtain a spatiotemporal sequence prediction network. For each of the target's three-dimensional coordinates, the target's three-dimensional coordinates are input into the spatiotemporal sequence prediction network. The local spatial features of the target's three-dimensional coordinates are extracted using a one-dimensional convolutional layer in the spatiotemporal sequence prediction network. Then, an encoder is used to perform temporal encoding on each of the local spatial features to obtain the complete trajectory of the target's three-dimensional coordinates in subsequent frames. The deviation parameters corresponding to the target's three-dimensional coordinates are extracted from the last frame of the complete trajectory, and a corresponding deviation score is constructed based on the deviation parameters: ; in, Indicates the deviation score. Indicates the deviation in landing point. Indicates speed deviation, Indicates the deviation in flight arc. , and They represent , and The weighting coefficients.

[0011] Optionally, the tennis serve trajectory prediction and correction method based on deep learning and binocular vision, wherein if the complete trajectory is determined to have a deviation based on the deviation parameter, a mapping relationship is established between the serve parameters and the deviation parameter to correct the complete trajectory, specifically including: Construct an error sliding window of a preset frame size, and detect the complete trajectory of the target tennis ball within the error sliding window; If the deviation score for no less than a preset number of frames is lower than the standard value, then the complete trajectory is determined to have a deviation. A mapping relationship is established between the serve parameters and the landing point deviation, the speed deviation, and the flight arc deviation, respectively, to obtain the corresponding correction amount and correct the complete trajectory.

[0012] Optionally, in the tennis serve trajectory prediction and correction method based on deep learning and binocular vision, the serve parameters include: serve distance, launch angle, serve speed, serve spin, left and right landing point deviation, and arc height. The process of establishing mapping relationships between the serve parameters and the landing point deviation, the speed deviation, and the flight arc deviation, respectively, to obtain corresponding correction amounts for correcting the complete trajectory, specifically includes: A mapping is constructed between the serve distance, the launch angle, and the landing point deviation to correct the landing point deviation, resulting in a landing point correction increment: ; in, This indicates the increment for correcting the landing point. This represents the landing point correction factor. Indicates the distance of the serve. Indicates the launch angle; A mapping is constructed between the serve speed, the serve spin speed, and the speed deviation to correct the speed deviation and obtain the speed correction increment: ; in, Indicates the speed correction increment. This represents the speed correction factor. Indicates the speed of the serve. Indicates the serve speed; A mapping is constructed between the left and right landing point deviations and the arc height and the flight arc deviation to correct the flight arc deviation and obtain the flight arc correction increment: ; in, Indicates the incremental correction for the flight arc. Indicates the flight arc correction factor. Indicate positive and negative function definitions. Indicates the deviation of the left and right landing points. Indicates radian height; Alternatively, a fully connected neural network can be trained as a mapping function, which can then be used to predict the landing point deviation, the velocity deviation, and the flight arc deviation, and output the single maximum landing point correction increment, the single maximum velocity correction increment, and the single maximum flight arc correction increment.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a tennis serve trajectory prediction and correction system based on deep learning and binocular vision, wherein the tennis serve trajectory prediction and correction system based on deep learning and binocular vision includes: The position detection module is used to acquire tennis ball flight images captured by a binocular high-speed camera, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively. The feature fusion module is used to filter out multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left views and all the right views according to all the detection confidence scores. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, the three-dimensional coordinate representation of the target tennis ball in different frames is obtained. The deviation estimation module is used to extract the target three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and to input all the target three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; The correction module is used to establish a mapping relationship between the serving parameters and the deviation parameters if the complete trajectory is determined to have a deviation based on the deviation parameters, so as to correct the complete trajectory and detect the serving trajectory of the ball machine again until the serving trajectory is accurate.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a tennis serve trajectory prediction and correction program based on deep learning and binocular vision stored in the memory and executable on the processor. When the tennis serve trajectory prediction and correction program based on deep learning and binocular vision is executed by the processor, it implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a tennis serve trajectory prediction and correction program based on deep learning and binocular vision, and when the tennis serve trajectory prediction and correction program based on deep learning and binocular vision is executed by a processor, it implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described above.

[0016] In this invention, tennis ball flight images captured by a binocular high-speed camera are acquired. These images are then input into a target detection network for matching, outputting the ball's center pixel coordinates and detection confidence scores for the left and right views of different frames. Based on all the detection confidence scores, multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images are selected from all the left and right views. Each pair of left-eye short-focus 4K images and right-eye long-focus HD images is aligned and fused to obtain the three-dimensional coordinate representation of the target tennis ball in different frames. The target's three-dimensional coordinates corresponding to all keyframes are extracted from all the three-dimensional coordinate representations, and all the target's three-dimensional coordinates are mapped to a pre-constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball. If the complete trajectory is found to have a deviation based on the deviation parameters, a mapping relationship between the serving parameters and the deviation parameters is established to correct the complete trajectory, and the serving trajectory of the ball machine is detected again until the serving trajectory is accurate. This invention enables automatic sensing, quantitative evaluation, and closed-loop correction of tennis serve quality, significantly reducing reliance on human experience and improving training consistency, efficiency, and accuracy. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision of the present invention; Figure 2 This is a binocular inference flowchart of a preferred embodiment of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision of the present invention. Figure 3 This is a structural diagram of a preferred embodiment of the tennis serve trajectory prediction and correction system based on deep learning and binocular vision of the present invention; Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. This invention is not intended to limit the invention.

[0019] Currently, mainstream ball-serving machines have a relatively limited measurement dimension, using only radar or photoelectric gates for speed measurement. However, this cannot provide key ball-serving quality indicators such as landing point, arc, and spin. Furthermore, during measurement, monocular solutions lack depth, while traditional binocular solutions have a high matching failure rate under high-speed ambiguity. This makes it impossible to automatically adjust the ball-serving machine parameters based on deviations, and training also relies heavily on manual intervention.

[0020] To effectively address these issues, this invention proposes a tennis serve trajectory prediction and correction method based on deep learning and binocular vision. A high-speed, synchronous binocular camera acquires tennis serve images. An improved deep learning target detection network locates the ball's center pixel in both left and right views. Stereo matching, combined with camera calibration parameters, yields a 3D coordinate sequence of the ball. This sequence is then fed into a spatiotemporal sequence prediction network to complete and predict the final flight trajectory. Deviations in serve parameters, such as landing point deviation, velocity deviation, and flight arc deviation, are quantified and calculated. Based on a pre-established mapping table between serve parameters (launch wheel speed, launch angle, sidespin) and trajectory deviations, an incremental correction strategy with continuous deviation exceeding a threshold and limited step size is employed to perform closed-loop correction of the serve machine parameters. After each correction, the trajectory is re-verified through a second serve to ensure the correct correction direction.

[0021] The preferred embodiment of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision described in this invention, such as... Figure 1 As shown, the tennis serve trajectory prediction and correction method based on deep learning and binocular vision includes the following steps: Step S10: Acquire tennis ball flight images captured by binocular high-speed cameras, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively.

[0022] The minimum frame rate requirement of the binocular high-speed camera used in this invention is 60 FPS, which can effectively ensure that the displacement between adjacent frames does not exceed 35 cm when the tennis ball is in high-speed flight, avoiding trajectory breakage and prediction errors. The resolution of the binocular module supports 4K (3840x2160) and HD (1280x720) pixels respectively, which can balance computing power and feature clarity.

[0023] Specifically, when the ball machine launches the target tennis ball, a binocular high-speed camera is used to capture the left and right views of the target tennis ball in each frame; For each frame, the left view and the right view are input into the constructed target detection network, and an attention module is introduced into the target detection network to perform edge feature enhancement processing on the left view and the right view; Small target detection is performed on the processed left and right views using a neck network. The left and right dual-stream branches are input separately for parallel inference to obtain the left view sphere center pixel coordinates, the right view sphere center pixel coordinates and the detection confidence for each frame in different frames.

[0024] In this invention, an improved deep learning object detection network is used for tennis ball detection. A CBAM attention module is introduced into the detection network to enhance the edge features of the tennis ball. A small object detection layer is added to the neck network to improve the recall rate of small balls at long distances. Finally, a weighted combination of CIoU (Complete Intersection over Union, the loss function used for bounding box regression in object detection) and Smooth-L1 (Smooth L1 Loss, which smooths the transition when the error is small and increases linearly when the error is large) loss functions is used as the loss function for bounding box regression, making the network training process more stable and converge faster.

[0025] Furthermore, such as Figure 2 As shown, the left and right views are input into the left and right dual-stream branches for parallel inference, respectively. The sphere center pixel coordinates and the detection confidence corresponding to each frame are extracted from the left and right views respectively. The left and right branches infer the sphere center pixel coordinates from the short-focus 4K wide field of view and the long-focus HD detail image respectively. Since the short-focus image has a wide field of view but low accuracy, and the long-focus image has high accuracy but limited field of view, after the dual branches extract in parallel, the left and right pixel coordinates can be fused by the principle of binocular parallax to reconstruct high-precision three-dimensional spatial coordinates and make up for the lack of depth information from a single viewpoint.

[0026] The ball center pixel coordinates serve as the basis for fusing 2D and 3D features, while the detection confidence score effectively filters out high-quality images. The detection confidence score output for each frame effectively quantifies image quality (such as interference from lighting, occlusion, motion blur, etc.). By setting a confidence threshold, low-quality frames can be automatically filtered out, ensuring that only ball center coordinates with high confidence scores enter subsequent tracking or 3D reconstruction modules, significantly improving the system's robustness and anti-interference capability. The left and right eye ball center coordinates, as 2D features, complement the 3D depth features derived from binocular parallax, providing a precise fusion basis for subsequent Kalman filtering, re-identification, or 3D trajectory generation, ultimately achieving stable and smooth tracking of a small ball at long distances.

[0027] Step S20: Based on all the detection confidence scores, select multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left views and all the right views respectively. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, obtain the three-dimensional coordinate representation of the target tennis ball in different frames.

[0028] In this process, after selecting multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images based on detection confidence, it is necessary to fuse the detection results of the binocular (long-focus and short-focus) cameras. The long-focus camera can capture the ball at a greater distance, while the short-focus camera can capture the global changes of the ball. Fusing the two results can stably obtain a three-dimensional representation of the tennis ball trajectory, thus presenting a reliable result.

[0029] Specifically, left-eye short-focus 4K images and right-eye long-focus HD images with a detection confidence score greater than a first confidence threshold are selected from all the left views and all the right views, respectively. Based on the intrinsic and extrinsic parameters of the binocular high-speed camera, the left-eye short-focus 4K image and the right-eye long-focus HD image are time-stamp aligned. Define the world coordinate system for the left-eye short-focus 4K image, and define the left eye of the binocular high-speed camera as the origin; The left view sphere center pixel coordinates and the right view sphere center pixel coordinates are mapped onto the world coordinate system to fuse the left eye short-focus 4K image and the right eye long-focus HD image to obtain a three-dimensional coordinate system; The three-dimensional coordinate representation of the target tennis ball in different frames is determined on the three-dimensional coordinate system.

[0030] The binocular camera has its extrinsic and intrinsic parameters pre-calibrated, and the timestamps are strictly aligned. Therefore, the left eye corresponding to the short-focus 4K image of the left eye can be defined as the origin of the world coordinate system. Then, the two images corresponding to each frame are mapped to this world coordinate system. In this way, the 2D image coordinates can be projected onto the 3D camera space coordinate system using the conversion formula of the binocular high-speed camera, so as to obtain the three-dimensional coordinate representation of the tennis ball.

[0031] This invention utilizes pre-calibrated extrinsic parameters (rotation matrix and translation vector) and intrinsic parameters (focal length, principal point coordinates, etc.) of a binocular camera, defining the optical center of the left-eye short-focal-length 4K camera as the origin of the world coordinate system. Because the left and right eye timestamps are strictly aligned, the left and right images of each frame can be correlated under the same spatiotemporal reference, ensuring spatiotemporal consistency of coordinate mapping. Simultaneously, using the triangulation formula of binocular vision, and leveraging the parallax of the ball's center pixel coordinates in the left and right eye images, combined with the known baseline distance and focal length, the precise three-dimensional coordinates of the tennis ball in the camera's spatial coordinate system can be calculated using the principle of similar triangles. This process transforms the pixel positions on the two-dimensional image plane into spatial points with a real physical scale. Since projection is directly based on a geometric model, it does not rely on complex motion prediction or filtering initial values, thus effectively avoiding the cumulative drift error caused by inter-frame integration in traditional methods. The final output three-dimensional coordinates accurately reflect the tennis ball's flight trajectory, providing highly reliable data support for subsequent landing point analysis, velocity calculation, or Hawk-Eye-assisted decision-making.

[0032] Step S30: Extract the target three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and input all the target three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball.

[0033] To address short-term occlusion and missed detection, this invention further employs a hybrid deep learning detection and Kalman filter tracking measurement approach to improve detection accuracy.

[0034] Specifically, after performing ID association on the three-dimensional coordinate representation of each frame, multiple missed detection frames are obtained; A constant-rate Kalman filter is established for all the three-dimensional coordinate representations to fill in each of the missed frames using the predicted values, thereby obtaining the three-dimensional coordinate representation of the target tennis ball throughout the entire time series; Taking the start of the ball-serving machine as the starting frame and the landing or netting of the target tennis ball as the ending frame, target frames with a velocity change rate greater than a velocity threshold, a detection confidence level greater than a second confidence threshold, and a sudden change in trajectory curvature are selected from all frames, and the three-dimensional coordinates corresponding to all the target frames are defined as target three-dimensional coordinates; All the target's three-dimensional coordinates are input into the constructed spatiotemporal sequence prediction network for trajectory prediction and deviation calculation, and the complete trajectory and deviation parameters of the target tennis ball are output.

[0035] In this invention, DeepSORT (SORT, Simple Online and Realtime Tracking) is used as a baseline. The three-dimensional coordinate representation of each frame is associated with an ID, and then a constant-speed Kalman filter is established on the three-dimensional coordinate representation sequence. This can fill in the three-dimensional coordinate representation of all missed detection frames (the filling content is predicted based on the detected three-dimensional coordinate representation), thereby obtaining a complete trajectory.

[0036] Using DeepSORT as a baseline, its cascaded matching and Hungarian algorithm are employed to associate the detected 3D coordinates of each frame with existing trajectories, assigning a unique ID to each frame. This ensures that even if the sphere experiences brief overlap or occlusion during movement, its identity remains stable through appearance features and motion consistency, avoiding frequent ID switching. For each trajectory with an established ID, a constant-velocity Kalman filter model is built. This not only smooths the coordinate noise of the current frame but, more importantly, allows the Kalman filter to make a one-step prediction based on historical states (position, velocity) when a detector misses a frame (confidence below the threshold or target loss), automatically generating the predicted 3D coordinates for that frame and filling in gaps in the trajectory. Finally, through the filter's prediction-update loop mechanism, the output trajectory not only covers all detected frames but also intelligently interpolates missed frames, forming a continuous and smooth 3D spatial curve. This provides high-quality, complete trajectory data without missing information for subsequent impact point calculations, velocity analysis, or tactical statistics.

[0037] Furthermore, an initial prediction network is constructed. A joint training set is built based on historical real serve data and physical synthetic data. The initial prediction network is trained using the joint training set to obtain a spatiotemporal sequence prediction network. For each of the target's three-dimensional coordinates, the target's three-dimensional coordinates are input into the spatiotemporal sequence prediction network. The local spatial features of the target's three-dimensional coordinates are extracted using a one-dimensional convolutional layer in the spatiotemporal sequence prediction network. Then, an encoder is used to perform temporal encoding on each of the local spatial features to obtain the complete trajectory of the target's three-dimensional coordinates in subsequent frames. The deviation parameters corresponding to the target's three-dimensional coordinates are extracted from the last frame of the complete trajectory, and a corresponding deviation score is constructed based on the deviation parameters: ; in, Indicates the deviation score. Indicates the deviation in landing point. Indicates speed deviation, Indicates the deviation in flight arc. , and They represent , and The weighting coefficients.

[0038] This invention uses the target's three-dimensional coordinates corresponding to keyframes as input to identify the deviation parameters of the target tennis ball. The spatiotemporal sequence prediction network is jointly trained using real serve data and synthetic data based on physical simulation. The training data incorporates perturbations such as spin, wind resistance, and humidity to enhance generalization, resulting in more realistic and reliable predictions. This allows for accurate modeling of the complex aerodynamic effects of the tennis ball in flight, predicting a theoretical trajectory closer to real physical laws, and providing a reliable benchmark for deviation calculation.

[0039] Furthermore, the deviation of the target tennis ball during a serve can be judged by three core deviation quantities at the end of its predicted trajectory: the landing point deviation is determined by calculating the Euclidean distance between the predicted landing point and the target landing point; the velocity deviation is determined by calculating the difference between the ball's instantaneous velocity at 1 meter and the target velocity; and the flight arc deviation is determined by calculating the height difference between the highest point of the trajectory and the target arc. These three deviations are then weighted and fused to obtain a trajectory deviation score for the serve. This comprehensive characterization of serve performance from three dimensions—landing point accuracy, power control, and arc quality—can intuitively reflect serve quality, provide data-driven guidance for users, and provide quantitative basis for targeted training.

[0040] Step S40: If the complete trajectory is determined to have a deviation based on the deviation parameter, a mapping relationship between the serving parameters and the deviation parameter is established to correct the complete trajectory, and the serving trajectory of the ball machine is detected again until the serving trajectory is accurate.

[0041] In this invention, the mapping relationship between the serving parameters and the deviation parameters is established based on physical prior approximation formulas or deep learning regression models to measure the deviation correction amount, ensuring that the serving machine parameters gradually approach the target value in small steps and preventing large jumps.

[0042] Specifically, an error sliding window of a preset frame size is constructed, and the complete trajectory of the target tennis ball is detected within the error sliding window; If the deviation score for no less than a preset number of frames is lower than the standard value, then the complete trajectory is determined to have a deviation. A mapping relationship is established between the serve parameters and the landing point deviation, the speed deviation, and the flight arc deviation, respectively, to obtain the corresponding correction amount and correct the complete trajectory.

[0043] After each parameter correction, the present invention will continue to perform multiple consecutive serves (3 times in the preferred embodiment of the present invention) to verify the correction. If the trajectory deviation score drops below the threshold, the correction can be confirmed as effective and the current parameter can be locked. If the trajectory deviation score does not decrease or increases in the opposite direction, the system will roll back to the previous set of parameters and mark the correction as failed, and re-enter the cumulative judgment. If multiple consecutive corrections (3 times in the preferred embodiment of the present invention) fail, the automatic correction will be paused and an alarm will be triggered to prompt manual intervention.

[0044] Through repeated testing, random disturbances in a single serve (such as sudden changes in wind speed or fluctuations in the feel of the ball) can be effectively filtered out, ensuring that the score changes truly reflect the effect of parameter correction, rather than interference from accidental factors. When the verification results show that the deviation score has not decreased or even increased, the system automatically rolls back to the previous set of parameters and marks the current correction as a failure. This mechanism effectively prevents erroneous parameters from continuously worsening the serve quality, ensuring the stability and security of the system. Finally, if multiple consecutive corrections (such as 3 times) fail, the system determines that the current problem exceeds the scope of automatic optimization (such as equipment malfunction, drastic environmental changes, or algorithm model failure), and immediately suspends automatic correction and issues an alarm to prompt manual intervention. This design not only avoids the waste of resources in an infinite loop, but also reserves a manual backup channel for extreme situations.

[0045] Furthermore, the serving parameters include: serving distance, launch angle, serving speed, serving spin, left and right landing point deviation, and arc height; A mapping is constructed between the serve distance, the launch angle, and the landing point deviation to correct the landing point deviation, resulting in a landing point correction increment: ; in, This indicates the increment for correcting the landing point. This represents the landing point correction factor. Indicates the distance of the serve. Indicates the launch angle; A mapping is constructed between the serve speed, the serve spin speed, and the speed deviation to correct the speed deviation and obtain the speed correction increment: ; in, Indicates the speed correction increment. This represents the speed correction factor. Indicates the speed of the serve. Indicates the serve speed; A mapping is constructed between the left and right landing point deviations and the arc height and the flight arc deviation to correct the flight arc deviation and obtain the flight arc correction increment: ; in, Indicates the incremental correction for the flight arc. Indicates the flight arc correction factor. Indicate positive and negative function definitions. Indicates the deviation of the left and right landing points. Indicates radian height; Alternatively, a fully connected neural network can be trained as a mapping function, which can then be used to predict the landing point deviation, the velocity deviation, and the flight arc deviation, and output the single maximum landing point correction increment, the single maximum velocity correction increment, and the single maximum flight arc correction increment.

[0046] In the process of establishing a mapping relationship between serving parameters and deviation parameters based on physical prior approximation formulas to measure deviation correction, this invention constructs mapping relationships between physical parameters such as serving distance, launch angle, speed, and spin rate, and three core evaluation indicators: landing point deviation, speed deviation, and flight arc deviation. This decoupled design ensures that each correction quantity can be clearly mapped to a specific physical control dimension, providing a clear mathematical basis for the fine-tuning of serving parameters. The flight arc correction increment introduces a positive and negative definition function to distinguish the sign of left and right landing point deviations, which can intelligently identify the deviation direction (left or right deviation) to ensure that the correction direction is opposite to the deviation direction, achieving precise correction. The calculated landing point correction increment, speed correction increment, and flight arc correction increment can be directly used as control commands for the serving machine or training guidance system. These increments correspond to adjusting the horizontal angle, ball speed, and topspin / backspin of the serving machine, respectively, transforming the abstract deviation score into specific executable parameter adjustments.

[0047] Furthermore, in the process of establishing the mapping relationship between serve parameters and deviation parameters based on a deep learning-based regression model, the increment for correcting the maximum landing point in a single instance cannot exceed 0.5°, and the increment for correcting the maximum speed in a single instance cannot exceed the serve spindle speed. The maximum correction increment for a single flight arc is 3%, and it cannot exceed 0.3. If the calculated correction exceeds the above limit, it will be truncated according to the limit value to ensure that it does not diverge.

[0048] This invention addresses the physical execution limits of the ball-serving machine by setting hard limits for three correction increments. These limits are based on the mechanical characteristics and kinematic principles of the equipment, ensuring that the correction commands remain within a safe and controllable range. When the correction calculated by the deep learning regression model exceeds the limit, the system automatically truncates it according to the limit instead of discarding it directly. This strategy preserves the effectiveness of the correction direction while avoiding severe oscillations caused by abnormal predictions (such as outliers or overfitting outputs), ensuring the asymptotic convergence of the closed-loop control system. With small deviations, the model is allowed to output precise corrections; with large deviations, the single-step correction amplitude is actively sacrificed for system stability. Combined with the preceding multi-round verification and rollback mechanism, this forms a robust strategy of "gradual correction and frequent verification," ensuring that no unexpected jumps occur during long-term optimization.

[0049] This invention enables automatic sensing, quantitative evaluation, and closed-loop correction of tennis serve quality, significantly reducing reliance on human experience and improving training consistency, efficiency, and accuracy.

[0050] Furthermore, such as Figure 3 As shown, based on the above-mentioned method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, this invention also provides a tennis serve trajectory prediction and correction system based on deep learning and binocular vision, wherein the tennis serve trajectory prediction and correction system based on deep learning and binocular vision includes: The position detection module 51 is used to acquire tennis ball flight images captured by a binocular high-speed camera, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively. The feature fusion module 52 is used to filter out multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left views and all the right views according to all the detection confidence scores. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, the three-dimensional coordinate representation of the target tennis ball in different frames is obtained. The deviation estimation module 53 is used to extract the target three-dimensional coordinates corresponding to all key frames from all the three-dimensional coordinate representations, and to input all the target three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; The correction module 54 is used to establish a mapping relationship between the serving parameters and the deviation parameters if it is determined that there is a deviation in the complete trajectory based on the deviation parameters, so as to correct the complete trajectory and detect the serving trajectory of the ball machine again until the serving trajectory is accurate.

[0051] Furthermore, such as Figure 4As shown, based on the above-mentioned method and system for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0052] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a tennis serve trajectory prediction and correction program 40 based on deep learning and binocular vision. This tennis serve trajectory prediction and correction program 40 based on deep learning and binocular vision can be executed by the processor 10, thereby implementing the tennis serve trajectory prediction and correction method based on deep learning and binocular vision in this application.

[0053] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the tennis serve trajectory prediction and correction method based on deep learning and binocular vision.

[0054] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display 30 is used for displaying descriptions. It should be understood that the specific embodiments described herein are merely illustrative of information on the terminal and for displaying a visual user interface. Components of the terminal communicate with each other via a system bus.

[0055] In one embodiment, when the processor 10 executes the tennis serve trajectory prediction and correction program 40 based on deep learning and binocular vision in the memory 20, it implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described above.

[0056] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a tennis serve trajectory prediction and correction program based on deep learning and binocular vision, and the tennis serve trajectory prediction and correction program based on deep learning and binocular vision, when executed by a processor, implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described above.

[0057] In summary, this invention provides a method and related device for predicting and correcting tennis serve trajectories based on deep learning and binocular vision. The method includes: acquiring tennis flight images captured by a binocular high-speed camera; inputting the tennis flight images into a target detection network for matching, and outputting the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively; based on all the detection confidence scores, selecting multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left and right views, respectively; aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images to obtain the three-dimensional coordinate representation of the target tennis ball in different frames; extracting the target three-dimensional coordinates corresponding to all key frames from all the three-dimensional coordinate representations, and mapping all the target three-dimensional coordinates to a pre-constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; if the complete trajectory is determined to have a deviation based on the deviation parameters, establishing a mapping relationship between the serve parameters and the deviation parameters to correct the complete trajectory, and detecting the serve trajectory of the ball machine again until the serve trajectory is accurate. This invention enables automatic sensing, quantitative evaluation, and closed-loop correction of tennis serve quality, significantly reducing reliance on human experience and improving training consistency, efficiency, and accuracy.

[0058] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0059] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0060] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision, characterized in that, The tennis serve trajectory prediction and correction method based on deep learning and binocular vision includes: Acquire tennis ball flight images captured by binocular high-speed cameras, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively; Based on all the detection confidence scores, multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images are selected from all the left views and all the right views respectively. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, the three-dimensional coordinate representation of the target tennis ball in different frames is obtained. Extract the target's three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and input all the target's three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; If the complete trajectory is determined to have a deviation based on the deviation parameter, a mapping relationship between the serving parameters and the deviation parameter is established to correct the complete trajectory, and the serving trajectory of the ball machine is detected again until the serving trajectory is accurate.

2. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 1, characterized in that, The process of acquiring tennis ball flight images captured by a binocular high-speed camera, inputting these images into a target detection network for matching, and outputting the ball center pixel coordinates and detection confidence scores for the left and right views of different frames, specifically includes: When the ball machine launches the target tennis ball, a binocular high-speed camera is used to capture the left and right views of the target tennis ball in each frame; For each frame, the left view and the right view are input into the constructed target detection network, and an attention module is introduced into the target detection network to perform edge feature enhancement processing on the left view and the right view; Small target detection is performed on the processed left and right views using a neck network. The left and right dual-stream branches are input separately for parallel inference to obtain the left view sphere center pixel coordinates, the right view sphere center pixel coordinates and the detection confidence for each frame in different frames.

3. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 2, characterized in that, The process involves selecting multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all left views and all right views based on all detection confidence scores. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, a three-dimensional coordinate representation of the target tennis ball in different frames is obtained. Specifically, this includes: Left-eye short-focus 4K images and right-eye long-focus HD images with a detection confidence greater than a first confidence threshold are selected from all the left views and all the right views, respectively. Based on the intrinsic and extrinsic parameters of the binocular high-speed camera, the left-eye short-focus 4K image and the right-eye long-focus HD image are time-stamp aligned. Define the world coordinate system for the left eye short-focus 4K image, and define the left eye of the binocular high-speed camera as the origin; The left view sphere center pixel coordinates and the right view sphere center pixel coordinates are mapped onto the world coordinate system to fuse the left eye short-focus 4K image and the right eye long-focus HD image to obtain a three-dimensional coordinate system; The three-dimensional coordinate representation of the target tennis ball in different frames is determined on the three-dimensional coordinate system.

4. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 1, characterized in that, The step of extracting the target's three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and then feeding all the target's three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball, specifically includes: After performing ID association on the three-dimensional coordinate representation of each frame, multiple missed detection frames are obtained; A constant-rate Kalman filter is established for all the three-dimensional coordinate representations to fill in each of the missed frames using the predicted values, thereby obtaining the three-dimensional coordinate representation of the target tennis ball throughout the entire time series; Taking the start of the ball-serving machine as the starting frame and the landing or netting of the target tennis ball as the ending frame, target frames with a velocity change rate greater than a velocity threshold, a detection confidence level greater than a second confidence threshold, and a sudden change in trajectory curvature are selected from all frames, and the three-dimensional coordinates corresponding to all the target frames are defined as target three-dimensional coordinates; All the target's three-dimensional coordinates are input into the constructed spatiotemporal sequence prediction network for trajectory prediction and deviation calculation, and the complete trajectory and deviation parameters of the target tennis ball are output.

5. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 4, characterized in that, The process of inputting all the target's three-dimensional coordinates into the constructed spatiotemporal sequence prediction network for trajectory prediction and deviation calculation, and outputting the complete trajectory and deviation parameters of the target tennis ball, specifically includes: An initial prediction network is constructed, a joint training set is built based on historical real serve data and physical synthetic data, and the initial prediction network is trained using the joint training set to obtain a spatiotemporal sequence prediction network. For each of the target's three-dimensional coordinates, the target's three-dimensional coordinates are input into the spatiotemporal sequence prediction network. The local spatial features of the target's three-dimensional coordinates are extracted using a one-dimensional convolutional layer in the spatiotemporal sequence prediction network. Then, an encoder is used to perform temporal encoding on each of the local spatial features to obtain the complete trajectory of the target's three-dimensional coordinates in subsequent frames. The deviation parameters corresponding to the target's three-dimensional coordinates are extracted from the last frame of the complete trajectory, and a corresponding deviation score is constructed based on the deviation parameters: ； in, Indicates the deviation score. Indicates the deviation in landing point. Indicates speed deviation, Indicates the deviation in flight arc. , and They represent , and The weighting coefficients.

6. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 5, characterized in that, If a deviation is determined to exist in the complete trajectory based on the deviation parameter, a mapping relationship is established between the serve parameters and the deviation parameter to correct the complete trajectory, specifically including: Construct an error sliding window of a preset frame size, and detect the complete trajectory of the target tennis ball within the error sliding window; If the deviation score for no less than a preset number of frames is lower than the standard value, then the complete trajectory is determined to have a deviation. A mapping relationship is established between the serve parameters and the landing point deviation, the speed deviation, and the flight arc deviation, respectively, to obtain the corresponding correction amount and correct the complete trajectory.

7. The method for predicting and correcting tennis serve trajectories based on deep learning and binocular vision according to claim 6, characterized in that, The serving parameters include: serving distance, launch angle, serving speed, serving spin, left and right landing point deviation, and arc height; The process of establishing mapping relationships between the serve parameters and the landing point deviation, the speed deviation, and the flight arc deviation, respectively, to obtain corresponding correction amounts for correcting the complete trajectory, specifically includes: A mapping is constructed between the serve distance, the launch angle, and the landing point deviation to correct the landing point deviation, resulting in a landing point correction increment: ； in, This indicates the increment for correcting the landing point. This represents the landing point correction factor. Indicates the distance of the serve. Indicates the launch angle; A mapping is constructed between the serve speed, the serve spin speed, and the speed deviation to correct the speed deviation and obtain the speed correction increment: ； in, Indicates the speed correction increment. This represents the speed correction factor. Indicates the speed of the serve. Indicates the serve speed; A mapping is constructed between the left and right landing point deviations and the arc height and the flight arc deviation to correct the flight arc deviation and obtain the flight arc correction increment: ； in, Indicates the incremental correction for the flight arc. Indicates the flight arc correction factor. Indicate positive and negative function definitions. Indicates the deviation of the left and right landing points. Indicates radian height; Alternatively, a fully connected neural network can be trained as a mapping function, which can then be used to predict the landing point deviation, the velocity deviation, and the flight arc deviation, and output the single maximum landing point correction increment, the single maximum velocity correction increment, and the single maximum flight arc correction increment.

8. A tennis serve trajectory prediction and correction system based on deep learning and binocular vision, characterized in that, The tennis serve trajectory prediction and correction system based on deep learning and binocular vision is used to implement the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described in any one of claims 1-7. The tennis serve trajectory prediction and correction system based on deep learning and binocular vision includes: The position detection module is used to acquire tennis ball flight images captured by a binocular high-speed camera, input the tennis ball flight images into a target detection network for matching, and output the ball center pixel coordinates and detection confidence scores corresponding to the left and right views of different frames, respectively. The feature fusion module is used to filter out multiple left-eye short-focus 4K images and multiple right-eye long-focus HD images from all the left views and all the right views according to all the detection confidence scores. After aligning and fusing each pair of left-eye short-focus 4K images and right-eye long-focus HD images, the three-dimensional coordinate representation of the target tennis ball in different frames is obtained. The deviation estimation module is used to extract the target three-dimensional coordinates corresponding to all keyframes from all the three-dimensional coordinate representations, and to input all the target three-dimensional coordinates into the constructed spatiotemporal sequence prediction network to output the complete trajectory and deviation parameters of the target tennis ball; The correction module is used to establish a mapping relationship between the serving parameters and the deviation parameters if the complete trajectory is determined to have a deviation based on the deviation parameters, so as to correct the complete trajectory and detect the serving trajectory of the ball machine again until the serving trajectory is accurate.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a tennis serve trajectory prediction and correction program based on deep learning and binocular vision stored in the memory and executable on the processor. When the tennis serve trajectory prediction and correction program based on deep learning and binocular vision is executed by the processor, it implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a tennis serve trajectory prediction and correction program based on deep learning and binocular vision. When the tennis serve trajectory prediction and correction program based on deep learning and binocular vision is executed by a processor, it implements the steps of the tennis serve trajectory prediction and correction method based on deep learning and binocular vision as described in any one of claims 1-7.

Images