A badminton tracking method and system based on three-dimensional vision

Abstract

The application discloses a badminton tracking method and system based on three-dimensional vision, relates to computer vision, and synchronously collects badminton motion video data of left and right views through a binocular camera, obtains a time-aligned left view frame sequence and a right view frame sequence, pre-processes the collected video data to obtain standardized left and right view input frame data; a 2D key frame detection model based on an improved YOLOv8 network is constructed according to the input frame data, and 2D key frame coordinates of the left and right views are extracted; according to the 2D key frame coordinates of the left and right views, 3D space positioning is performed on the badminton through stereo matching and trajectory prediction, 3D trajectory data is obtained, and according to the 3D trajectory data, a broken trajectory is repaired by using a frame missing compensation mechanism, and finally, 3D trajectory is obtained. In view of the fact that in the prior art, the key frame is easily lost when the badminton is in high-speed motion such as smashing, thereby causing the trajectory to be broken, the application repairs the broken trajectory of the badminton.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a badminton shuttlecock tracking method and system based on three-dimensional vision. Background Technology

[0002] As a high-speed ball sport, badminton trajectory analysis has significant applications in areas such as live broadcasts of professional matches, athlete training analysis, and tactical review. Accurate 3D trajectory reconstruction relies on the precise detection and positioning of key frames during badminton movements, especially in high-speed scenarios such as smashes and flat drives, where the shuttlecock speed can reach 35–42 m / s, placing extremely high demands on the real-time performance and accuracy of the detection system.

[0003] Existing badminton keyframe extraction techniques mainly include: (1) traditional frame difference method combined with feature matching, such as the frame difference + SIFT feature matching scheme used by a certain sports live streaming platform; (2) optical flow method combined with deep learning, such as the optical flow method + ResNet50 feature extraction scheme used by a certain training system; (3) target detection algorithms based on traditional deep learning such as YOLOv3; (4) semi-automatic scheme of fixed threshold screening combined with manual correction.

[0004] However, existing technologies suffer from a serious keyframe loss problem when processing high-speed badminton scenes, resulting in broken reconstructed 3D trajectories.

[0005] First, the problem of missed detections due to high-speed motion is particularly prominent. In high-speed motion scenarios such as badminton smashes (speed 35-42 m / s) and flat drives (speed 25-30 m / s), the traditional frame difference method uses a fixed sampling interval of 10 frames / sample, which easily misses key action frames, such as the instant the racket hits the shuttlecock. The measured missed detection rate is 18%-25%. Although commercial solutions based on YOLOv3 have introduced deep learning, they have not been optimized for the characteristics of badminton as a small target (the percentage of badminton pixels in a single frame is ≤0.5%). When the target positioning deviation exceeds 10 pixels during high-speed motion, the missed detection rate is still as high as 12%. This high missed detection rate directly leads to discontinuity in trajectory data, forming multiple broken trajectory segments.

[0006] Secondly, the false detection problem caused by changes in lighting exacerbates the trajectory breakage. In indoor event lighting switching (e.g., brightness fluctuation ±30% after a pause) and outdoor training scenarios with alternating sunshine and cloudy conditions (light intensity change ±5000 lux), the optical flow method is overly sensitive to grayscale fluctuations, easily misidentifying flickering light frames and cloud reflection frames as keyframes, with a false detection rate of 22%–28%. This false detection not only increases invalid data but also interferes with the judgment of the continuity of the true trajectory.

[0007] Furthermore, the shortcomings of existing solutions in terms of real-time performance and scene adaptability also affect the quality of trajectory reconstruction. The high complexity of computation, generally ≥80MFLOPs / frame, results in system latency of 320-350ms, which cannot meet the requirements of real-time editing for live sports broadcasts (industry requirement ≤100ms), leading to the inability to capture all keyframes in a timely manner during fast and continuous ball-hitting actions. At the same time, existing solutions are mostly optimized for standard indoor sports scenarios. In complex scenarios such as strong outdoor light and low light at night (light intensity ≤500lux), the accuracy of keyframes drops by more than 40%, further exacerbating the trajectory breakage problem.

[0008] In summary, existing technologies for handling keyframe loss during high-speed badminton movement severely impact the integrity and accuracy of 3D trajectory reconstruction. There is an urgent need for a new technical solution that can effectively detect high-speed small targets, suppress background interference, and possess trajectory repair capabilities. Summary of the Invention

[0009] In view of the fact that keyframes are easily lost and the trajectory is broken when the badminton shuttlecock is smashed or moved at high speed in the existing technology, this application provides a badminton shuttlecock tracking method and system based on three-dimensional vision, which repairs the broken trajectory of the badminton shuttlecock.

[0010] One aspect of this application provides a badminton tracking method based on three-dimensional vision, comprising: S1, simultaneously acquiring badminton motion video data from left and right views using a binocular camera to obtain time-aligned left view frame sequences and right view frame sequences; S2, preprocessing the acquired video data to obtain standardized left and right view input frame data. Where W represents the frame width and H represents the frame height. S3 indicates the number of input channels; S3, based on the input frame data A 2D keyframe detection model based on an improved YOLOv8 network was constructed to extract the 2D keyframe coordinates of the left and right views. and S4, based on the 2D keyframe coordinates of the left and right views. and S5. Through stereo matching and trajectory prediction, the badminton shuttlecock is spatially located in 3D to obtain 3D trajectory data; S6. Based on the 3D trajectory data, the broken trajectory is repaired using a frame loss compensation mechanism to obtain the final 3D trajectory.

[0011] Furthermore, S3, based on the input frame data This paper describes the construction of a 2D keyframe detection model based on an improved YOLOv8 network. The model involves constructing a 2D keyframe detection model based on the YOLOv8 network, improving the YOLOv8 network backbone using the CoT2f module, and using the improved backbone network to process the input frame data. Feature extraction is performed to obtain 3D enhanced feature map, where, This represents the number of feature map channels after enhancement by the CoT2f module. Its value is determined based on the depth configuration of the YOLOv8 network and the channel expansion coefficient of the CoT2f module. , This refers to the base number of channels in the last layer of the YOLOv8 Backbone. represents the channel expansion coefficient of the CoT2f module, with a value range of [1.5, 2.0], used to enhance feature representation capabilities. This is the network depth scaling factor, which takes values ​​in the range [0.33, 1.0] depending on the model size. The enhanced feature map of dimensions integrates local detail features and global contextual information, increasing the number of channels by 50%-100% compared to the original YOLOv8 Backbone output, thereby improving the feature representation capability of small targets in badminton.

[0012] The SANeck structure is used to improve the neck network of the YOLOv8 network. The improved neck network is then used to perform multi-scale feature fusion on the enhanced feature maps to obtain multi-scale fused features. A decoupling head with SGE is used to improve the head detection head of the YOLOv8 network. This improved head is then used to perform classification and regression operations on the multi-scale fused features to extract the 2D keyframe coordinates of the left view. 2D keyframe coordinates of the right view .

[0013] Furthermore, a 2D keyframe detection model based on the YOLOv8 network is constructed. The CoT2f module is used to improve the YOLOv8 network's backbone, and the improved backbone network is used to process the input frame data. Feature extraction is performed to obtain 3D enhanced feature maps, including: input frame data The YOLOv8 network's backbone is input, and a CoT2f module is embedded in the feature extraction layer of the backbone. Within the CoT2f module, the input feature data is processed: a 5×5 convolution kernel is used to perform convolution operations on the input features, resulting in... The input feature mapping of the channel is as follows The intermediate feature map of the channel is obtained. The first intermediate feature of the channel dimension is obtained by equally splitting the first intermediate feature along the channel dimension, resulting in two... Sub-features of Dimension Sub-features Inputting a CoT-Bottleneck structure, 3×3 convolution operations are used to capture local contextual information, generating contextual features containing global-local correlations. Sub-features As a residual connection branch; context features With residual branch By stitching along the channel dimension, we obtain The concatenated features are obtained; a 3×3 convolution operation is applied to the concatenated features for feature fusion, and the features are then... Channel mapping as Channel; Output 3D enhanced feature map, where, To output the number of channels, the enhanced feature map contains badminton target information enhanced by global-local features;

[0014] Furthermore, the SANeck structure is used to improve the Neck network of the YOLOv8 network. The improved Neck network is then used to perform multi-scale feature fusion on the enhanced feature maps, resulting in multi-scale fused features. This includes: inputting the W×H×C 2-dimensional enhanced feature map into the Neck network of the YOLOv8 network; embedding an SGE-2f spatial attention module on the C2f structure of the Neck network to construct the SANeck structure; and performing multi-scale processing on the enhanced feature map within the SANeck structure. The channel-enhanced feature map is adjusted using a 1×1 convolution to obtain a channel-adjusted feature map. This channel-adjusted feature map is then split into two branches along the channel dimension: the first branch features... Second branch features ; the first branch feature Spatial attention enhancement is performed on n cascaded SGE-2f modules to obtain the first branch features after spatial attention enhancement. ; the first branch feature Second branch features Channel-level concatenation is performed to obtain a concatenated feature map; a 1×1 convolution is then applied to the concatenated feature map for feature fusion to restore the original number of channels. The Feature Pyramid Network (FPN) is used to aggregate the output features of SANeck at different levels at multiple scales to obtain multi-scale fused features.

[0015] Furthermore, the first branch features Spatial attention enhancement is performed on n cascaded SGE-2f modules to obtain the first branch features after spatial attention enhancement. This includes: for each SGE-2f module, the first branch feature... The input feature map is divided into G spatial groups according to spatial dimensions, and each spatial group is processed separately.

[0016] The features within each spatial group are processed as follows: global average pooling is performed on the features within the spatial group to obtain the global statistical features of that group. Extract local features at each location (i, j) within the spatial group. Among them, local features Let C be the feature vector at this location; calculate the difference between the local features and the global statistical features at each spatial location: ; for difference values Apply a 1×1 convolution and a sigmoid activation function to generate initial weights: For difference values For regions smaller than a preset threshold θ, the weights will be adjusted. The inhibition coefficient Used to suppress regular backgrounds such as field lines; for difference values Greater than or equal to the preset threshold In the region, the weight will be adjusted to The enhancement coefficient Used to highlight the badminton objective; the adjusted weights Features of the first branch The corresponding positions are multiplied element-wise; the processing results of the G spatial groups are combined to obtain the output of the current SGE-2f module; the n SGE-2f modules are concatenated for processing, and the first branch feature after spatial attention enhancement is finally output. .

[0017] Furthermore, the YOLOv8 network's head detection head is improved using a decoupled head with SGE. The improved detection head is then used to perform classification and regression operations on multi-scale fused features to extract the 2D keyframe coordinates of the left view. 2D keyframe coordinates of the right view This includes: inputting the multi-scale fusion features into corresponding decoupling heads with SGE, wherein each decoupling head contains a classification branch and a regression branch;

[0018] The input features are processed in each decoupling head: the scale features of the input are processed. Applying 1×1 convolution to adjust the feature dimensions yields intermediate features. ; intermediate features Inputting into the SGE spatial gating module, spatial attention enhancement is performed: for intermediate features Global average pooling and global max pooling are performed to obtain average pooling features and max pooling features, respectively. These features are then concatenated and a spatial attention map is generated through a 1×1 convolution. Spatial attention map With intermediate features Element-wise multiplication yields spatially enhanced features. ;

[0019] Spatial enhancement features Decoupling process: Spatial augmentation features The input classification branch is processed through a sequence of 3×3 and 1×1 convolutions to generate a category prediction feature map. , dimension Where K is the number of classes and 1 represents the background class; spatial augmentation features The input regression branch generates bounding box regression feature maps through a sequence of 3×3 and 1×1 convolutions. , dimension Corresponding center point coordinates Width and height (w, h); predict feature maps for categories Apply the Sigmoid function to obtain the class confidence score at each location; filter out detection boxes with confidence scores greater than a threshold τ, where τ ∈ [0.5, 0.7]; apply Non-Maximum Suppression (NMS) to the filtered detection boxes to remove overlapping detections; convert the bounding box regression parameters into image coordinates: , The center coordinates of the badminton detection box in the left view are obtained by processing the input frame data of the left view through the above steps. The center coordinates of the badminton detection box in the right view are obtained by processing the input frame data of the right view through the same process. Output time-aligned left and right views Figure 3 D keyframe coordinates .

[0020] Furthermore, S4, based on the 2D keyframe coordinates of the left and right views... and By using stereo matching and trajectory prediction, the badminton shuttlecock is spatially located in 3D to obtain 3D trajectory data, including: left and right view data based on the aforementioned time alignment. Figure 3 D keyframe coordinates Perform stereo matching and calculate 3D world coordinates: Set the camera intrinsic parameter matrix, where the focal length f = 1200 pixels, and the principal point coordinates... , Set the camera extrinsic parameters, where the baseline distance between the binocular cameras is b = 0.8m; calculate the parallax based on the u-coordinates of the left and right views: ; Calculate the depth coordinates based on the triangulation principle: z = f × b / d; Calculate the 3D world coordinates based on the depth z and the 2D coordinates: , Output the 3D world coordinates of the current frame. ;

[0021] Trajectory prediction for 3D world coordinates across multiple consecutive frames: Input historical 3D coordinate sequence ,in, The timestamp of the i-th frame;

[0022] Linear fitting was performed on the x, y, and z coordinate axes respectively: x-axis fitting: y-axis fitting: z-axis fitting: Based on the fitted parameters Predict 3D coordinates for the next 10 frames Update the detection region based on predicted 3D coordinates: update the predicted 3D coordinates... The camera projection matrix is ​​used to back-project onto the left and right view image planes; the predicted coordinates of the left view are then obtained. And right view predicted coordinates A region of interest (ROI) of 640×640 pixels is defined centered on the predicted coordinates; the ROI information is passed to the S3 2D keyframe detection model to constrain the detection range of subsequent frames; 3D trajectory data is output, including the 3D world coordinates of each frame. The timestamp t and the predicted future trajectory point.

[0023] Furthermore, S5, based on the 3D trajectory data, a missing frame compensation mechanism is used to repair the broken trajectory to obtain the final 3D trajectory, including: identifying missing frames in the 3D trajectory data; and calculating the displacement between adjacent frames based on the 3D coordinates of consecutive frames in the 3D trajectory data. ; Calculate the velocity based on the displacement D and the time interval: When displacement or speed When a frame is missing between frame i and frame i+1, it is determined that there is a missing frame. Displacement threshold Given a velocity threshold, a trajectory mask M is generated, where, Indicates the location of the missing frame. Indicates a normal frame;

[0024] Constructing an interpolation compensation model based on a 1D convolutional U-shaped network: A U-shaped network structure is established, comprising a 4-layer 1D convolutional encoder and a 4-layer 1D deconvolutional decoder, with a kernel size of 3 and a stride of 2. 3D coordinates are back-projected onto the left and right views to obtain the corresponding 2D trajectory coordinates. 2D trajectory data from 5 frames before and after the missing frame position are extracted as network input. A trajectory mask M is used as an auxiliary input to indicate the missing frame position. The network outputs the compensated 2D coordinates of the missing frame position. and ;

[0025] 3D trajectory reconstruction and fusion: 2D coordinate pairs based on compensation Reconstructing and compensating 3D coordinates using stereo matching method Spline interpolation is performed on the 3D coordinate sequence of normal frames to generate continuous trajectories. ; Calculate compensation coordinates Distance to neighboring normal frames: Where j is the sequence number of the nearest normal frame; when At that time, a weighted fusion method is used: Where α is the compensation weight. For distance threshold; when At that time, directly use the compensated coordinates. Output the repaired complete 3D trajectory sequence, including the fusion result of the original detection coordinates and the compensated coordinates.

[0026] Another aspect of this application provides a badminton shuttlecock tracking system based on three-dimensional vision for performing a badminton shuttlecock tracking method based on three-dimensional vision of this application.

[0027] Compared to existing technologies, the advantages of this application are:

[0028] To address the challenges of detecting badminton shuttlecocks as high-speed, small targets in complex environments, their susceptibility to interference from court lines, and trajectory breakage due to high-speed motion in existing technologies, this application provides a badminton shuttlecock tracking method based on 3D vision. By making multi-level improvements to the YOLOv8 network (including enhancing the small target feature extraction capability of the backbone with the CoT2f module, suppressing background interference in the neck with the SANeck structure, and improving the detection accuracy of the head with a decoupling head with SGE), combined with 3D reconstruction using binocular stereo vision, a trajectory prediction feedback mechanism, and a frame loss compensation mechanism based on a 1D convolutional neural network, this method can achieve high-precision detection of badminton shuttlecock trajectory, continuous 3D trajectory reconstruction, and automatic repair of broken trajectories, effectively improving the accuracy and completeness of badminton shuttlecock 3D trajectory detection in complex scenes. Attached Figure Description

[0029] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0030] Figure 1 This is a schematic diagram of an overall process according to some embodiments of this application;

[0031] Figure 2 This is another overall process diagram shown according to some embodiments of this application;

[0032] Figure 3 This is a hardware-software collaboration flowchart shown according to some embodiments of this application;

[0033] Figure 4 This is a schematic diagram of the YO-CSA detection network process according to some embodiments of this application. Detailed Implementation

[0034] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0035] like Figure 1 As shown, badminton motion video data in both left and right views is acquired simultaneously using a binocular camera to obtain time-aligned left and right view frame sequences. The acquired video data is then preprocessed to obtain standardized left and right view input frame data. Where W represents the frame width and H represents the frame height. Indicates the number of input channels; based on the input frame data A 2D keyframe detection model based on an improved YOLOv8 network was constructed to extract the 2D keyframe coordinates of the left and right views. and Based on the 2D keyframe coordinates of the left and right views and By using stereo matching and trajectory prediction, the badminton shuttlecock is spatially located in 3D to obtain 3D trajectory data. Based on the 3D trajectory data, the broken trajectory is repaired using a frame loss compensation mechanism to obtain the final 3D trajectory.

[0036] A stereo vision system is built using a low-cost monocular camera, with a structure as follows: Figure 2 and Figure 3 As shown:

[0037] Video acquisition module: Two Huairui A7200CU130 monocular cameras (1920×1080 resolution, 160fps frame rate, 1 / 1000s shutter speed), installed behind the robot's field of view, with a baseline (distance between the two cameras) of 0.8m and a height of 1.8m (simulating adult perspective), covering the entire field trajectory; each camera is connected to the input interface of the Xilinx Artix-7 FPGA to ensure stable transmission of frame synchronization signals (the FPGA needs to be configured with frame synchronization logic in advance, with a time error ≤1ms).

[0038] Connect the FPGA's output interface to the computing device (desktop i7-13700H+RTX4060 or mobile Snapdragon 888 terminal) via a PCIe 4.0 cable to transmit synchronized video frames; connect a 1TB NVMe SSD to the computing device via the NVMe interface to ensure high-speed storage of key frame data (read / write speed ≥3000MB / s); connect the computing device to the output terminal (event live broadcast control console, training and analysis computer, robot controller) with a gigabit network cable to test network bandwidth ≥1000Mbps.

[0039] Start all hardware and check the frame synchronization status of the two cameras using the FPGA's accompanying tools (time difference ≤ 1ms is normal); open the camera preview software on the computing device to confirm that there is no distortion in the left and right views and that the image covers the entire field; adjust the camera focus so that the badminton shuttlecock is clearly visible at a resolution of 1920×1080 (single frame pixel ratio ≥ 0.3%).

[0040] For camera calibration, prepare a checkerboard calibration board (checkerboard size 20mm×20mm). Take 15 clear checkerboard images (covering near / mid / far field) from different angles and distances within the camera's field of view. Run the OpenCV Zhang Zhengyou calibration program, input the 15 checkerboard images, calculate the camera's intrinsic parameters (focal length f=1200 pixels, principal point coordinates u0=320, v0=320) and extrinsic parameters (rotation matrix R is the identity matrix, translation vector T=[-0.8, 0, 0]m), and save the calibration results to the configuration file. Verify the calibration accuracy by back-projecting the checkerboard corner points through the calibration results, ensuring that the reprojection error is ≤0.5 pixels (if the error is too large, the calibration image needs to be retaken).

[0041] like Figure 4 As shown, the algorithm detection network (an improved version of YOLOv8s for high-precision 2D keyframe detection) optimizes the backbone, neck, and head parts to address the small target characteristics of badminton shuttlecocks.

[0042] Step A1: Backbone Improvement (Adding the CoT2f Module):

[0043] Input frame Output after 5×5 convolutional layer 1 Split into using chunk operations ; The data is fed into CoT-Bottleneck (which includes 3×3 convolutions to capture contextual information, replacing traditional 1×1 convolutions) to extract global features. The data is directly fed into the splicing layer, and after splicing, it is fused through a 3×3 convolutional layer 2 to output the final result. Enhance the correlation between global and local features of small targets;

[0044] Step A2: Neck Improvement (SANeck Structure):

[0045] Based on the C2f structure of YOLOv8s, an SGE-2f module (lightweight spatial attention) is inserted: the output of the first layer is split into two parts after a 1×1 convolution. Through n SGE-2f modules Direct splicing; finally, multi-scale features are fused through Feature Pyramid (FPN) to reduce site line interference;

[0046] Step A3: Head Improvement (Decoupling Head with SGE):

[0047] After the single-branch feature is spatially enhanced by the SGE module, it is split into two parallel branches: ① Classification branch (two 3×3 convolutions, output...) , ① Badminton category only); ② Regression branch (two 3×3 convolutions, output [H, W, 64], including a 16-channel DFL module and four bounding box parameters), to decouple classification and regression parameters and improve positioning accuracy.

[0048] 3D trajectory mapping and prediction (from 2D keyframes to 3D coordinates):

[0049] Step B1: Stereo Matching (Obtaining 3D Coordinates):

[0050] 2D keyframe coordinates output from the left and right cameras , The intrinsic parameters (focal length f, principal point coordinates) obtained based on Zhang Zhengyou's calibration method Using extrinsic parameters (baseline b, rotation matrix R, translation vector T), 3D coordinates are calculated using triangulation formulas. : , , ;

[0051] Step B2: 3D trajectory prediction:

[0052] Historical 3D keyframe sequences By timestamp Perform triaxial linear fitting to predict the next 10 frames. (j>n); every 10 frames will By projecting back onto the left and right views, we obtain Update the 640×640 ROI region to reduce the amount of detection computation and constrain the positioning range.

[0053] Frame loss compensation module (to ensure trajectory integrity):

[0054] Step C1: Missing Frame Identification:

[0055] Based on threshold (Displacement threshold, set to 50 pixels) (Speed ​​threshold, set to 200 pixels / second), if consecutive frames... Displacement or speed exceeding If a frame is detected as missing, a trajectory mask M is generated. This indicates a missed detection in the i-th frame. (Indicates normal)

[0056] Step C2: Interpolation compensation:

[0057] A U-shaped network (an encoder-decoder structure based on 1D convolution) is used as the input, consisting of 2D trajectory sequences before and after a frame loss. Using the mask M, shallow high-dimensional spatial features and deep semantic features are extracted, and interpolation is used to generate the 2D coordinates of the missing frames. , ;

[0058] Step C3: 3D trajectory integration:

[0059] Will , Mapped to 3D coordinates Simultaneously, trajectories are generated through spline interpolation. Integrate complete 3D trajectory: Where α = 0.6 (compensation trajectory weight). (3D displacement threshold), j is the sequence number of the nearest normal frame to the missing frame.

[0060] If you need to adapt to special scenarios (such as indoor lighting or outdoor rainy days), you can expand the dataset by following these steps: Collect 30 minutes of video (160fps) in the target scene, extract about 28,800 frames by taking 1 frame out of every 10 frames, label the 2D bounding boxes with LabelImg and label the 3D coordinates with Zhang Zhengyou method, perform data augmentation, divide the training / validation / test sets into 8:1:1, and retrain the algorithm model (3-5 days) to ensure that the scene accuracy is ≥85%.

[0061] The proposed YO-CSA detection network significantly improves the detection accuracy of small targets like badminton shuttlecocks by integrating contextual attention (CoT2f module) and spatial attention (SANeck, SGE decoupling head) on top of YOLOv8s. Experimental results show that YO-CSA achieves an mAP@0.75 of 90.43% on a custom dataset (32,539 frames, including multiple scenes), a 7.76% improvement over YOLOv8s (82.67%) and a 3.49% improvement over YOLO11s (86.94%). Simultaneously, it achieves a recall rate of 99.02%, effectively distinguishing badminton shuttlecocks from court lines, light reflections, and other distractions, thus addressing the insufficient accuracy of traditional models for small targets (single-frame pixel percentage ≤0.5%) and high-dynamic badminton shuttlecocks. Furthermore, ablation experiments verify that YO-CSA improves accuracy while reducing network GFLOPs from 23.6 in YOLOv8s to 20.9, achieving a balance between high accuracy and lightweight design.

[0062] This application ensures high real-time performance while maintaining high-precision detection through model lightweighting and hardware adaptation optimization. On one hand, YO-CSA itself adopts lightweight modules (such as the efficient design of SGE-2f and CoT2f), achieving an FPS of 243.9 without acceleration. On the other hand, after acceleration with ONNX+TensorRT, the single-frame inference time of YO-CSA is reduced from 6.67ms to 5.82ms, a speed improvement of 12.74%, while still maintaining 73.94% mAP@0.5:0.95 accuracy.

[0063] This application significantly improves the accuracy and consistency of 3D trajectories through end-to-end optimization of 2D detection-3D mapping-trajectory prediction. In the evaluation of 3D trajectory smoothness, the velocity smoothness using the complete strategy of this application (Strategy D: ROI constraint + 3D prediction + compensation module) is assessed. Acceleration smoothness It is far superior to strategies without prediction or compensation. And strategies with only ROI constraints The trajectory more closely approximates the actual motion patterns. Furthermore, based on a triangulation method using binocular stereo vision, combined with camera calibration and frame synchronization design, the 3D positioning error is significantly reduced. This meets the high-precision requirements of badminton robot interception (≤5cm) and 3D event analysis, overcoming the deficiency of traditional 2D detection solutions that lack 3D trajectory output.

[0064] The compensation module designed in this application (based on 2D temporal convolution TCN or 3D convolution) can 100% complete the missing frames, ensuring trajectory integrity. This module interpolates the 2D trajectory sequences and trajectory masks before and after the missing frame, generates the 2D coordinates of the missed frame, maps them to 3D coordinates, and then combines them with the trajectory generated by spline interpolation, using a fusion function. The data is integrated into a complete 3D trajectory. Experiments show that the compensation module can achieve 100% coverage for both missed detection frames due to occlusion and abnormal frames discarded due to spatiotemporal constraints, far exceeding the commercial ZED system (60% compensation rate) and the TrackNetV3+ monocular solution (85% compensation rate), avoiding trajectory breakage and eliminating the need for manual correction.

[0065] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A badminton tracking method based on three-dimensional vision, characterized by, Comprise: S1, the video data of badminton movement of left and right view is acquired by synchronous acquisition of binocular camera, and the time alignment left view frame sequence and right view frame sequence are obtained; S2, pre-processing the collected video data to obtain standardized left and right view input frame data wherein W represents frame width, H represents frame height, represents the number of input channels; S3, according to the input frame data , a 2D key frame detection model based on the improved YOLOv8 network is constructed to extract the 2D key frame coordinates of the left and right views and ; S4, 2D key frame coordinates of left and right views and , through stereo matching and trajectory prediction, badminton is positioned in 3D space to obtain 3D trajectory data; S5, according to 3D trajectory data, the broken trajectory is repaired by using frame missing compensation mechanism, and the final 3D trajectory is obtained; Wherein, according to 3D trajectory data, the broken trajectory is repaired by using frame missing compensation mechanism, comprising: According to the continuous frame coordinates and time information in 3D trajectory data, the interframe displacement and speed characteristics are calculated; Based on the preset displacement threshold and speed threshold, the discontinuous frame in 3D trajectory data is identified; According to the discontinuous frame, the trajectory mask indicating the frame missing position is generated; A 1D convolutional neural network including an encoder-decoder structure is established; The 3D coordinates in 3D trajectory data are mapped to left and right 2D view planes through projection transformation, and corresponding 2D trajectory coordinates are obtained; The 2D trajectory coordinates before and after the frame missing position are extracted as context information; The context trajectory information and trajectory mask are input into the 1D convolutional neural network; The compensation 2D coordinates of the frame missing position are output by the 1D convolutional neural network; Based on the compensation 2D coordinates, the compensation 3D coordinates are reconstructed by using the stereo matching method in S4; The normal frame sequence in 3D trajectory data is interpolated to generate a reference trajectory; According to the spatial distance relationship between the compensation 3D coordinates and the reference trajectory, an adaptive fusion strategy is used to integrate the trajectory data; The complete 3D trajectory sequence of the fusion of the original detection result and the compensation result is output.

2. The badminton tracking method based on three-dimensional vision according to claim 1, wherein: S3, a 2D key frame detection model based on an improved YOLOv8 network is constructed, comprising: A 2D key frame detection model based on a YOLOv8 network is constructed, a CoT2f module is used to improve a Backbone main network of the YOLOv8 network, and an improved Backbone network is used to perform feature extraction on input frame data to obtain enhanced feature maps; wherein, represents the number of feature channels after feature extraction. The Neck network of YOLOv8 network is improved by using SANeck structure, and multi-scale feature fusion is performed on enhanced feature maps by using the improved Neck network to obtain multi-scale fusion features; The head detection head of the YOLOv8 network is improved by using a decoupling head with an SGE, and the improved detection head is used for classification and regression operation on multi-scale fusion features to extract 2D key frame coordinates of a left view and 2D key frame coordinates of a right view .

3. The badminton tracking method based on three-dimensional vision according to claim 2, wherein: A 2D key frame detection model based on a YOLOv8 network is constructed, a CoT2f module is used to improve a Backbone main network of the YOLOv8 network, and an improved Backbone network is used to perform feature extraction on input frame data to obtain enhanced feature maps, comprising: input frame data input the YOLOv8 network Backbone backbone network embedded with the CoT2f module A double-branch processing mechanism is adopted in the CoT2f module: The input features are expanded in channels and transformed in features to generate intermediate features; The intermediate features are split into two parallel branches: the first branch is used for context feature extraction, and the second branch is used for residual connection; The first branch captures global-local context association through CoT-Bottleneck structure to enhance the feature expression ability for small targets; The features of the two branches are fused to combine the context enhanced information and the original feature information; Output The enhanced feature map enhances the feature expression ability of the badminton target by fusing local detailed features and global context information.

4. The badminton tracking method based on three-dimensional vision according to claim 2, wherein: The Neck network of YOLOv8 network is improved by using SANeck structure, comprising: The WxHxC2-dimensional enhanced feature map is input into the improved Neck network, wherein the SANeck structure is constructed by embedding SGE-2f spatial attention module in the C2f structure of YOLOv8 network; A double-branch processing mechanism is adopted in the SANeck structure: perform feature transformation and channel dimension splitting on the enhanced feature map to generate a first branch feature and a second branch feature ; The first branch feature The spatial attention enhancement processing is performed, background interference is suppressed through a plurality of SGE-2f modules connected in series, target features are enhanced, and the first branch feature after spatial attention enhancement is obtained ; Maintaining second branch features As original feature information; fusing the enhanced first branch feature with the second branch feature performing feature fusion; The output features of the SANeck structure at different depth layers are subjected to multi-scale aggregation processing; and multi-scale fusion features suitable for target detection at different scales are generated.

5. The badminton tracking method based on three-dimensional vision according to claim 4, characterized in that: The first branch feature Inputting n serial SGE-2f modules for spatial attention enhancement, comprising: For each SGE-2f module, the input features are divided into multiple spatial groups according to the spatial dimension, and each spatial group is processed separately; Global statistical features and local features of each spatial group are extracted; The difference value between the local features and the global statistical features is calculated, and the spatial attention weight is generated based on the difference value; For regions with a difference value lower than a preset threshold, an inhibition coefficient is applied to reduce the corresponding weight and inhibit the regular background such as the court line; For regions with a difference value higher than a preset threshold, an enhancement coefficient is applied to increase the corresponding weight and highlight the badminton target; The adjusted weight is subjected to element-wise multiplication with the input features; The processing results of each spatial group are integrated to obtain the output of the current module; Through the processing of the n SGE-2f modules in series, the target features are gradually enhanced, the background interference is suppressed, and finally the first branch features enhanced by spatial attention are output .

6. The badminton tracking method based on three-dimensional vision according to claim 4, characterized in that: extracting 2D keyframe coordinates of the left view and 2D keyframe coordinates of the right view comprising: Step A: The multi-scale fusion features are input into corresponding SGE-equipped decoupling heads, and each decoupling head includes an independent classification branch and a regression branch; Step B: In each decoupling head, the SGE spatial gating module is used to perform spatial attention enhancement on the input features: the scale features are subjected to feature dimension adjustment to obtain intermediate features; the global information of the intermediate features is extracted through global pooling operation to generate a spatial attention map; and the spatial attention map is subjected to feature modulation with the intermediate features to obtain spatial enhanced features; Step C: The spatial enhanced features are subjected to decoupling processing: the classification branch is used to perform classification operation on the spatial enhanced features to generate a class prediction feature map containing target classes and background classes; and the regression branch is used to perform regression operation on the spatial enhanced features to generate a bounding box regression feature map containing target position information; Step D: Confidence calculation is performed on the class prediction feature map to filter high-confidence detection results; Step E: Non-maximum suppression is performed on the filtered detection results to remove redundant detections; Step F: The bounding box parameters are converted into 2D coordinates in the image coordinate system; The above processing is performed on the input frame data of the left view and the right view respectively to extract the corresponding badminton center coordinates; Outputting time-aligned left and right view 3D keyframe coordinate pairs .

7. The badminton tracking method based on three-dimensional vision according to any one of claims 2 to 6, characterized in that: S4: The badminton is located in 3D space through stereo matching and trajectory prediction to obtain 3D trajectory data, including: A geometric model of the binocular vision system is established according to a preset camera intrinsic matrix and extrinsic matrix; The disparity value d is calculated based on the left and right view coordinates; The depth coordinate z is calculated based on the disparity value d, the focal length f, and the baseline distance b using the triangulation principle; According to the depth coordinate z and the 2D pixel coordinate, the 3D world coordinate is calculated by coordinate transformation ; Obtaining 3D world coordinates calculated from the current frame and n-1 previous frames, forming a historical 3D coordinate sequence ; wherein, denotes the timestamp of the first frame in the historical 3D coordinate sequence; denotes the timestamp of the last frame in the historical 3D coordinate sequence; The parameters of each coordinate axis in the historical sequence are fitted to establish a functional relationship between the coordinates and time; and the 3D trajectory points in the future preset number of frames are predicted based on the fitting parameters; The predicted 3D coordinates are mapped to left and right view planes through projection transformation; a region of interest (ROI) is generated with the predicted coordinates as the center; and the ROI information is fed back to the 2D detection model to optimize the detection range of the subsequent frame; The 3D trajectory data containing the current frame 3D coordinates, the historical 3D coordinate sequence and the predicted trajectory are output. 8.The badminton tracking method based on three-dimensional vision according to claim 7, characterized in that: S2, preprocessing the collected video data to obtain standardized left and right view input frame data comprising: The left view frame sequence and the right view frame sequence obtained in S1 are subjected to size normalization processing; For the size-normalized image, color space conversion is performed to convert the image data into a target format having channels. output pre-processed normalized left and right view input frame data . 9.A badminton tracking system based on three-dimensional vision, characterized in that, It comprises: At least one processing unit for executing instructions to implement the badminton tracking method based on three-dimensional vision according to any one of claims 1 to 8.

Images