Shuttlecock detection and tracking method based on deep learning

Abstract

The invention discloses a shuttlecock detection and tracking method based on deep learning, and relates to the technical field of computer vision, and the method comprises the following modules: an image sequence collection module, which is used for deploying a camera device at a side line of a badminton court to capture an image sequence in real time and transmitting the image sequence to an edge device; the image sequence preprocessing module is used for carrying out normalization processing on the image sequence and scaling a pixel value to a range from 0 to 1; the real-time detection module is used for extracting badminton features through a neural network, refining a feature focusing area layer by layer by using a multi-layer attention encoder, and outputting bounding box parameters of the detected badminton; and the target tracking module is used for extracting the feature vector of the detection frame, then matching to generate a track, entering a data-physical dual-drive architecture to obtain a preliminary coordinate, and finally obtaining a final prediction coordinate through a self-adaptive fusion module. According to the badminton detecting and tracking method based on deep learning, the badminton detecting precision is improved, and the problem of badminton tracking robustness under various emergencies can be solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a badminton detection and tracking method based on deep learning. Background Art

[0002] Badminton is a competitive sport that combines speed, power, skill, and strategy. It demands flexible and complex technical and tactical skills. Meeting these demands requires more scientific and comprehensive training methods and a more intelligent technical and tactical analysis platform. During badminton competitions and training, the shuttlecock's trajectory, speed, and landing point vary greatly. Therefore, one of the keys to meeting these technical and tactical requirements is shuttlecock detection and position tracking.

[0003] Traditional methods for detecting and tracking badminton balls may lack speed, accuracy, and robustness. For example, a high-speed shuttlecock typically occupies only a very small number of pixels in an image (e.g., less than 10×10 pixels), and is accompanied by motion blur and occlusion. Traditional detection methods (such as YOLO and Faster R-CNN) are prone to missed or false detections. Furthermore, the difficulty in tracking a badminton ball lies in temporary occlusions (such as those caused by the racket) and sudden changes in motion (such as the shuttlecock's sudden change of direction after being struck). Traditional Kalman filtering relies on linear assumptions and struggles to handle nonlinear motion. Summary of the Invention

[0004] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a badminton detection and tracking method based on deep learning, aiming to solve the problems of low detection accuracy and poor tracking robustness in the existing badminton detection and tracking process.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for detecting and tracking badminton based on deep learning, comprising the following steps:

[0007] Image sequence acquisition: Deploy cameras on the sidelines of the badminton court, adjust the pitch angle to cover the entire court, and connect the cameras to edge computing devices. The cameras capture image sequences in real time and transmit them to the edge devices.

[0008] Image sequence preprocessing: normalize the image and scale the pixel values to the range [0,1], where μ and σ are the mean and standard deviation of the original data set respectively;

[0009] .

[0010] Real-time detection: Based on the detection results of the previous frame, the dynamic ROI is cropped to the 200×200 pixel search area of the current frame. If it is the first frame, the pixel search area of the current frame is directly cropped;

[0011] Input the cropped ROI image to the HRNet-W18 backbone network, which is designed to preserve high-resolution features and output 4-level feature maps, i.e., images with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size;

[0012] Applying the Deformable Transformer encoder on the highest resolution feature map, the encoder can dynamically focus on the badminton motion area, where is the learnable offset, is the attention weight, calculated as:

[0013] DeformAtt(Q,K,V) = ;

[0014] After the enhanced features are input into the detection head, the target position and confidence level are output. The target position is output in the form of a detection frame. If the detection frame speed is greater than 50m / s or the acceleration is abnormal, it is judged as a false detection and filtered.

[0015] Target tracking: Extract the detection frame features of the current frame and calculate the similarity matrix with the historical trajectory features. Then use the Hungarian algorithm to achieve optimal matching. For unmatched detection frames, if they appear in the pixel area with a displacement of less than 5 for three consecutive frames, a new trajectory is created.

[0016] =cos(ReID( ),ReID( ));

[0017] The tracking module is mainly divided into data-driven flow and physical-driven flow. The former inputs the historical 5-frame coordinate sequence to the LSTM network and outputs the position offset of the next 3 frames. For physical-driven prediction, we established an aerodynamic model, the key of which is based on the current speed. and location , solve the following differential equation:

[0018] = + + ;

[0019] After obtaining the prediction results of data-driven and physical-driven respectively, the adaptive fusion module is used to dynamically fuse the prediction results of the two driving flows to achieve accurate tracking of high-speed moving badminton.

[0020] The adaptive fusion module dynamically adjusts the weights of the data and physical drive prediction results based on the target's real-time motion state. The final predicted coordinates are the weighted sum of the two results.

[0021] = + (1- ) ;

[0022] If the badminton is occluded during its motion and the target is not detected for two consecutive frames, the physical model is enabled for continuous prediction, which lasts for up to five frames, and the search area is expanded around the predicted position. If there is still no match, the target is considered lost.

[0023] Based on this method, the pre-processed data passes through multiple modules to generate real-time data: velocity v, acceleration a, trajectory curvature k, and the determined badminton position visualization output: the detection box, predicted trajectory, and velocity vector diagram are superimposed on the original image.

[0024] In the technical solution of the present invention, the HRNet-W18 network model is pre-trained on the data set to learn the general image feature extraction capability, the Transformer encoder part is pre-trained, the weights of the deformable attention layer are retained, the query vector is reset to adapt to the target characteristics of the badminton, the moving objects in the video scene are detected, the badminton target is determined, and the current frame detection box features are extracted and input into the physical-data dual-drive module to track the badminton.

[0025] In general, the present invention has the following beneficial effects:

[0026] The method of the present invention adopts a CNN-Transformer hybrid feature extraction network to dynamically focus on the badminton motion area, realizes the coordinated optimization of local feature extraction and global context modeling, and significantly improves the detection accuracy in real-time badminton detection.

[0027] The physics-data dual-driven tracking method provided by the present invention integrates LSTM time series prediction and aerodynamic equations to deal with the tracking robustness problem caused by short-term occlusion and motion mutation of the badminton, and can adapt to badminton detection and tracking in various special situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a system diagram of a badminton detection and tracking method based on deep learning in the present invention;

[0029] Figure 2 Schematic diagram of a real-time detection method for badminton detection and tracking based on deep learning in the present invention;

[0030] Figure 3 Schematic diagram of target tracking of a badminton detection and tracking method based on deep learning in the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] The core of the present invention is to provide a badminton detection and tracking method based on deep learning, which can overcome the various disadvantages of traditional methods to achieve the goal of improving the accuracy of badminton detection and tracking.

[0033] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] The first embodiment of the present invention provides a method for detecting and tracking badminton based on deep learning, and the system diagram thereof is shown in FIG. Figure 1 As shown, the following steps are included:

[0035] 100: image sequence acquisition module;

[0036] 200: image sequence preprocessing module;

[0037] 300: real-time detection module;

[0038] 400: target tracking module;

[0039] In this embodiment, the image sequence acquisition module requires the deployment of a camera device on the sideline of the badminton court, adjusting the pitch angle to cover the entire court, and connecting the camera to the edge computing device. The camera captures the image sequence in real time and transmits it to the edge device, that is, entering the image sequence preprocessing module. After the image is normalized and the pixel value is scaled to the range of [0,1], it will be transmitted to the real-time detection module. This module is mainly divided into three areas. The front area network extracts the badminton features, the middle area is responsible for spatiotemporal modeling, stacking multiple layers of deformable attention encoders, and refining the feature focus area layer by layer. The back area detects the badminton and outputs the bounding box parameters, and then enters the target tracking module. This module adopts a "data-physical dual-drive architecture". After extracting the multi-dimensional feature vector of the detection box, the trajectory is generated by matching and enters the data-driven predictor and the physical-driven predictor respectively to obtain two position results, and finally enters the adaptive fusion to output the final prediction result.

[0040] The second embodiment of the present invention provides a real-time detection module and a target tracking module method for a badminton detection and tracking method based on deep learning, wherein the real-time detection module schematic diagram is as follows: Figure 2 As shown, compared with the previous embodiment, this embodiment first specifically describes the real-time detection module technology:

[0041] This example addresses the characteristics of small targets in high-speed badminton, such as those smaller than 10×10 pixels, motion blur, and background interference. To preserve high-resolution features, avoid loss of detail and dynamic focus during downsampling, and accurately capture the trajectory and direction of high-speed targets, we designed a hybrid HRNet+Deformable Transformer network architecture. The processing is divided into three stages:

[0042] In this embodiment, the HRNet network can output a feature map with 1 / 4 original image resolution and gradually increase the downsampling branches while always retaining a 1 / 4 high-resolution main branch. In cross-stage feature fusion, through repeated multi-scale information exchange, the semantic consistency of features at different scales is enhanced. After multiple downsampling, high-resolution details can still be retained, and the response strength of small target features is significantly improved.

[0043] In this embodiment, the deformable Transformer encoder only uses the 1 / 4 resolution feature map output by HRNet for input feature processing to avoid loss of details of high-level features, flattens the feature map into a sequence, and embeds positional encoding. A deformable attention layer is placed on the encoder, which evenly generates grid points on the feature map as initial sampling reference points. Each query predicts the offsets Δx and Δy of K sampling points. The offset range is limited to ±16 pixels to avoid excessive divergence. The offset positions are then bilinearly interpolated and sampled, and the weighted sum is used to generate enhanced features. The deformable attention encoder stacks 6 layers to refine the feature focus area layer by layer.

[0044] In this embodiment, the last layer of the target detection module is the detection head, which receives the sequence features output by the Transformer encoder and compresses the features, retaining key information, and then outputs the bounding box parameters and confidence scores. Finally, it filters once, and only the detection boxes with confidence > 0.7 can enter the subsequent processing. The bounding box here is displayed in normalized coordinates, and the box contains the coordinates of the badminton detected by the module.

[0045] In this embodiment, the target detection module schematic diagram is as follows: Figure 3 As shown, the following is a detailed description of the module technology:

[0046] In this invention, the target tracking module contains a spatiotemporal attention associator, which extracts the feature vector of the detection box through the network and normalizes it. Then, it calculates the similarity matrix with the historical trajectory features to obtain the cosine similarity and the IoU distance between the predicted position and the current detection box based on the Kalman filter. The weights are then assigned to calculate the joint score. The above calculation formula is as follows:

[0047] = IoU ;

[0048] S = + (1 - ) ;

[0049] In this embodiment, the optimal matching between detection frames is achieved through the Hungarian algorithm, and unreliable associations with a score of less than 0.4 are eliminated. Unmatched detection frames must appear in the same area for three consecutive frames (with a displacement of less than 10 pixels) before they are initialized as new tracks.

[0050] In this embodiment, the data-driven predictor selects the 3D coordinates and velocities of the most recent five frames, normalizes the coordinates to the court space, and then inputs them into the LSTM network to output the coordinate offsets for the next three frames. The physics-driven predictor builds a model based on aerodynamics and solves differential equations. Accordingly, the predictor outputs the coordinate offsets for the next three frames. The prediction results of the two predictors are input into the adaptive fusion module, which dynamically adjusts the weights of the data and physics predictor results based on the real-time motion state. The final predicted coordinates are the weighted sum of the two results.

[0051] The above describes in detail the deep learning-based badminton detection and tracking method provided by the present invention. This embodiment is merely an exemplary embodiment of the technical solution of the present invention and does not limit the scope of protection of the present invention. Based on the core design concept of the present invention, those skilled in the art may make changes in the specific implementation and scope of application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A badminton detection and tracking method based on deep learning, characterized in that: It comprises an image sequence acquisition module (100), an image sequence preprocessing module (200), a real-time detection module (300), and a target tracking module (400), wherein: The image sequence acquisition module (100) deploys a camera device at the sideline of the badminton court to capture the image sequence in real time and transmits it to the edge device; The image sequence preprocessing module (200) performs normalization processing on the image and scales the pixel values to the range of [0, 1]; The real-time detection module (300) extracts badminton features through a deep learning neural network, then uses a multi-layer deformable attention encoder to refine the feature focus area layer by layer, and then outputs the bounding box parameters of the detected badminton; The target tracking module (400) extracts the multi-dimensional feature vector of the detection frame and generates a trajectory through matching, enters the data-physical dual-drive architecture, and obtains two position results respectively, and finally obtains the final predicted coordinates through the adaptive fusion module.

2. The method according to claim 1, characterized in that The implementation of the real-time detection module (300) comprises: A monocular high-speed camera is used to capture target motion video streams, and the HRNet backbone network is used to extract multi-scale feature maps to retain high-resolution detail information. To avoid loss of high-level feature details on the 1 / 4 resolution feature map output by HRNet, the feature map is flattened into a sequence and embedded into positional encoding; Deploy a deformable Transformer encoder on the multi-scale feature map to focus on the edges and trajectory directions of high-speed moving targets by dynamically adjusting the positions of attention sampling points; Generate grid points uniformly on the feature map as initial sampling reference points, and predict the dynamic offsets of K sampling points for each query vector, with the offset range constrained to ±16 pixels; The offset position features are obtained through bilinear interpolation, and the enhanced features are generated by weighted aggregation.

3. The method according to claim 1, characterized in that The target tracking module (400) is implemented in such a way that: The target tracking module (400) contains a spatiotemporal attention associator, which extracts the feature vector of the detection frame through the network and normalizes it. Then, it calculates the similarity matrix with the historical trajectory features to obtain the cosine similarity and the IoU distance between the predicted position and the current detection frame based on the Kalman filter, and then distributes the weights to calculate the joint score; The target tracking module (400) combines the LSTM time series prediction model with the solution of the aerodynamic differential equation to construct a physical-data dual-driven tracking engine. The aerodynamic differential equation is specifically expressed as: = + + ； in The drag coefficient is set to 0.6, m is the mass of the badminton, is the air density, A is the windward area; The physical model weights are dynamically generated based on real-time velocity and acceleration, and the two prediction results are weighted and fused to output the target trajectory and motion parameters.

4. The method according to claim 1, wherein The process of outputting the bounding box parameters of the detected badminton, where: The last layer of the target detection module (300) is the detection head, which receives the sequence features output by the Transformer encoder and compresses the features, retains the key information, and then outputs the bounding box parameters and confidence scores, and finally filters them once; Only detection boxes with a confidence level greater than 0.7 can enter subsequent processing. The bounding box here is displayed in normalized coordinates, and the coordinates of the badminton detected by the module are within the box.

5. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of any method described in claims 1-4 are implemented, including processes such as deformable attention dynamic focusing and dual-drive trajectory prediction.

Images