A badminton detection and tracking method based on a time series coding network

Abstract

The application provides a badminton real-time detection and tracking method based on a time sequence coding and decoding network, relates to the fields of intelligent image processing and machine vision, and combines a channel attention mechanism and a time sequence network structure to design a badminton detection and tracking model based on a coding and decoding network architecture, recalibrates input features through the channel attention mechanism, and fully utilizes interframe information of image sequences through the time sequence network structure. The badminton real-time accurate detection and stable tracking of the badminton with motion blur in the image sequence can be realized in a complex background, the detection accuracy and tracking stability of the badminton are effectively improved, and the coding and decoding network is first applied to the badminton detection and tracking task. In addition, in order to improve the positioning accuracy of the badminton detection, the network uses a binary heat map contour detection method, avoids the defect that the positioning coordinate accuracy of an end-to-end network is insufficient, and further improves the accuracy of the badminton detection.

Description

Technical Field

[0001] This invention relates to the fields of intelligent image processing and machine vision, and in particular to a badminton shuttlecock detection and tracking method based on a temporal codec network. Background Technology

[0002] Badminton has become a global phenomenon, one of the most popular sports in the world. Simultaneously, tactical analysis and outcome assessment in badminton matches are receiving increasing attention. Among these, shuttlecock detection and tracking are core tasks, as detection information can assist in many operations, such as serving, hitting, and landing identification. Several sophisticated detection and tracking systems exist, such as the Hawk-Eye system, which uses multiple high-end cameras to acquire detection information about badminton movements. This information can assist referees in their decisions and aid athletes in professional analysis and training. However, this system is proprietary and has high deployment costs. Therefore, implementing badminton detection and tracking algorithms on low-frame-rate cameras has practical application value.

[0003] Badminton detection and tracking algorithms can be broadly categorized into traditional visual processing algorithms and deep learning algorithms based on convolutional neural networks. Traditional badminton detection and tracking methods primarily utilize optical flow. Optical flow leverages the temporal changes of pixels in a badminton match image sequence and the correlation between adjacent frames. Based on the correspondence between the previous and current frames, it calculates the motion information of objects between adjacent frames and then uses the characteristics of the badminton shuttlecock, such as its shape and color, for detection. However, many moving objects in badminton match video images share similar characteristics with the shuttlecock, such as the player's racket, shoes, socks, and clothing. These objects can be misidentified as the shuttlecock. Furthermore, the motion blur of the shuttlecock is severe under low frame rate cameras, which can significantly interfere with detection and tracking. In the field of deep learning, a common method is to track the target by detecting its position information in each frame. However, because this method obtains position information frame by frame, the correlation between consecutive frames is not fully utilized, making it prone to losing the target during tracking. Therefore, it is of practical significance to study an algorithm that can perform real-time accurate detection and stable tracking of badminton shuttlecocks in the presence of severe motion blur and complex backgrounds. Summary of the Invention

[0004] This invention provides a badminton shuttlecock detection and tracking method based on a temporal codec network to solve the problems mentioned in the background section. The specific technical solution is as follows:

[0005] A badminton shuttlecock detection and tracking method based on a temporal codec network, characterized by the following steps:

[0006] Step (1) Preprocess the image data; obtain continuous... … , A total of k RGB images are generated, and then the image sizes are adjusted and normalized. Finally, the images are superimposed in the channel dimension to form a normalized feature map with 3k channels, a height of h, and a width of w.

[0007] Step (2) Create the dataset required for model training; first, collect badminton match video images, divide the video images into training and validation sets according to the proportion, and decompose the video images to obtain image sequence data; then, mark the center of the badminton shuttlecock cap shape; at the same time, generate binary single-channel JPG heat map labels and store them in the document;

[0008] Step (3) Construct a temporal codec network model; the network model consists of an input layer, an encoding layer, a decoding layer, and an output layer.

[0009] The input layer adopts a temporal network structure. After obtaining k consecutive preprocessed feature maps that have been cropped and normalized, the input layer is combined with the temporal output F of the network model. hot (n-3k+1) to F hot (nk) are superimposed on the channels to form a normalized feature map with 5k channels, a height of h, and a width of w; if there are empty cases in the timing output, they are filled with a zero matrix;

[0010] The encoding layer consists of four feature extraction modules and three layers of two-dimensional max pooling downsampling operations. The kernel size of the three layers of two-dimensional max pooling downsampling operations is set to 2×2. The number of input channels and output channels of the four feature extraction modules are set to (5k, 32), (32, 64), (64, 128), and (128, 256) respectively. The three layers of two-dimensional max pooling downsampling are applied between the feature extraction modules. The first and second / fourth layers of feature extraction modules contain two basic convolutional modules, and the third and fourth / fourth layers contain three basic convolutional modules. Due to the three layers of downsampling, the height and width of the output feature map are compressed to 1 / 8 of the input feature map, thus achieving the compression encoding function. The input feature map passes through the encoding layer to obtain the encoded output.

[0011] The decoding layer consists of three feature extraction modules, three channel stacking operations, and three upsampling operations. The spatial multiplier for upsampling is set to 2, and the sampling method is nearest neighbor. The number of input and output channels for the three feature extraction modules 1 / 3, 2 / 3, and 3 / 3 are (384, 128), (192, 64), and (96, 32), respectively. The first / third feature extraction module contains three basic convolutional modules, and the others contain two. The encoded output first undergoes upsampling, and then is superimposed on the input features of the third / fourth feature extraction modules in the encoding layer to obtain the output features. Figure 1 Next, the features Figure 1 After upsampling, the output features are then superimposed on the input feature extraction module 2 / 3 along with the output of the feature extraction module 2 / 3 in the coding layer to obtain the output features. Figure 2 Then the features Figure 2 After upsampling, the output of the feature extraction module in the first / fourth layer of the coding layer is superimposed on the input feature extraction module 3 / 3 on the channel to finally obtain the decoded output;

[0012] The output layer first passes the decoded output through a two-dimensional convolution operation with 32 input channels and k output channels; then it passes through a sigmoid activation function, and finally outputs a normalized heatmap with k channels, h height and w width.

[0013] Step (4) Train the temporal encoder-decoder network model; set the data input path and hyperparameter information of the network model; set data loading to sequential sampling, and perform data preprocessing using the method in step (1); calculate the output heatmap loss using the binary cross-entropy loss function; perform iterative optimization using the Adam optimizer, and initialize the convolutional layer parameters using the Kaiming normal distribution; the learning rate parameter adopts a linear decreasing strategy, decreasing once after each training cycle, and the learning rate lr(n) in the nth round is:

[0014]

[0015] Where a is the initial learning rate, b is a small constant less than 1e-8, epochs is the total number of training iterations, and epoch is the current number of training iterations; during validation, contour detection is performed on the heatmap of the network model's predicted output and the real label to obtain the relative coordinates of the badminton shuttlecock, and the model weights with the highest accuracy on the validation set are saved during training.

[0016] Step (5) Real-time detection and tracking of badminton shuttlecocks: The temporal encoding and decoding network model obtained in steps (3) and (4) is used to detect real-time images of badminton matches and obtain the pixel coordinates of the badminton shuttlecocks.

[0017] In step (3), the feature extraction module is to input the feature map and then pass it through a basic convolution module to obtain the output y1. y1 passes through a channel attention module and N basic convolution modules to obtain y2. Finally, y1 and y2 are added pixel by pixel to achieve the function of feature extraction.

[0018] The basic convolution module consists of two-dimensional convolution operations, a ReLU activation function, and a group normalization sequence; wherein the kernel size of the two-dimensional convolution is 3×3.

[0019] The channel attention module consists of one layer of adaptive average pooling based on the width and height of the feature map, two fully connected layers, and ReLU and Sigmoid activation functions. After the feature map is input, it is first reduced to 1×1 by adaptive average pooling. Then it passes through a fully connected layer, a ReLU activation function, another fully connected layer, and a Sigmoid activation function in sequence. Finally, it is weighted channel by channel by matrix multiplication and applied to each channel of the input feature map to complete the channel attention recalibration of the input feature map.

[0020] The specific steps for badminton shuttlecock detection and tracking in step (5) are as follows:

[0021] Step (5.1) Obtain real-time images of the badminton match; Deploy a camera on one side of the badminton court to acquire video frames, and after the preprocessing shown in step (1), obtain a normalized feature map with 5k channels, a height of h, and a width of w.

[0022] Step (5.2) Load the temporal codec network model and perform inference; load the network model and training weights saved in step (4), perform model inference, and obtain k normalized heatmaps with single channel, height h and width w;

[0023] Step (5.3) binarizes the heatmap and detects the target contour to obtain the relative coordinates of the badminton shuttlecock in the pixel coordinate system. First, binarize k normalized heatmaps with a threshold t. Then, detect the bounding rectangle of the non-zero region in the binarized heatmap and calculate the minimum rectangle of the vertical boundary of each contour. Finally, use the center point coordinates of the bounding rectangle with the largest area as the pixel coordinates of the badminton shuttlecock. If all pixels in the binarized heatmap are equal to 0, it means that no badminton shuttlecock has been detected.

[0024] Step (5.4) outputs the detection results; the downsampled coordinates of the badminton shuttlecock obtained in real time in step (5.3) are mapped to the badminton shuttlecock coordinates in the original k-frame image; and the coordinate information is marked in the original image sequence frames by white solid circles, while the badminton shuttlecock coordinate information of the previous m frames of the frame is marked to achieve the visual tracking effect.

[0025] The beneficial effects of this invention are as follows:

[0026] (1) This invention uses a time-series coding and decoding network model to realize an algorithm for real-time accurate detection and stable tracking of badminton shuttlecocks in the presence of severe motion blur and complex background. This algorithm effectively reduces the stringent performance requirements of camera frame rate for badminton shuttlecock detection and tracking tasks.

[0027] (2) This invention introduces a temporal network structure and a channel attention mechanism. The temporal network structure can overlay the heat map output by the network model with the badminton match image input by the network model in a temporal manner, so that the network can effectively utilize the correlation information between the previous and next frames of the video, thereby avoiding the loss of the target caused by frame-by-frame detection. The channel attention mechanism allows the network model to learn and use global information, selectively emphasize informative features and suppress less useful features, effectively improving the accuracy of badminton detection and the stability of tracking.

[0028] (3) The method of introducing graphic contour detection into the model output avoids the disadvantage of low accuracy of end-to-end networks due to direct output of detection results. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the badminton shuttlecock detection and tracking method in this embodiment of the invention.

[0031] Figure 2 This is a schematic diagram of the temporal encoding and decoding network model of the badminton shuttlecock detection and tracking method in this embodiment of the invention.

[0032] Figure 3 This is a schematic diagram of the basic convolutional module of the temporal encoding and decoding network in an embodiment of the present invention.

[0033] Figure 4 This is a schematic diagram of the channel attention module of the timing codec network in an embodiment of the present invention.

[0034] Figure 5 This is a schematic diagram of the feature extraction module of the temporal encoding and decoding network in an embodiment of the present invention.

[0035] Figure 6 This is a schematic diagram showing the size of the output feature maps of each layer module of the temporal encoding and decoding network in an embodiment of the present invention.

[0036] Figure 7This is a visual output example of the badminton shuttlecock detection and tracking method in this embodiment of the invention.

[0037] Figure 8 This is the actual effect of adding a temporal network structure and a channel attention mechanism in the embodiments of the present invention. Detailed Implementation

[0038] To better understand the above-described objects, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention; however, the invention may be practiced in other ways different from those described herein, and therefore, the invention is not limited to the specific embodiments disclosed below.

[0039] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art described herein. The terms “first,” “second,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, terms such as “connected” or “linked” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, which change accordingly when the absolute position of the described object changes.

[0040] Example:

[0041] This embodiment presents a badminton shuttlecock detection and tracking method based on a temporal codec network. Its main workflow is as follows: Figure 1 As shown, it includes the following steps:

[0042] Step (1) Preprocess the image data; obtain continuous... , , A total of 3 RGB images are generated, where n represents the sequence number of the video frame; then the images are cropped to a height of 288 pixels and a width of 512 pixels, and each pixel value is divided by 255 to achieve normalization; finally, the normalized feature map with 9 channels, a height of 288, and a width of 512 is superimposed in the channel dimension.

[0043] Step (2) Create an image dataset; collect badminton match video images with different backgrounds from the internet and offline, divide the video images into training and validation sets in an 8:2 ratio, and decompose the collected badminton match video images to obtain JPG image sequence data; then, use the LabelMe annotation tool to annotate the center of the badminton shuttlecock cap shape in the image; next, obtain the relative coordinates of the shuttlecock in the corresponding image according to the LabelMe annotation JSON file, and then generate a binary single-channel JPG heatmap with a height of 288 pixels, a width of 512 pixels, and pixel values ​​of 0 and 255 according to the relative coordinates of the shuttlecock. Among them, the circular area with a radius of 8 pixels centered on the shuttlecock coordinates has a pixel value of 255, and the pixel value of other positions is 0; finally, store the images obtained from different badminton match video images and the corresponding binary heatmap label paths in a txt document of the training and validation data path set;

[0044] Step (3) Construct a temporal codec network model; the temporal codec network model is as follows: Figure 2 As shown; the network model consists of an input layer, an encoding layer, a decoding layer, and an output layer in sequence; the input layer uses a temporal network structure, the encoding layer contains 4 feature extraction modules and 3 downsampling operations, the decoding layer contains 3 feature extraction modules and 3 upsampling operations, and the output layer contains 1 convolution operation and 1 sigmoid activation function.

[0045] First, a feature extraction module is constructed that is used multiple times in both the encoding and decoding layers. The feature extraction module consists of a basic convolutional module and a channel attention module.

[0046] Basic convolutional modules such as Figure 3 As shown, the basic convolution module consists of two-dimensional convolution operation, ReLU activation function, and group normalization; the kernel size of the two-dimensional convolution is 3×3, the stride is 1, and the boundary padding is 1; if the number of input channels for group normalization is a multiple of 3, then the number of groups is 1 / 3 of the number of input channels; if the number of input channels for group normalization is a multiple of 4, then the number of groups is 1 / 4 of the number of input channels.

[0047] Channel attention module such as Figure 4 As shown, the channel attention module consists of one layer of adaptive average pooling based on the width and height of the feature map, two fully connected layers, and ReLU and Sigmoid activation functions. After the feature map is input, the spatial features are first reduced to 1×1 through adaptive average pooling. Then, fully connected layers, ReLU activation function, fully connected layers, and Sigmoid activation function are applied sequentially. Finally, matrix multiplication is used to weight each channel of the input feature map, completing the channel attention recalibration of the input feature map.

[0048] The feature extraction module, consisting of a basic convolutional module and a channel attention module, is as follows: Figure 5 As shown, firstly, the feature map is input and then passes through one basic convolutional module to obtain the output y1. y1 passes through one channel attention module and N basic convolutional modules to obtain y2. Finally, y1 and y2 are added pixel by pixel.

[0049] Construct an input layer; the input layer uses a temporal network structure; after obtaining three consecutive temporally cropped and normalized preprocessed feature maps, it is combined with the temporal output F of the network model. hot (n-8) to F hot (n-3) are superimposed on the channels to form a normalized feature map with 15 channels, a height of 288, and a width of 512; if there are empty cases in the timing output, they are filled with a zero matrix;

[0050] An encoding layer is constructed, consisting of four feature extraction modules and three layers of two-dimensional max-pooling downsampling operations. The kernel size of the three-layer two-dimensional max-pooling downsampling operations is set to 2×2, with a stride of 2. The number of input and output channels of the four feature extraction modules are set to (15, 32), (32, 64), (64, 128), and (128, 256) respectively, and the three-layer two-dimensional max-pooling downsampling interval is applied between the feature extraction modules. The first and second / fourth feature extraction modules contain two basic convolutional modules, and the third and fourth / fourth feature extraction modules contain three basic convolutional modules. Due to the three-layer downsampling, the height and width of the output feature map are compressed to 1 / 8 of the input feature map, thus achieving the compression encoding function. The input feature map passes through the encoding layer to obtain the encoded output.

[0051] A decoding layer is constructed; the decoding layer consists of 3 feature extraction modules, 3 channel stacking operations, and 3 upsampling operations; the spatial multiplier for the upsampling operation is set to 2, and the sampling method is nearest neighbor. The number of input channels and output channels of the 3 feature extraction modules 1 / 3, 2 / 3, and 3 / 3 are (384, 128), (192, 64), and (96, 32) respectively; the 1 / 3 feature extraction module contains 3 basic convolutional modules, and the rest contain 2; the encoded output first undergoes upsampling operation, and then is superimposed on the input feature extraction module 1 / 3 along with the output of the 3 / 4 feature extraction modules in the encoding layer to obtain the output feature. Figure 1 Next, the features Figure 1 After upsampling, the output features are then superimposed on the input feature extraction module 2 / 3 along with the output of the feature extraction module 2 / 3 in the coding layer to obtain the output features. Figure 2 Then the features Figure 2 After upsampling, the output of the feature extraction module in the first / fourth layer of the coding layer is superimposed on the channel of the input feature extraction module 3 / 3, and finally the decoded output is obtained.

[0052] The output layer is constructed as follows: First, the decoded output is passed through a 2D convolution with 32 input channels, 3 output channels, a stride of 1, and padding of 1. Then, it is passed through a sigmoid activation function, finally outputting a normalized heatmap with 3 channels, a height of 288, and a width of 512. The size of the output feature maps of each module of the network is as follows. Figure 6 As shown;

[0053] Step (4) Train the network model. Set the data input path and hyperparameter information of the network model; set the number of training rounds to 25, the initial learning rate to 1e-3, and the allowable relative error to be 1.5%, i.e., 5 pixels; set the data loading to sequential sampling, and use the method in step (1) for data preprocessing; use the binary cross-entropy loss function to calculate the output heatmap loss; use the Adam optimizer for iterative optimization, and use the Kaiming normal distribution to initialize the convolutional layer parameters; the learning rate parameter adopts a linear decreasing strategy, decreasing once after each training cycle, and the learning rate lr(n) of the nth round is:

[0054]

[0055] Where a is the initial learning rate, b is a small constant less than 1e-8, epochs is the total number of training iterations, and epoch is the current number of training iterations; during validation, contour detection is performed on the heatmap of the network model's predicted output and the real label to obtain the relative coordinates of the badminton shuttlecock, and the accuracy, precision, and recall of the validation set are statistically analyzed. During training, the model weights with the highest accuracy of the validation set are saved.

[0056] Step (5) Real-time detection and tracking of badminton shuttlecocks; using the temporal encoding and decoding network model obtained in steps (3) and (4), detect real-time images of badminton matches and obtain the pixel coordinates of the shuttlecocks; the specific steps are as follows:

[0057] Step (5.1) Obtain real-time images of the badminton match; Deploy cameras at appropriate locations around the badminton match venue so that the camera's field of view can cover the entire badminton court; In this embodiment, a camera with a resolution of 3840×2160 and a refresh rate of 25FPS is used; Obtain video frames and obtain normalized feature maps through the preprocessing shown in step (1), resulting in a normalized feature map with 15 channels, a height of 288, and a width of 512;

[0058] Step (5.2) Load the temporal codec network model and perform inference; load the network model training weights saved in step (4), perform model inference, and obtain F. hot (n-2), F hot (n-1), F hot(n) A total of 3 normalized heatmaps with a single channel, a height of 288, and a width of 512, where n represents the sequence number of the video frame;

[0059] Step (5.3) binarizes the heatmap and detects the target contour to obtain the relative coordinates of the badminton shuttlecock in the pixel coordinate system. First, binarize the three normalized heatmaps with a threshold of 0.5. Then, detect the bounding rectangle of the non-zero region in the binarized heatmap and calculate the minimum rectangle of the vertical boundary of each contour. Then, use the center point coordinates of the bounding rectangle with the largest area as the pixel coordinates of the badminton shuttlecock. If all pixels in the binarized heatmap are equal to 0, it means that no badminton shuttlecock has been detected.

[0060] Step (5.4) outputs the detection results; the downsampled coordinates of the badminton shuttlecock obtained in real time in step (5.3) are mapped to the badminton shuttlecock coordinates in the original 3-frame images; and the coordinate information is marked in the original image sequence frames using white solid circles, while also marking the badminton shuttlecock coordinate information of the previous 7 frames of this frame to achieve visual tracking effect. The results are as follows. Figure 7 As shown; finally, the results are output to the display terminal according to the video frame rate to present the real-time video tracking effect.

[0061] Under a hardware environment with an Intel Core i5 12600kf CPU and an NVIDIA GeForce RTX 3060 graphics card, the detection accuracy and detection time of the badminton shuttlecock detection and tracking method based on a time-series codec network were tested, and the tracking stability was verified in practice. The results are as follows:

[0062] 1. Accuracy test. In the network training in step (4), the allowable error between the predicted and actual values ​​of the badminton ball coordinates is 1.5%, which is 5 pixels. The training and testing use the Google Badminton Public Dataset, which contains 130 badminton match videos with different backgrounds, and a total of 66,000 training images and 12,000 verification images. The number of training iterations is set to 25, and the initial learning rate is set to 1e-3. The following table shows the test results of the accuracy, precision, and recall of badminton ball detection.

[0063] Table 1. Detection accuracy test results

[0064] accuracy Precision Recall rate 91.5% 93.8% 95.9%

[0065] 2. Detection Time Test. After acquiring badminton match video image data from the camera, the time required to continuously detect 300 frames of images until the terminal displays the results was recorded. The time required to detect one frame was then calculated. Specifically, a video containing badminton shuttlecocks and a video without badminton shuttlecocks were input, and the detection and visualization were performed on each. The experimental results are shown in the table below. In this embodiment, a 25FPS camera was selected, while the algorithm's processing speed can reach at least 71FPS, thus enabling real-time detection and tracking of badminton shuttlecocks.

[0066] Table 2 Detection Time Test Results

[0067] Video Image Categories Average processing time per frame No badminton shuttlecocks 10.0 ms Includes badminton shuttlecock 14.3 ms

[0068] 3. Tracking stability verification. Badminton detection is performed on the same pre-prepared video by comparing it with trained models that do not contain temporal network structures. For example... Figure 8 As shown, when there are multiple badminton targets within the rectangular frame in the figure, adding a temporal network structure can effectively utilize the information between consecutive frames, avoid losing the tracking target due to frame-by-frame detection, and effectively increase the tracking stability of the target badminton.

[0069] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A badminton shuttlecock detection and tracking method based on a temporal codec network, characterized in that, The method includes the following steps: Step (1) Preprocess the image data; obtain k consecutive frames of images, then adjust the image size and perform normalization; finally, stack them in the channel dimension to form a normalized feature map with 3k channels, a height of h, and a width of w. Step (2) Create the dataset required for model training; first, collect badminton match video images, divide the video images into training and validation sets according to the proportion, and decompose the video images to obtain image sequence data; then, mark the center of the badminton shuttlecock cap shape; at the same time, generate binary single-channel JPG heat map labels and store them in the document; Step (3) Construct a temporal codec network model; the network model consists of an input layer, an encoding layer, a decoding layer, and an output layer in sequence; The input layer adopts a temporal network structure. After obtaining k consecutive preprocessed feature maps that have been cropped and normalized, the input layer is combined with the temporal output F of the network model. hot (n-3k+1) to F hot (nk) are superimposed on the channels to form a normalized feature map with 5k channels, a height of h, and a width of w; if there are empty cases in the timing output, they are filled with a zero matrix; The encoding layer consists of four feature extraction modules and three layers of two-dimensional max pooling downsampling operations. The kernel size of the three layers of two-dimensional max pooling downsampling operations is set to 2×2. The number of input channels and output channels of the four feature extraction modules are set to (5k, 32), (32, 64), (64, 128), and (128, 256) respectively. The three layers of two-dimensional max pooling downsampling are applied between the feature extraction modules. The first and second / fourth layers of feature extraction modules contain two basic convolutional modules, and the third and fourth / fourth layers contain three basic convolutional modules. Due to the three layers of downsampling, the height and width of the output feature map are compressed to 1 / 8 of the input feature map, thus achieving the compression encoding function. The input feature map passes through the encoding layer to obtain the encoded output. The decoding layer consists of three feature extraction modules, three channel stacking operations, and three upsampling operations. The spatial multiplier for upsampling is set to 2, and the sampling method is nearest neighbor. The number of input and output channels for the three feature extraction modules 1 / 3, 2 / 3, and 3 / 3 are (384, 128), (192, 64), and (96, 32), respectively. The first / third layer feature extraction module contains three basic convolutional modules, and the rest contain two. The encoded output first undergoes upsampling, and then is stacked on the channel with the output of the third / fourth layer feature extraction module in the encoding layer, inputting to feature extraction module 1 / 3 to obtain output feature map 1. Next, feature map 1 undergoes upsampling, and then is stacked on the channel with the output of the second / fourth layer feature extraction module in the encoding layer, inputting to feature extraction module 2 / 3 to obtain output feature map 2. Feature map 2 then undergoes upsampling, and then is stacked on the channel with the output of the first / fourth layer feature extraction module in the encoding layer, inputting to feature extraction module 3 / 3, finally obtaining the decoded output. The output layer first passes the decoded output through a two-dimensional convolution operation with 32 input channels and k output channels; then it passes through a sigmoid activation function, and finally outputs a normalized heatmap with k channels, a height of h, and a width of w. Step (4) Train the temporal encoder-decoder network model; set the data input path and hyperparameter information of the network model; set data loading to sequential sampling, and perform data preprocessing using the method in step (1); calculate the output heatmap loss using the binary cross-entropy loss function; perform iterative optimization using the Adam optimizer, and initialize the convolutional layer parameters using the Kaiming normal distribution; the learning rate parameter adopts a linear decreasing strategy, decreasing once after each training cycle, and the learning rate lr(n) in the nth round is: ； Where a is the initial learning rate, b is a small constant less than 1e-8, epochs is the total number of training iterations, and epoch is the current number of training iterations; during validation, contour detection is performed on the heatmap of the network model's predicted output and the real label to obtain the relative coordinates of the badminton shuttlecock, and the model weights with the highest accuracy on the validation set are saved during training. Step (5) Real-time detection and tracking of badminton shuttlecocks: The temporal encoding and decoding network model obtained in steps (3) and (4) is used to detect real-time images of badminton matches and obtain the pixel coordinates of the badminton shuttlecocks.

2. The method according to claim 1, characterized in that, In step (3), the feature extraction module is to input the feature map and then pass it through a basic convolution module to obtain the output y1. y1 passes through a channel attention module and N basic convolution modules to obtain y2. Finally, y1 and y2 are added pixel by pixel to achieve the function of feature extraction. The basic convolution module consists of two-dimensional convolution operations, a ReLU activation function, and a group normalization sequence; wherein the kernel size of the two-dimensional convolution is 3×3. The channel attention module consists of one layer of adaptive average pooling based on the width and height of the feature map, two fully connected layers, and ReLU and Sigmoid activation functions. After the feature map is input, it is first reduced to 1×1 by adaptive average pooling. Then it passes through a fully connected layer, a ReLU activation function, another fully connected layer, and a Sigmoid activation function in sequence. Finally, it is weighted channel by channel by matrix multiplication and applied to each channel of the input feature map to complete the channel attention recalibration of the input feature map.

3. The method according to claim 1, characterized in that, The specific steps for badminton shuttlecock detection and tracking in step (5) are as follows: Step (5.1) Obtain real-time images of the badminton match; Deploy a camera on one side of the badminton court to acquire video frames, and after the preprocessing shown in step (1), obtain a normalized feature map with 5k channels, a height of h, and a width of w. Step (5.2) Load the temporal codec network model and perform inference; load the network model and training weights saved in step (4), perform model inference, and obtain k normalized heatmaps with single channel, height h and width w; Step (5.3) binarizes the heatmap and detects the target contour to obtain the relative coordinates of the badminton shuttlecock in the pixel coordinate system. First, binarize k normalized heatmaps with a threshold t. Then, detect the bounding rectangle of the non-zero region in the binarized heatmap and calculate the minimum rectangle of the vertical boundary of each contour. Finally, use the center point coordinates of the bounding rectangle with the largest area as the pixel coordinates of the badminton shuttlecock. If all pixels in the binarized heatmap are equal to 0, it means that no badminton shuttlecock has been detected. Step (5.4) outputs the detection results; the downsampled coordinates of the badminton shuttlecock obtained in real time in step (5.3) are mapped to the badminton shuttlecock coordinates in the original k-frame image; and the coordinate information is marked in the original image sequence frames by white solid circles, while the badminton shuttlecock coordinate information of the previous m frames of the frame is marked to achieve the visual tracking effect.

Images