Badminton motion detection algorithm based on improved YOWO
By introducing CAA attention mechanism and 3D depth separation convolution in the YOWO network, the challenges of existing spatiotemporal action detection methods in complex action tube generation and computing resource consumption are solved, and more efficient and accurate action detection is achieved.
Patent Information
- Application Number
- CN202411729562.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-02-25
AI Technical Summary
The existing spatiotemporal action detection methods have challenges in dealing with complex 3D action tube generation and high computing resource consumption, resulting in insufficient detection accuracy and speed, making it difficult to achieve real-time application.
A badminton movement action detection algorithm based on improved YOWO is proposed. By introducing a CAA attention mechanism into the YOWO network and adding 3D depth separation convolution and residual connections before the pooling layer, the network's ability to extract spatiotemporal features is improved.
It significantly improves the accuracy and efficiency of action detection, enhances the model's ability to extract shallow and deep features, reduces computing resource consumption, and achieves faster detection speed.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a badminton motion detection algorithm based on improved YOWO, in particular, utilizing an improved CAA attention mechanism to enhance the accuracy and efficiency of motion detection. Background Art
[0002] The core task of spatiotemporal action detection is to detect the action category of the target in the video and determine the time and location of the action. This field occupies an important position in computer vision research. In recent years, deep learning has made significant progress in action recognition, and researchers are no longer satisfied with basic coarse-grained action recognition. Nowadays, many related studies combine action detection with sports events, and use deep learning to achieve intelligent detection of key actions in sports videos.
[0003] Existing spatiotemporal action detection methods are mainly divided into two-stage and 3D convolution-based networks, and have made significant progress in recent years. However, since the generation of action tubes is composed of bounding boxes across frames, this process is more complicated and time-consuming than the 2D case. The classification performance is highly dependent on these proposals, and the detected bounding boxes may not be the best choice for subsequent classification tasks. Although 3D convolution performs well in extracting temporal information, it consumes huge computational resources and has high running time cost, so it becomes particularly important to lightweight it for real-time applications. Therefore, improving the accuracy and speed of spatiotemporal action detection is of great research value. Summary of the invention
[0004] In order to solve the above technical problems, the present invention proposes a badminton motion detection algorithm based on improved YOWO. The invention is based on the YOWO network, which inputs the RGB video continuous frames and RGB key frame information into the temporal detection module and the spatial detection module for processing respectively; in the temporal detection module, the attention mechanism CAA module is introduced to enhance the sensitivity of the model to channel features, and at the same time, continuous 3D depth separable convolution and residual connection are added before the pooling layer of the CAA attention mechanism, so that the network pays more attention to the shallow and deep feature information, ensuring that the key information is retained, thereby enhancing the spatiotemporal feature extraction capability of the 3D convolutional network; in the spatial detection module, the Darknet-19 network is used for spatial feature extraction, and then the spatiotemporal features and spatial features are feature fused, and finally these fused features are detected to obtain the category, duration and location information of the target behavior of the video. In order to achieve the above purpose, the technical scheme of the present invention is as follows.
[0005] A badminton motion detection algorithm based on improved YOWO includes the following steps.
[0006] Step 1: Obtain a badminton motion detection video dataset, and modify the badminton motion dataset according to the AVA dataset annotation format so that it can run successfully in the YOWO network structure.
[0007] Step 2: In the time series detection model, 3D-Resnext-101 is used to extract the spatiotemporal features of 30 consecutive frames of badminton motion. In order to improve the model's sensitivity to channel features and deeper features, the CAA attention mechanism and continuous 3D depth-separable convolution are introduced.
[0008] Step 3: In the spatial detection model, Darknet-19 is used as the basic network of the spatial feature extraction module to extract the badminton motion features of key frames.
[0009] Step 4: Use the self-attention mechanism CFAM to perform channel fusion on the extracted spatiotemporal features, and finally regress the bounding box and output the detection results.
[0010] The specific situation in the step 1 is as follows.
[0011] (1) Obtain a public badminton dataset (VideoBadminton) and divide the video to be tested into 30 consecutive frames.
[0012] (2) Use YOLOV5 to identify the target person in each frame and delete the label boxes of irrelevant people.
[0013] (3) Use the VIA annotation tool to manually annotate the target person's actions, annotate the target person's actions and spatial position information in each frame; process the action labels annotated by experts as network labels. According to the expert annotations, eight basic actions are classified as follows: Jump Smash, Overhead Clear, Backhand Drive, Forehand Drive, Backhand Lift, Forehand Lift, Forehand Net Shot, and Backhand Net Shot. Based on the eight basic actions and the actions in the video, 10 common postures are summarized: Up, Down, Sidepart, Flat part, Turn around, Front posture, Stride, Jump, Bend over, and Standing.
[0014] (4) The images cropped from the video and the coordinates detected by YOLOV5 are sent to Deepsort for detection to associate the ID of the target person.
[0015] The specific situation in the step 2 is as follows.
[0016] The 3D CNN branch is used to extract timing features, and the timing information of the previous and next frames of the video is analyzed through the 3D-Resnext-101 network model.
[0017] (1) In order to improve the model’s sensitivity to channels and increase the time series for consecutive frames, the CAA attention mechanism is added to the time series to make it a three-dimensional module.
[0018] (2) Based on the idea of connecting multiple convolutions and a convolution with a pooling layer in the C3D network, this idea is applied to the CAA attention mechanism. Continuous 3D depthwise separable convolution and residual connection are used before the pooling layer. This method can extract deeper features and retain more key information.
[0019] The specific situation in the step three is as follows.
[0020] The 2D CNN branch is used to extract spatial features, and the spatial information of the target person is analyzed through the Darknet-19 network model. The Darknet-19 network model achieves effective learning of features by alternating between 3*3 and 1*1 filters. The initial feature size of the input is C*H*W, and features are extracted through multiple convolutional modules consisting of 32, 64, 128, 256, 512, 1024 and a specific output filter number of 4. The maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolutional modules.
[0021] The specific situation in the step 4 is as follows.
[0022] The outputs of the two-dimensional and three-dimensional networks have the same shape in the last two dimensions, and the channel stacking feature is used to fuse the two feature maps. The information is then passed as input to the CFAM module. The self-attention mechanism can highlight the relationship between the previous and next frames and enhance the distinguishability of the features. Finally, following the idea of YOLO on bounding box regression, the last layer uses a 1*1 convolution kernel to generate the target channel number and output the target person's coordinate box information, behavior category, and confidence.
[0023] Compared with the prior art, the technical solution of the present invention has the following beneficial effects.
[0024] (1) The present invention changes the CAA attention mechanism from two-dimensional to three-dimensional and adds a 3D CNN branch with a YOWO network model, which not only enhances the network's sensitivity to channels but also makes the network pay more attention to the time dimension and strengthens the connection between previous and next frames.
[0025] (2) Adding continuous 3D depth-wise separable convolutions and residual connections before the pooling layer of the CAA attention mechanism enables the network to extract shallow and deep features, retain more key information, and enhance the network's sensitivity to details. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is the overall flow chart of the present invention.
[0027] Figure 2 This is the improved YOWO network model used in the present invention.
[0028] Figure 3 The improved CAA attention mechanism module used in the present invention. DETAILED DESCRIPTION
[0029] For those skilled in the art, some well-known structures and their descriptions in the drawings may be omitted. The technical solution of the present invention is further described below in conjunction with the drawings and embodiments.
[0030] The present invention provides a badminton motion detection algorithm based on improved YOWO, which significantly improves the ability of 3D CNN branches to process complex data, enhances the spatial perception ability of the model, and also improves the model's ability to extract shallow and deep features.
[0031] Figure 1 The flowchart of the method of the present invention is as follows: first, the video is preprocessed, and the VideoBadminton data set is modified into the input and annotation files required by the network; then an improved YOWO network model is constructed and used to train the video data; secondly, the weight file that appears in the training is evaluated, and the network weight with the best effect is saved to realize video action detection.
[0032] Figure 2 This is the improved YOWO network model of the present invention, which includes a 2D CNN branch and an improved 3D CNN branch. The output feature maps of the two branches are fused through channel stacking features and passed to the CFAM module. Finally, the bounding box regression is used to output the action detection task.
[0033] Figure 3The improved CAA attention mechanism module of the present invention enhances the network's sensitivity to channels through the CAA module, while adding the temporal sequence between consecutive frames. Secondly, continuous convolution and residual connection improve the model's ability to extract shallow and deeper features.
[0034] The specific implementation steps are:
[0035] Step 1.1 Obtain a public badminton sports dataset (VideoBadminton) and divide the video to be tested into 30 consecutive frames.
[0036] Step 1.2 Use YOLOV5 to identify the target person in each frame and delete the label boxes of irrelevant people.
[0037] Step 1.3 Use the VIA annotation tool to manually annotate the target person's actions, annotate the target person's actions and spatial position information in each frame; process the action labels annotated by experts as network labels. According to the expert annotations, eight basic actions are classified as follows: Jump Smash, Overhead Clear, Backhand Drive, Forehand Drive, Backhand Lift, Forehand Lift, Forehand Net Shot, and Backhand Net Shot. Based on the eight basic actions and the actions in the video, 10 common postures are summarized: Up, Down, Side part, Flat part, Turn around, Front posture, Stride, Jump, Bend over, and Standing.
[0038] Step 1.4 Send the image cropped from the video and the coordinates detected by YOLOV5 to Deepsort for detection to associate the ID of the target person.
[0039] Step 2.1 uses the 3D CNN branch to extract timing features and analyzes the timing information of the previous and next frames of the video through the 3D-Resnext-101 network model.
[0040] Step 2.2 Change the CAA attention mechanism from two dimensions to three dimensions and add it to the 3D CNN branch with the YOWO network model.
[0041] Step 2.3 Based on the idea of connecting multiple convolutions and a convolution with a pooling layer in the C3D network, this idea is applied before the pooling layer of the CAA attention mechanism, using continuous 3D depth-separable convolutions and residual connections. This method can extract deeper features, capture more details, and retain more key information.
[0042] Step 3.1 Use the 2D CNN branch to extract spatial features and analyze the spatial information of the target person through the Darknet-19 network model.
[0043] Step 3.2 The Darknet-19 network model achieves effective feature learning by alternating between 3*3 and 1*1 filters. The initial feature size of the input is C*H*W. Features are extracted through multiple convolutional modules consisting of 32, 64, 128, 256, 512, 1024 and a specific number of output filters, and the maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolutional modules.
[0044] Step 4.1 uses the channel stacking feature to fuse the two feature maps, as shown in formula (1): (1) Among them, A is the feature map of 2D CNN and 3D CNN fusion. However, since such fusion has no connection relationship, it is necessary to connect the feature maps together through a convolutional layer, as shown in formula (2): (2) Among them, B is the newly generated feature map, and F is the reshaped tensor, that is, the feature vector of each channel is quantized into one dimension; The information is then passed as input to the CFAM module. This module maps the dependencies between channels based on the Gram matrix and uses the self-attention mechanism to highlight the relationship between the previous and next frames and enhance the distinguishability of features, as shown in formula (3): (3) Among them, G is the Gram matrix, G ij represents the inner product between the vectorized feature maps i and j.
[0045] Step 4.2 follows YOLO’s idea of bounding box regression and uses a 1*1 convolution kernel in the last layer to generate the target channel number. Finally, the coordinate box information, behavior category and confidence of the target person are output.
Claims
1. A badminton motion detection algorithm based on improved YOWO, characterized in that: The following steps are involved: Step 1: Obtain a badminton motion recognition video dataset, and modify the badminton dataset according to the AVA dataset annotation format so that it can run successfully in the YOWO network structure; Step 2: In the time series detection model, 3D-Resnext-101 is used to extract the spatiotemporal features of 30 consecutive frames of badminton motion, and the CAA attention mechanism and continuous 3D depth-separable convolution are introduced; Step 3: In the spatial detection model, Darknet-19 is used as the basic network of the spatial feature extraction module to extract the badminton motion features of the key frames; Step 4: Use the self-attention mechanism to perform channel fusion on the extracted spatiotemporal features, and finally regress the bounding box and output the detection results.
2. The badminton motion detection algorithm based on improved YOWO according to claim 1, characterized in that: The specific process in Step 1 is as follows: Step 1.1 Obtain a public badminton dataset (VideoBadminton) and divide the video to be tested into 30 consecutive frames; Step 1.2 Use YOLOV5 to identify the target person in each frame and delete the annotation boxes of irrelevant people; Step 1.3 Use the VIA annotation tool to manually annotate the target person’s actions, annotate the action and spatial position information of each frame of the target person; process the action labels annotated by experts as network labels; Step 1.4 Send the cropped image from the video and the coordinates detected by YOLOV5 to Deepsort for detection to associate the ID of the target person.
3. The badminton motion detection algorithm based on improved YOWO according to claim 1, characterized in that: The specific process in Step 2 is as follows: Step 2.1 Build a badminton motion detection algorithm based on the improved YOWO, add the CAA attention mechanism in the 3D CNN branch, focus on the information that is more critical to the current task among a large amount of input information, reduce the attention to other information, and improve the efficiency and accuracy of task processing; Step 2.2 Before the pooling layer of the added CAA attention mechanism, continuous 3D depth-separable convolution and residual structure are used to extract deeper features and retain more key information.
4. The badminton motion detection algorithm based on improved YOWO according to claim 1, characterized in that: The specific process in Step 3 is as follows: Step 3.1 uses 2D CNN branches to extract spatial features and analyzes the spatial information of the target person through the Darknet-19 network model; Step 3.2 The Darknet-19 network model achieves effective feature learning by alternating between 3*3 and 1*1 filters. The initial feature size of the input is C*H*W. Features are extracted through multiple convolutional modules consisting of 32, 64, 128, 256, 512, 1024 and a specific number of output filters, and the maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolutional modules.
5. The badminton motion detection algorithm based on improved YOWO according to claim 1, characterized in that: The specific process in Step 4 is as follows: Step 4.1 The outputs of the 2D and 3D networks have the same shape in the last two dimensions, and the channel stacking feature is used to fuse the two feature maps; Step 4.2 The self-attention mechanism can highlight the relationship between the previous and next frames and enhance the distinguishability of features; Step 4.3 follows YOLO’s idea of bounding box regression. The last layer uses a 1*1 convolution kernel to generate the target channel number and outputs the target person’s coordinate box information, behavior category, and confidence.