Football motion detection algorithm based on improved YOWO
By introducing improved SE attention mechanism and 3D depth separation convolution into the football video action detection algorithm, the problems of low detection accuracy and high computing resource consumption in the existing technology are solved, and more efficient video action detection effect is achieved .
Patent Information
- Application Number
- CN202411653026.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-02-18
AI Technical Summary
The prior art has problems of low detection accuracy and high computing resource consumption in football video action detection. In particular, the limited features learned by YOWO network structure on self-made behavior data sets, resulting in high misjudgment rate and high running time cost. .
A football action detection algorithm based on improved SE attention mechanism is proposed. By introducing SE attention mechanism and continuous 3D depth separation convolution and residual connections in the timing detection module, the network's ability to extract spatiotemporal features is enhanced; YOLOV2 is used in the spatial detection module for spatial feature extraction and feature fusion, and finally detection is carried out through the CFAM module.
It significantly improves the ability of 3D CNN branches to process complex data, enhances the spatial perception ability of the model, improves the ability to extract shallow and deep features, reduces computing resource consumption, and achieves more efficient video action detection.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing and computer vision, and in particular relates to a football video action detection algorithm based on an improved SE attention mechanism. Background Art
[0002] Spatiotemporal behavior detection is an important field of computer vision. Its main task is to detect the behavior category of the target in the video and accurately locate the time and location of the behavior. In recent years, deep learning has made breakthrough progress in action recognition, so that researchers are no longer satisfied with simple coarse-grained action recognition. Today, researchers have combined action recognition with sports events and used deep learning technology to realize the intelligent recognition and analysis of key actions in sports event videos. YOWO is applied to the field of video analysis because of its simple model structure and the ability to quickly identify video information. However, the original YOWO network is used in the field of video analysis because of its simple model structure and the ability to quickly identify video information. However, the original YOWO network is based on a self-made behavior dataset. Due to the limited sample features, the features learned by the model are also limited, resulting in low detection accuracy of the model. Through analysis, it is found that there are two problems with the network structure of YOWO: one is that the 2D feature extraction network of YOWO does not have the ability to obtain temporal context information, so it is easy to misjudge similar behaviors; the other is that the 3D feature extraction network of YOWO has made great progress in extracting temporal information, but it consumes a lot of computing resources and has a high running time cost. How to make it lightweight, truly apply it to practice, and realize real-time operation is very important. Therefore, improving the accuracy and speed of spatiotemporal behavior detection has important research significance. Summary of the invention
[0003] In order to solve the above technical problems, the present invention proposes a football action detection algorithm based on an improved SE attention mechanism. The invention is based on the YOWO network, which inputs the RGB video continuous frames and RGB key frame information into the temporal detection module and the spatial detection module for processing respectively; in the temporal detection module, the attention mechanism SE module is introduced to improve the model's sensitivity to channel features, and at the same time, continuous 3D depth-separable convolution and residual connection are added before the pooling layer of the SE attention mechanism, so that the network pays more attention to the shallow and deep feature information, ensuring that the key information is retained, thereby improving the spatiotemporal feature extraction capability of the 3D convolutional network; in the spatial detection module, YOLOV2 is used to extract spatial features, and then the spatiotemporal features and spatial features are fused, and finally these fused features are detected to obtain the category, duration and location information of the target behavior in the video.
[0004] In order to achieve the above object, the technical solution of the present invention is as follows:
[0005] The football video action detection algorithm based on the improved SE attention mechanism includes the following steps: Step 1: Obtain a football action recognition video dataset, and modify the football video dataset according to the AVA dataset annotation format so that it can run successfully in the YOWO network structure; Step 2: In the time series detection model, 3D-Resnext-101 is used to extract the spatiotemporal features of 30 consecutive frames of football actions. In order to improve the model's sensitivity to channel features and deeper features, SE attention mechanism and continuous 3D depth-separable convolution are introduced; Step 3: In the spatial detection model, Darknet-19 is used as the basic network of the spatial feature extraction module to extract the football action features of the key frames; Step 4: Use the self-attention mechanism CFAM to perform channel fusion on the extracted spatiotemporal features, and finally regress the bounding box and output the detection results.
[0006] The specific situation in step one is as follows: football video data is collected and football videos are searched, the videos are divided into continuous frame images at 30 frames per second through video processing software, and relevant motion information is manually annotated, and the annotated data set is divided into a training set and a test set.
[0007] The specific situation in the step 2 is as follows: the 3D CNN branch is used to extract the timing features, and the timing information of the previous and next frames of the video is analyzed through the 3D-Resnext-101 network model.
[0008] (1) In order to improve the model’s sensitivity to channels and increase the time series for consecutive frames, the SE attention mechanism is added to the time series to make it a three-dimensional module; (2) Based on the idea of connecting multiple convolutions and a convolution with a pooling layer in the C3D network, this idea is applied to the SE attention mechanism. Continuous 3D depthwise separable convolution and residual connection are used before the pooling layer. This method can extract deeper features and retain more key information.
[0009] The specific situation in step three is as follows: The 2D CNN branch is used to extract spatial features, and the spatial information of the target person is analyzed through the Darknet-19 network model. The Darknet-19 network model achieves effective feature learning by alternating between 3*3 and 1*1 filters. The initial feature size of the input is C*H*W, and features are extracted through multiple convolution modules consisting of 32, 64, 128, 256, 512, 1024 and a specific output filter number of 4. The maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolution modules.
[0010] The specific situation in the step 4 is as follows: The outputs of the two-dimensional and three-dimensional networks have the same shape in the last two dimensions, and the channel stacking feature is used to fuse the two feature maps. The information is then passed as input to the CFAM module. The self-attention mechanism can highlight the relationship between the previous and next frames and enhance the distinguishability of the features. Finally, following the idea of YOLO on bounding box regression, the last layer uses a 1*1 convolution kernel to generate the target channel number and output the target person's coordinate box information, behavior category, and confidence.
[0011] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: (1) The present invention changes the SE attention mechanism from two-dimensional to three-dimensional and adds a 3D CNN branch with a YOWO network model, which not only enhances the network's sensitivity to channels but also makes the network pay more attention to the time dimension and strengthens the connection between previous and next frames.
[0012] (2) Adding continuous 3D depth-wise separable convolutions and residual connections before the pooling layer of the SE attention mechanism enables the network to extract shallow and deep features, retain more key information, and enhance the network's sensitivity to details. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is the overall flow chart of the present invention; Figure 2 The improved YOWO network model used in the present invention; Figure 3 Improved SE attention mechanism module used in the present invention; DETAILED DESCRIPTION
[0014] For those skilled in the art, some well-known structures and their descriptions in the accompanying drawings may be omitted. The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments. The present invention provides a football action detection algorithm based on an improved SE attention mechanism, which significantly improves the ability of the 3D CNN branch to process complex data, enhances the spatial perception ability of the model, and also improves the model's ability to extract features from shallow and deep layers.
[0015] Figure 1 The flowchart of the method of the present invention is as follows: first, the video is preprocessed to modify the data set into the input and annotation files required by the network; then, an improved YOWO network model is constructed and used to train the video data; secondly, the weight file that appears in the training is evaluated, and the network weight with the best effect is saved to realize video action detection. Figure 2This is the improved YOWO network model of the present invention, which includes a 2D CNN branch and an improved 3D CNN branch. The output feature maps of the two branches are fused through channel stacking features and passed to the CFAM module. Finally, the bounding box regression is used to output the action detection task. Figure 3 The SE attention mechanism module is improved by the present invention. The SE module enhances the network's sensitivity to channels and adds the temporal sequence between consecutive frames. Secondly, continuous convolution and residual connection improve the model's ability to extract shallow and deeper features.
[0016] The specific implementation steps are: Step 1.1 Obtain a football sports data set and divide the video to be detected into 30 consecutive frames; Step 1.2 Use YOLOV5 to identify the target person in each frame and delete the annotation boxes of irrelevant people; Step 1.3 Use VIA annotation tool to manually annotate the target person’s actions, annotating the action and spatial position information of the target person in each frame; Step 1.4 Send the image cropped from the video and the coordinates detected by YOLOV5 to Deepsort for detection to associate the ID of the target person.
[0017] Step 2.1 uses 3D CNN branches to extract timing features and analyzes the timing information of the previous and next frames of the video through the 3D-Resnext-101 network model; Step 2.2 Change the SE attention mechanism from 2D to 3D and add the 3D CNN branch with the YOWO network model; Step 2.3 Based on the idea of connecting multiple convolutions and a convolution with a pooling layer in the C3D network, this idea is applied before the pooling layer of the SE attention mechanism, using continuous 3D depth-separable convolutions and residual connections. This method can extract deeper features, capture more details, and retain more key information.
[0018] Step 3.1 uses 2D CNN branches to extract spatial features and analyzes the spatial information of the target person through the Darknet-19 network model; Step 3.2 The Darknet-19 network model achieves effective learning of features by alternating between 3*3 and 1*1 filters. The initial feature size of the input is C*H*W. Features are extracted through multiple convolutional modules consisting of 32, 64, 128, 256, 512, 1024 and a specific number of output filters, and the maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolutional modules.
[0019] Step 4.1 uses the channel stacking feature to fuse the two feature maps, as shown in formula (1): (1) Among them, A is the feature map of 2D CNN and 3D CNN fusion. However, since such fusion has no connection relationship, it is necessary to connect the feature maps together through a convolutional layer, as shown in Formula 2: (2) Among them, B is the newly generated feature map, F is the reshaped tensor, that is, the feature vector of each channel is quantized into one dimension: Then the information is passed as input to the CFAM module. This module maps the dependencies between channels based on the Gram matrix. The self-attention mechanism can highlight the relationship between the previous and next frames and enhance the distinguishability of features, as shown in Formula 3. (3) Among them, G is the Gram matrix, and Gij represents the inner product between the vectorized feature maps i and j.
[0020] Step 4.2 follows YOLO’s idea of bounding box regression and uses a 1*1 convolution kernel in the last layer to generate the target channel number. Finally, the coordinate box information, behavior category and confidence of the target person are output.
Claims
1. A football motion detection algorithm based on an improved SE attention mechanism, characterized in that: The following steps are involved: Step 1: Obtain a football motion recognition video dataset, and modify the football motion dataset according to the AVA dataset annotation format so that it can run successfully in the YOWO network structure; Step 2: In the temporal detection model, 3D-Resnext-101 is used to extract the spatiotemporal features of football motion for 30 consecutive frames, and the SE attention mechanism and continuous 3D depth-separable convolution are introduced; Step 3: In the spatial detection model, Darknet-19 is used as the basic network of the spatial feature extraction module to extract the football motion features of the key frames; Step 4: Use the self-attention mechanism to perform channel fusion on the extracted spatiotemporal features, and finally regress the bounding box and output the detection results.
2. The football motion detection algorithm based on the improved SE attention mechanism according to claim 1 is characterized in that: The specific process in Step 1 is as follows: Step 1.1 Obtain a football sports data set and divide the video to be detected into 30 consecutive frames; Step 1.2 Use YOLOV5 to identify the target person in each frame and delete the annotation boxes of irrelevant people; Step 1.3 Use VIA annotation tool to manually annotate the target person’s actions, annotate the action and spatial position information of the target person in each frame; Step 1.4 Send the cropped image from the video and the coordinates detected by YOLOV5 to Deepsort for detection to associate the ID of the target person.
3. The football motion detection algorithm based on the improved SE attention mechanism according to claim 1 is characterized in that: The specific process in Step 2 is as follows: Step 2.1 Build and improve the football motion detection algorithm based on the improved SE attention mechanism. Add the SE attention mechanism in the 3D CNN branch to focus on the information that is more critical to the current task among a large amount of input information, reduce the attention to other information, and improve the efficiency and accuracy of task processing; Step 2.2 Before the pooling layer of the added SE attention mechanism, continuous 3D depth-separable convolution and residual structure are used to extract deeper features and retain more key information.
4. The football motion detection algorithm based on the improved SE attention mechanism according to claim 1 is characterized in that: The specific process in Step 3 is as follows: Step 3.1 uses 2D CNN branches to extract spatial features and analyzes the spatial information of the target person through the Darknet-19 network model; Step 3.2 The Darknet-19 network model achieves effective feature learning by alternating between 3×3 and 1×1 filters. The initial feature size of the input is C*H*W. Features are extracted through multiple convolutional modules consisting of 32, 64, 128, 256, 512, 1024 and a specific number of output filters, and the maximum pooling layer, normalization operation BN and activation function LeakyReLU are embedded between the convolutional modules.
5. The football motion detection algorithm based on the improved SE attention mechanism according to claim 1 is characterized in that: The specific process in Step 4 is as follows: Step 4.1 The outputs of the 2D and 3D networks have the same shape in the last two dimensions, and the channel stacking feature is used to fuse the two feature maps; Step 4.2 The self-attention mechanism can highlight the relationship between the previous and next frames and enhance the distinguishability of features; Step 4.3 follows YOLO’s idea of bounding box regression. The last layer uses a 1*1 convolution kernel to generate the target channel number and outputs the target person’s coordinate box information, behavior category, and confidence.