An Action Recognition Method Based on Two-Stream Convolutional Attention

Abstract

The invention discloses an action recognition method based on two-stream convolutional attention. The method of the present invention first preprocesses the video to obtain the frame image sequence and optical flow image sequence, and extracts the appearance feature representation and action feature representation of the video respectively; then constructs a convolution attention module to obtain the attention feature representation of the frame image and optical flow image , and perform information fusion on the two attention representations through the two-stream fusion module; then train the action recognition model using the convolutional attention mechanism and the two-stream fusion method, and output the action category of the preprocessed new video according to the model. The method of the present invention not only uses channel attention and spatio-temporal attention to capture the potential pattern and spatio-temporal relationship of video action content, but also fuses the appearance features and motion features of the video from a global perspective through dual-stream fusion, effectively alleviating the long-term timing dependence of video The lack of timing information improves the accuracy of action recognition.

Description

technical field [0001] The invention belongs to the technical field of computers, especially the technical field of action recognition in video analysis, and specifically relates to an action recognition method based on dual-stream convolution attention. Background technique [0002] In recent years, all kinds of video data have been increasing day by day, how to identify the action content of video has become a basic research topic for many video processing tasks. Action recognition technology mainly gives video action categories based on video content, and has very important social value in multiple application scenarios such as assisted driving, video content review, and personalized recommendation. For example, in vehicle assisted driving scenarios, motion recognition technology can help users issue instructions to the navigation system through gestures to improve people's driving comfort; in video content review, the motion recognition system can assist manual video con...

AI Technical Summary

Problems solved by technology

[0004] There are still some deficiencies in the existing video action recognition methods: first, there are differences in the key information in different video frames, and the importance of different frames is not the same, so a single visual attention cannot effectively capture the key information; second, three-dimensional The convolutional neural network is limited by the size of the convolution kernel, and can only extract short-term dependent timing information of multiple frames in a small range, and lacks the extraction of long-term dependent timing information; third, most methods based on dual streams directly combine the actions of two features The recognition results are weighted and summed, and the information fusion of spatial features and motion features is not considered

Images