Action recognition method based on double-flow convolution attention

Abstract

The invention discloses an action recognition method based on double-flow convolution attention. The method comprises the following steps: firstly, preprocessing a video to obtain a frame image sequence and an optical flow image sequence, and respectively extracting appearance feature representation and action feature representation of the video; then constructing a convolution attention module to obtain attention feature representations of the frame image and the optical flow image, and performing information fusion on the two attention representations through a double-flow fusion module; and then training an action recognition model using a convolution attention mechanism and a double-flow fusion method, and outputting the action category of the preprocessed new video according to the model. According to the method, channel attention and space-time attention are utilized to capture a potential mode and a space-time relationship of video action contents, and information fusion is carried out on appearance features and action features of the video from a global perspective through double-flow fusion, so that the problem of time sequence information loss of long-term time sequence dependence of the video is effectively relieved; and the accuracy of action recognition is improved.

Description

technical field [0001] The invention belongs to the technical field of computers, especially the technical field of action recognition in video analysis, and specifically relates to an action recognition method based on dual-stream convolution attention. Background technique [0002] In recent years, all kinds of video data have been increasing day by day, how to identify the action content of video has become a basic research topic for many video processing tasks. Action recognition technology mainly gives video action categories based on video content, and has very important social value in multiple application scenarios such as assisted driving, video content review, and personalized recommendation. For example, in vehicle assisted driving scenarios, motion recognition technology can help users issue instructions to the navigation system through gestures to improve people's driving comfort; in video content review, the motion recognition system can assist manual video con...

AI Technical Summary

Problems solved by technology

[0004] There are still some deficiencies in the existing video action recognition methods: first, there are differences in the key information in different video frames, and the importance of different frames is not the same, so a single visual attention cannot effectively capture the key information; second, three-dimensional The convolutional neural network is limited by the size of the convolution kernel, and can only extract short-term dependent timing information of multiple frames in a small range, and lacks the extraction of long-term dependent timing information; third, most methods based on dual streams directly combine the actions of two features The recognition results are weighted and summed, and the information fusion of spatial features and motion features is not considered

Images