Action detection method based on asymmetric multi-flow

Abstract

The invention discloses an action detection method based on asymmetric multi-stream, which comprises the following steps: extracting an RGB image and an optical stream from a prior video, and training to obtain a trained RGB image single-stream network and an optical stream single-stream network; extracting image flow characteristic information and optical flow characteristic information of each frame in the priori video, and training an asymmetric double-flow network by combining an action label; respectively extracting image flow characteristic information and optical flow characteristic information of each frame in the target video to be detected through the trained RGB image single-flow network and optical flow single-flow network, obtaining segment characteristics of the target video, inputting the segment characteristics into the trained asymmetric double-flow network, and calculating to obtain a video classification vector; selecting potential actions from the video classification vectors to obtain an action recognition sequence of the potential actions; and completing the action detection is completed through the action recognition sequence. According to the action detection method, the asymmetry between the image flow and the optical flow is considered, and the accuracy of action recognition and action detection can be improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision and pattern recognition, and in particular relates to an action detection method based on asymmetric multi-flow. Background technique [0002] The discovery and segmentation of video motion is an important research content in the field of video processing, and it is widely used in motion tracking and detection, which has great theoretical research value and practical application value. Among them, the action detection method implemented with weakly supervised data can achieve good performance with only a small amount of manual annotation. Action detection methods implemented with weak supervision are modeled from video-level labels, and judge whether a video frame contains an action through an attention mechanism. [0003] In order to model the frame, most methods first process the video sequence, extract and fuse the image flow and optical flow, then use the video label training, and the...

AI Technical Summary

Problems solved by technology

In the weakly supervised action detection method, the traditional two-stream combination method regards the two streams as symmetric, which brings strong overfitting and cannot accurately locate the action.

Images