Video action detection method based on scale attention hole convolution network

Abstract

The invention discloses a video action detection method based on a scale attention hole convolutional network. The method comprises the following steps: firstly, sampling a video to obtain a frame image sequence, and marking according to fragment positions to obtain video fragments; and then respectively constructing a layer-scale attention action fragment model and a frame position attention action recognition model, and according to the models and in combination with a watershed algorithm, sequentially obtaining weighted feature representation of a frame image and an action category to whicha video fragment belongs, thereby finishing a video action detection task. According to the method, the space-time motion information capable of better reflecting the intrinsic structures of the timedimension and the space dimension of the video data is extracted by utilizing the hole convolution network; the intrinsic relation of the time sequence context relation of the video frames along withthe change of the size of the scale is more properly described through the layer scale attention, the designed frame position attention mechanism gives the video frames of the action clips a weight more accurately representing the key content of the action clips, the accuracy of video action detection is improved, and the efficiency of action detection is improved.

Description

technical field [0001] The invention belongs to the technical field of video analysis, in particular to the technical field of time series action detection, and relates to a video action detection method based on a scale-attention hole convolution network. Background technique [0002] The understanding of human motion video plays an important role in many fields such as security monitoring and behavior analysis, and has become a frontier research topic in the field of computer vision. However, unedited real videos often contain background segments unrelated to human actions, which will affect the correct understanding of video content. To solve this problem, the video action detection method not only classifies the actions in the video, but also locates the start and end time of the action instance in the video. Video action detection tasks usually use video frame sequences as input, and output the detection results of multiple groups of segments in the form of "action typ...

AI Technical Summary

Problems solved by technology

[0004] The existing video action detection methods mainly have the following deficiencies: First, in the feature extraction stage, the three-dimensional convolution operation used to extract the timing features of the action will fixedly reduce the timing dimension of the input video layer by layer in the constructed network model, Constrains the scale size of the extracted features in time series. Too small a scale may cause contextual semantic segmentation, and too large a scale may cause interference of different semantics. The key points of whether or not and its type, that is, the key frame position and its duration (such as continuous key frames) are often different, and the conventional average pooling operation ignores the weight of the key points; third, the existing methods for different scales Fragments use different network structures (such as dilated convolutional networks) to extract feature representations of action clips, which will greatly increase the time and space costs of network construction and training

Images