Video behavior identification method based on hybrid multi-scale time sequence separable convolution operation

Abstract

The invention discloses a video behavior identification method based on hybrid multi-scale time sequence separable convolution operation. The method comprises the following steps: extracting an original video into a picture sequence, dividing the picture sequence into a plurality of intervals, extracting a picture from each interval to form a picture sub-sequence, performing feature extraction onthe picture sub-sequence, and classifying the features of the picture sub-sequence to obtain a classification result serving as the category of behaviors occurring in the video; adopting a convolutional neural network model in which hybrid multi-scale time sequence separable convolution is added as a learner, extracting semantic features of picture sub-sequences, and using full connection layers in the convolutional neural network model for classifying the extracted picture sub-sequence features. The method is used for solving the problems that the lengths of actions in the video are differentand different semantic features in the space have different time sequence change scales, can be applied to video understanding tasks in the aspects of video behavior recognition and the like, and canefficiently realize video behavior recognition.

Description

technical field [0001] The invention belongs to the technical field of machine vision and deep learning, and relates to video behavior recognition technology, in particular to a method for efficient video behavior recognition using mixed multi-scale timing depth separable convolution. Background technique [0002] The purpose of video behavior recognition is to, for a given piece of video, analyze the action category that occurs in the video and give the corresponding label. This task can be considered as a kind of video classification task. For subsequent tasks of understanding other video content, the video classification task is the foundation, so it plays a very important role. As a basic task, video behavior recognition is widely used in human-computer interaction, urban security monitoring and other scenarios. [0003] In the process of video behavior recognition, existing methods use neural networks for feature extraction. It is mainly divided into three types: 1) ...

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a spatio-temporal modeling method based on hybrid multi-scale temporal depth separable convolution operation, which is used to solve the different lengths of actions in videos and different temporal changes of different semantic features in space. The problem of scale can be applied to but not limited to the video understanding task of video behavior recognition, and can efficiently realize video behavior recognition

[0005] The present invention fuses depth-separable convolutions of different sizes into one depth-separable convolution operation, and performs timing modeling of different scales on the features of different channels, so as to solve the different lengths of actions in videos and different semantic features in space. Problems with different scales of temporal variation

Images