Behavior classification method based on twin three-dimensional convolutional neural network

Abstract

The invention discloses a behavior classification method based on a twin three-dimensional convolutional neural network. The method comprises the following steps: 1, adjusting the length and width ofan image frame to the size required by the twin three-dimensional convolutional neural network; 2, grouping an image frame sequence in groups of continuous 16 frames, inputting each group of image frames into the twin three-dimensional convolutional neural network, extracting abstract time features and abstract space features, and inputting the abstract time features into a deconvolution network to obtain an optical flow field; 3, calculating to obtain a total loss function L, wherein the expression is as follows: L = Lcls + alpha Lflow; and 4, optimizing network parameters by using a back propagation technology, so that the network performance is optimal. The algorithm provided by the invention has advantages in both speed and accuracy.

Description

technical field [0001] The present invention relates to the neighborhood of video speech understanding, in particular to a method for classifying behaviors in videos. Background technique [0002] Behavior classification is an important branch in the field of video understanding. Higher-level tasks such as generating video collections and describing video content in text are all based on behavior classification. Behavior classification is a technology that extracts the abstract semantic information of the video and judges the action category contained in the video according to the semantic information. At present, there is no optimal solution to the video classification problem. One type of algorithm only uses the spatial characteristics of the video, ignoring the temporal characteristics of the video, which makes the effect of the classification algorithm unsatisfactory; the other type of algorithm uses both time and space. This kind of method describes the temporal featur...

AI Technical Summary

Problems solved by technology

At present, there is no optimal solution to the video classification problem. One type of algorithm only uses the spatial characteristics of the video, ignoring the temporal characteristics of the video, which makes the effect of the classification algorithm unsatisfactory; the other type of algorithm uses both time and space. feature, this type of method describes the temporal characteristics of the video through the optical flow field, and abstracts the temporal characteristics for behavior classification, but the calculation speed of the optical flow field cannot be applied to the scene of real-time behavior classification, and the optical flow field does not describe the temporal characteristics of the video The best method, the performance of the behavior classification algorithm based on the optical flow field still has a lot of room for improvement

Images