Video behavior recognition method based on spatial-temporal feature fusion deep learning network

Abstract

The invention discloses a video behavior identification method based on a spatio-temporal feature fusion deep learning network, which adopts two independent networks to extract time and space information of a video respectively, adds LSTM learning video time information to each network on the basis of a CNN, and fuses the time and space information with a certain strategy. The accuracy of the FSTFN on a data set is improved by 7.5% compared with a network model without introducing a space-time network proposed by Tran, the accuracy on the data set is improved by 4.7% compared with a common double-flow network model, a segmentation mode is adopted for a video, each video sample samples a network composed of a plurality of segments and inputs the segments into a CNN and an LSTM, and by covering the time range of the whole video. The problem of long-term dependence in video behavior recognition is solved, a visual attention mechanism is introduced to the tail end of the CNN, the weight ofa non-visual subject in a network model is reduced, the influence of the visual subject in a video image frame is improved, and the spatial features of the video are better utilized.

Description

【Technical field】 [0001] The invention relates to a video behavior recognition method, in particular to a video behavior recognition method based on spatio-temporal feature fusion deep learning network. 【Background technique】 [0002] Video content behavior recognition aims to classify video clips to determine behavior types. At present, video content behavior recognition technology is mainly divided into two directions: the traditional way of extracting features and the way of using deep learning to establish an end-to-end predictive network model. [0003] Based on the traditional behavior recognition method, the relevant visual features are first designed and extracted, and then these features are encoded, and finally the relevant classification methods in statistical machine learning are used to obtain the predicted classification results. [0004] Most of the deep learning network models are end-to-end models, using convolutional neural network (Convolutional Neural Ne...

AI Technical Summary

Problems solved by technology

[0009] The above methods all have the problem of long-term dependence and insufficient mining of spatial features. Due to the persistence of behavior, in order to accurately identify a behavior, it often depends on a longer time segment. Without long-term analysis, it is easy to put a Behavior recognition is other behaviors. Although long-term segment analysis can improve the recognition accuracy, too long time analysis will bring more complex network structure and double the amount of training; and the current research is on the extraction of video dynamic temporal features. Obvious deficiencies, only obtained from RGB (Red-Green-Blue) through C3D or cyclic neural network, but a single feature is still not enough to fully extract the dynamic features of the time dimension of the video, and the extraction of spatial features is also not sufficient, the existing network model easy to confuse and misjudgment

Images