A human body behavior recognition method based on a multi-space-time information fusion convolutional neural network

Abstract

The invention provides a human body behavior recognition method based on a multi-space-time information fusion convolutional neural network, and the method comprises the steps: firstly building a (2 +1) D convolutional neural network, carrying out the training until the evaluation accuracy of a network model reaches a stable value, and carrying out the video human body behavior recognition through employing a grid model. According to the (2 + 1) D convolutional neural network provided by the invention, spatial convolutional layers of spatial receptive fields with different scales are used forspatial information extraction; meanwhile, a plurality of time domain convolution layers of different scale time domain receptive fields are used for carrying out time domain information extraction;the extracted feature information is fused and then is used as the input of the next layer; a convolution kernel containing n scale space receptive fields and a convolution kernel containing m scale time domain receptive fields are connected in series, a multi-space-time fusion convolution layer containing k space-time receptive fields is designed, modeling can be carried out by using feature information in a long and short time range of a video at the same time, and human body behaviors can be identified more accurately.

Description

technical field [0001] The invention relates to a video human behavior recognition method, in particular to a human behavior recognition method based on multi-temporal information fusion convolutional neural network. Background technique [0002] Video human action recognition is one of the most challenging tasks in computer vision and can have wide applications in many fields, such as video surveillance, motion retrieval, human-computer interaction, smart home, and healthcare. Traditional video action recognition methods generally use artificially designed video spatio-temporal features, such as SIFT-3D, STIPs, HOG3D, HOF, dense trajectory (iDT), etc. With the impressive progress of convolutional neural networks in the field of still image recognition, a large number of network structures with powerful feature extraction capabilities have emerged. In recent years, the application of convolutional neural networks to video behavior recognition has become a research hotspot a...

AI Technical Summary

Problems solved by technology

[0003] However, the above methods all have certain defects: first, the calculation is complex and there is parameter redundancy; second, these methods only model the information of a single spatio-temporal receptive field, which has certain limitations, and it is difficult to extract variable spatio-temporal information. Affects the performance of the convolutional network and cannot effectively identify complex human behaviors

Images