A Human Behavior Recognition Method Based on Two-Stream Deep Neural Network

Abstract

The invention discloses a human action recognition method based on a dual-stream deep neural network, aiming at solving the technical problems existing in the application of deep learning in video recognition. It includes: obtaining multiple RGB image sequences according to the original video data set; using the trained spatial domain behavior recognition model and time domain behavior recognition model to process the RGB image sequences respectively to obtain the spatial domain recognition type probability matrix and the time domain recognition type probability matrix ; Use the mean fusion model to perform probability fusion on the spatial domain recognition type probability matrix and the time domain recognition type probability matrix to obtain the human behavior recognition results corresponding to the original video data set. The invention can effectively improve the accuracy of human action recognition, and has good generalization performance.

Description

technical field [0001] The invention relates to a human action recognition method based on a dual-stream deep neural network, and belongs to the technical field of computer vision. Background technique [0002] In recent years, with the development of surveillance video, Internet video retrieval processing, human-computer interaction, virtual reality, medical care, intelligent security and other technologies, the recognition of human behavior from video has received more and more attention from computer vision researchers. , due to the influence of factors such as video occlusion, dynamic background, moving camera, perspective and illumination changes, video human behavior recognition is difficult and challenging. [0003] With the successful application of CNN to still image classification and recognition, the continuous improvement of computer performance, the rapid development of the GPU industry, and the proliferation of video datasets on the Internet, more and more rese...

AI Technical Summary

Problems solved by technology

[0002] In recent years, with the development of monitoring video, Internet video retrieval processing, human-computer interaction, virtual reality, medical care, intelligent security and other technologies, recognizing human behavior from video has attracted more and more attention from computer vision researchers. , due to the influence of factors such as video occlusion, dynamic background, moving camera, viewing angle and lighting changes, video human behavior recognition is difficult and very challenging

[0003] With the successful application of CNN to classification and recognition of static images, the continuous improvement of computer performance, the rapid development of the GPU industry, and the surge in video data sets on the Internet, more and more researchers have begun to apply deep learning to the recognition of video fields. Among them, the most commonly used video recognition network architectures are 3D convolutional network and dual-stream network. However, the 3D convolutional network generally has a large number of parameters, which is difficult to train, and the small training data set is easy to cause problems such as overfitting; while the dual-stream network is There is a long time-consuming optical flow extraction, and the overall network cannot achieve end-to-end and other deficiencies

Images