Deep learning based crowd emotion recognition method

Abstract

The invention provides a deep learning based video crowd emotion analysis method, and mainly relates to classification for crowd emotions in the video by using a multi-stream neural network. The method comprises the steps of building a multi-stream neural network (pixel, optical flow and saliency), concurrently extracting features in pixel information, superposition optical flow information and saliency information of a video sequence by using the network, and finally integrating the three types of features to obtain classification for the crowd emotions. The self-learning ability of deep learning is given into full play, the limitation of manual feature extraction is avoided, and the method is enabled to be higher in adaptability. Training and prediction are performed by using structural features of the multi-stream deep learning network, classification results of multi-stream sub-networks are integrated finally, and the accuracy and the work efficiency are improved.

Description

technical field [0001] The invention relates to the problem of crowd emotion recognition in the field of video analysis, in particular to a video analysis method for classifying crowd emotions based on a deep learning multi-stream neural network. Background technique [0002] The emotional analysis of the crowd is to judge the emotional state of the crowd by analyzing the behavior and clothing of the crowd, such as excitement, excitement, normal, boring, etc. A large number of videos exist in real life, such as drone video surveillance, network sharing video, 3D video, etc. Analyzing the emotions of the crowd in the video will help to dynamically understand the emotions and changes of the crowd in the video, and has broad application prospects. Taking the Stampede on the Bund as an example, by analyzing the emotional changes of the crowd, administrators can intervene to prevent such incidents from happening again before emergencies occur. [0003] Traditional crowd emotion...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a method for crowd emotion recognition in video, combining deep learning with crowd emotion in video, giving full play to the advantages of deep learning self-learning, and solving the difficulty of adjusting the parameters of shallow learning at present, requiring manual selection of features , low accuracy and other issues

Images