Individual emotion recognition method fusing expressions and postures

Abstract

The invention provides an individual emotion recognition method fusing expressions and postures, and mainly relates to classification of individual sequence emotions by using a multi-mode individual emotion recognition network. The method comprises the following steps: constructing a multi-modal individual emotion recognition network (including two channels for processing an expression sequence and a posture sequence), extracting expression features and posture features in a video sequence in parallel by using the network, and finally fusing the two features to obtain individual sequence emotion classification. According to the method, the self-learning ability of deep learning is fully exerted, the limitation of manual feature extraction is avoided, and the adaptive capacity of the method is higher. And the structural features of the multi-stream deep learning network are used for parallel training and prediction, and finally classification results of multiple sub-networks are fused, so that the accuracy and the working efficiency are improved.

Description

technical field [0001] The invention relates to the problem of individual sequence emotion recognition in the field of video analysis, in particular to a video analysis method for classifying individual sequence emotions by a multi-stream neural network integrating expressions and gestures. Background technique [0002] Emotion recognition aims to enable computers to have the ability to perceive and analyze human emotions and intentions, so as to play a role in entertainment, medical care, education, public safety and other fields. Emotional expressions do not exist in isolation, and among them, the combined visual channel of facial expression and body posture is considered to be an important channel for judging human behavior cues. Facial expressions can most intuitively reflect people's emotional state and mental activities, and are an important way to express emotions. However, irrelevant factors in the real environment will have a great impact on the recognition of facia...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a method for individual emotion recognition in a video sequence, which combines deep learning with video individual emotion, gives full play to the advantages of deep learning self-learning, and effectively integrates emotional information expressed by facial expressions and body postures, which can solve the problem of At present, the parameters of shallow learning are difficult to adjust, and features need to be manually selected, and the accuracy of individual emotion recognition in public spaces is not high.

Images