Expression recognition system and method based on enhancement CNN and cross-layer LSTM

Abstract

The invention discloses an expression recognition system and method based on an enhancement CNN and a cross-layer LSTM. The system comprises a feature enhancement CNN module, a cross-layer LSTM module, and a full connection layer. The method comprises: the feature enhancement CNN module and the cross-layer LSTM module are in cascade connection for end-to-end training; the feature enhancement CNN module leads out a feature enhancement branch in the middle layer of the backbone CNN network, and fuses the output of the feature enhancement branch with the output of the backbone CNN network; and the cross-layer LSTM module inputs the output of the feature enhancement CNN module to the first layer LSTM network on the basis of cascading of at least two layers of LSTM networks, and bridges the output of the feature enhancement CNN module to the input end of the rear layer LSTM network. Accurate video sequence expression time information can be acquired, the accuracy of non-constraint facial expression recognition is effectively improved, and the method has wide application prospects in the fields of human-computer interaction, wisdom education, patient monitoring and the like.

Description

technical field [0001] The invention relates to the technical field of expression recognition, in particular to an expression recognition system and method based on enhanced CNN and cross-layer LSTM. Background technique [0002] Facial expressions contain rich emotional information, which is one of the important ways of expressing human emotions and an effective means for people to communicate non-verbal emotions. People can express their emotions through facial expressions, and can also accurately identify the inner emotional changes of the other party. Therefore, accurate recognition of facial expressions has important research value and application prospects, and has become a research hotspot in the field of artificial intelligence in recent years. [0003] A facial expression recognition system generally includes four steps: image preprocessing, face detection and face region segmentation, expression feature extraction, and expression classification. Among them, expres...

AI Technical Summary

Problems solved by technology

[0008] (1) Most of the existing research focuses on static single-frame images, and there are not many researches on facial expression recognition based on video sequences, and most of the research results are verified on the video database collected in the experimental environment, for example, CK+, MMI, Oulu-CASIA, etc., the facial expressions in these data are exaggerated and less disturbed by noise, which has little reference value for practical applications;

[0009] (2) The existing facial expression video data collected in the real environment is less, which leads to insufficient training samples of the deep neural network, which seriously affects the performance of the network; at the same time, due to individual differences such as age, gender, and race As well as intra-individual changes in lighting, posture, occlusion, accessories, etc., the quality of the collected facial expression samples is uneven

[0010] These all increase the difficulty of designing a real-time and accurate unconstrained facial expression recognition system. The existing research on facial expression recognition based on deep neural network still has a lot of room for improvement in performance.

Images