Video sequence expression recognition system and method based on self-attention enhanced CNN

Abstract

The invention discloses a video sequence expression recognition system and method based on a self-attention enhanced CNN. The system comprises a feature enhancement CNN module, a self-attention mechanism module and a full connection layer. A video sequence is input into a feature enhancement CNN module, feature vectors output by the feature enhancement CNN module are input into the self-attentionmechanism module, feature vectors output by the self-attention mechanism module are input into the full connection layer, and the full connection layer maps the feature vectors into a sample marking space to realize classification; the feature enhancement CNN module adds a plurality of convolution layers in a backbone network, leads out a feature enhancement branch from a middle layer of the backbone network, fuses the output of the feature enhancement branch with the output of the backbone network, and replaces a full connection layer in the network with a global flat pooling layer. The system provided by the invention is lower in complexity, can effectively improve the accuracy of video sequence expression recognition, and has a wide application prospect in the fields of human-computer interaction, wisdom education, patient monitoring and the like.

Description

technical field [0001] The invention relates to the technical field of facial expression recognition, in particular to a video sequence facial expression recognition system and method based on self-attention enhanced CNN. Background technique [0002] Facial expressions contain rich emotional information, which is one of the important ways of expressing human emotions and an effective means for people to communicate non-verbal emotions. People can express their emotions through facial expressions, and can also accurately identify the inner emotional changes of the other party. Therefore, accurate recognition of facial expressions has important research value and application prospects, and has become a research hotspot in the field of artificial intelligence in recent years. [0003] A facial expression recognition system generally includes four steps: image preprocessing, face detection and face region segmentation, expression feature extraction, and expression classificati...

AI Technical Summary

Problems solved by technology

[0008] (1) Most of the existing research focuses on static single-frame images, and there are not many researches on facial expression recognition based on video sequences, and most of the research results are verified on the video database collected in the experimental environment, for example, CK+, MMI, Oulu-CASIA, etc., the facial expressions in these data are exaggerated and less disturbed by noise, which has little reference value for practical applications;

[0009] (2) The existing facial expression video data collected in the real environment is less, which leads to insufficient training samples of the deep neural network, which seriously affects the network performance

At the same time, due to inter-individual differences such as age, gender, and race, as well as intra-individual changes in lighting, posture, occlusion, and accessories, the quality of collected facial expression samples is uneven.

[0010] These all increase the difficulty of designing a real-time and accurate unconstrained facial expression recognition system. The existing research on facial expression recognition based on deep neural network still has a lot of room for improvement in performance.

Images