Video expression recognition method based on C3D-SA

Abstract

A C3D-SA-based video expression recognition method comprises the steps of S100, performing expression feature extraction on a video sequence through a three-dimensional convolutional neural network to obtain an expression feature matrix; s200, connecting the correlation between features in the self-attention mechanism layer learning expression feature matrix to obtain an attention weight value, and weighting the expression feature matrix to obtain a weighted expression feature matrix; s300, connecting a global mean pooling layer to perform feature mapping and dimension reduction on the weighted expression feature matrix, and randomly discarding part of feature values in the weighted expression feature matrix through a loss layer to obtain a new expression feature matrix; and S400, connecting a full connection layer to carry out feature mapping on the new expression feature matrix to obtain a final feature matrix, and outputting an expression recognition label from the obtained final feature matrix through a softmax layer. According to the method, the 3D convolutional neural network and the self-attention mechanism are combined to improve the recognition accuracy of the video facial expression.

Description

technical field [0001] The present disclosure belongs to the technical fields of computer vision and neural networks, and particularly relates to a video expression recognition method based on C3D-SA. Background technique [0002] Facial expression recognition is one of the research hotspots in the fields of computer vision, pattern recognition and human emotion understanding, and plays an increasingly important role in public security, criminal investigation, medical care, education, retail and other fields. [0003] Convolutional Neural Networks (CNN) is a classic deep learning neural network, which is widely used in facial expression recognition. Multilayer Perceptron (MLP) is more robust. The basic layer of the convolutional neural network includes the convolutional layer, the pooling layer, the activation layer and the fully connected layer, of which the convolutional layer is the most important part, which contains multiple convolution kernels for convolution operatio...

AI Technical Summary

Problems solved by technology

[0004] 2D CNN can perform different levels of spatial feature extraction on a single static image, but it cannot take into account the motion information between frames in the video, so it is not effective for video sequence processing

[0008] The above methods can recognize video facial expressions to a certain extent, but there are deficiencies: 1) The traditional feature extraction is not intelligent enough, the algorithm processing data is not efficient, and the performance is weak; 2) 3D CNN can only extract local spatio-temporal information and cannot Learn about long-term dependencies

Images