Micro-expression recognition method for representative AU region extraction based on multi-task learning

Abstract

The invention relates to a micro-expression recognition method for representative AU region extraction based on multi-task learning. The method comprises the following steps: A, preprocessing a micro-expression video; B, acquiring the position of the AU region to obtain the most representative AU region; C, dividing a training set and a test set; D, training an AU mask feature extraction network model; E, sending the face image to the trained AU mask feature extraction network to obtain a face image sequence only containing representative AU; F, training a 3D-ResNet network comprising a non-local module; G, sending to a 3D-ResNet network containing a non-local module to obtain classification accuracy. According to the method, contributions of different AUs to micro-expression recognition are considered, the problem of insufficient micro-expression samples is solved, the number of training samples is increased, and the micro-expression recognition performance is improved.

Description

technical field [0001] The invention relates to a micro-expression recognition method for extracting representative AU regions based on multi-task learning, and belongs to the technical field of deep learning and pattern recognition. Background technique [0002] The study of facial emotions began with Charles Darwin, who pointed out the main rules of emotion generation, introduced in detail the external manifestations of different emotions and the relationship between emotions and the nervous system, and laid the foundation for emotion research. When Ekman and Friesen observed a video of a patient with severe depression hiding his suicide intentions, they found a picture containing a desperate expression, which lasted only 2 / 25 of a second, so Ekman and others named this short expression micro-expression . [0003] Micro-expressions are different from ordinary macro-expressions. Micro-expressions are short-duration and unconsciously made facial expressions that reveal the ...

AI Technical Summary

Problems solved by technology

The AGACN proposed by Xie et al. combines AU and micro-expression labels, and models different AUs based on the muscle movement of the face and its relationship information. Due to the shortcomings of limited and unbalanced micro-expression training samples, Xie et al. A data enhancement method is proposed to effectively improve the performance of micro-expression recognition. However, due to the large number of AUs, the introduction of graph convolutional network needs to consider the relationship between multiple AU nodes, and the calculation amount is relatively large, which leads to the experiment low efficiency

Puneet Gupta proposed MERASTCMERASTC to alleviate the problem of easy over-fitting of micro-expressions, combine AU, key points and appearance features to encode the subtle deformation of micro-expression video sequences, and propose a new neutral face normalization method to speed up The efficiency of micro-expression recognition, but this method requires a neutral frame in the video sequence, so it has relatively large limitations

Lo et al. proposed MERGCN, which extracts AU features through a 3D convolutional neural network, and then uses a graph convolutional network to discover dependencies between AU nodes to help classify micro-expressions. This method uses all AUs to construct a graph convolutional network, without Select the most representative AU, making the computational cost higher

Images