A Micro-Expression Recognition Method Based on Multi-task Learning for Representational au Region Extraction

Abstract

The invention relates to a micro-expression recognition method for extracting representative AU regions based on multi-task learning, comprising: A. preprocessing micro-expression videos; B. obtaining the position of the AU region to obtain the most representative AU region; C. Divide the training set and the test set; D. Train the AU mask feature extraction network model; E. Send the trained AU mask feature extraction network to obtain a face image sequence containing only representative AUs; F. The training contains 3D-ResNet network with non-local modules; G. Feed into the 3D-ResNet network containing non-local modules to get the classification accuracy. The present invention considers the contribution of different AUs to micro-expression recognition, also solves the problem of insufficient micro-expression samples, increases the number of training samples, and improves the micro-expression recognition performance.

Description

technical field [0001] The invention relates to a micro-expression recognition method for extracting representative AU regions based on multi-task learning, and belongs to the technical field of deep learning and pattern recognition. Background technique [0002] The research on facial emotions began with Charles Darwin, who pointed out the main rules of emotion production, introduced the external manifestations of different emotions and the relationship between emotions and the nervous system in detail, and laid the foundation for emotion research. When Ekman and Friesen observed a video of a patient with severe depression hiding suicidal intentions, they found a picture containing a desperate expression. The picture lasted only 2 / 25 of a second. Ekman and others named this short expression as a micro-expression. . [0003] Micro-expressions are different from ordinary macro-expressions. Micro-expressions are short-duration and unconscious facial expressions that reveal th...

AI Technical Summary

Problems solved by technology

The AGACN proposed by Xie et al. combines AU and micro-expression labels, and models different AUs based on the muscle movement of the face and its relationship information. Due to the shortcomings of limited and unbalanced micro-expression training samples, Xie et al. A data enhancement method is proposed to effectively improve the performance of micro-expression recognition. However, due to the large number of AUs, the introduction of graph convolutional network needs to consider the relationship between multiple AU nodes, and the calculation amount is relatively large, which leads to the experiment low efficiency

Puneet Gupta proposed MERASTCMERASTC to alleviate the problem of easy over-fitting of micro-expressions, combine AU, key points and appearance features to encode the subtle deformation of micro-expression video sequences, and propose a new neutral face normalization method to speed up The efficiency of micro-expression recognition, but this method requires a neutral frame in the video sequence, so it has relatively large limitations

Lo et al. proposed MERGCN, which extracts AU features through a 3D convolutional neural network, and then uses a graph convolutional network to discover dependencies between AU nodes to help classify micro-expressions. This method uses all AUs to construct a graph convolutional network, without Select the most representative AU, making the computational cost higher

Images