Facial expression motion unit identification method based on space-time diagram convolutional network

Abstract

The invention discloses a facial expression motion unit identification method based on a space-time diagram convolutional network. The method comprises the steps: performing feature extraction on a facial motion unit AU area through a convolutional auto-encoder, then constructing a spatio-temporal relationship diagram of an AU sequence according to the AU spatio-temporal relationship closeness degree, and finally performing AU identification based on ST-GCN. According to the method, the facial motion unit is recognized based on the space-time diagram convolutional network, modeling is carried out on the space-time dependence relationship between the AUs by using the undirected space-time diagram model, and learning of AU deep representation features is carried out by using the space-time diagram convolutional network, so that the AU recognition accuracy is improved. The method can effectively solve the problems of poor robustness, low accuracy and the like of an AU detection model, and can be widely applied to expression analysis, emotion calculation and man-machine interaction application.

Description

technical field [0001] The present invention relates to the technical fields of computer vision and affective computing, and in particular to a human facial expression movement unit recognition (ActionUnit, AU) based on a Spatial-Temporal Graph Convolutional Networks (ST-GCN). It is widely used in emotion recognition, human-computer interaction and other fields. Background technique [0002] Facial expressions can reveal people's inner activities, mental state, and social behaviors that are communicated outwards. With the development of artificial intelligence, human-centered facial expression recognition has gradually attracted widespread attention in the industry and academia. Expression analysis using facial motion coding system is one of the common methods for facial expression recognition. [0003] The Facial Action Coding System (FACS) divides the human face into 44 facial movement units according to the movement of muscles from an anatomical point of view, which is ...

AI Technical Summary

Problems solved by technology

But at the same time, AU detection still faces many challenges. In actual application scenarios, AU data has high complexity. Factors such as head posture, occlusion, and complex lighting will cause a significant decline in the performance of the AU recognition model.

In addition, due to individual differences in race, skin color, age, gender, etc., there are huge intra-class differences in AU recognition, which will also significantly affect the accuracy of AU recognition.

At the same time, because the labeling of AU requires professional personnel to spend a lot of time to complete, the data set available for training is far from meeting the needs of various race detection in complex scenarios, and the data sample is small, which is prone to overfitting

[0005] Most of the existing AU detection models only consider the correlation between different AUs at the same time point, while ignoring the correlation of space and time

Images