Facial action unit recognition method and device based on adaptive attention and space-time correlation

Abstract

The invention discloses a facial action unit recognition method and device based on adaptive attention and space-time correlation, and the method comprises the steps: firstly extracting original continuous image frames needed by model training from video data to form a training data set, and then carrying out the preprocessing of the original image frames, and obtaining an amplified image frame sequence; a convolutional neural network module I is constructed to extract layered multi-scale region features of the amplified image frame, a convolutional neural network module II is constructed to carry out adaptive attention regression learning of the facial action unit, and an adaptive space-time diagram convolutional neural network module III is constructed to learn space-time association of the facial action unit; and finally, constructing a full connection layer module IV to perform facial action unit recognition. According to the method, an end-to-end deep learning framework is adopted for learning action unit recognition, the motion condition of facial muscles in a two-dimensional image can be effectively recognized by utilizing the mutual dependency relationship and the space-time correlation between facial action units, and construction of a facial action unit recognition system is achieved.

Description

technical field [0001] The invention relates to a facial action unit recognition method and device based on adaptive attention and space-time correlation, and belongs to computer vision technology. Background technique [0002] In order to study human facial expressions more carefully, the famous American emotional psychologist Ekman first proposed the Facial Action Coding System (FACS) in 1978, and made important improvements in 2002. The facial action coding system is divided into several facial action units that are both independent and interconnected according to the anatomical characteristics of the face. The facial expression can be reflected by the action features of these facial action units and the main areas they control. [0003] With the development of computer technology and information technology, deep learning technology has been widely used. In the field of AU (Facial Action Unit) recognition, research on AU recognition based on deep learning models has beco...

AI Technical Summary

Problems solved by technology

[0004] Although the automatic recognition of facial action units has made impressive progress, current AU detection methods based on region learning only utilize AU labels because AUs do not have obvious contours and textures and may vary with people and expressions. To supervise neural networks to adaptively learn implicit attention that often captures irrelevant regions

However, in the AU detection method based on relational reasoning, all AUs share parameters during inference, ignoring the specificity and dynamics of each AU, so that the recognition accuracy is not high, and there is room for further improvement.

Images