Joint emotion recognition method and system based on action unit driven attention

Abstract

This application provides a joint emotion recognition method and system based on action unit-driven attention, relating to the fields of computer vision and affective computing. The method includes: inputting a feature map into an action unit (AU)-driven attention branch, processing it through a sigmoid activation function to obtain an activation vector, and inputting this vector into a learnable mapping layer; projecting the AU information back into the spatial geometric space to generate a single-channel spatial attention map, which is then multiplied element-wise with the feature map to obtain a weighted feature map; determining continuous values ​​for discrete emotion category probabilities, valence, and arousal; constructing a total loss function; and training the model using the acquired AffectNet and Aff-wild2 datasets to determine the target model for emotion recognition. This application achieves end-to-end joint optimization of AU detection and emotion recognition, significantly enhancing the interpretability and generalization ability of the model while improving emotion recognition accuracy, enabling it to better adapt to the emotion recognition needs in complex scenarios.

Images