Expression recognition method and system based on cross attention and depth center loss

Abstract

The invention provides an expression recognition method and system based on cross attention and depth center loss, and relates to the technical field of computer vision, and the method comprises the following steps: obtaining a facial expression data set, and carrying out the preprocessing of the facial expression data set, and obtaining a preprocessed expression data set; constructing a facial expression recognition model, and training the facial expression recognition model based on the preprocessed expression data set to obtain a trained facial expression recognition model; and performing expression recognition by using the trained facial expression recognition model to obtain an expression recognition result. According to the method, double-branch feature optimization is performed through depth feature weighting and multi-head cross attention, and joint loss is constructed to supervise and train the model, so that the problems of insufficient local expression feature capture, serious feature redundancy, fuzzy classification boundary and weak model generalization ability in the existing expression recognition technology are effectively solved; and cooperative improvement of expression recognition precision, robustness and training efficiency is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to an expression recognition method and system based on cross-attention and deep center loss. BACKGROUND

[0002] Face expression recognition, as one of the core technologies in human-computer interaction, intelligent security, and affective computing, aims to automatically identify the emotional state (such as happy, sad, angry, surprised, etc.) of a face in an image or video through a computer, realize emotional communication and intelligent response between human and computer, and has become a research hotspot in the field of computer vision and pattern recognition. With the rapid development of deep learning technology, expression recognition technology has gradually transitioned from the traditional hand-crafted feature extraction (such as LBP, HOG, etc.) stage to the end-to-end learning stage based on deep neural networks, and the recognition accuracy and application scenario coverage have been significantly improved.

[0003] The development of existing expression recognition technology mainly focuses on two core directions: feature extraction optimization and model training improvement. In terms of feature extraction, researchers often use convolutional neural networks (CNN) as the basic architecture, stack convolutional layers and pooling layers to capture high-level semantic features of faces, and some schemes introduce attention mechanisms (such as spatial attention and channel attention) to try to strengthen the feature attention to key areas of expressions (such as mouth corners, forehead, and eyes). In terms of model training, to solve the problem of fuzzy classification boundaries, researchers have proposed improved loss functions such as center loss and triplet loss, which constrain the aggregation of similar features and the separation of different features to improve the discriminability of features. In addition, some schemes also use data preprocessing (such as face alignment and normalization) and pre-trained network transfer learning to shorten the model training period and improve the generalization performance in small sample scenarios.

[0004] Although the existing technology has made certain progress, expression recognition still faces many technical challenges in complex real-world scenarios (such as pose changes, lighting differences, and varying expression intensities), which are specifically manifested as follows: Firstly, in the prior art, most attention mechanisms use a single attention head or a fixed scale convolution kernel design, which can only focus on single-dimensional or fixed-range features, and cannot cover multi-scale and multi-region local expression features. Although some solutions introduce multi-head attention, they lack cross collaboration between spatial and channel dimensions, resulting in insufficient comprehensiveness and pertinence of local feature capture, making it difficult to accurately depict subtle changes in expressions. Secondly, the spatial convolution feature map extracted by the convolutional neural network not only contains effective features related to expressions, but also contains redundant information such as background noise and irrelevant regions of the face (such as hair and cheek contours). The prior art lacks a precise feature selection mechanism, and simple pooling or channel compression cannot effectively filter out irrelevant features, resulting in a large amount of redundant information that needs to be processed in the subsequent classification stage, which not only increases the model's computational load but also reduces the efficiency of feature discrimination, thereby affecting the recognition accuracy. Finally, the training of existing expression recognition models relies on a single loss function (such as Softmax loss and single center loss), which can only guide model training from a single dimension of "classification result optimization" or "feature distribution constraint". For example, Softmax loss only focuses on the correctness of class prediction and cannot effectively constrain the aggregation of features of the same class; traditional center loss can optimize feature distribution, but lacks collaborative constraints on classification results and feature complementarity, resulting in problems such as dispersion of features of the same class and overlap of features of different classes during model training, blurred classification boundaries, and reduced recognition robustness in complex scenarios.

[0005] How to solve the above technical problems is the task faced by the present application. SUMMARY

[0006] To solve the problems of the prior art, the present application provides a method for recognizing expressions based on cross-attention and deep center loss, which uses deep feature weighting and multi-head cross-attention to optimize features in two branches and constructs a joint loss to supervise the training of the model, effectively solving the problems of insufficient capture of local expression features, severe feature redundancy, blurred classification boundaries, and weak model generalization ability in existing expression recognition technology, and achieving the coordinated improvement of expression recognition accuracy, robustness, and training efficiency.

[0007] The technical solution adopted by the present application to solve its technical problems is: the present application provides a method for recognizing expressions based on cross-attention and deep center loss, comprising the following steps Obtain a facial expression dataset and preprocess it to obtain a preprocessed expression dataset; A facial expression recognition model is constructed by training the model on a preprocessed expression dataset. The trained model includes a convolutional feature extraction network, a deep feature weighting network, a multi-head cross-attention network, a feature fusion network, and a classifier. The convolutional feature extraction network takes the preprocessed expression dataset as input, and its output is connected to the inputs of the deep feature weighting network and the multi-head cross-attention network. The outputs of the deep feature weighting network and the multi-head cross-attention network are connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the classifier. The trained facial expression recognition model is used to perform expression recognition, and the expression recognition results are obtained.

[0008] Preferably, the preprocessing includes face detection and alignment, image normalization, and data augmentation of the facial expression dataset.

[0009] Preferably, the convolutional feature extraction network uses a pre-trained ResNet-18 network to extract spatial convolutional feature maps from the preprocessed facial expression dataset.

[0010] Preferably, the deep feature weighting network includes a context encoder, a multi-head binary classifier, and a feature weighting unit; The context encoder consists of three trainable linear fully connected layers stacked together, which encode the spatial convolutional feature map to obtain the latent feature vector. The multi-head binary classifier calculates the feature inclusion probability and feature exclusion probability for each latent feature vector; the formula is expressed as follows:

[0011] in, and The learnable weights and biases for each classification head, with subscripts. Indicates inclusion, subscript Indicates exclusion. and They represent The Middle The feature inclusion probability and feature exclusion probability of the dimension are calculated. Based on the feature inclusion probability and feature exclusion probability, the attention weight is calculated using the Softmax function, as shown in the following formula:

[0012] in, It is an exponential function; The feature weighting unit weights the spatial convolutional feature map according to the attention weights to obtain the weighted depth feature map.

[0013] Preferably, the multi-head cross-attention network includes several parallel cross-attention heads, each of which includes a spatial attention unit and a channel attention unit connected in sequence; The spatial attention unit uses multi-scale convolution kernels to extract and weight features from the spatial convolution feature map to obtain a spatial attention feature map. The channel attention unit includes a globally average pooling layer and an adaptive convolutional layer connected in sequence. The spatial attention feature map is pooled by the globally average pooling layer to generate a pooled feature map. The size of the adaptive convolutional kernel is calculated based on the pooled feature map, and a one-dimensional convolution is performed by the adaptive convolutional layer based on the size of the adaptive convolutional kernel to obtain the channel attention. The spatial attention feature map is weighted according to the channel attention to obtain the attention feature map.

[0014] Preferably, the calculation of the adaptive convolutional kernel size based on the pooled feature map includes calculation based on the channel dimension of the spatial attention feature map, combined with a nonlinear function; the formula is expressed as follows:

[0015] in, Indicates and The odd numbers that are closest to each other The bias constant is This is a scaling constant.

[0016] Preferably, the feature fusion network includes a feature scaling layer, a weighted fusion layer, and a feature stitching layer; The feature scaling layer uses the Log-Softmax function to scale all attention feature maps to obtain scaled attention feature maps; The weighted fusion layer performs a weighted summation on the scaled attention feature maps to obtain a fused feature map. The feature stitching layer stitches the fused feature map with the weighted depth feature map through channels to obtain a joint feature map.

[0017] Preferably, the training of the facial expression recognition model based on the preprocessed expression dataset includes: Construct a joint loss function and use the joint loss function to supervise the training of the facial expression recognition model; The joint loss function includes sparse center loss, partition loss, and Softmax classification loss, which is calculated by weighted summation of sparse center loss, partition loss, and Softmax classification loss.

[0018] Preferably, the sparse center loss formula is expressed as follows:

[0019] in, This indicates element-wise multiplication. For along the first The first dimension embedded in space Weights of each deep feature; The partitioning loss formula is expressed as follows:

[0020] in, This represents the total number of samples in the current training batch. The number of channels in the attention feature map. To note the number of cross-heads, For the first The sample at the th On each channel, all The variance of the attention weight values ​​generated by each attention head on this channel; The Softmax classification loss formula is expressed as follows:

[0021] in, This represents the total number of samples in the current training batch. The total number of emoji categories. For the first The true label of each sample For the model to predict the first The sample belongs to the first The probability of each emoji category.

[0022] This invention also provides an expression recognition system based on cross-attention and deep center loss, including... The data acquisition and preprocessing module is used to acquire and preprocess facial expression datasets to obtain preprocessed expression datasets. The model building and training module is used to build a facial expression recognition model. It trains the facial expression recognition model based on the preprocessed expression dataset to obtain the trained facial expression recognition model. The facial expression recognition execution module is used to perform facial expression recognition using the trained facial expression recognition model and obtain the expression recognition results.

[0023] The beneficial effects of this invention are as follows: By optimizing dual-branch features through deep feature weighting and multi-head cross-attention, and constructing a joint loss to supervise model training, this invention effectively solves the problems of insufficient local expression feature capture, severe feature redundancy, blurred classification boundaries, and weak model generalization ability in existing expression recognition technologies, achieving a synergistic improvement in expression recognition accuracy, robustness, and training efficiency. The deep feature weighting network, through a collaborative architecture of "context encoder - multi-head binary classifier - feature weighting unit," first encodes the spatial convolutional feature map using a context encoder composed of three trainable fully connected layers, generating a latent feature vector containing key semantic information; then, the multi-head binary classifier calculates the feature inclusion / exclusion probability, and combines it with the Softmax function to generate accurate channel attention weights; finally, the feature weighting unit performs weighted filtering on the original spatial convolutional feature map according to the weights, retaining effective features strongly correlated with expression recognition and filtering redundant noise features. The multi-head cross-attention network employs a design with several parallel cross-attention heads. The spatial attention unit uses multi-scale convolutional kernels to process the spatial convolutional feature map, accurately capturing local facial expression features at different scales (such as fine lines around the eyes and the curvature of the corners of the mouth), thus enhancing features. The channel attention unit combines global average pooling with adaptive convolutional layers, adaptively calculating the convolutional kernel size based on the channel dimension of the spatial attention feature map. It generates precise channel attention weights through one-dimensional convolution, further filtering effective features along the channel dimension. This parallel processing and cross-collaboration design of multiple attention heads comprehensively covers multi-scale features in key facial expression regions, solving the problem of incomplete local feature capture in traditional single attention mechanisms and significantly improving the completeness and relevance of facial expression features. A joint loss function is constructed, comprising sparse center loss, partitioning loss, and Softmax classification loss. This function provides multi-dimensional supervision of model training through weighted summation: the sparse center loss constrains the distribution of weighted deep features, making similar expression features more clustered and dissimilar features more separated; the partitioning loss constrains multiple feature maps of the multi-head cross-attention network, strengthening the complementarity and discriminative power of features from different attention heads; and the Softmax classification loss directly optimizes the classifier's category prediction accuracy. Compared to a single loss function, the joint loss function guides model training from three core dimensions: feature distribution, feature complementarity, and classification results. This effectively addresses issues such as ambiguous classification boundaries and discrete feature distribution, further improving the model's expression classification accuracy and stability. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating the method steps of the present invention.

[0025] Figure 2 This is a system module diagram of the present invention.

[0026] Figure 3 This is a network structure diagram of the facial expression recognition model of the present invention.

[0027] Figure 4 This is a network model framework diagram of the deep feature weighted network of the present invention.

[0028] Figure 5 This is a structural diagram of the spatial attention unit of the multi-head cross-attention network of the present invention.

[0029] Figure 6 This is a structural diagram of the channel attention unit of the multi-head cross-attention network of the present invention.

[0030] Figure 7 This is a schematic diagram of the Softmax loss, sparse center loss, and confusion matrix of the method of the present invention on the RAF-DB dataset in Embodiment 3 of the present invention.

[0031] Figure 8 This is a schematic diagram of the RAF-DB dataset confusion matrix in Embodiment 4 of the present invention.

[0032] Figure 9 This is a schematic diagram illustrating the recognition accuracy of the number of head branches under different values ​​in Embodiment 4 of the present invention. Detailed Implementation

[0033] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0034] Example 1: See Figure 1 , Figures 3-6 As shown, this embodiment is an expression recognition method based on cross-attention and depth center loss, including the following steps. S1. Obtain the facial expression dataset and preprocess it to obtain the preprocessed expression dataset; Preprocessing includes face detection and alignment, image normalization, and data augmentation on the facial expression dataset.

[0035] It should be noted that face detection and alignment involves using mature face detectors (such as MTCNN and RetinaFace) to automatically locate face regions in the input image. Affine or similarity transformations are then performed based on facial key points (such as the eyes, nose tip, and corners of the mouth) to normalize all detected faces to a uniform scale and pose (e.g., facing forward). This step ensures spatial consistency of the input for subsequent network processing, reducing interference introduced by variations in pose and scale.

[0036] Image normalization: The pixel values ​​of the aligned face image are linearly scaled from the original range (e.g., 0-255) to a range suitable for the model input (e.g., [-1,1] or [0,1]), and channel normalization is performed (e.g., subtracting the mean and dividing by the standard deviation). This helps accelerate the convergence process of model training and improves numerical stability.

[0037] Data augmentation: During the training phase, a series of random transformations are applied online to the normalized images, including but not limited to: random horizontal flipping, small-amplitude random rotation (e.g., ±10°), brightness / contrast fine-tuning, and slight random cropping. These operations effectively expand the diversity of the training data without altering the semantics of facial expressions.

[0038] S2. Construct a facial expression recognition model. Train the facial expression recognition model based on the preprocessed expression dataset to obtain the trained facial expression recognition model. The facial expression recognition model includes a convolutional feature extraction network, a deep feature weighting network (DACL), a multi-head cross-attention network (MAN), a feature fusion network (AFN), and a classifier. The convolutional feature extraction network takes the preprocessed expression dataset as input, and its output is connected to the input of the deep feature weighting network and the input of the MAN. The outputs of the deep feature weighting network and the MAN are connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the classifier. The convolutional feature extraction network uses a pre-trained ResNet-18 network to extract spatial convolutional feature maps from the pre-processed facial expression dataset.

[0039] Deep feature-weighted networks include a context encoder, a multi-head binary classifier, and feature weighting units; The context encoder consists of three trainable linear fully connected layers stacked together. It encodes the spatial convolutional feature map to obtain the latent feature vector; the formula is expressed as follows:

[0040] in, This is the last spatial convolutional feature map of the convolutional feature extraction network. and The attention network is the first The weights and biases of the linear layer, where Batch normalized normalized (BN) and modified linear units (ReLU) are inserted between layers to capture nonlinear relationships between them. Finally, the hyperbolic tangent function tanh (.) is used to smooth the gradients in the network. The weights of the linear layers are reset using a normal distribution initialization method, and the biases are reset to 0. This is achieved by extracting the... The latent feature vector of each sample This is used to eliminate features irrelevant to facial expression recognition while retaining important features. The network can also adjust the parameters of each layer according to specific tasks. In this embodiment, the three fully connected layers are designed with dimensions of 3584, 512, and 64, respectively, and the latent feature vector is 64-dimensional.

[0041] The multi-head binary classifier calculates the feature inclusion probability and feature exclusion probability for each latent feature vector; the formula is expressed as follows:

[0042] in, and The learnable weights and biases for each classification head, with subscripts. Indicates inclusion, subscript Indicates exclusion. and They represent The Middle The feature inclusion probability and feature exclusion probability of the dimension are calculated. Based on the feature inclusion probability and feature exclusion probability, the attention weight is calculated using the Softmax function, as shown in the following formula:

[0043] in, It is an exponential function.

[0044] It should be noted that the role of the multi-head binary classifier is to classify the most recently obtained binary data. The dimensional feature vectors are shared among the linear units, meaning each head has two outputs.

[0045] The feature weighting unit weights the spatial convolutional feature map according to the attention weights to obtain the weighted depth feature map.

[0046] It should be noted that the context encoder consists of three fully connected layers stacked sequentially, using Batch Normalization (BN), ReLU, and finally the Tanh activation function between layers. The feature maps are flattened and encoded into 64-dimensional latent feature vectors, achieving abstraction and dimensionality reduction of high-order features, with the aim of filtering out noise information irrelevant to expression recognition.

[0047] The multi-head binary classifier comprises 512 parallel classification heads (corresponding to the number of backbone feature channels). Each head receives and outputs two scalars through a lightweight linear layer: an inclusion probability and an exclusion probability. These two scores are then normalized to an attention weight between 0 and 1 using a Softmax function. Higher weight values ​​indicate that the feature channel is more important for recognizing the current expression.

[0048] The feature weighting unit multiplies the obtained 512-dimensional attention weight vector with the original spatial convolutional feature map channel by channel (Hadamard product). Important channels are enhanced, while unimportant channels are suppressed, thus generating a "weighted depth feature map".

[0049] The multi-head cross-attention network consists of several parallel cross-attention heads, each of which includes sequentially connected spatial attention units and channel attention units; The spatial attention unit uses multi-scale convolutional kernels to extract and weight features from the spatial convolutional feature map to obtain the spatial attention feature map. The channel attention unit includes a globally average pooling layer and an adaptive convolutional layer connected in sequence. The spatial attention feature map is pooled by the globally average pooling layer to generate a pooled feature map. The size of the adaptive convolutional kernel is calculated based on the pooled feature map, and a one-dimensional convolution is performed by the adaptive convolutional layer based on the size of the adaptive convolutional kernel to obtain the channel attention. The spatial attention feature map is weighted according to the channel attention to obtain the attention feature map.

[0050] The adaptive convolution kernel size is calculated based on the pooled feature map, including the channel dimension based on the spatial attention feature map, and is calculated using a non-linear function; the formula is expressed as follows:

[0051] in, Indicates and The odd numbers that are closest to each other This is a bias constant used to adjust the offset of the mapping relationship. In this embodiment, it is set as follows: . This is a scaling constant used to control the number of channels. Sensitivity to the influence of kernel size. In this embodiment, it is set... .

[0052] It should be noted that the "multi-head" design aims to allow the model to simultaneously focus on multiple key facial expression regions, avoiding omissions; "cross-attention" (combining spatial and channel order) can more finely model the importance of regions; and adaptive channel attention improves efficiency. It provides the model with comprehensive and detailed spatial feature representations, solving the problem of insufficient local subtle facial expression capture in traditional methods. This invention improves the multi-head cross-attention network, which contains many lightweight, independent, parallel cross-attention heads. Each cross-attention head consists of spatial attention units and channel attention units in sequence. Different sized convolutional kernels are designed in the spatial attention units, and the channel attention units are connected to the ends of the spatial attention units, strengthening the attention map through cross-channel interaction. In the channel attention units, this paper adopts a partial channel interaction method that does not reduce dimensionality, and automatically selects the dimensionality of the convolutional kernels adaptively based on the number of channels. Reducing the dimensionality between the two fully connected layers of the SE block would negatively affect the learning of channel attention weight parameters; during weight learning, the channel dimensions directly correspond.

[0053] The feature fusion network includes a feature scaling layer, a weighted fusion layer, and a feature concatenation layer; The feature scaling layer uses the Log-Softmax function to scale all attention feature maps, resulting in scaled attention feature maps. The weighted fusion layer performs a weighted summation on the scaled attention feature maps to obtain a fused feature map. The feature concatenation layer concatenates the fused feature map with the weighted depth feature map through channels to obtain a joint feature map.

[0054] Training a facial expression recognition model based on a preprocessed expression dataset includes: Construct a joint loss function and use the joint loss function to supervise the training of the facial expression recognition model; The joint loss function includes sparse center loss, partitioning loss, and softmax classification loss, and is calculated by weighted summation of these three losses. The formula is as follows:

[0055] in, For depth center loss, For Softmax classification loss, For partition loss, The hyperparameter weights of the depth center loss are... The hyperparameter weights are the partition loss parameters.

[0056] For one Classification problem, dividing the embedding space into There are clusters. Given that there are The training batch of each sample, then , belongs to the The class of There are 3 deep feature vectors. The formula for minimizing the center loss is as follows:

[0057] in, for The center of the corresponding category.

[0058] Our goal is to select only a subset of important feature vectors from a deep feature vector to aid in identification. During the identification process, irrelevant features need to be filtered out. The calculated center distances in each dimension are weighted to obtain the sparse center loss formula, which is expressed as follows:

[0059]

[0060] in, This indicates element-wise multiplication. For along the first The first dimension embedded in space Weights of each deep feature; The partitioning loss formula is expressed as follows:

[0061] in, This represents the total number of samples in the current training batch. The number of channels in the attention feature map. To note the number of cross-heads, For the first The sample at the th On each channel, all The variance of the attention weight values ​​generated by each attention head on this channel reflects the degree of difference in the response of different attention heads to this channel. The larger the variance, the more dispersed the attention areas of the attention heads are. The Softmax classification loss formula is expressed as follows:

[0062] in, This represents the total number of samples in the current training batch. The total number of emoji categories. For the first The true label of a sample is usually in one-hot encoded form. For the model to predict the first The sample belongs to the first The probability of each emoji category.

[0063] It should be noted that the end-to-end supervised learning strategy is employed, and its specific training process includes: The constructed facial expression recognition model is used to perform forward propagation on the input samples, and the sparse center loss, partition loss and Softmax classification loss are calculated respectively. The total value of the joint loss function is then calculated according to the formula.

[0064] The model is optimized using stochastic gradient descent. Specifically, the gradient of the joint loss with respect to all trainable parameters of the model (including the parameters of the convolutional feature extraction network, deep feature weighting network, multi-head cross-attention network, feature fusion network, and classifier) ​​is calculated. Based on the gradient, all network parameters are updated using backpropagation.

[0065] In each training batch, the class centers in the sparse center loss are updated according to a moving average strategy, with the following update formula:

[0066] The Kronecker function is defined as: when hour, ,otherwise .

[0067] The training process is iterative, using a validation set to monitor model performance. When the model's recognition accuracy on the validation set no longer significantly improves, it is considered to have converged, training stops, and the final usable trained model is obtained. Hyperparameters are used to balance the contributions of the three loss parameters and are typically preset before training based on validation set performance or determined through cross-validation.

[0068] S3. Use the trained facial expression recognition model to perform expression recognition and obtain the expression recognition results.

[0069] Deploy the trained and converged model to the application environment. For a new input face image, perform the same preprocessing as S1 (detection, alignment, normalization). Then, directly input the processed image into the trained model. The model automatically performs forward propagation, processing all the above network modules, and finally, the classifier outputs a probability distribution across all preset expression categories. The category with the highest probability is taken as the model's expression recognition result.

[0070] Example 2: See Figure 2 As shown, this embodiment is an expression recognition system based on cross-attention and deep center loss, including... The data acquisition and preprocessing module is used to acquire and preprocess facial expression datasets to obtain preprocessed expression datasets. The model building and training module is used to build a facial expression recognition model. It trains the facial expression recognition model based on the preprocessed expression dataset to obtain the trained facial expression recognition model. The facial expression recognition execution module is used to perform facial expression recognition using the trained facial expression recognition model and obtain the expression recognition results.

[0071] Example 3: To verify the recognition performance of the proposed deep center loss method, this embodiment uses the publicly available natural facial expression recognition dataset RAF-DB. Experiments on this dataset are conducted to verify the recognition performance of the proposed deep center loss method. The method is also evaluated on this dataset and compared with two base networks (Softmax loss and center loss) and current state-of-the-art methods.

[0072] This article is in Figure 7 The paper presents confusion matrices obtained using baseline methods (Softmax loss and sparse center loss) and the proposed method on two natural facial expression datasets to evaluate the recognition accuracy of individual categories. On the RAF-DB dataset, compared to the Softmax loss method, the proposed method improves the recognition accuracy for all expression categories except for surprise and disgust, with improvements of 0.03% and 0.05% for anger and fear, respectively. Compared to the sparse center loss method, the proposed method performs better overall on RAF-DB, as the recognition accuracy for surprise, fear, and disgust is higher than that of the sparse center loss method.

[0073] Example 4: To verify the impact of the multi-head cross-attention network proposed in this invention on recognition accuracy, this embodiment presents the recognition results of the network model on the RAF-DB dataset under natural environmental conditions.

[0074] Details are as follows: In the accuracy comparison on the RAF-DB dataset, our method outperforms the basic network method (ResNet-18) and other state-of-the-art methods. Our model achieves a recognition accuracy of 89.31% and an average recognition accuracy of 82.50% on the RAF-DB dataset, representing improvements of 3.06% and 5.18% respectively compared to the basic network ResNet-18. The confusion matrix for RAF-DB is shown below. Figure 8 As shown.

[0075] Furthermore, this embodiment conducts ablation experiments on the RAF-DB dataset. Details are as follows: The number of cross-attention heads affects the recognition performance of the network model. Figure 9The accuracy results are shown for different numbers of cross-attention heads. The cross-attention network structure proposed in this invention outperforms a single attention module in terms of recognition accuracy. When the number of attention heads is too small, the effect of cross-attention is lost; while when the number of attention heads is too large, the image segmentation becomes too scattered, losing its overall coherence. Therefore, using four cross-attention heads achieves the highest performance.

[0076] The technical features of this invention not described can be implemented by or using existing technology, and will not be repeated here. Of course, the above description is not a limitation of this invention, and this invention is not limited to the examples above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of this invention should also be within the protection scope of this invention.

Claims

1. An expression recognition method based on cross-attention and depth center loss, characterized in that, Includes the following steps Obtain the facial expression dataset and preprocess it to obtain the preprocessed expression dataset; A facial expression recognition model is constructed by training the model on a preprocessed expression dataset. The trained model includes a convolutional feature extraction network, a deep feature weighting network, a multi-head cross-attention network, a feature fusion network, and a classifier. The convolutional feature extraction network takes the preprocessed expression dataset as input, and its output is connected to the inputs of the deep feature weighting network and the multi-head cross-attention network. The outputs of the deep feature weighting network and the multi-head cross-attention network are connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the classifier. The trained facial expression recognition model is used to perform expression recognition, and the expression recognition results are obtained.

2. The facial expression recognition method based on cross-attention and depth center loss according to claim 1, characterized in that, The preprocessing includes face detection and alignment, image normalization, and data augmentation on the facial expression dataset.

3. The facial expression recognition method based on cross-attention and depth center loss according to claim 2, characterized in that, The convolutional feature extraction network uses a pre-trained ResNet-18 network to extract spatial convolutional feature maps from the pre-processed facial expression dataset.

4. The facial expression recognition method based on cross-attention and depth center loss according to claim 3, characterized in that, The deep feature weighting network includes a context encoder, a multi-head binary classifier, and a feature weighting unit; The context encoder consists of three trainable linear fully connected layers stacked together, which encode the spatial convolutional feature map to obtain the latent feature vector. The multi-head binary classifier calculates the feature inclusion probability and feature exclusion probability for each latent feature vector; the formula is expressed as follows: in, and The learnable weights and biases for each classification head, with subscripts. Indicates inclusion, subscript Indicates exclusion. and They represent The Middle The feature inclusion probability and feature exclusion probability of the dimension are calculated. Based on the feature inclusion probability and feature exclusion probability, the attention weight is calculated using the Softmax function, as shown in the following formula: in, It is an exponential function; The feature weighting unit weights the spatial convolutional feature map according to the attention weights to obtain the weighted depth feature map.

5. The facial expression recognition method based on cross-attention and depth center loss according to claim 4, characterized in that, The multi-head cross-attention network includes several parallel cross-attention heads, each of which includes sequentially connected spatial attention units and channel attention units; The spatial attention unit uses multi-scale convolution kernels to extract and weight features from the spatial convolution feature map to obtain a spatial attention feature map. The channel attention unit includes a globally average pooling layer and an adaptive convolutional layer connected in sequence. The spatial attention feature map is pooled by the globally average pooling layer to generate a pooled feature map. The size of the adaptive convolutional kernel is calculated based on the pooled feature map, and a one-dimensional convolution is performed by the adaptive convolutional layer based on the size of the adaptive convolutional kernel to obtain the channel attention. The spatial attention feature map is weighted according to the channel attention to obtain the attention feature map.

6. The facial expression recognition method based on cross-attention and depth center loss according to claim 5, characterized in that, The calculation of the adaptive convolutional kernel size based on the pooled feature map includes calculation based on the channel dimension of the spatial attention feature map, combined with a nonlinear function; the formula is expressed as follows: in, Indicates and The odd numbers that are closest to each other The bias constant is This is a scaling constant.

7. The facial expression recognition method based on cross-attention and depth center loss according to claim 6, characterized in that, The feature fusion network includes a feature scaling layer, a weighted fusion layer, and a feature splicing layer; The feature scaling layer uses the Log-Softmax function to scale all attention feature maps to obtain scaled attention feature maps; The weighted fusion layer performs a weighted summation on the scaled attention feature maps to obtain a fused feature map. The feature stitching layer stitches the fused feature map with the weighted depth feature map through channels to obtain a joint feature map.

8. The facial expression recognition method based on cross-attention and depth center loss according to claim 7, characterized in that, The training of the facial expression recognition model based on the preprocessed expression dataset includes: Construct a joint loss function and use the joint loss function to supervise the training of the facial expression recognition model; The joint loss function includes sparse center loss, partition loss, and Softmax classification loss, which is calculated by weighted summation of sparse center loss, partition loss, and Softmax classification loss.

9. The facial expression recognition method based on cross-attention and depth center loss according to claim 8, characterized in that, The sparse center loss formula is expressed as follows: in, This indicates element-wise multiplication. For along the first The first dimension embedded in space Weights of each deep feature; The partitioning loss formula is expressed as follows: in, This represents the total number of samples in the current training batch. The number of channels in the attention feature map. To note the number of cross-heads, For the first The sample at the th On each channel, all The variance of the attention weight values ​​generated by each attention head on this channel; The Softmax classification loss formula is expressed as follows: in, This represents the total number of samples in the current training batch. The total number of emoji categories. For the first The true label of each sample For the model to predict the first The sample belongs to the first The probability of each emoji category.

10. An expression recognition system based on cross-attention and depth center loss, characterized in that, The steps for performing the facial expression recognition method based on cross-attention and deep center loss as described in any one of claims 1 to 9 include: The data acquisition and preprocessing module is used to acquire and preprocess facial expression datasets to obtain preprocessed expression datasets. The model building and training module is used to build a facial expression recognition model. It trains the facial expression recognition model based on the preprocessed expression dataset to obtain the trained facial expression recognition model. The facial expression recognition execution module is used to perform facial expression recognition using the trained facial expression recognition model and obtain the expression recognition results.

Images