Facial expression recognition method based on feature deentanglement and self-distillation

Abstract

The invention discloses a facial expression recognition method based on feature deentanglement and self-distillation. The method comprises the following steps: S1, extracting an initial feature map from an input facial image through a backbone network; s2, generating a weight map based on the initial feature map, and decomposing the initial feature map into expression-related features and expression-independent features by using the weight map; s3, extracting supplementary features related to expressions from the expression irrelevant features, and fusing the supplementary features with the expression related features to obtain enhanced expression features; and S4, guiding the backbone network to carry out optimization training through a self-distillation mechanism by utilizing the enhanced expression features, so that the trained backbone network can directly output robust expression recognition features. According to the method, through characteristic decomposition, supplementary fusion and self-distillation optimization, the model is effectively separated and irrelevant interference information is inhibited, so that the accuracy and robustness of expression recognition are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a face expression recognition method based on feature disentanglement and self-distillation. BACKGROUND

[0002] The face expression recognition task aims to automatically analyze and recognize the emotional state in face images or video sequences through algorithms. However, in practical applications, the information contained in the face image is extremely complex, and in addition to the expression itself, there are also a large number of irrelevant interference factors such as individual identity features, head posture changes, local occlusions (such as glasses, masks), and light condition differences. In the training process, although the deep neural network model can learn to extract and classify features related to expressions, it is difficult to completely and spontaneously avoid learning about these irrelevant interference information. Due to the differences in the collection environment and the composition of the crowd of different data sets, the patterns of their interference factors are also different. When the model is trained on a specific data set, it will inevitably learn the interference patterns specific to that data set. When the model is applied to a test set or real scene with different interference patterns, its recognition performance will often decrease significantly. This problem not only reflects the overfitting of the model to the training data, but also reflects the lack of robustness of the model when facing changes in data distribution.

[0003] At present, the mainstream methods aimed at improving the robustness of the model and alleviating the influence of irrelevant interference mainly include two categories: The first category of methods is based on attention mechanisms. This kind of method guides the model to adaptively focus on the spatial regions or feature channels in the image that are related to expressions by constructing attention modules. However, existing methods usually generate attention maps through single or a small number of convolutional operations, which cannot effectively integrate global context information and do not fully utilize different scale features, resulting in insufficiently fine and accurate characterization of key regions by the generated attention weights.

[0004] The second category of methods is based on feature disentanglement. The core idea is to separate expression features from identity, posture and other interference features in the feature space by introducing additional constraints or designing specific network structures. However, this kind of method usually has obvious limitations: first, they often rely on data sets with detailed interference factor labels (such as explicit identity ID, posture angle) for supervised training, and the acquisition of such labeled data is costly and extremely difficult in engineering practice; second, existing methods usually only disentangle one or a few pre-set interference factors, and have limited generalization ability, making it difficult to cope with the diverse and unknown mixed interference in real scenes.

[0005] In addition, many model structures constructed based on the above mechanism are complex in structure and large in parameter quantity, resulting in high demand for computing resources and storage space, slow inference speed, and limiting the deployment and application of the model in computing resource limited environments (such as mobile devices and embedded systems).

[0006] Therefore, there is an urgent need for a new method that can efficiently disentangle expression features and irrelevant interference features without the need for interference factor labels, while keeping the model lightweight, to improve the accuracy and robustness of the facial expression recognition model. SUMMARY

[0007] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a facial expression recognition method based on feature disentanglement and self-distillation, which can effectively separate and suppress irrelevant interference information by feature decomposition, supplementary fusion and self-distillation optimization without relying on interference factor labels, thereby improving the accuracy and robustness of expression recognition.

[0008] The facial expression recognition method based on feature disentanglement and self-distillation comprises the following steps: Step S1: extracting an initial feature map from an input facial image through a backbone network; Step S2: generating a weight map based on the initial feature map, and decomposing the initial feature map into expression-related features and expression-independent features using the weight map; Step S3: extracting supplementary features related to expression from the expression-independent features, and fusing the supplementary features with the expression-related features to obtain enhanced expression features; Step S4: using the enhanced expression features to guide the optimization training of the backbone network through a self-distillation mechanism, so that the trained backbone network can directly output robust expression recognition features.

[0009] Further, the step S2 specifically comprises: inputting the initial feature map into a weight map generation module to generate a three-dimensional weight map through multi-scale convolution; weighting the initial feature map using the three-dimensional weight map to obtain an expression-related feature map and an expression-independent feature map, respectively; converting the expression-related feature map and the expression-independent feature map into an expression-related feature vector and an expression-independent feature vector, respectively.

[0010] Further, the step S2 further comprises: processing the expression-independent feature map and the expression-independent feature vector through a reconstruction and classification adversarial module, so that the expression-independent feature map can reconstruct the input image, and the expression-independent feature vector cannot be correctly recognized by the classifier.

[0011] Furthermore, step S3 specifically includes: Calculate the variance of each feature dimension based on the expression-independent feature vectors of multiple samples belonging to the same expression category in the same training batch. Based on the variance, the top T dimensions with the smallest variance are identified, and corresponding binary mask vectors are generated. The supplementary features are extracted from the expression-independent feature vector by using an attention mechanism and combining the binary mask vector.

[0012] Further, step S4 specifically involves: during the training phase, using the enhanced facial feature vector as the teacher signal and the feature vector directly output by the backbone network as the student signal, and optimizing the parameters of the backbone network by minimizing the consistency loss between the two.

[0013] Furthermore, during the training phase, the facial expression recognition method based on feature deentanglement and self-distillation also includes calculating an orthogonal loss to constrain the expression-related feature vectors to be orthogonal to the expression-independent feature vectors in the feature space.

[0014] Furthermore, during the training phase, the facial expression recognition method based on feature deentanglement and self-distillation also includes calculating a classification adversarial loss to guide the weight graph generation module in generating a weight graph that prevents expression-irrelevant feature vectors from being classified.

[0015] Furthermore, during the training phase, the facial expression recognition method based on feature deentanglement and self-distillation also includes calculating a reconstruction loss to guide the decoder in the reconstruction and classification adversarial module and the weight map generation module, so that the expression-independent feature map contains more visual information needed to reconstruct the original image.

[0016] Furthermore, during the training phase, the total loss function of this facial expression recognition method based on feature deentanglement and self-distillation is obtained by weighted summation of expression classification loss, orthogonality loss, classification adversarial loss, reconstruction loss, and self-distillation consistency loss.

[0017] Furthermore, during the testing phase, only the trained backbone network and the weight graph generation module are used to extract expression-related features, and facial expression recognition is completed based on these features.

[0018] Compared with existing technologies, the beneficial effects of this invention are as follows: 1. This invention achieves more robust and accurate facial expression recognition. Through feature decomposition and fusion mechanisms, it can effectively separate expression-related features from irrelevant interference features, and further extract expression-related information from the separated irrelevant features for supplementary fusion, forming a more complete and pure expression representation. This allows the model to directly output robust expression recognition features after training, significantly improving recognition accuracy. Experiments show that the recognition accuracy on two public datasets is improved by 2.84% and 2.07% respectively compared to the baseline model, and the recognition balance across various expressions is better. 2. This invention does not rely on labeled data for interference factors. Through the mechanisms of weighted graph generation and feature decomposition, this invention can automatically achieve effective separation of expression-related features from irrelevant features without labels for identity, posture, or other interference factors. This overcomes the limitations of existing feature deentanglement methods that rely on strongly labeled data, reduces data acquisition costs, and improves the practicality and generalization ability of the method. 3. This invention improves inference efficiency by decoupling training and testing. Through a self-distillation mechanism, enhanced facial expression features are used to guide the optimization of the backbone network during the training phase, enabling the backbone network to learn stronger feature extraction capabilities. During the testing phase, facial expression recognition can be completed using only the optimized backbone network and weight graph generation module, significantly reducing computational overhead and memory usage during inference, facilitating deployment in practical applications. 4. This invention improves the interpretability and discriminability of features. Through multi-stage feature processing and distillation optimization, the facial expression features learned by the model are more compact within classes and more separated between classes in the feature space. Visual analysis shows that the features have better clustering and discriminative properties, which helps in understanding the model's decision-making process. Attached Figure Description

[0019] Figure 1 This is a flowchart of a facial expression recognition method based on feature deentanglement and self-distillation in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of the facial expression recognition method based on feature deentanglement and self-distillation in this embodiment of the invention. Figure 3 This is a schematic diagram of the significance dimension identification method based on batch variance in an embodiment of the present invention; Figure 4 This is a schematic diagram of the complete facial expression-related feature extraction process in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the principle of the self-distillation mechanism in an embodiment of the present invention; Figure 6 This is a confusion matrix diagram of the model in this embodiment of the invention on the RAF-DB and FERPlus datasets; Figure 7 This is a t-SNE visualization distribution of features extracted by the model in this embodiment of the invention. Detailed Implementation

[0020] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0021] like Figures 1-7 As shown, this embodiment of the invention provides a facial expression recognition method based on feature deentanglement and self-distillation, including the following steps: Step S1: Extract the initial feature map from the input face image through the backbone network.

[0022] Step S2: Based on the initial feature map, generate a weight map, and use the weight map to decompose the initial feature map into expression-related features and expression-independent features.

[0023] Step S3: Extract expression-related supplementary features from the expression-independent features, and fuse the supplementary features with the expression-related features to obtain enhanced expression features.

[0024] Step S4: Using the enhanced facial expression features, guide the backbone network to perform optimized training through a self-distillation mechanism so that the trained backbone network can directly output robust facial expression recognition features.

[0025] The facial expression recognition method based on feature deentanglement and self-distillation according to embodiments of the present invention will be further described in detail below: The facial expression recognition method based on feature deentanglement and self-distillation in this invention is a method that can improve the facial expression feature representation capability of neural networks without interference labels while simultaneously achieving deentanglement of expression-independent features. This method includes two main mechanisms: a reconstruction and classification adversarial deentanglement mechanism and an expression-related feature self-distillation mechanism. The reconstruction and classification adversarial deentanglement mechanism includes a weight map generation module and a reconstruction and classification adversarial module. The expression-related feature self-distillation mechanism includes a saliency dimension recognition module and a self-distillation mechanism. The weight map generation module generates a three-dimensional weight map through multi-scale convolution. This indicates the positions and weights of expression-related and expression-independent features in the feature map, used to decompose the feature map into expression-related and expression-independent features. The reconstruction and classification adversarial module contains a classifier and a decoder. The classifier... To achieve facial expression classification, the adversarial relationship between the module and the weight graph generation module results in feature vectors of expression-independent branches. Unable to be correctly classified; decoder The expression-independent feature map and the weighted map generation module work together to reconstruct the original image, ensuring that the expression-independent features contain as much facial visual information as possible. Through these multiple processes, the expression-independent features... Because the adversarial classification effect of the weighted graph generation module cannot be correctly classified, it can still reconstruct as many visual features of the input samples as possible. This means that expression-independent features contain a large amount of visual information from the input samples but not expression-related semantic information, i.e., they can express expression-independent interference factors in the input samples to the maximum extent. The saliency dimension identification module identifies their common dimensions by the variance of the expression-independent feature vectors of similar expression samples in a batch. Dimensions with similar values ​​in the features of similar samples imply their common information, while the common information in similar samples with different interference factors is related to the expression features of the same category they belong to. The attention mechanism attempts to extract information from these common dimensions. To address the issue of incorrect separation of facial expression features, facial expression-related features are extracted again. These re-extracted facial expression-related features and the de-entangled facial expression-related feature vector are then processed. The samples are fused to form a more complete facial expression feature vector, which is used to guide the backbone network to learn to extract facial expression features and suppress other irrelevant facial features.

[0026] In this embodiment, the specific details of reconstructing and classifying the adversarial unentanglement mechanism are as follows: The reconstruction and classification adversarial deentanglement mechanism includes a weight graph generation module and a reconstruction and classification adversarial module.

[0027] The weight map generation module generates a 3D weight map that characterizes the relevance of each element of the input features to the facial expression. Each element of the 3D weight map not only integrates global and local information but also provides precise location information ("where") and semantic judgment ("what it is"). Specifically, for an input sample... Its encoded features are ,in , , These represent the number of channels, height, and width of the feature map, respectively. The weight map generation module first... Divided from the channel dimension Each feature submap ,in , Each feature sub-map has a... The convolutional layer. Correspondingly, after convolution, there is Each feature weight subgraph is represented as... . It is derived from formula (1).

[0028] in, Indicates the kernel size as Convolution operation, This represents a nonlinear transformation. (All) Consolidated according to channel dimensions Then through The final 3D feature weight map is obtained after processing with convolution and the Sigmoid function. .

[0029] 3D feature weight maps are used to decompose feature maps into expression-related features. Features unrelated to facial expressions . and The results are derived from formulas (2) and (3) respectively.

[0030] Among them, symbols This indicates digit-wise multiplication. Indicate size and The same tensor of all "1".

[0031] Facial expression related feature map Feature maps unrelated to facial expressions The expression-related feature vectors are obtained by mean pooling and transformation respectively. Feature vectors unrelated to facial expressions Simultaneously, this invention introduces expression-related feature vectors. Feature vectors unrelated to facial expressions The orthogonal loss between them enables the weight graph generation module to generate weight graphs that can orthogonally decompose features.

[0032] The reconstruction and classification adversarial module contains a decoder and a classifier. Its main purpose is to work in conjunction with the weight map to make the expression-independent feature map... Preserve as much information as possible from the original image and use expression-independent feature vectors. It exhibits indistinguishability. On the one hand, expression-independent feature vectors... It should not be able to be classified correctly. The classifier in this module... Attempting to obtain the most accurate expression-independent feature vectors Classification is performed. Meanwhile, the weight map generation module, under the influence of classification adversarial loss, attempts to make... Unable to be correctly classified. On the other hand, this invention argues that in a face image, facial expressions are only a small part of the vast amount of information contained in the image, and the vast majority of visual information in the image is redundant and irrelevant to facial expressions. Therefore, to make expression-irrelevant feature maps... Characterize as much as possible This invention designs a decoder that carries facial expression-irrelevant information from the original image and does not learn noise information unrelated to the original image during training. The decoder uses As input, with input samples The image is reconstructed using labels. During training, the reconstruction loss guides the decoder's learning and is backpropagated to the weight map generation module, enabling the weight map generation module to generate images capable of decomposing most visual information into... The weighted graph.

[0033] In this embodiment, the specific details of the self-distillation mechanism for expression-related features are as follows: The self-distillation mechanism for facial expression-related features includes a saliency dimension recognition module and a self-distillation mechanism.

[0034] The saliency dimension recognition module is responsible for identifying expression-independent feature vectors. The dimension related to facial expressions. Under the action of the adversarial deentanglement mechanism in reconstruction and classification, facial expression-independent feature vectors... It can include as much original image information as possible and as little expression-related information as possible. However, a single decomposition based on the weighted graph generation module may not be able to accurately separate expression-related features from expression-independent features. Expression-independent feature vector Inevitably, it still carries a small amount of expression-related information. To address this issue, the present invention aims to transform expression-independent feature vectors... The facial expression-related features were extracted again.

[0035] Among these, the common information among different samples of the same type of expression is related to the features of their respective categories. Based on this judgment, the design goal of the saliency dimension recognition module is to identify dimensions with smaller variances among similar sample vectors in the same training batch. Indicates that in a training batch, the element belonging to the first... The class of The expression-independent feature vectors of each sample. For multiple sample feature vectors of a specific category. Due to the different interference factors present in the samples, these vectors show significant differences. Furthermore, because... It's unavoidable that facial expressions are included, so their feature values ​​in certain dimensions are very similar. These dimensions with similar feature values ​​are those highly correlated with facial expressions. To find these dimensions and guide the model to extract the corresponding features, during training, features belonging to the same category in the current batch are compared. All classes Find their variances The smaller the variance value of a dimension, the higher the variance of that category of samples. The eigenvalues ​​of these dimensions are closer. The variance is calculated. Then, the model generates a similar... Binary mask vectors of the same size .vector The smallest median value One dimension, The corresponding dimension is set to 1, and the rest are set to 0. For a given category... , It is derived from formula (5).

[0036] in, Indicates that in a batch, the element belonging to the first... The total number of samples in the class, Represents the first in the vector One portion, Representing vectors The smallest median value A set of indices for each component. The value was set to 64 through experimentation.

[0037] like Figure 4 As shown, SDRM stands for Salient Dimension Recognition Module. For a specific category in the current batch of samples... The corresponding saliency binary mask vector output by SDRM is During model training, this invention will use different categories of... Arrange and combine the sample labels in the batch to obtain the binary mask vector corresponding to each sample. To extract expression-related features carried in expression-independent feature vectors, this invention employs an attention mechanism to enable the model to adaptively obtain expression-independent feature vectors. The positions and corresponding weights related to facial expressions are used to extract facial expression-independent feature vectors. The expression-related information in the image. Meanwhile, the saliency binary mask vector... The extracted features are further restricted to prevent the attention mechanism from mistakenly re-extracting features unrelated to facial expressions, ensuring that only the small subset of features relevant to facial expressions are extracted. The extracted features are then compared to facial expression-related features. Adding them together yields a feature vector containing more complete semantic information about facial expressions. . It is derived from formula (6).

[0038] in, This represents a positional multiplication between vectors. This indicates a fully connected operation. This represents the Sigmoid function.

[0039] To ensure the model maintains good feature representation capabilities while not relying on information outside the test samples during testing, this invention introduces a self-distillation mechanism into the model. Specifically, the feature vector... It includes facial expression-related features obtained from the decomposition. Compared to expression-related features extracted from expression-independent feature vectors, expression-related features contain more complete and richer semantic information about facial expressions. For example... Figure 5 As shown, the present invention provides a simplified representation of the entire model. It is a complete facial expression feature vector extracted by a network consisting of the backbone network and the aforementioned newly added modules. Feature maps extracted from the backbone network The feature vector obtained through mean pooling. During model training, this invention will... Separate a copy from the computation graph as a label, and then... The consistency loss between the core and newly added modules guides the backbone network's learning. This is equivalent to using the backbone network and the newly added modules as the teacher network, and the backbone network as the student network, achieving dynamic distillation of the network model from a global perspective to local aspects. During testing and application, the model only needs to extract facial expression-related features from the test samples. No need to calculate facial expression-irrelevant features Features related to complete facial expressions .

[0040] In this embodiment, the specific content of the loss function is as follows: The facial expression recognition method based on feature deentanglement and self-distillation in this invention includes three expression classifiers, which are classifiers for classifying expression-related features. A classifier for classifying features related to complete facial expressions. Classifiers for classifying facial expression-irrelevant features This invention guides the learning of these classifiers using three classification loss functions. All classification losses employ the cross-entropy function. Specifically, for a batch of input samples in the training set... There are corresponding tags. , For batch size, Let be the total number of expression categories. The classification loss function is shown in formula (7).

[0041] in, Indicates the first The first classifier prediction value Each component. express , express , express .

[0042] The weight generator learns to generate a weight map that decomposes input features into expression-related features and expression-independent features. This invention designs an orthogonal loss to ensure that expression-related features... Features unrelated to facial expressions The vectors should be as orthogonal as possible in the feature space. Orthogonality of vectors in the feature space indicates that they are linearly uncorrelated. From the perspective of semantic features, orthogonality of feature vectors indicates semantic uncorrelation. Under the guidance of orthogonal loss, the model can ultimately achieve the deentanglement of expression-related features and expression-independent features. The orthogonal loss adopts the cosine similarity function, as shown in formula (8).

[0043] The model makes expression-independent feature vectors The feature vectors cannot be correctly classified to ensure accurate separation of interfering information. Therefore, this invention designs a specialized classification adversarial loss function to make the expression-independent feature vectors... It exhibits indiscriminability. The loss function should be able to make the classifier... The predicted vector should have components corresponding to the true label that are as close to zero as possible, while other components should be distributed as evenly as possible. Under the influence of the Softmax function, when training converges, samples tend to be randomly predicted as categories other than the true label. For the true label... ,have ,in The classification adversarial loss is shown in Equation (9).

[0044] Expression-independent feature map It should be able to represent the interference information of the original image, that is, the vast majority of visual information. Therefore, The original image should be able to be reconstructed. This invention uses the MSE function as the reconstruction loss function to guide the decoder learning, while simultaneously enabling the weight generator to learn to generate feature maps that can decompose interference information into expression-independent feature maps. The weighted graph. For the input sample decoder The reconstruction result is expressed as The reconstruction loss is shown in formula (10).

[0045] To enable the model to operate without referencing batch data during testing, as it does during training, and to improve the backbone network's ability to characterize facial features, the model employs a self-distillation mechanism for dynamic compression during training. This invention compresses the complete facial expression-related feature vectors. A copy is extracted from the computation graph and used as a label to guide the backbone network in learning to encode feature vectors that more accurately represent facial expression-related information. Therefore, the present invention designs a consistency loss, as shown in formula (11).

[0046] The facial expression recognition method based on feature deentanglement and self-distillation in this invention can be divided into the following components: a backbone network, a weight graph generation module, a decoder and classifier for reconstruction and classification adversarial modules. 1. Saliency dimension recognition module (including attention mechanism) 2. Expression-related branch classifier Complete facial expression feature classifier During training, the optimization targets of the loss function are as follows: Classification loss Optimize the emoji-related branch classifier Weight graph generation module; backbone network; Classification loss Optimize the complete facial expression feature classifier 1. Saliency dimension identification module (including attention mechanism); Classification loss Optimize the classifier for the reconstruction and classification adversarial modules. ; Orthogonal loss Optimize the weight graph generation module; Classification of counter-loss Optimize the weight graph generation module; Reconstruction losses Optimize the decoder and weight map generation module for reconstruction and classification adversarial modules; Consistency loss Optimize the backbone network.

[0047] The overall optimization objective of the model is shown in equation (12). , , As hyperparameters, this invention experimentally sets them to 4.0, 1.0, and 1.0 respectively.

[0048] Experimental example: The facial expression recognition method based on feature disentanglement and self-distillation in this invention can achieve more refined facial expression feature representation and expression-independent interference disentanglement even when the training dataset has no interfering labels. This invention uses ResNet-18 as a baseline and conducts experiments on two public datasets, RAF-DB and FERPlus, under real-world conditions. The experimental results are shown in Table 1. The experimental results show that on the RAF-DB and FERPlus datasets, the facial expression recognition method based on feature disentanglement and self-distillation in this invention can improve the accuracy by 2.84% and 2.07% respectively compared to the baseline model, and exhibits better performance than state-of-the-art (SOTA) methods using the same or equivalent baseline models.

[0049] Table 1. Accuracy comparison between the method of this invention and the SOTA method. like Figure 6 As shown, Figure 6 (a) and Figure 6 (b) represents the confusion matrices of the baseline model and the model of this invention on the RAF-DB dataset, respectively. Figure 6 (c) and Figure 6 (d) Represents the confusion matrices of the baseline model and the model of this invention on the FERPlus dataset, respectively. In the confusion matrix, "SU" represents "surprise", "FE" represents "fear", "DI" represents "disgust", "HA" represents "happiness", "SA" represents "sadness", "AN" represents "anger", "NE" represents "neutrality", and "CO" represents "contempt". The confusion matrices show that the model of this invention achieves better or equal recognition accuracy than the baseline model in any category on both datasets. Figure 5 Comparative analysis shows that the model proposed in this invention significantly outperforms the baseline model in recognizing six expressions (surprise, fear, disgust, happiness, sadness, and neutrality) in the RAF-DB dataset and six expressions (neutrality, surprise, sadness, anger, disgust, and contempt) in the FERPlus dataset.

[0050] like Figure 7 The image shows the distribution of the feature vectors of the test set samples extracted by the backbone network after dimensionality reduction using the t-SNE algorithm. Different categories are labeled with different colors. Figure 7 (a) and Figure 7 (b) shows the feature vector distributions of the RAF-DB test set samples extracted by the baseline model and the model proposed in this invention, respectively. The features extracted by the baseline network have a high degree of confusion, with large intra-class distances and small inter-class distances. In contrast, the features extracted by the model of this invention have greater inter-class distances, forming seven distinct clusters corresponding to the classes. Figure 7 (c) and Figure 7 (d) shows the feature vector distributions of the FERPlus test set samples extracted by the baseline model and the model of this invention, respectively. A comparison of these two distribution plots also demonstrates that the features extracted by the model of this invention have better discriminability.

[0051] As shown in Table 2, this invention measures model complexity using two metrics: number of parameters and floating-point operations (FLOPs). The table shows that the proposed method has significantly fewer parameters during training than models such as IPA2LT, gACNN, MA-Net, and ADDL, demonstrating a clear advantage in memory usage. Compared to RAN and SCN, the model of this invention has slightly more parameters during training; however, because the reconstruction and classification adversarial modules and saliency dimension recognition blocks can be pruned during testing and application, the pruned parameter count is only 0.54M higher than theirs. Therefore, the actual memory overhead is essentially on par with them. During training, the model of this invention has 3.74G of floating-point operations, only higher than MA-Net and SCN. During testing, it is only slightly higher than SCN. Overall, the model of this invention has relatively low parameter count and floating-point operation costs compared to other methods during training, while during testing, it approaches the lower limit of methods using ResNet-18 as the baseline model.

[0052] Table 2 Comparison of the operating costs of the method of the present invention with other methods Therefore, the facial expression recognition method based on feature deentanglement and self-distillation in this embodiment of the invention can achieve better facial expression feature extraction results and improve the model's accuracy in recognizing facial expressions without relying on an expression dataset with interference labels and with only a small increase in parameters and floating-point operation overhead.

[0053] The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A face expression recognition method based on feature disentangling and self-distillation, characterized in that, The method comprises the following steps: Step S1: extracting an initial feature map from an input face image through a backbone network; Step S2: generating a weight map based on the initial feature map, and decomposing the initial feature map into an expression-related feature and an expression-independent feature by using the weight map; Step S3: extracting a supplementary feature related to an expression from the expression-independent feature, and fusing the supplementary feature with the expression-related feature to obtain an enhanced expression feature; Step S4: guiding the backbone network to perform optimization training through a self-distillation mechanism by using the enhanced expression feature, so that the trained backbone network can directly output a robust expression recognition feature. 2.The facial expression recognition method based on feature disentangling and self-distillation according to claim 1, wherein, The step S2 specifically comprises: inputting the initial feature map into a weight map generation module to generate a three-dimensional weight map through multi-scale convolution; weighting the initial feature map by using the three-dimensional weight map to obtain an expression-related feature map and an expression-independent feature map respectively; converting the expression-related feature map and the expression-independent feature map into an expression-related feature vector and an expression-independent feature vector respectively. 3.The facial expression recognition method based on feature disentangling and self-distillation according to claim 2, characterized in that, The step S2 further comprises: processing the expression-independent feature map and the expression-independent feature vector through a reconstruction and classification confrontation module, so that the expression-independent feature map can reconstruct the input image, and the expression-independent feature vector cannot be correctly recognized by a classifier.

4. The method of facial expression recognition based on feature disentangling and self-distillation according to claim 2 or 3, characterized in that, The step S3 specifically comprises: calculating the variance of each feature dimension according to the expression-independent feature vectors of multiple samples belonging to the same expression category in the same training batch; identifying the first T dimensions with the smallest variance based on the variance to generate a corresponding binary mask vector; extracting the supplementary feature from the expression-independent feature vector through an attention mechanism and in combination with the binary mask vector.

5. The method for facial expression recognition based on feature disentangling and self-distillation according to claim 1, characterized in that, The step S4 specifically comprises: in the training phase, taking the enhanced expression feature vector as a teacher signal and taking a feature vector directly output by the backbone network as a student signal, and optimizing the parameters of the backbone network by minimizing the consistency loss between the two.

6. The method of facial expression recognition based on feature disentangling and self-distillation according to claim 2, characterized in that, In the training phase, the face expression recognition method based on feature disentanglement and self-distillation further comprises calculating an orthogonal loss for constraining the expression-related feature vector and the expression-independent feature vector to be orthogonal in a feature space.

7. The method of facial expression recognition based on feature disentangling and self-distillation according to claim 3, characterized in that, In the training phase, the face expression recognition method based on feature disentanglement and self-distillation further comprises calculating a classification confrontation loss for guiding the weight map generation module to generate a weight map that makes the expression-independent feature vector unable to be classified. 8.The facial expression recognition method based on feature disentangling and self-distillation of claim 3, wherein, In the training phase, the face expression recognition method based on feature disentanglement and self-distillation further comprises calculating a reconstruction loss for guiding the decoder in the reconstruction and classification confrontation module and the weight map generation module to make the expression-independent feature map contain more visual information required for reconstructing the original image. 9.The facial expression recognition method based on feature disentangling and self-distillation of claim 1, wherein, In the training phase, the total loss function of the face expression recognition method based on feature disentanglement and self-distillation is obtained by weighted summation of an expression classification loss, an orthogonal loss, a classification confrontation loss, a reconstruction loss and a self-distillation consistency loss.

10. The method of facial expression recognition based on feature disentangling and self-distillation according to claim 1, characterized in that, In the test stage, only the trained backbone network and the weight map generation module are used to extract expression-related features, and the facial expression recognition is completed based on the features.

Images