Expression recognition method fusing uncertainty estimation and active learning

By integrating uncertainty estimation and active learning methods, dynamically selecting samples for annotation and training, solving the problem of performance degradation of expression recognition in the existing technology under complex scenes and noise labels, and improving the accuracy and robustness of expression recognition and enhances the ability to generalize unseen expressions.

CN119049113AActive Publication Date: 2024-11-29杨勇

Patent Information

Application Number
CN202411235694.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-29
Estimated Expiration
2044-09-04

AI Technical Summary

Technical Problem

Existing expression recognition methods perform poorly when processing large-scale data and complex scenarios, and deep learning models are highly sensitive to data distribution and quality, especially in the case of uneven distribution of noise labels or data, and lack effective combination uncertainty. Sex estimation and active learning methods for optimal performance.

Method used

A method of expression recognition that integrates uncertainty estimation and active learning is proposed. By constructing a SimCLR pre-training framework, a re-marking module based on uncertainty estimation and an active learning strategy, dynamically select samples for annotation and training, including self-attention weight , log-weighted cross-entropy loss, regularized sorting and dynamic threshold re-notation reduce labeling costs and improve model performance.

Benefits of technology

It significantly improves the accuracy and robustness of expression recognition, enhances the generalization ability of unseen expressions, adapts to data imbalance and noise labeling, reduces the risk of overfitting, and reduces the cost and time of data labeling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049113A_ABST
    Figure CN119049113A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of deep learning and image processing, and discloses an expression recognition method fusing uncertainty estimation and active learning. The expression recognition method fusing uncertainty estimation and active learning comprises the steps of 1, performing comparison self-supervision pre-training; 2, a dynamic re-annotation module (DRM) which is used for integrating uncertainty estimation is adopted, and a dynamic re-annotation module (DRM) which is used for estimating the uncertainty is adopted; and 3, performing model fine tuning based on the pre-training model. According to the expression recognition method fusing uncertainty estimation and active learning, firstly, underlying feature representation based on an overall unmarked data set is utilized for comparative learning, and the feature extraction capability of a model is further enhanced through an improved uncertainty estimation method; subsequently, the active learning performs model fine tuning based on a pre-training model so as to more efficiently select and mark a sample with the most information content, and compared with the prior art, the recognition accuracy is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing technology, and in particular to an expression recognition method that integrates uncertainty estimation and active learning. Background Technology

[0002] Facial expression recognition has important applications in computer vision and human-computer interaction. Traditional methods mainly rely on manual feature extraction and machine learning algorithms, which often perform poorly when dealing with large-scale data and complex scenes. With the development of deep learning technology, methods based on deep neural networks have made significant progress in facial expression recognition tasks. However, training deep learning models usually requires a large amount of labeled data, and the data labeling process is time-consuming, labor-intensive, and expensive.

[0003] Furthermore, deep learning models are typically highly sensitive to the distribution and quality of training data during training and inference. Especially in the presence of noisy labels or uneven data distribution, model performance can significantly degrade. To address these issues, uncertainty estimation and active learning techniques have been introduced. Uncertainty estimation, by quantifying the model's prediction confidence for each sample, helps identify samples that are difficult to classify or may be misclassified. Active learning techniques, on the other hand, intelligently select the most valuable samples for labeling, thereby improving model performance while reducing labeling costs.

[0004] However, existing research on how to effectively combine these two methods to achieve optimal performance remains limited. Therefore, a novel facial expression recognition method is needed that organically combines uncertainty estimation with active learning techniques to improve the accuracy and robustness of facial expression recognition. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes an expression recognition method that integrates uncertainty estimation and active learning to reduce annotation costs, improve model performance, and enhance the generalization ability to unseen expressions.

[0006] The facial expression recognition method that integrates uncertainty estimation and active learning provided by this invention includes the following steps:

[0007] Step 1: Constructing the first stage: Comparing pre-trained learning frameworks;

[0008] Step 2: Construct SimCLR (Simple Framework for Contrastive Learning of Visual Representations): Compared to pre-trained facial expression recognition networks, it consists of a backbone network, a global average pooling layer, a projection head, and a dynamic re-annotation module;

[0009] Step 3: Construct a relabeling module based on uncertainty estimation, which mainly includes a self-attention weight module, log-weighted cross-entropy loss, regularized sorting, and dynamic threshold relabeling.

[0010] Step 4: Active learning.

[0011] Preferably, step 1 includes the following sub-steps:

[0012] Step 1.1, Data Sampling: Select a dataset and obtain a face image dataset X = {X1, X2, ..., X...} by randomly sampling face images. k ···X K}, X k Represent the image of the k-th face, and store the label value corresponding to the face separately as y = {y1, y2, ..., y...} k ···y K}, y k This represents the label value corresponding to the k-th face image;

[0013] Step 1.2: For each face image in dataset X, resize the image, normalize the pixel values, and then apply random transformations (rotation, cropping, color adjustments, etc.) to obtain the data-augmented image X' = {X'1, X'2, ..., X'...} k ···X' K},X”={X”1,X”2,···,X” k ···X” K}, where X' k and X” k Let represent the random data augmentation views of the k-th sample, and let represent these two views as a positive sample pair.

[0014] Preferably, step 2 includes the following sub-steps:

[0015] Step 2.1: The backbone network is a ResNet18 network, which includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, and a sixth fully connected block in sequence.

[0016] The first convolutional block has an m1×m2 kernel, the second convolutional block consists of one residual block, the third convolutional block consists of one residual block, the fourth convolutional block consists of two residual blocks, the fifth convolutional block consists of two residual blocks, and the sixth fully connected block contains global average pooling and a fully connected layer; each residual block contains two 3×3 convolutional kernels.

[0017] Step 2.2: Input X' and X” into the backbone network for processing, and after passing through five convolutional blocks, output the k-th enhanced graphic feature map F'. k F”k The global average pooling layer compresses the spatial dimension of each feature map to a single value, thus obtaining two fixed-length feature vectors z. i ,z j ;

[0018] Step 2.3: Compare the loss calculation with the training, and calculate the positive sample pair z. i ,z j The cosine similarity; selecting negative samples from the current batch; each view's feature vector is a negative sample for the feature vectors of other views; using formula (1), for each positive sample pair, maximize its similarity while minimizing its similarity with all negative sample pairs.

[0019]

[0020] Equation (1)z i ,z j z represents the embedding representation of sample i and positive sample p, respectively. a For the embedding representation of negative samples, τ is the temperature parameter, 1 [k≠i] It is an indicator function, which is 1 when k ≠ i, and 0 otherwise; P(i) = {p ∈ {1, ..., N} | y p =y i ,p≠i} is the positive sample set.

[0021] Preferably, step 3 includes the following sub-steps:

[0022] Step 3.1, Calculate the self-attention weights: (The text abruptly ends here, likely due to an incomplete sentence or a formatting error.) k After backpropagation in the Softmax classifier, the feature weights of the fully connected (FC) layers are extracted. The feature weights are then applied to the sample X, and the feature weights corresponding to each image sample are obtained by formula (2).

[0023]

[0024] In equation (2), α i It is the feature weight of the i-th sample. σ represents the weight coefficients of the fully connected layer, and σ is the Sigmoid function.

[0025] Step 3.2: Calculate the log-weighted cross-entropy loss L using formula (3). WCE

[0026]

[0027] Among them W j It is the weight of the j-th classifier;

[0028] Step 3.3, Regularized Sorting: First, sort the attention weights in descending order, then proportionally divide them into attention weight regroups with high standard deviations, group a. s and low standard deviation attention weight reorganization a u Two groups; then the rank regularization of formulas (4) and (5) is used to ensure that the standard deviation attention weight of the high importance group is slightly higher than that of the low importance group;

[0029] L RR =max{0,δ1-(α)} s -α u )}(4)

[0030] In equation (4), δ1 is the marginal value controlling the proportion of the total number of low-importance groups. During training, the comprehensive loss function is defined as:

[0031] L all =γL RR +(1-γ)L WCE (5)

[0032] In equation (5), γ is the trade-off ratio;

[0033] Step 3.4, Dynamic Threshold Relabeling: Uncertain samples typically exhibit lower importance weights, i.e., a u The re-annotation module will a u The sample data in the algorithm uses the Softmax probability method to evaluate samples with lower importance weights; the highest predicted probability and the probability assigned to the original label are compared using formulas (6) and (7); when the former exceeds the latter, the sample is reassigned a new pseudo-label; the dynamic threshold relabeling process operates as follows:

[0034]

[0035] In equation (6), δ2 is the threshold, and P max To predict the value with the highest probability, P gtInd Represents the probability of a given label value; l max and l org These are the indexes of the original given label and the index of the maximum predicted value, respectively.

[0036] In equation (7), increase and decrease represent the magnitude of each adjustment;

[0037] Step 3.5: Based on the overall loss function L all The contrastive supervised pre-trained network is trained until the number of training iterations reaches a set number or the overall loss function L. all Upon convergence, training stops, thus obtaining the optimal facial expression feature extraction network model M. pThis is used for subsequent active learning.

[0038] Preferably, step 4 includes the following sub-steps:

[0039] Step 4.1: Divide the face dataset X into a small-scale labeled dataset.

[0040] and a large number of unlabeled datasets D L The final position `initial_points` in the dataset is the starting point for distinguishing between labeled and unlabeled data.

[0041] Step 4.2, Least Confidence Method:

[0042] The learning rounds are set to 7. In each round of active learning, model M... p Unlabeled dataset D U Select the batch size of the samples, for D U Each sample in Each feature value was processed by a Softmax classifier. Determine the expression category to obtain D. U Corresponding expression category prediction probability According to the highest confidence level P max =max(P U Sort all unlabeled samples from smallest to largest, and select the sample with the highest confidence but lowest ranking as the sample selected for active learning. This is denoted as... in This represents the k-th highest confidence level sample selected in the batch; the newly labeled data D new Add to labeled dataset D L In the dataset D, which has never been labeled. U Remove these samples from the middle;

[0043] Step 4.3: Use the updated labeled dataset D L Retraining or fine-tuning model M P Generate a new model M P+1 ; Evaluate the model’s performance on the validation set until the budget is met or 7 rounds are completed.

[0044] Compared with related technologies, the facial expression recognition method that integrates uncertainty estimation and active learning provided by this invention has the following beneficial effects:

[0045] 1. Improve annotation efficiency: By combining uncertainty estimation with active learning strategies, the most informative samples are dynamically selected for annotation, effectively reducing the amount of data required for annotation and lowering the cost and time of data annotation.

[0046] 2. Improve model performance: Identify samples that are difficult for the model to classify or may be misclassified by using uncertainty estimation methods, and focus on training these samples through active learning strategies to significantly improve the model's recognition accuracy and robustness.

[0047] 3. Enhanced generalization ability for unseen expressions: During multiple rounds of training and model updates, the model continuously adapts to newly added samples, improving its generalization ability for unseen expressions and making it more robust in practical applications.

[0048] 4. Adapting to Imbalanced Data and Noisy Labels: Even when the dataset is imbalanced or contains noisy labels, the method can still effectively improve model performance and reduce the performance degradation caused by imbalanced data or noisy labels through uncertainty estimation and active learning strategies.

[0049] 5. Reduce overfitting risk: Active learning strategies prioritize labeling and training on samples with high uncertainty, avoiding overtraining on known samples, thereby reducing the risk of overfitting and improving the model's generalization ability. Attached Figure Description

[0050] Figure 1 This is a diagram of the contrastive learning pre-training framework in the facial expression recognition method that integrates uncertainty estimation and active learning proposed in this invention.

[0051] Figure 2 This is a diagram of the dynamic tag re-annotation structure of the present invention;

[0052] Figure 3 This is a structural diagram of the active learning part in the method of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] This invention proposes an expression recognition method that integrates uncertainty estimation and active learning. The first stage of its pre-training process is as follows: Figure 1 As shown, the procedure is as follows:

[0055] Step 1: Constructing the first stage: Comparing pre-trained learning frameworks;

[0056] Step 1.1, Data Sampling: Select a dataset and obtain a face image dataset X = {X1, X2, ..., X...} by randomly sampling face images. k ···X K}, X k Represent the image of the k-th face, and store the label value corresponding to the face separately as y = {y1, y2, ..., y...} k ···y K}, y k This represents the label value corresponding to the k-th face image.

[0057] Step 1.2: For each face image in dataset X, resize the image, normalize the pixel values, and then apply random transformations (rotation, cropping, color adjustments, etc.) to obtain the data-augmented image X' = {X'1, X'2, ..., X'...} k ···X' K},X”={X”1,X”2,···,X” k ···X” K}, where X' k and X” k Let represent the random data augmentation views of the k-th sample, and let represent these two views as a positive sample pair; in this example, the training data was augmented using random cropping and random horizontal flipping.

[0058] Step 2: Construct SimCLR (Simple Framework for Contrastive Learning of Visual Representations): Compared to pre-trained facial expression recognition networks, it consists of a backbone network, global average pooling layers, a projection head, and a dynamic re-annotation module.

[0059] Step 2.1: The backbone network is a ResNet18 network, which includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, and a sixth fully connected block in sequence.

[0060] The first convolutional block has an m1×m2 kernel, the second convolutional block consists of one residual block, the third convolutional block consists of one residual block, the fourth convolutional block consists of two residual blocks, the fifth convolutional block consists of two residual blocks, and the sixth fully connected block contains global average pooling and a fully connected layer; each residual block contains two 3×3 convolutional kernels.

[0061] Step 2.2: Input X' and X” into the backbone network for processing, and after passing through five convolutional blocks, output the k-th enhanced graphic feature map F'. k F” k The global average pooling layer compresses the spatial dimension of each feature map to a single value, thus obtaining two fixed-length feature vectors z. i ,z j In this example, the enhanced image is input as a 224x224x3 face image, and after passing through a convolutional layer and four residual blocks, a 7x7x512 feature map is obtained. Then, it is downsampled through a global average pooling layer, and the final model output is a feature map of size 1x1x512.

[0062] Step 2.3: Compare the loss calculation with the training, and calculate the positive sample pair z. i ,z j The cosine similarity. Negative samples are selected from the current batch. The feature vector of each view is a negative sample to the feature vectors of other views. Using formula (1), for each positive sample pair, maximize its similarity while minimizing its similarity with all negative sample pairs.

[0063]

[0064] In equation (1) z i ,z j These are the embedding representations of sample i and positive sample p, respectively, z a For the embedding representation of negative samples, τ is the temperature parameter, 1 [k≠i] It is an indicator function, which is 1 when k ≠ i, and 0 otherwise. P(i) = {p ∈ {1, ..., N} | y p =y i ,p≠i} is the positive sample set.

[0065] Step 3: Construct a relabeling module based on uncertainty estimation, the main structure of which is as follows: Figure 2 As shown, it mainly includes a self-attention weight module, log-weighted cross-entropy loss, regularized sorting, and dynamic threshold relabeling.

[0066] Step 3.1, Calculate the self-attention weights: (The text abruptly ends here, likely due to an incomplete sentence or a formatting error.) k After backpropagation in the Softmax classifier, the feature weights of the fully connected (FC) layers are extracted. The features are weighted with those of sample X, and the feature weights corresponding to each image sample are obtained by formula (2).

[0067]

[0068] In equation (2), α i It is the feature weight of the i-th sample. σ is the weight coefficient of the fully connected layer, and σ is the Sigmoid function.

[0069] Step 3.2: Calculate the log-weighted cross-entropy loss L using formula (3). WCE

[0070]

[0071] Among them W j These are the weights of the j-th classifier. Establishing a log-weighted cross-entropy loss accurately reflects the loss weighting process utilizing attention weights.

[0072] Step 3.3, Regularized Sorting: First, sort the attention weights in descending order, then proportionally divide them into attention weight regroups with high standard deviations, group a. s and low standard deviation attention weight reorganization a u Two groups. Then, rank regularization of formulas (4) and (5) is used to ensure that the standard deviation attention weight of the high importance group is slightly higher than that of the low importance group; in this example, the ratio of high to low is 8:2.

[0073] L RR =max{0,δ1-(α)} s -α u )}(4)

[0074] In equation (4), δ1 is the marginal value controlling the proportion of the total number of low-importance groups. During training, the comprehensive loss function is defined as:

[0075] L all =γL RR +(1-γ)L WCE (5)

[0076] In equation (5), γ is the trade-off ratio.

[0077] In this example, δ1 is set to 0.07.

[0078] Step 3.4, Dynamic Threshold Relabeling: Uncertain samples typically exhibit lower importance weights, i.e., a u The re-annotation module will a u The sample data in the dataset uses the Softmax probability method to evaluate samples with lower importance weights. Equations (6) and (7) are used to compare the highest predicted probability with the probability assigned to the original label. When the former exceeds the latter, the sample is reassigned a new pseudo-label. The dynamic threshold relabeling process operates as follows:

[0079]

[0080] In equation (6), δ2 is the threshold, and P max To predict the value with the highest probability, P gtInd This represents the probability of a given label value. max and l org These are the indexes of the original given label and the index of the maximum predicted value, respectively.

[0081] In equation (7), increase and decrease represent the magnitude of each adjustment.

[0082] In this example, δ2 is set to 0.2, and increase and decrease are set to 0.05.

[0083] Step 3.5: Based on the overall loss function L all The contrastive supervised pre-trained network is trained until the number of training iterations reaches a set number or the overall loss function L. all Upon convergence, training stops, thus obtaining the optimal facial expression feature extraction network model M. p This is used for subsequent active learning. In this example, batch_size is set to 1024.

[0084] Step 4, Active Learning, its main structure is as follows: Figure 3 As shown:

[0085] Step 4.1: Divide the face dataset X into a small-scale labeled dataset.

[0086] and a large number of unlabeled datasets D L The initial_points position in the dataset is the starting point for distinguishing between labeled and unlabeled data. In this example, initial_points is set to 10% of the dataset.

[0087] Step 4.2, Least Confidence Method:

[0088] The learning rounds are set to 7. In each round of active learning, model M... p Unlabeled dataset D U Select the batch size of the samples, for D U Each sample in Each feature value was processed by a Softmax classifier. Determine the expression category to obtain D. U Corresponding expression category prediction probability According to the highest confidence level P max =max(P U Sort all unlabeled samples from smallest to largest, and select the sample with the highest confidence but lowest ranking as the sample selected for active learning. This is denoted as... in This represents the k-th sample selected with the lowest highest confidence level in the batch. The newly labeled data D... new Add to labeled dataset D L In the dataset D, which has never been labeled. UThese samples are removed from the dataset. In this embodiment, the model's output feature map is fed into a global average pooling layer (AavgPool), and the predicted class probability is finally given by the softmax activation function. Active learning can be used to select facial expression samples with the highest current model uncertainty for further annotation and training. In this way, the model can handle complex and uncertain samples more effectively, improving the overall model performance and robustness.

[0089] Step 4.3: Use the updated labeled dataset D L Retraining or fine-tuning model M P Generate a new model M P+1 The model's performance on the validation set is evaluated until the budget is reached or 7 rounds are completed. In this embodiment, the Adam optimizer is used, with an initial learning rate of 1e-3, which decays to 0.1 times its original value every 100 iterations. The trained model is obtained after 500 iterations.

[0090] To verify the effectiveness of the method of the present invention, existing active learning methods on FER13, RAF-DB, and KDEF were selected for comparative verification. Table 1 shows the comparison of the accuracy of facial expression recognition with various active learning methods.

[0091]

[0092] Table 1 Comparison with other algorithms

[0093] Compared with related technologies, the facial expression recognition method that integrates uncertainty estimation and active learning provided by this invention has the following beneficial effects:

[0094] This invention consists of two main stages: First, the model undergoes self-supervised pre-training through contrastive learning and uncertainty estimation to enhance its feature extraction capabilities; then, the model is fine-tuned using the knowledge gained in the pre-training stage. This method makes full use of limited data labels, helps alleviate the dependence on a large amount of labeled data and the cold start problem in active learning on small datasets, thereby significantly improving recognition accuracy.

[0095] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A facial expression recognition method integrating uncertainty estimation and active learning, characterized in that: The steps include: Step 1: Build the first stage: Compare the pre-training learning framework; Step 2: Build SimCLR: Compare the pre-trained expression recognition network, which is composed of the backbone network, global average pooling layer, projection head, and dynamic re-labeling module; Step 3: Construct a re-labeling module based on uncertainty estimation, which mainly includes self-attention weight module, logarithmic weighted cross entropy loss, regularized sorting and dynamic threshold re-labeling; Step 4: Active learning.

2. The method for facial expression recognition integrating uncertainty estimation and active learning as claimed in claim 1, characterized in that: The step 1 comprises the following sub-steps: Step 1.1, data sampling: select a data set and obtain a face image data set X = {X1, X2, ···, X k ···X K }, X k represents the kth face image, and saves the label value corresponding to its face separately as y={y1,y2,···,y k ···y K },y k Indicates the label value corresponding to the k-th face image; Step 1.2: Resize each face in the dataset X, normalize the pixel values, and then apply random transformations (rotation, cropping, color adjustment, etc.) to obtain the data augmented image X' = {X'1, X'2, ···, X' k ···X' K },X”={X”1,X”2,···,X” k ···X” K }, where X' k and X” k They represent the random data augmented views of the k-th sample respectively, and these two views are regarded as positive sample pairs.

3. The method for facial expression recognition integrating uncertainty estimation and active learning as claimed in claim 1, characterized in that: The step 2 includes the following sub-steps: Step 2.1, the backbone network is a ResNet18 network, and includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, and a sixth fully connected block in sequence; The convolution kernel of the first convolution block is m1×m2, the second convolution block consists of a residual block, the third convolution block consists of a residual block; the fourth convolution block consists of two residual blocks; the fifth convolution block consists of two residual blocks; the sixth fully connected block contains global average pooling and a fully connected layer; each residual block contains two 3×3 convolution kernels; Step 2.2: Input X', X" into the backbone network for processing, and after being processed by five convolution blocks in sequence, output the kth enhanced graphic feature map F'. k , F” k , the global average pooling layer compresses the spatial dimension of each feature map to a single value, thereby obtaining two fixed-length feature vectors z i ,z j ; Step 2.3: Compare the loss and train the training to calculate the positive sample pair z i ,z j cosine similarity; select negative samples from the current batch; the feature vector of each view is a negative sample of the feature vector of other views; Using formula (1), for each positive sample pair, maximize its similarity and minimize its similarity with all negative sample pairs. Formula (1) i ,z j where are the embedding representations of sample i and positive sample p, respectively, z a is the embedding representation of negative samples, τ is the temperature parameter, 1 [k≠i] is an indicator function, which is 1 when k≠i and 0 otherwise; P(i)={p∈{1,...,N}|y p =y i ,p≠i} is the positive sample set.

4. The method for facial expression recognition integrating uncertainty estimation and active learning as claimed in claim 1, characterized in that: The step 3 comprises the following sub-steps: Step 3.

1. Calculate the self-attention weight: F” k After input into the Softmax classifier for back propagation, the feature weight W of the fully connected FC layer is extracted a T , and perform feature weighting with sample X, and obtain the feature weight corresponding to each image sample through formula (2); In formula (2), α i is the feature weight of the i-th sample, is the weight coefficient of the fully connected layer, σ is the Sigmod function; Step 3.2: Calculate the log-weighted cross entropy loss L using formula (3) WCE Where W j is the weight of the jth classifier; Step 3.3, regularized sorting: First, sort the attention weights in descending order, and then divide them into high standard deviation attention weight groups a s and low standard deviation attention weights group a u two groups; then use rank regularization of formulas (4) and (5) to ensure that the standard deviation attention weight of the high-importance group is slightly higher than the standard deviation attention weight of the low-importance group; L RR =max{0,δ1-(α s -a u )} (4) In formula (4), δ1 is the marginal value that controls the proportion of the total number of low-importance groups. During training, the comprehensive loss function is defined as: L all =γL RR +(1-γ)L WCE (5) Formula (5), where γ is the trade-off ratio; Step 3.4, Dynamic Threshold Relabeling: Uncertain samples usually show lower importance weights, i.e., a u , the re-labeling module will u The sample data in the Softmax probability method is used to evaluate samples with lower importance weights; the highest predicted probability and the probability assigned to the original label are compared using formulas (6) and (7); when the former exceeds the latter, the sample is reassigned a new pseudo label; the operation of the dynamic threshold relabeling process is as follows: In formula (6), δ2 is the threshold, P max is the value with the largest predicted probability, P gtInd Represents the probability of a given label value; l max and l org are the index of the original given label and the index of the maximum predicted value, respectively; In formula (7), increase and decrease represent the magnitude of each adjustment; Step 3.5: Based on the overall loss function L all The contrast supervised pre-trained network is trained. When the number of training iterations reaches the set number or the overall loss function L all When convergence occurs, the training stops, and the optimal facial expression feature extraction network model M is obtained. p , used for subsequent active learning.

5. The method for facial expression recognition integrating uncertainty estimation and active learning as claimed in claim 4, characterized in that: The step 4 comprises the following sub-steps: Step 4.1: Divide the face dataset X into a small-scale labeled dataset and a large unlabeled dataset D L The last position initial_points in the data set is the starting point for distinguishing whether there is labeled data or not; Step 4.2, minimum confidence method: Set the number of learning rounds to 7. In each round of active learning, the model M p From the unlabeled dataset D U Select the batch size of the sample in U Each sample D in k U All feature values ​​are respectively classified by Softmax classifier Determine the expression category and get D U Corresponding expression category prediction probability According to the highest confidence level P max =max(P U ) Sort all unlabeled samples from small to large, and select the sample with the highest confidence and the lowest ranking as the sample selected for active learning, and record it as in represents the selected sample with the smallest k-th highest confidence in the batch; the newly labeled data D new Add to the labeled dataset D L In the unlabeled dataset D U Remove these samples from the Step 4.3: Use the updated labeled dataset D L Retrain or fine-tune model M P , generate a new model M P+1 ; Evaluate the performance of the model on the validation set until the budget is reached or 7 epochs are completed.

Citation Information

Patent Citations

  • Expression recognition method and system with noise robustness

    CN115376181A

  • Training method of image classification network, image classification method and system

    CN115482413A

  • Cross-subject electroencephalogram emotion recognition method based on label symmetry

    CN116702003A

  • Inter-bank fund face emotion analysis method based on hierarchical attention

    CN118069841A

Cited By

  • PreDyn-ST model-based EHA performance degradation prediction method and system

    CN119720761A

  • Chest DR image quality evaluation method based on feature fusion

    CN119832400A

  • Abnormal behavior analysis and prediction system and method based on unmanned aerial vehicle monitoring system

    CN121121575A