Multi-stage progressive expression recognition method and system based on improved residual network

Abstract

The invention relates to the technical field of image and expression recognition, in particular to a multi-stage progressive expression recognition method and system based on an improved residual network, and solves the problems of unbalanced expression recognition model data set and poor model training effect in the prior art. The method comprises the following steps: constructing an expression recognition model comprising a first-order sub-model, a second-order positive sub-model and a second-order negative sub-model; the first-order sub-module is used for recognizing the expression as positive, neutral or negative; the second-order positive sub-model classifies expressions which are recognized to be positive; and the second-order negative sub-model classifies expressions which are recognized as negative expressions. According to the method, expression categories are highly summarized into three emotion categories, and then positive categories and negative categories are gradually decomposed into smaller sub-categories for model training. During reasoning, a corresponding execution strategy is designed to recognize the expression image, so that the influence of similarity between different types of data on the model can be indirectly reduced, the problem of confusion caused by the similarity of the types is greatly reduced, and high-precision recognition from coarse to fine is realized. And meanwhile, different models can learn more discriminative features, so that the accuracy of online expression recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image and expression recognition, and in particular to a multi-stage progressive expression recognition method and system based on an improved residual network. Background Art

[0002] At present, deep learning-based methods occupy a dominant position in many computer vision tasks. They can obtain robust algorithm models by utilizing large-scale training data and powerful representation capabilities. However, there are still some problems in the research of facial expression recognition algorithms: 1) The distribution of expression data of different categories is uneven, and there is a long tail phenomenon, which leads to inaccurate recognition of certain categories by the model; 2) The uncertainty of labeled expressions. Since different expression categories themselves have certain uncertainties, model training is difficult to converge; 3) There is a similarity problem in facial expression image data of different categories, which makes it difficult for the trained model to distinguish different categories, affecting the recognition effect. Summary of the Invention

[0003] In order to overcome the defects of the above-mentioned prior art in the expression recognition model data set imbalance and poor model training effect, the present invention proposes a multi-stage progressive expression recognition method based on an improved residual network, which adopts a two-stage model to perform expression recognition in stages, greatly improving the expression recognition accuracy.

[0004] The present invention proposes a multi-stage progressive expression recognition method based on an improved residual network:

[0005] First, an expression recognition model is constructed, which includes a first-order sub-model, a second-order positive sub-model, and a second-order negative sub-model. The first-order sub-module is used to identify expressions as positive, neutral, or negative; the second-order positive sub-model classifies expressions identified as positive; and the second-order negative sub-model classifies expressions identified as negative.

[0006] The second-order positive sub-model and the second-order negative sub-model have the same structure, both including a channel attention unit, a spatial attention unit, a multi-scale feature extraction unit, and a dimension superposition unit. The channel attention unit is used to extract the channel attention feature CA(X) of the input data X, the spatial attention unit is used to extract the spatial attention feature SA(X) of the input data X, and the multi-scale feature extraction unit extracts the convolution features of the input data X at different scales and concatenates them into a multi-scale feature F(X, W). CA(X), SA(X), and F(X, W) are dot-multiplied, and then dimensionally superimposed with the input data X to obtain feature Y, which is then activated to generate the predicted label of the input data X.

[0007] Input the expression image to be recognized into the first-order sub-model to obtain the first-order label of the expression image;

[0008] If the first-order label is positive, the second-order positive sub-model is used to determine the classification label of the expression image under positive emotions; if the first-order label is negative, the second-order negative sub-model is used to determine the classification label of the expression image under negative emotions.

[0009] Preferably, the method comprises the following steps:

[0010] S1. Input the expression image to be recognized into the first-order sub-model to obtain the emotion probability;

[0011] S2, determine whether the probability of neutral emotion is the highest;

[0012] If the probability of neutral emotion is greater than the set threshold, it is identified as neutral emotion; otherwise, step S3 is executed;

[0013] No, if the probability of positive emotion is the highest, then step S4 is executed to determine the positive label; otherwise, it is identified as negative emotion, and step S5 is executed to determine the negative label;

[0014] S4, inputting the expression image to be recognized into the second-order positive sub-model to obtain a positive label;

[0015] S5. Input the facial expression image to be recognized into the second-order negative sub-model to obtain a negative label.

[0016] Preferably, the loss function used for second-order sub-model training is L = λL pre +(1-λ)L fine ;L pre represents contrast loss; L fine represents the target task loss, and λ represents the weight coefficient;

[0017] The training process of the second-order sub-model is:

[0018] Let λ→1;

[0019] Extract training samples from the dataset and substitute them into the second-order sub-model to calculate the loss function L. Update the parameters of the second-order sub-model according to the loss function L. Repeat this step until the loss function converges.

[0020] Let λ→0;

[0021] Extract training samples from the data and substitute them into the second-order sub-model to calculate the loss function L, and update the parameters of the second-order sub-model according to the loss function L; repeat this step until the second-order sub-model converges.

[0022] Preferably, the dataset used for training the second-order sub-model is {expression image; fine-grained emotion label}, where the fine-grained emotion label is different expression labels under the same emotion;

[0023] The contrast loss calculation formula is:

[0024]

[0025] Among them, sim represents the modeling function; represents the second-order submodel, express Output features; x i represents the i-th training sample in this round, x j represents the jth training sample in this round, x k represents the kth training sample in this round, and x i and x j Belong to different fine-grained sentiment labels, x i and x k Belong to the same fine-grained sentiment label; Represents x i go through The obtained feature Y; Represents x j go through The obtained feature Y; Represents x i go through The obtained feature Y; Represents x i go through Predicted fine-grained sentiment label, y i is x i The true positive labels; log represents the logarithmic function; τ represents the temperature parameter.

[0026] Preferably, the training method of the expression recognition model comprises the following steps:

[0027] First, train the first-order sub-model on the first-order dataset {expression images; positive, neutral, negative};

[0028] Then the second-order sub-model training method is used to train the second-order positive sub-model and the second-order negative sub-model respectively;

[0029] The second-order positive sub-model is trained on the positive dataset {expression images; positive labels}, where positive labels include: happy and surprised;

[0030] The second-order negative sub-model is trained on the negative dataset {expression images; negative labels}, where negative labels include sadness, fear, anger, and disgust.

[0031] Preferably, the target task loss L fine The cross entropy loss function is used.

[0032] Preferably, the first-order sub-model adopts cross entropy loss or mean absolute error loss in the neural network model training process.

[0033] Preferably, the channel attention unit is composed of a convolution layer with a convolution kernel of 1×1 and an activation function, and the spatial attention unit is composed of a convolution layer with a convolution kernel of 3×3 and an activation function; the multi-scale feature extraction unit extracts convolution features of different scales of the input data X through convolution kernels of different sizes and splices them into multi-scale features.

[0034] The present invention proposes a multi-stage progressive expression recognition system based on an improved residual network, comprising a memory and a processor, wherein a computer program is stored in the memory, the processor is connected to the memory, and the processor is used to execute the computer program to implement the multi-stage progressive expression recognition method based on the improved residual network.

[0035] The present invention proposes a storage medium storing a computer program, which is used to implement the multi-stage progressive expression recognition method based on an improved residual network when executed.

[0036] The advantages of the present invention are:

[0037] (1) The multi-stage progressive expression recognition method based on the improved residual network proposed in this invention first highly summarizes the expression categories into three emotion categories, and then gradually decomposes the positive and negative categories into smaller subcategories for model training. During inference, a corresponding execution strategy is designed to recognize expression images, which can indirectly reduce the impact of similarities between data of different categories on the model, greatly reduce the confusion caused by category similarity, and achieve high-precision recognition from coarse to fine. At the same time, different models can learn more discriminative features, thereby improving the accuracy of online expression recognition.

[0038] (2) This paper improves the residual network ResNet-18 to obtain a second-order sub-model. The proposed AAF module, which integrates CA and SA, improves the ability to focus on key features. At the same time, the parallel convolutional structures (1×1, 3×3, 5×5) enhance the model's ability to recognize facial expressions of different scales. This paper introduces an adaptive attention mechanism and multi-scale feature fusion, effectively enhancing the model's feature extraction capabilities.

[0039] (3) This invention introduces contrastive loss during the training of the second-order sub-model. By maximizing the distance between samples of different categories and minimizing the distance between samples of the same category, the model is able to achieve clearer boundary demarcation in the feature space. This provides key support for distinguishing between similar categories (such as "surprise" and "fear"), significantly reducing the confusion rate and enhancing the model's ability to distinguish between expression categories.

[0040] This paper proposes a progressive training strategy and loss function design during the training of the second-order sub-model, which gradually optimizes the model and makes it easier to converge. The progressive loss function design used in this paper allows the model to focus on basic discriminative capabilities such as contrastive learning in the early stages of training, gradually transitioning to specific classification tasks in the later stages, effectively addressing the difficulty in training convergence caused by imbalanced data.

[0041] (4) This invention also proposes a phased reasoning process strategy design to ensure full recognition coverage and reduce the risk of misidentification. This invention proposes a phased execution logic, using prediction probability and threshold judgment strategies to ensure no missed or misidentified judgments, taking into account both coverage and accuracy, significantly outperforming traditional single-model direct classification methods.

[0042] (5) Compared with the average recognition rate of the seven-category model of 84%, the two-stage method proposed in this invention reaches 93.43%, an average improvement of nearly 10 percentage points, which reflects the effectiveness of the model design. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the multi-stage progressive expression recognition model based on the improved residual network proposed in the present invention;

[0044] Figure 2 Flowchart of the training process of the second-order sub-model proposed in the present invention;

[0045] Figure 3 Flowchart of the multi-stage progressive expression recognition method based on the improved residual network proposed by the present invention;

[0046] Figure 4 2 is a comparison chart of the model test results in the embodiment. DETAILED DESCRIPTION

[0047] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] like Figure 1 As shown, a multi-stage progressive expression recognition model based on an improved residual network includes a first-order sub-model, a second-order positive sub-model, and a second-order negative sub-model;

[0049] The first-order submodule is used to identify expressions as positive, neutral, or negative;

[0050] The second-order positive sub-model is used to classify the expressions identified as positive by the first-order sub-model, and the classification labels include: happy and surprised;

[0051] The second-order negative sub-model is used to classify expressions identified as negative by the first-order sub-model, with the classification labels including sadness, fear, anger, and disgust.

[0052] The first-order sub-model adopts a deep neural network model, specifically a Transformer network model, which is trained on the first-order dataset {expression images; positive, neutral, negative}.

[0053] The second-order positive sub-model is trained on the positive dataset {expression images; positive labels}, where positive labels include: happy and surprised.

[0054] The second-order negative sub-model is trained on the negative dataset {expression images; negative labels}, where negative labels include sadness, fear, anger, and disgust.

[0055] The second-order positive sub-model and the second-order negative sub-model have the same results and are collectively referred to as the second-order sub-model, which uses an improved residual network model.

[0056] The improved residual network model includes: channel attention unit, spatial attention unit, multi-scale feature extraction unit and dimension superposition unit.

[0057] The channel attention unit is used to extract the channel attention feature CA(X) of the input data X (i.e., the expression image). The channel attention unit consists of a convolution layer with a convolution kernel of 1×1 and an activation function. The channel attention unit processes the input data X, and the formula is expressed as follows:

[0058] CA(X)=σ(Wc*X)

[0059] Among them, Wc represents a 1×1 convolution kernel, σ represents an activation function, specifically a Sigmoid activation function; CA(X) is the output of the channel attention unit.

[0060] The spatial attention unit is used to extract the spatial attention feature SA(X) of the input data X (i.e., the expression image). The spatial attention unit consists of a convolution layer with a 3×3 convolution kernel and an activation function. The process of the spatial attention unit on the input data X is expressed as follows:

[0061] SA(X)=σ(Ws*X)

[0062] Where Ws represents a 3×3 convolution kernel, σ represents an activation function, and specifically a Sigmoid activation function can be used.

[0063] The multi-scale feature extraction unit extracts convolution features of different scales of the input data X through convolution kernels of different sizes, and then splices them into multi-scale features F(X,W). The processing formula is expressed as:

[0064] F(X,W)=Concat(W1*X, W2*X, W3*X)

[0065] W1, W2, and W3 represent three convolution kernels of increasing size. Specifically, W1 can be set as a 1×1 convolution kernel to capture local features; W2 as a 3×3 convolution kernel to capture medium-scale features; and W3 as a 5×5 convolution kernel to capture large-scale features).

[0066] The output of the channel attention unit CA(X), the output of the spatial attention unit SA(X), the output of the multi-scale feature extraction unit F(X,W) are multiplied by the dot product AAF(X) and the input data X through the dimension superposition unit to form the predicted feature Y. The predicted feature Y is activated by the softmax activation function to obtain the label probability distribution, and the label corresponding to the maximum probability is taken as the predicted label; that is:

[0067] Y=X+AAF(X)

[0068] AAF(X)=CA(X)·SA(X)·F(X,W)

[0069] The improved residual network model is trained on the dataset {expression images; fine-grained emotion labels}. Fine-grained emotion labels refer to emotion classifications under the same emotional tendency, such as happiness and surprise under positive emotions, or sadness, fear, anger, and disgust under negative emotions.

[0070] The loss function during the training of the improved residual network model is recorded as:

[0071] L=λL pre +(1-λ)L fine

[0072] Among them, L pre represents contrast loss; L fine Represents the target task loss, specifically the cross entropy loss, and λ represents the weight coefficient.

[0073] Reference Figure 2 , the training process of the improved residual network model (i.e., the second-order sub-model) includes the following steps:

[0074] Set λ→1, specifically the value can be in the interval [0.9, 1); extract training samples from the dataset {expression images; fine-grained emotion labels} and substitute them into the improved residual network model to calculate the loss function L. Update the improved residual network model according to the loss function L until the loss function converges;

[0075] Let λ→0, specifically the value can be in the interval (0,0.1); extract training samples from the dataset {expression images; fine-grained emotion labels} and substitute them into the improved residual network model to calculate the loss function L, and update the improved residual network model according to the loss function L until the improved residual network model converges.

[0076] The present invention proposes a multi-stage progressive expression recognition model based on an improved residual network, referred to as an expression recognition model, and its training method includes the following steps.

[0077] First, a first-order dataset {expression images; positive, neutral, negative} is constructed, and a machine learning method is used to train the first-order sub-model on the first-order dataset until it converges; cross entropy loss or mean absolute error loss can be used in the training process of the first-order sub-model.

[0078] Then the second-order positive sub-model and the second-order negative sub-model are trained separately.

[0079] Specifically, the loss function of the first-order sub-model can adopt cross entropy loss, and its convergence condition can be set as: the loss function converges or the number of iterations reaches a set value.

[0080] The training process of the second-order active submodel includes the following steps:

[0081] SA1. Construct a positive dataset {expression image Xm; positive label Ym}, where positive labels include: happy and surprised;

[0082] SA2, let λ→1;

[0083] SA3, extract training samples from the positive data set and substitute them into the second-order positive sub-model to calculate the loss function;

[0084] L=λL pre +(1-λ)L fine

[0085]

[0086] Among them, sim represents the modeling function; represents the second-order active submodel, express Output feature Y; x mi represents the i-th training sample in this round, x mj represents the jth training sample in this round, x mk represents the kth training sample in this round, and x mi and x mj Belong to different positive labels, x mi and x mk belong to the same positive label; Represents x mi go through The obtained feature Y; Represents x mj go through The obtained feature Y; Represents x mi go through The obtained feature Y; Represents x mi go through The predicted positive label, y mi is x mi True positive labels; log represents a logarithmic function, whose base can be any positive constant; τ represents a temperature parameter, which is used to adjust the smoothness of the similarity distribution and is set to 0.1 in this embodiment. In specific implementations, τ can also take values in the interval (0, 1).

[0087] SA4, determine whether the loss function converges;

[0088] If not, the loss function is used to update the second-order active sub-model, and then the process returns to step SA3;

[0089] If yes, set λ→0 and then execute step SA3 until the second-order active submodel converges.

[0090] The training process of the second-order passive sub-model includes the following steps:

[0091] St1. Construct a negative dataset {expression image Xn; negative label Yn}, where negative labels include: happy and surprised;

[0092] St2, let λ→1;

[0093] St3, extract training samples from the negative data set and substitute them into the second-order negative sub-model to calculate the loss function;

[0094] L=λL pre +(1-λ)L fine

[0095]

[0096] Among them, sim represents the modeling function; represents the second-order passive submodel, express Output feature Y; x ni represents the i-th training sample in this round, x nj represents the jth training sample in this round, x nk represents the kth training sample in this round, and x mi and x nj Belong to different negative labels, x ni and x nkbelong to the same negative label; Represents x ni go through The obtained feature Y; Represents x nj go through The obtained feature Y; Represents x ni go through The obtained feature Y; Represents x ni go through The predicted negative label, y ni is x ni True negative labels;

[0097] St4, determine whether the loss function converges;

[0098] If not, the loss function is used to update the second-order passive sub-model, and then the process returns to step St3;

[0099] If yes, set λ→0, and then execute step St3 until the second-order passive sub-model converges.

[0100] In specific implementation, the predicted label output by the second-order sub-model can be represented as a 01 vector, that is, the label dimension corresponding to the maximum probability is 1, and the remaining label dimensions are 0.

[0101] Taking the second-order positive sub-model as an example, happiness can be set to correspond to the first dimension and surprise to correspond to the second dimension. In this way, 10 represents the happy label and 01 represents the surprised label.

[0102] Reference Figure 3 The multi-stage progressive expression recognition method based on the improved residual network proposed in the present invention comprises the following steps:

[0103] S1. First, input the facial expression image to be recognized into the first-order sub-model to obtain the probability of being recognized as each type of emotion; let P ps Indicates the probability of identifying as positive emotion, P nt Indicates the probability of identifying as neutral emotion, P ng Represents the probability of identifying as negative emotion, the sum of the three is 1;

[0104] S2. If P nt If the value is the largest and greater than or equal to the threshold P0, it is identified as a neutral emotion;

[0105] If P nt If the value is the largest and smaller than the threshold value P0, then step S3 is executed;

[0106] If P ps If the value is the largest, it is identified as a positive emotion, and the process goes to step S4 to determine the positive label;

[0107] If P ng If the value is the largest, it is identified as negative emotion, and the process goes to step S5 to determine the negative label;

[0108] S3. Determine P ps Is it greater than or equal to P ng ;

[0109] If yes, it is identified as positive emotion and the process goes to step S4 to determine the positive label;

[0110] If not, it is identified as negative emotion and the process goes to step S5 to determine the negative label;

[0111] S4, inputting the expression image to be recognized into the second-order positive sub-model to obtain a positive label;

[0112] S5. Input the facial expression image to be recognized into the second-order negative sub-model to obtain a negative label.

[0113] It is worth noting that if equal probabilities occur in step S2, emotions are identified in the order of neutral, positive, and negative; that is:

[0114] If P nt With another value P ps or P ng If the maximum is tied, then according to P nt Maximum processing;

[0115] If P ps and P ng If the maximum is tied, then according to P ps Maximum processing.

[0116] The above expression recognition model is verified below in conjunction with specific embodiments.

[0117] In this embodiment, facial expression images of teenagers in the classroom are collected and labeled by experts as shown in Tables 1 and 2 below.

[0118] Table 1: Three types of datasets

[0119]

[0120] Table 2: 7-category dataset

[0121]

[0122] In this embodiment, two models are trained for expression recognition, one is the expression recognition model provided by the present invention, and the other is a 7-category model.

[0123] The facial expression recognition model was trained as follows: a first-order sub-model for emotion recognition was trained on the training datasets of the three datasets shown in Table 1; a second-order positive sub-model was trained on the training datasets of the positive dataset; and a second-order negative sub-model was trained on the training datasets of the negative dataset. During the training of the second-order sub-model, the value of λ was 0.99 when λ→1 and 0.01 when λ→0.

[0124] The training method of the 7-category model is as follows: 8,000 data are extracted from the neutral training dataset in Table 1, and merged with the training dataset of the 6 categories of expressions in Table 2 to form a 7-category dataset. The 7-category model is trained on the 7-category dataset, and the 7-category model uses the Transformer network model.

[0125] In this embodiment, the neutral test data set in Table 1 and the 6-category test data set in Table 2 are extracted and merged into a 7-category test data set.

[0126] In this embodiment, the performance of the expression recognition model and the 7-category model are tested on 7 test data sets. The expression recognition model (hereinafter referred to as the present invention model) is applied as shown in the above steps S1-S5. The test results of the two models are shown in Table 3. Figure 4 shown.

[0127]

[0128]

[0129] Combined with the test results, it can be seen that compared with the average recognition rate of the seven-category model of 84%, the two-stage method proposed in this invention reached 93.43%, an average improvement of nearly 10 percentage points, which reflects the effectiveness of the model design.

[0130] Of course, it will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also encompasses the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and it is intended that all variations that fall within the meaning and range of equivalents of the claims be encompassed within the present invention. Any reference signs in the claims should not be construed as limiting the claim to which they relate.

[0131] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0132] The technology, shape, and structure not described in detail in the present invention are all well-known technologies.

Claims

1. A multi-stage progressive expression recognition method based on an improved residual network, characterized by: First, an expression recognition model is constructed, which includes a first-order sub-model, a second-order positive sub-model, and a second-order negative sub-model. The first-order sub-module is used to identify expressions as positive, neutral, or negative; the second-order positive sub-model classifies expressions identified as positive; and the second-order negative sub-model classifies expressions identified as negative. The second-order positive sub-model and the second-order negative sub-model have the same structure, both including a channel attention unit, a spatial attention unit, a multi-scale feature extraction unit, and a dimension superposition unit. The channel attention unit is used to extract the channel attention feature CA(X) of the input data X, the spatial attention unit is used to extract the spatial attention feature SA(X) of the input data X, and the multi-scale feature extraction unit extracts the convolution features of the input data X at different scales and concatenates them into a multi-scale feature F(X, W). CA(X), SA(X), and F(X, W) are dot-multiplied, and then dimensionally superimposed with the input data X to obtain feature Y, which is then activated to generate the predicted label of the input data X. Input the expression image to be recognized into the first-order sub-model to obtain the first-order label of the expression image; If the first-order label is positive, the second-order positive sub-model is used to determine the classification label of the expression image under positive emotions; if the first-order label is negative, the second-order negative sub-model is used to determine the classification label of the expression image under negative emotions.

2. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 1, wherein The following steps are involved: S1. Input the expression image to be recognized into the first-order sub-model to obtain the emotion probability; S2, determine whether the probability of neutral emotion is the highest; If the probability of neutral emotion is greater than the set threshold, it is identified as neutral emotion; otherwise, step S3 is executed; No, if the probability of positive emotion is the highest, then step S4 is executed to determine the positive label; Otherwise, it is identified as negative emotion and the process goes to step S5 to determine the negative label; S4, inputting the expression image to be recognized into the second-order positive sub-model to obtain a positive label; S5. Input the facial expression image to be recognized into the second-order negative sub-model to obtain a negative label.

3. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 1, wherein The loss function used for training the second-order sub-model is L = λL pre +(1-λ)L fine ;L pre represents contrast loss; L fine represents the target task loss, and λ represents the weight coefficient; The training process of the second-order sub-model is: Let λ→1; Extract training samples from the dataset and substitute them into the second-order sub-model to calculate the loss function L. Update the parameters of the second-order sub-model according to the loss function L. Repeat this step until the loss function converges. Let λ→0; Extract training samples from the data and substitute them into the second-order sub-model to calculate the loss function L, and update the parameters of the second-order sub-model according to the loss function L; repeat this step until the second-order sub-model converges.

4. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 3, wherein Let the dataset used for second-order sub-model training be {expression image; fine-grained emotion label}, where fine-grained emotion label is different expression labels under the same emotion; The contrast loss calculation formula is: Among them, sim represents the modeling function; represents the second-order submodel, express Output features; x i represents the i-th training sample in this round, x j represents the jth training sample in this round, x k represents the kth training sample in this round, and x i and x j Belong to different fine-grained sentiment labels, x i and x k Belong to the same fine-grained sentiment label; Represents x i go through The obtained feature Y; Represents x j go through The obtained feature Y; Represents x i go through The obtained feature Y; Represents x i go through Predicted fine-grained sentiment label, y i is x i The true positive labels; log represents the logarithmic function; τ represents the temperature parameter.

5. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 4, wherein The training method of the expression recognition model includes the following steps: First, train the first-order sub-model on the first-order dataset {expression images; positive, neutral, negative}; Then the second-order sub-model training method is used to train the second-order positive sub-model and the second-order negative sub-model respectively; The second-order positive sub-model is trained on the positive dataset {expression images; positive labels}, where positive labels include: happy and surprised; The second-order negative sub-model is trained on the negative dataset {expression images; negative labels}, where negative labels include sadness, fear, anger, and disgust.

6. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 3, wherein Target task loss L fine The cross entropy loss function is used.

7. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 1, wherein The first-order sub-model uses cross entropy loss or mean absolute error loss in the neural network model training process.

8. The multi-stage progressive expression recognition method based on the improved residual network as claimed in claim 1, wherein The channel attention unit consists of a convolution layer with a 1×1 convolution kernel and an activation function, and the spatial attention unit consists of a convolution layer with a 3×3 convolution kernel and an activation function. The multi-scale feature extraction unit extracts convolution features of different scales of the input data X through convolution kernels of different sizes and splices them into multi-scale features.

9. A multi-stage progressive expression recognition system based on an improved residual network, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, the processor is connected to the memory, and the processor is used to execute the computer program to implement the multi-stage progressive expression recognition method based on the improved residual network as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: A computer program is stored, and when the computer program is executed, it is used to implement the multi-stage progressive expression recognition method based on the improved residual network according to any one of claims 1 to 8.

Images