A facial expression recognition method based on multi-level and multi-scale attention mechanism

Abstract

The application discloses a facial expression recognition method based on a multi-level and multi-scale attention mechanism, which first splits four public data sets into a training set, a test set and a verification set. Secondly, a facial expression recognition model containing a feature pre-extraction module, a multi-level attention module and a multi-scale attention module is constructed, and face detection and alignment are performed on the images of the data set input into the model, and random image enhancement is performed. Then, the processed training set images are input into the constructed facial expression recognition model, and the model is trained and optimized by using cross-entropy loss and limited center loss. Finally, the images in the test set and the verification set are inferred to obtain the corresponding classification results and recognize the expressions. The application increases the robustness of the features extracted by the model, improves the expression ability of the output features of the model, and accurately recognizes facial expressions.

Description

Technical Field

[0001] This invention relates to the field of facial expression recognition, and more particularly to a facial expression recognition method based on a multi-level and multi-scale attention mechanism. Background Technology

[0002] Facial expressions are among the simplest and most natural forms of communication, vividly conveying human emotions and playing a vital role in daily life. Facial expression recognition is an important research area in artificial intelligence. Over the past few decades, its widespread application in human-computer interaction, digital entertainment, healthcare, and driver detection has drawn increasing attention to this research direction.

[0003] The goal of facial expression recognition is to accurately classify the emotion expressed in an image into one of the following basic emotions: happiness, surprise, sadness, disgust, fear, anger, neutrality, and contempt. Traditional facial expression recognition methods mainly utilize handcrafted features or shallow features to identify facial expression images. Due to the high similarity between facial expression classes, facial occlusion, and pose variations, traditional methods struggle to correctly identify the category of an image. Although many advanced attention-based methods have been proposed in recent years to improve the performance of facial expression recognition, these methods still suffer from the following problems:

[0004] 1. Since different emotions may have high similarities, while the same emotion may have high differences, these images can only be distinguished by detailed features such as the degree of forehead wrinkles, the size of the eyes, and the degree of closure of the corners of the mouth. Existing methods make it difficult for models to pay attention to these important detailed features.

[0005] 2. Most current facial expression recognition models only use cross-entropy loss as the final loss function. Cross-entropy loss only focuses on the difference between the predicted score and the correct label, while ignoring the difference with other incorrect labels. Summary of the Invention

[0006] To address the above problems, this invention provides a facial expression recognition method based on a multi-level and multi-scale attention mechanism. First, a multi-level attention module is designed, which assigns different weights to features extracted at different stages of the feature pre-extraction module according to their importance. Second, a multi-scale attention module is designed, which performs multi-scale fusion of deep features to improve their semantic information and representational power. Finally, a restricted center loss is designed, which brings features of the same class closer together and widens the distance between features of different classes, thereby increasing the diversity of features of different classes. Furthermore, by adding constraints to limit intra-class and inter-class distances, excessively large loss values ​​are prevented.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows:

[0008] Step 1: Split several public datasets into training set, test set and validation set. Each of the training set, test set and validation set contains several images of 7 basic facial expressions.

[0009] Step 2: Construct a facial expression recognition model, which includes a feature pre-extraction module, a multi-level attention module, and a multi-scale attention module.

[0010] Step 3: Perform face detection and alignment on the images in the training, test, and validation sets input to the model, and crop them to a fixed size. Randomly select 10% of the images in the training set input to the model for image enhancement. Image enhancement operations such as horizontal flipping, random cropping, random removal, and random rotation are used to prevent the model from overfitting.

[0011] Step 4: Input the training set images processed in Step 3 into the constructed facial expression recognition model. Use cross-entropy loss and restricted center loss to train and optimize the learnable parameters in the model until the accuracy of the model no longer improves. The specific training process includes the following sub-steps:

[0012] Step 4.1: Input the image processed in Step 3 into the feature pre-extraction module, which contains a total of n basic blocks. The output feature map X of each basic block in the feature pre-extraction module... i Features are the output of each stage of the feature pre-extraction module.

[0013] Step 4.2: The weight allocation module in the multi-level attention module assigns different weights to the feature maps output by the first n-1 basic blocks according to their importance. In this module, the average pooling layer is first used to assign different weights to the feature maps X output by the first n-1 basic blocks. i The channel information is aggregated to obtain the average pooled channel vector. Secondly, the channel vectors are processed through fully connected layers and activation layers. Dimensionality reduction is performed to obtain the final weight α. i The calculation formula is as follows:

[0014]

[0015]

[0016] Where AvgPool represents the average pooling operation; and These represent the ReLU activation function and the Sigmoid activation function, respectively. and The parameters represent the fully connected layer. The subscript indicates that it corresponds to the i-th basic block, and the superscript indicates which fully connected layer or activation layer it is. and α i These represent the average pooling channel vector and weights corresponding to the output features of the i-th basic block, respectively.

[0017] Step 4.3: By flattening the multidimensional feature map into a single dimension and using a fully connected layer, the feature maps output from the first n-1 basic blocks are mapped to the same dimension to obtain the feature map Z. i The calculation formula is as follows:

[0018] Z i =f(X) i )w i ,

[0019] Where f(*) represents the tie-break operation, w i Z represents the parameters of the fully connected layer. i The vector is obtained by reshaping the feature map output by the i-th basic block.

[0020] Step 4.4: Place α i With the corresponding Z i After multiplying and adding, we obtain the output Y of the multi-level attention module. out1 .

[0021] Where n represents the number of basic blocks, in the facial expression recognition model of this method, the feature pre-extraction module contains 4 basic blocks.

[0022] Step 4.5: Due to SEBlock's excellent channel aggregation capability, an SEBlock consisting of an average pooling layer, a fully connected layer, a sigmoid activation layer, and a ReLU activation layer is used to process the output X of the nth basic block of the feature pre-extraction module. n In this module, channel-level weighting is first applied, using an average pooling layer to weight the output X of the nth basic block. n The channel information is aggregated to obtain the average pooled channel vector. Secondly, using fully connected layers and activation layers to... After dimensionality increase, dimensionality reduction is obtained. A vector W of uniform size; finally, the resulting W is compared with X. n Multiplying them together gives S, thus realizing X. n Weighting on the channel dimension.

[0023] Step 4.6: Use the multi-scale fusion module in the multi-scale attention module to perform multi-scale fusion on the feature map S. In this module, firstly, the feature map S is passed through three dilated convolutional layers, a normalization layer, and an activation layer to obtain the corresponding feature map D.j Secondly, the obtained feature map D j The feature map D is obtained by addition and fusion. Its calculation formula is:

[0024] D j =σ j (BN j (μ j (S;j))),

[0025]

[0026] Where, σ j Represents the ReLU activation function, BN j (*) represents the normalization operation, μ j (·;j) represents the dilated convolution operation, and j represents the dilation ratio.

[0027] Step 4.7: Map the feature map D to Y using a flattening operation and a fully connected layer. out1 The final output Y of the multi-scale attention module is obtained on the same dimension. out2 .

[0028] Step 4.8: Y out1 and Y out2 The summations yield the final feature output Y. Its calculation formula is:

[0029] Y = β1Y out1 +β2Y out2 ,

[0030] Here, β1 and β2 are hyperparameters, representing the contributions of the outputs of the multi-level attention module and the multi-scale attention module to the final output features of the model.

[0031] Step 4.9: Construct the loss function, calculate the loss, and optimize the facial expression recognition model. To reduce the gap between the predicted score and the true label, this invention uses cross-entropy loss L. c1 The calculation formula is as follows:

[0032]

[0033] Where m represents the number of samples in each mini-batch, and k represents the number of classes. The label representing the g-th category corresponding to the h-th sample. This represents the predicted score for the g-th category corresponding to the h-th sample.

[0034] To reduce the distance between features of the same class and increase the distance between features of different classes, and to limit the magnitude of the loss value, this invention designs a restricted center loss L. lc The calculation formula is as follows:

[0035]

[0036] Where Y h c represents the predicted score for each category corresponding to the h-th sample. h This represents the center vector of the class corresponding to the h-th sample. c represents the square of the L2 norm. g This represents the center vector corresponding to the g-th category. γ1, γ2, ω1, and ω2 are hyperparameters, with γ1 and γ2 used to adjust L. lc The value of ω1 is used to constrain intra-class distance, ω2 is used to constrain inter-class distance, and δ represents a minimum value to prevent the denominator from being 0.

[0037] Adding the above loss functions together yields the total loss L. total .

[0038] Step 5: Perform inference on the images in the test set and validation set. Input the images from the training set and test set processed in Step 3 into the model to obtain the corresponding classification results. Recognize facial expressions and classify them as one of the seven basic facial expressions: disgust, happiness, anger, fear, surprise, sadness, and neutral.

[0039] The network architecture of this invention consists of a feature pre-extraction module, a multi-level attention module, and a multi-scale attention module. The feature pre-extraction module employs a multi-stage feature extraction approach, ensuring that the final extracted features contain both low-level spatial information from the shallow network (the first n-1 basic blocks) and high-level semantic information from the deep network (the nth basic block), increasing the robustness of the extracted features. The multi-level attention module operates on the shallow basic blocks of the feature pre-extraction module, assigning different weights to the features extracted from these blocks according to their importance, thus allowing the model to focus on important detailed features. The multi-scale attention module operates on the deepest basic blocks of the feature pre-extraction module, helping the model better integrate multi-scale contextual information, thereby improving the expressive power of the model's output features. Furthermore, this invention designs a restricted center loss, which, based on the center loss function, penalizes the center distance between different classes, widening the distance between them. By adding constraints on intra-class and inter-class distances, it effectively prevents excessively large loss values, thereby better optimizing the model and recognizing accurate facial expressions. Attached Figure Description

[0040] Figure 1 This is a flowchart of the facial expression recognition method based on multi-level and multi-scale attention mechanisms of the present invention;

[0041] Figure 2 This is a schematic diagram of the overall network architecture of an example of the present invention;

[0042] Figure 3 This provides an attention visualization of the multi-scale attention module and multi-level attention module of this method on the original image. Detailed Implementation

[0043] This invention proposes a facial expression recognition method based on multi-level and multi-scale attention mechanisms. For example... Figure 1 As shown, the overall network architecture consists of a feature pre-extraction module, a multi-level attention module, and a multi-scale attention module. First, the feature pre-extraction module extracts preliminary features from the input image. Research shows that in convolutional networks, deeper convolutions have larger receptive fields and learn richer semantic features, while shallower convolutions have smaller receptive fields and learn richer geometric features (such as shape, contour, and edges). Therefore, the feature pre-extraction module adopts a staged feature extraction approach to allow the model to capture more comprehensive information. Next, a multi-level attention module further extracts the shallow features from the feature pre-extraction module, assigning different weights based on the importance of the extracted features. The resulting features are multiplied by their corresponding weights and then summed to obtain the final output of the multi-level attention module. Then, a multi-scale attention block performs multi-scale fusion on the features output from the deepest layer of the feature pre-extraction module to improve the semantic information and representational power of the deep features. The output features are then fused with the final output of the multi-level attention module to obtain the final output of the model. Finally, cross-entropy loss and restricted center loss are used to calculate the loss on the original labels, the center vectors of each category, and the final output of the model. This step is only used during training to optimize the overall parameters of the network, enabling the model to predict accurate classification scores in subsequent inference.

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0045] Specifically, this invention is a facial expression recognition method based on multi-level and multi-scale attention mechanisms, such as... Figure 1 As shown, the main steps include the following:

[0046] Step 1: Split the four public datasets Affect-Net, RAF-DB, FERPlus, and SFEW2.0 into training, testing, and validation sets. Each of the training, testing, and validation sets contains several images of the seven basic facial expressions.

[0047] RAF-DB is a real-world dataset containing 30,000 facial expression images from thousands of people of different ages and from different countries. The occlusion, pose variations, and brightness of these images are uncontrolled, and each image is independently labeled by approximately 40 annotators. The dataset is divided into seven basic labels and contains 15,339 images (12,271 as the training set and 3,068 as the test set). The FERPlus dataset is an extension of the FER2013 dataset, a large-scale real-world dataset collected by the Google search engine. In the FER2013 dataset, all images were cropped to 48x48 after face alignment. However, due to the presence of many mislabeled images in the FER2013 dataset, the FERPlus dataset was expanded to include these images. The FERPlus dataset contains a total of 35,887 images (28,709 for training, 3,589 for testing, and 3,589 for validation). The Affect-Net dataset is the largest public dataset for facial expression recognition, collected from three major search engines, and contains 287,401 images (283,901 for training and 3,500 for testing). The SFEW2.0 dataset is extracted from the AFEW dataset and covers unconstrained facial expressions, variations in poses, different age ranges, and different facial resolutions, containing 1,766 images (958 for training, 372 for testing, and 436 for validation).

[0048] Step 2: As Figure 2 As shown, a facial expression recognition model is constructed, which includes a feature pre-extraction module, a multi-level attention module, and a multi-scale attention module.

[0049] Step 3: Perform face detection and alignment on the images in the training, test, and validation sets input to the model, and crop them to a fixed size. Randomly select 10% of the images in the training set input to the model for image enhancement. Image enhancement operations such as horizontal flipping, random cropping, random removal, and random rotation are used to prevent the model from overfitting.

[0050] Step 4: Input the processed training set images from Step 3 into the constructed facial expression recognition model. Use cross-entropy loss and restricted center loss to train and optimize the learnable parameters in the model until the model's accuracy no longer improves. The specific training process includes the following sub-steps:

[0051] Step 4.1: Input the image processed in Step 3 into the feature pre-extraction module, which is composed of ResNet-50. ResNet-50 contains four basic blocks, each containing 3, 4, 6, and 3 Bottlenecks respectively. Each Bottleneck contains one 3x3 convolution and two 1x1 convolutions. When the image is input into the feature pre-extraction module, the spatial scale of the feature map is halved and the number of channels doubles with each basic block. The output feature map X of each basic block in the feature pre-extraction module is then processed. i These are the features output at each stage of the feature pre-extraction module. Their calculation formula is:

[0052] X i =r(X) i-1 ;θ i ),

[0053] Where r(·) represents the convolution and activation operations, θ i This represents the parameters in each basic block of the feature pre-extraction module, where i indicates the number of the basic block, starting from 1.

[0054] Step 4.2: The feature maps output by the first three basic blocks of the feature pre-extraction module are assigned different weights according to their importance using a weight allocation module. This module consists of an average pooling layer, a fully connected layer, and an activation layer. In this module, the average pooling layer is first used to assign different weights to the feature maps X output by the first n-1 basic blocks. i The channel information is aggregated to obtain the average pooled channel vector. Secondly, the channel vectors are processed through two fully connected layers and activation layers. Dimensionality reduction is performed to obtain the final weight α. i The calculation formula is as follows:

[0055]

[0056]

[0057] Where σ represents the ReLU activation function, and AvgPool represents the average pooling operation. and These represent the ReLU activation function and the Sigmoid activation function, respectively. and The parameters represent the fully connected layer. The subscript indicates that it corresponds to the i-th basic block, and the superscript indicates which fully connected layer or activation layer it is. and α i These represent the average pooling channel vector and weights corresponding to the output features of the i-th basic block, respectively.

[0058] Step 4.3: Flatten the multidimensional feature map into a one-dimensional shape using a flattening operation and a fully connected layer to output the feature map X from the first n-1 basic blocks. i The feature map Z is obtained by mapping to the same dimension. i The calculation formula is as follows:

[0059] Z i =f(X) i )w i ,

[0060] Where f(*) represents the tie-break operation, w i Z represents the parameters of the fully connected layer. i The vector is obtained by reshaping the feature map output by the i-th basic block.

[0061] Step 4.4: Place α i With the corresponding Z i After multiplying and adding, we obtain the output Y of the multi-level attention module. out1 :

[0062]

[0063] Where n represents the number of basic blocks, in the facial expression recognition model of this method, the feature pre-extraction module contains 4 basic blocks.

[0064] Step 4.5: Due to SEBlock's excellent channel aggregation capability, this method uses SEBlock to process the output X of the fourth basic block of the feature pre-extraction module. n Weighting is performed along the channel dimension. SEBlock is primarily composed of an average pooling layer, a fully connected layer, a sigmoid activation layer, and a ReLU activation layer connected sequentially. In this module, firstly, the average pooling layer is used to weight the output X of the nth basic block. n The channel information is aggregated to obtain the average pooled channel vector. Secondly, using fully connected layers and ReLU activation layers to... After dimensionality increase, dimensionality reduction is performed using fully connected layers and Sigmoid activation layers to obtain the result. A vector W of uniform size; finally, the resulting W is compared with X. n Multiplying them together gives S, thus realizing X. n Weighting along the channel dimension. The calculation formula is:

[0065]

[0066]

[0067] S = WX n ,

[0068] Step 4.6: Use a multi-scale fusion module to perform multi-scale fusion on the feature map S. This module mainly consists of three 3x3 dilated convolutional layers with different dilation ratios, a normalization layer, and an activation layer. In this module, the feature map S is first passed through three different dilated convolutional layers, normalization layers, and activation layers, with the dilation ratios of the three dilated convolutions set to 1, 2, and 3, respectively, to obtain the corresponding feature map D. j Secondly, the obtained feature map D j The feature map D is obtained by addition and fusion. Its calculation formula is:

[0069] D j =σ + (BN j (μ j (S;j))),

[0070]

[0071] Where, σ j Represents the ReLU activation function, BN j (*) represents the normalization operation, μ j (·;j) represents the dilated convolution operation, and j represents the dilation ratio.

[0072] Step 4.7: Map the feature map D to Y using a flattening operation and a fully connected layer. out1 The final output Y of the multi-scale attention module is obtained on the same dimension. out2 The calculation formula is as follows:

[0073] Y out2 =f(D)w n ,

[0074] Step 4.8: Y out1 and Y out2 The summations yield the final feature output Y. Its calculation formula is:

[0075] Y = β1Y out1 +β2Y out2 ,

[0076] Here, β1 and β2 are hyperparameters, representing the contributions of the outputs of the multi-level attention module and the multi-scale attention module to the final output features of the model.

[0077] Step 4.9: Construct the loss function, calculate the loss, and optimize the facial expression recognition model. To reduce the gap between the predicted score and the true label, this invention uses cross-entropy loss L. ce The calculation formula is as follows:

[0078]

[0079] Where m represents the number of samples in each mini-batch, and k represents the number of classes. The label representing the g-th category corresponding to the h-th sample. This represents the predicted score for the g-th category corresponding to the h-th sample.

[0080] Traditional center loss only considers bringing features of the same class closer together, but not those of different classes, and is prone to producing large loss values. Therefore, this invention proposes a restricted center loss L. lc It builds upon center loss by penalizing the distance between center points of different classes, thereby widening the center distance between them. Furthermore, it limits intra-class and inter-class distances through additional constraints, thus preventing excessively large loss values. Its calculation formula is:

[0081]

[0082] Where Y h c represents the predicted score for each category corresponding to the h-th sample. h c represents the center vector of the class corresponding to the h-th sample. g This represents the center vector corresponding to the g-th category. The L2 norm squared is represented by γ1, γ2, ω1, and ω2, which are hyperparameters. γ1 and γ2 are used to adjust L2 norm. lc The value of ω1 is used to constrain intra-class distance, ω2 is used to constrain inter-class distance, and δ represents a minimum value to prevent the denominator from being 0.

[0083] Combining the above loss functions yields the total loss L. total The calculation formula is as follows:

[0084] L total =L ce +L lc .

[0085] Step 5: Perform inference on the images in the test set and validation set. Input the processed training set and test set images into the model to obtain the corresponding classification results, which are one of the seven basic expressions: disgust, happiness, anger, fear, surprise, sadness, and neutrality.

[0086] To demonstrate the effectiveness of the method, this method utilizes heatmaps to visualize the regions of interest in the model, such as... Figure 3As shown in the figure, the first row represents anger, the second row represents fear, the first column represents the original image, and the second and third columns represent the attention visualizations of the multi-scale attention module and the multi-level attention module's final outputs to the original image. The figure clearly shows that the multi-scale attention module focuses more on global features, while the multi-level attention module focuses more on important detailed features. Through the cooperation of these two modules, the accuracy of facial expression recognition is effectively improved.

[0087] Table 1 shows the experimental results comparing our method with other advanced methods on the RAF-DB dataset.

[0088]

[0089] Table 2 shows the experimental results comparing our method with other advanced methods on the Affect-Net dataset.

[0090]

[0091] Table 3 shows the experimental results comparing our method with other advanced methods on the FERPlus dataset.

[0092]

[0093] To demonstrate the superiority of the method, this invention was compared with other state-of-the-art methods on three datasets: RAF-DB, Affect-Net, and FERPlus. The experimental results are shown in Tables 1, 2, and 3. The experimental results demonstrate that this method has a high recognition accuracy.

Claims

1. A facial expression recognition method based on multi-level and multi-scale attention mechanism, characterized in that, The method comprises the following steps: Step 1: split a plurality of public data sets into a training set, a test set and a validation set; Step 2: construct a facial expression recognition model comprising a feature pre-extraction module, a multi-level attention module and a multi-scale attention module; Step 3: perform face detection and alignment on the images in the training set, test set and validation set input to the model, and crop them to a fixed size; Step 4: input the images in the training set processed in step 3 to the constructed facial expression recognition model, train and optimize the parameters in the model using cross-entropy loss and center loss restriction until the accuracy of the facial expression recognition model no longer improves, and the specific process is as follows: Step 4.6: use the multi-scale fusion module in the multi-scale attention module to perform multi-scale fusion on the feature map S to obtain a feature map D; Step 4.1: input the processed image in step 3 into a feature pre-extraction module, wherein the feature pre-extraction module comprises n basic blocks, output feature maps X i as the features output by each stage of the feature pre-extraction module Step 4.2: The feature maps output by the first n-1 basic blocks are assigned different weights a according to their importance by the weight assignment module in the multi-level attention module i ; Step 4.3: The flattened operation of unfolding the multi-dimensional feature map into one dimension and the fully connected layer are used to map the feature maps X output by the first n-1 basic blocks to the same dimension to obtain the feature map Z i i ;​ Step 4.4: multiply the obtained a i with the corresponding Z i and add them to obtain the final output Y of the multi-level attention module out1 ; Step 4.5: the output X of the nth basic block of the feature pre-extraction module is subjected to SEBlock composed of sequentially connected average pooling layer, full connection layer, Sigmoid activation layer and ReLu activation layer n Weighting in the channel dimension is performed to obtain the feature map S; Where β1 and β2 are hyperparameters; Step 4.7: Map the feature map D to Y by a flatten operation and a fully connected layer out1 The final output Y of the multi-scale attention module on the same size dimension out2 ; Step 4.8: Add Y out1 and Y out2 to get the final feature output Y, which is calculated as: Y = β1Y + β2Y out1 + β2Y out2 , Step 4.9: construct a loss function to calculate the loss and optimize the facial expression recognition model, and the loss function for calculating the loss is as follows: Step 5: input the images in the training set and test set processed in step 3 to the facial expression recognition model to obtain the corresponding classification results and recognize the expressions. (1) Cross-entropy loss L ce The formula is: where m represents the number of samples in each mini-batch, and k represents the number of classes, represents the label of the gth class corresponding to the hth sample, represents the predicted score of the gth class corresponding to the output of the hth sample. (2) the center loss L lc The formula is: where Y h represents the prediction score of the hth sample corresponding to each category, c h represents the center vector of the hth sample corresponding to each category, c represents the square of the L2 norm, c g represents the center vector of the gth category, γ1, γ2, ω1, ω2 are hyperparameters, γ1 and γ2 adjust the size of L lc , ω1 restricts the intra-class distance, and ω2 restricts the inter-class distance, and δ represents a minimum value; (3) the cross-entropy loss L ce and the center loss L lc are added to obtain the total loss L total ; In step 1, the training set, test set and validation set each contain a plurality of images of 7 basic expressions; 2. The method of claim 1, wherein the method is based on a multi-level and multi-scale attention mechanism. The 7 basic expressions are disgust, happiness, anger, fear, surprise, sadness and neutral. In step 3, the image enhancement operation is horizontal flipping, random cropping, random removal and random rotation.

3. The method of claim 1, wherein the method is based on a multi-level and multi-scale attention mechanism. ​ 4. The method of claim 3, wherein the method is based on a multi-level and multi-scale attention mechanism. In the weight distribution module described in step 4.2, first, the channel information of the feature maps X i output by the first n-1 basic blocks is aggregated by using an average pooling layer to obtain an average pooling channel vector Second, the channel vector is reduced in dimension through a fully connected layer and an activation layer to obtain the weight α i .​ 5. The method of claim 4, wherein the method is based on a multi-level and multi-scale attention mechanism. In the SEBlock described in step 4.5, first, the output X of the nth basic block is aggregated by using an average pooling layer on the channel information of X to obtain an average pooling channel vector n ​ Secondly, the full connection layer and the activation layer are used to process the vector W obtained by the dimensionality reduction of the vector X to obtain a vector W with the same dimension as the vector X. the vector X.​ Finally, the obtained W is multiplied with X n to obtain the feature map S.

6. The method of claim 5, wherein the method is based on a multi-level and multi-scale attention mechanism. In the multi-scale fusion module described in step 4.6, the feature map S is first passed through three hollow convolution layers, normalization layers and activation layers respectively to obtain the corresponding feature map D j ; Secondly, the obtained feature map D j addition fusion to obtain the feature map D.

Images