Facial expression recognition method, system and device based on spatial channel convolution and enhanced compression incentive attention

Abstract

The present invention relates to a facial expression recognition method based on spatial channel convolution and enhanced compression excitation attention, including: S1 using spatial channel convolution to expand the receptive field and capture the global information of the image; S2 performing feature fusion on feature information of different scales and different levels; S3 using a compression excitation module of global attention to adaptively adjust feature mapping weights to enhance the model's attention to important features. The spatial channel convolution of the present invention can enhance the perception of multi-scale and cross-level features; the feature fusion module integrates feature information of different scales and levels in the channel dimension, and realizes the optimal combination of feature information through an adaptive weight acquisition mechanism; the compression excitation module of global attention can alleviate the problem of information loss and improve recognition accuracy. The method of the present invention has achieved significant improvements in the accuracy of facial expression recognition, while the computational cost and network scale are much smaller than similar existing technologies, and has higher accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and machine learning, and in particular to a facial expression recognition method and system based on spatial channel convolution and enhanced compression-stimulated attention. Background Art

[0002] With the development and application of deep learning technology, facial expression recognition has made significant progress. However, facial expression recognition still faces several challenges, primarily due to the diversity and complexity of facial expressions. Specifically, the diversity of facial expressions is reflected in the differences between individuals and situations, making it difficult for models to generalize to all people and scenarios. The complexity of facial expressions is reflected in subtle changes in expression, such as micro-expressions, which are very difficult to capture and recognize.

[0003] In terms of technical implementation, traditional convolutional neural networks, limited by their local receptive field, struggle to fully capture the deep semantic information of images. Furthermore, issues such as inconsistent multi-scale information and missing cross-layer features make it difficult for the model to accurately capture the complex characteristics of facial expressions, severely impacting its generalization and robustness. Furthermore, traditional convolutional neural networks are unable to effectively learn and utilize the importance of each channel's features, causing the model to fail when processing complex data. Furthermore, as the number of network layers increases, key information from the original input is easily lost, impacting model performance.

[0004] To address these issues, researchers have attempted to introduce various technologies and methods, such as multi-scale feature extraction, multi-view feature fusion, and attention mechanisms, in order to improve the accuracy and robustness of facial expression recognition. However, these methods still have certain limitations in practical applications, such as high computational cost, high model complexity, and limited ability to recognize complex emotional changes. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to design a facial expression recognition method and system based on spatial channel convolution and enhanced compression incentive attention, aiming to solve the problem of insufficient facial expression feature extraction caused by the diversity and complexity of facial expressions in the existing technology, improve the accuracy and robustness of facial expression recognition, and thus solve the existing technical problems.

[0006] To solve the above technical problems, the facial expression recognition method based on spatial channel convolution and enhanced compression incentive attention of the present invention comprises the following steps:

[0007] Step S1: Use spatial channel convolution to expand the receptive field and capture the global information of the image.

[0008] Step S2: Feature fusion of feature information of different scales and levels.

[0009] Step S3: Use a global attention compression excitation module to adaptively adjust the feature mapping weights to enhance the model's attention to important features. The global attention compression excitation module includes a dual-branch structure, in which one branch is used to capture global information and the other branch is used to capture local feature detail information.

[0010] Furthermore, in step S1, for the input features of the residual network , first obtain the spatial refinement features through spatial feature reconstruction , and then obtain channel refinement features through channel feature reconstruction .

[0011] Furthermore, spatial feature reconstruction includes:

[0012] Feature map segmentation: Segment feature maps with more information from feature maps with less information, and use the group normalization scaling factor to evaluate the different information contents of feature maps;

[0013] Feature map reconstruction: information-rich features Features with less information Sum, by adopting the cross reconstruction strategy to fully combine different information features, the cross reconstruction features and Splicing to obtain optimized spatial feature map .

[0014] Furthermore, the specific method of feature map segmentation is: given feature ,in Indicates batch processing, Indicates the number of channels, and Represents height and width respectively, specifically including the following steps:

[0015] Step 1: Input features Standardized processing to obtain ,

[0016] ,in, is the mean, is the standard deviation, is a trainable affine transformation, is a constant;

[0017] Step 2: Obtain the importance of different feature maps :

[0018]

[0019] Step 3: Use the Sigmod function to map the weighted feature map to the (0, 1) range and gate it by the threshold: set the weight of the feature map greater than the threshold to 1, and construct the important content weight ; At the same time, the weight of the feature map smaller than the threshold is set to 0, and the weight of non-important content is constructed ;

[0020] Step 4: Input features Multiply by and Get two weighted features: informative and less informative features .

[0021] Furthermore, channel feature reconstruction includes:

[0022] segmentation:

[0023] For spatial optimization features , first divide its channels into Channel and The channel has two parts, , then use 1×1 convolution to compress the feature map, and control the feature channel by introducing the compression ratio r. At this time, the spatial optimization feature Divided into important features and unimportant features ;

[0024] Mapping:

[0025] Important features Perform feature mapping, including:

[0026] Use group convolution and point convolution instead of k×k convolution to extract high-level features;

[0027] Add the two parts of output features to construct a representative feature map :

[0028] ;

[0029] in, and Group convolution and point convolution replace k×k convolution to extract high-level features; and are input features and output feature maps respectively;

[0030] For non-important features Perform feature mapping, including:

[0031] Use 1×1 point convolution to generate feature maps with shallow hidden details as a supplement to the important information feature maps;

[0032] The generated feature map is concatenated with the initial non-important information features to form the output of non-important information features. :

[0033] ;

[0034] in, is the weight matrix of point convolution; For splicing; and are input features and output features respectively;

[0035] Fusion:

[0036] After the mapping is completed, the two types of features are adaptively merged using a simplified residual structure, including:

[0037] Use global average pooling to collect global spatial information and perform channel statistics :

[0038]

[0039] in, Indicates the spatial position in the cth channel Feature map at ;

[0040] The channel features of the upper and lower branches are stacked as and , and use channel attention to generate the corresponding feature importance and :

[0041]

[0042] In feature importance and Under the guidance of and Merge in a channel manner to obtain refined channel features : .

[0043] Furthermore, in step S2, feature fusion specifically includes the following steps:

[0044] Step S21: Global enhancement features and local attention features Perform concatenation and dimensionality reduction on the channel dimension and output features ;

[0045] Step S22: Using two sets of linear calculations And Sigmoid function to obtain adaptive fusion weights:

[0046]

[0047] in, is the adaptive weight for learning, It includes batch normalization, ReLU activation function and convolution operation;

[0048] Step S23: According to the adaptive weight Realize weighted fusion of global enhancement features and local attention features:

[0049]

[0050] is the adaptive fusion feature, Represents element-wise multiplication.

[0051] Furthermore, the adaptive fusion feature Input to the global average pooling layer (GAP layer), fully connected layer and Softmax classifier to obtain its category vector and the final expression category :

[0052]

[0053]

[0054] in, is the number of expression categories that the model can recognize, is the eigenvector Judged as the The probability of expression-like Indicates the final predicted expression category.

[0055] Furthermore, step S3 specifically includes:

[0056] Step S31: Input features Divided into Sub-features , , and and They are respectively used as input for proposing two branches;

[0057] Step S32: using the proposed dual-branch capture feature to reduce the complexity of the model and prevent information loss;

[0058] Step S33: Capture different semantic information through cross-channel learning without losing generality , and the learned attention weights are used to strengthen the feature representation of the region of interest in each sub-feature.

[0059] Furthermore, in step S3, the global attention compression excitation module uses two parallel branches to extract the attention weights of feature compression, where the 1×1 branch is used to capture the global information of the input features and perform cross-channel interactive learning in a global range; the 3×3 branch is used to capture local feature detail information and perform cross-channel interactive learning in a local range.

[0060] Furthermore, in the first branch, each group of features is first normalized using group normalization; then the feature map of the 1×1 branch is encoded using the global average pooling shown in formula (1):

[0061]

[0062] in, Represents the spatial position in the cth channel of the input feature map X Feature map at ;

[0063] Secondly, softmax is introduced to improve computational efficiency; finally, the average pooling layer is used to fit the linear transformation, and the obtained feature map is multiplied element-wise with the feature map of the 3×3 branch to obtain the first spatial attention;

[0064] Furthermore, in the second branch, the global information in the 3×3 branch is first encoded using the global average pooling shown in formula (1); then the obtained encoded features are element-wise multiplied with the feature map of the 1×1 branch to obtain the second spatial attention;

[0065] In the final stage of the enhanced compression excitation module, based on the fusion of the two spatial attention features, the RELU activation function is used to obtain nonlinear features, and the weights are reweighted so that the final feature map has the same dimension as the input feature map.

[0066] Furthermore, it also includes a data augmentation step, and uses a small number of sample data to fine-tune the pre-trained model. Data augmentation is to generate more diverse training samples by performing a series of transformation operations on the original training data, thereby increasing the model's adaptability to different inputs. The pre-trained model is a model trained on a large-scale dataset, specifically including:

[0067] Choose a pre-trained model: Choose a convolutional neural network model pre-trained on a large-scale image dataset (such as ImageNet), such as ResNet and VGG.

[0068] Freeze some layers: During fine-tuning, freeze some layers of the pre-trained model (usually the bottom convolutional layers) to retain its strong feature extraction capabilities;

[0069] Add a custom layer and add a custom fully connected layer on top of the pre-trained model to adapt to the output requirements of the facial expression recognition task;

[0070] Fine-tuning training: Use a small amount of facial expression sample data to train the model. In this invention, a small amount of facial expression sample data refers to the sample data amount in the range of [800, 1200], and adjust the weights of the model to achieve better performance on specific tasks;

[0071] Learning rate adjustment: During the fine-tuning process, a lower learning rate is used. In the present invention, a lower learning rate refers to a learning rate in the range of [0.0001, 0.001] to avoid excessive adjustments to the weights of the pre-trained model, thereby retaining the common features learned in its pre-training stage.

[0072] Furthermore, it also includes an image quality assessment step, and combines the predictions of multiple models to reduce recognition bias and improve generalization ability. The specific methods of image quality assessment include image clarity assessment, lighting condition assessment, noise level assessment, face integrity assessment, and image resolution assessment.

[0073] The present invention also provides a facial expression recognition system based on spatial channel convolution and enhanced compression to stimulate attention, comprising:

[0074] The spatial channel convolution module includes a spatial reconstruction unit and a channel reconstruction unit. The spatial reconstruction unit is used to input features and separate features with larger information content from features with smaller information content, and reconstruct them to enhance representative features and suppress redundant features. The channel reconstruction unit adopts a segmentation-mapping-fusion strategy to extract rich representative features through lightweight convolution, and at the same time processes redundant features through a feature reuse scheme.

[0075] The feature fusion module performs splicing and dimensionality reduction in the channel dimension, and uses two sets of linear calculations and Sigmoid functions to obtain adaptive fusion weights, integrating feature information of different scales and levels, and enhancing the perception and representation capabilities of multi-scale, cross-level, and complex environments.

[0076] The compression excitation module, based on global attention, adaptively adjusts the feature mapping weights through a dual-branch structure, enhancing the model's attention to important features and improving recognition accuracy.

[0077] The present invention further provides an electronic device, comprising:

[0078] at least one processor; and

[0079] at least one memory in communication with the processor;

[0080] The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to enable the electronic device to perform the aforementioned facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention.

[0081] Beneficial effects of the present invention:

[0082] The facial expression recognition method and system based on spatial channel convolution and enhanced compression to stimulate attention can effectively solve the problem of insufficient facial expression feature extraction in the prior art due to the diversity and complexity of facial expressions, which is specifically manifested in the following aspects:

[0083] 1. This paper proposes a spatial channel convolution technology, which expands the receptive field of the model by combining the spatial reconstruction unit and the channel reconstruction unit, effectively captures the global information of the image, and enhances the perception of multi-scale and cross-level features.

[0084] 2. The present invention designs a feature fusion module that can integrate feature information of different scales and levels in the channel dimension, enhance the perception and representation capabilities of multi-scale, cross-level, and complex environments, and achieve optimal combination of feature information through an adaptive weight acquisition mechanism.

[0085] 3. The present invention introduces a global attention compression excitation module, which adaptively adjusts the feature mapping weights through a dual-branch structure, can alleviate the problem of information loss and enhance the model's attention to important features, thereby improving recognition accuracy.

[0086] 4. This invention improves sample diversity through data enhancement technology, and fine-tunes the pre-trained model with a small amount of sample data to enhance the generalization and robustness of the model.

[0087] 5. The present invention further combines the image quality assessment step to reduce the impact of image quality on recognition performance, and improves the accuracy and robustness of recognition through multi-model fusion technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The specific embodiments of the present invention will be further explained below with reference to the accompanying drawings.

[0089] Figure 1 Flowchart of the facial expression recognition method of the present invention.

[0090] Figure 2 Schematic diagram of the framework of the facial expression recognition method in the present invention.

[0091] Figure 3 Schematic diagram of the structure of spatial channel convolution in the present invention.

[0092] Figure 4 Schematic diagram of the structure of the spatial feature reconstruction submodule in the present invention.

[0093] Figure 5 Schematic diagram of the structure of the channel feature reconstruction submodule in the present invention.

[0094] Figure 6 Schematic diagram of the structure of the feature fusion module in the present invention.

[0095] Figure 7 This is a structural diagram of an existing compression excitation module.

[0096] Figure 8 Schematic diagram of the structure of the enhanced compression excitation module of global attention in the present invention.

[0097] Figure 9 This is a block diagram of the facial expression recognition system based on spatial channel convolution and enhanced compression-stimulated attention in the present invention.

[0098] in, Figure 2 and Figure 3 The SFRU in the figure represents the spatial feature reconstruction submodule; the CFRU represents the channel feature reconstruction submodule. DETAILED DESCRIPTION

[0099] Example 1

[0100] Combine Figure 1 and Figure 2 The facial expression recognition method based on spatial channel convolution and enhanced compression-stimulated attention in this embodiment specifically includes the following steps:

[0101] Step S1: Use spatial channel convolution to expand the receptive field and capture the global information of the image.

[0102] In this embodiment, preferably, in step S1, the input feature of the residual network , first obtain the spatial refinement features through spatial feature reconstruction , and then obtain channel refinement features through channel feature reconstruction , the structure of spatial channel convolution is as follows Figure 3 As shown in Figure 3, the spatial channel convolution module can be seamlessly integrated into any convolutional neural network architecture to reduce spatial feature redundancy and channel feature redundancy and enhance feature representation.

[0103] In this embodiment, preferably, the spatial feature reconstruction submodule is as follows: Figure 4 As shown, spatial feature reconstruction includes:

[0104] Feature map segmentation: Segment feature maps with more information from feature maps with less information, and use the group normalization scaling factor to evaluate the different information contents of feature maps.

[0105] In this embodiment, the specific method of feature map segmentation is as follows: given a feature ,in Indicates batch processing, Indicates the number of channels, and Represents height and width respectively, specifically including the following steps:

[0106] Step 1: Input features Standardized processing to obtain :

[0107] ,in, is the mean, is the standard deviation, For a trainable affine transformation, the richer the spatial information, the greater the spatial pixel changes, resulting in a larger , is a constant, is a small positive constant added for stability, The value range of is: In this embodiment, The value is 0.01.

[0108] Step 2: Obtain the importance of different feature maps :

[0109]

[0110] Step 3: Use the Sigmod function to map the weighted feature map to the (0, 1) range and gate it by the threshold: set the weight of the feature map greater than the threshold to 1, and construct the important content weight ; At the same time, the weight of the feature map smaller than the threshold is set to 0, and the weight of non-important content is constructed The whole process of obtaining weights can be expressed as follows:

[0111]

[0112] Step 4: Input features Multiply by and Get two weighted features: informative and less informative features . Therefore, the input feature is successfully It is divided into two parts: Informative and expressive spatial content, while There is almost no information and can be considered redundant.

[0113] Feature map reconstruction: information-rich features Features with less information Sum, by adopting the cross reconstruction strategy to fully combine different information features, the cross reconstruction features and Splicing to obtain optimized spatial feature map .

[0114] To reduce spatial feature redundancy, the reconstruction step of the spatial feature reconstruction submodule is used to sum information-rich features with less information-rich features to generate more information-rich features. The present invention adopts a cross-reconstruction strategy to fully combine different information features and enhance the information flow between them. Cross-reconstruction not only avoids the information loss that may result from simple addition but also promotes the flow and fusion of different information features, making the optimized feature map more representative in the spatial dimension, thereby improving the model's perception of complex expression features and generalization performance.

[0115] The entire process of spatial feature reconstruction can be expressed as follows:

[0116]

[0117] in:

[0118] Information-rich features Sub-features generated during cross-reconstruction;

[0119] Features with less information Sub-features generated during cross-reconstruction;

[0120] Features with less information and The cross-combination results;

[0121] Information-rich features and The cross-combination results;

[0122] 、 is the cross-reconstructed feature map, where is an information-rich feature map, It is a feature map with sparse information;

[0123] represents element-wise multiplication, represents the sum of elements, Indicates element splicing.

[0124] Apply the spatial feature reconstruction submodule to the input features After that, not only are the features with more information separated from the features with less information, but they are also reconstructed to enhance the representative features and suppress the redundant features. There is still redundancy in the channels.

[0125] To reduce channel feature redundancy, the convolution is replaced by the channel feature reconstruction submodule, which adopts the segmentation-mapping-fusion strategy. Using k×k convolution kernel to extract features can easily lead to redundant feature maps in the channel dimension direction. represents the k×k convolution kernel, , , , and Represent the input features and convolution output features respectively.

[0126] In this embodiment, preferably, the channel feature reconstruction submodule is as follows Figure 5 As shown in the figure, GWC (Group-Wise Convolution) represents group convolution, PWC (Point-Wise Convolution) represents point convolution, and channel feature reconstruction includes:

[0127] segmentation:

[0128] For spatial optimization features , first divide its channels into Channel and The channel has two parts, , then 1×1 convolution is used to compress the feature map, and the compression ratio r is introduced to control the feature channel to improve the computational efficiency.

[0129] At this time, the spatial optimization feature Divided into important features and unimportant features ;

[0130] Mapping:

[0131] Important features Perform feature mapping, including:

[0132] Efficient convolutions (including group convolution and point convolution) are used instead of k×k convolution to extract high-level features and reduce computational costs. Due to sparse connections, group convolution effectively reduces the number of parameters, but it also cuts off the flow of information between channels; while point convolution effectively compensates for information loss and facilitates cross-channel information flow.

[0133] Add the two parts of output features to construct a representative feature map :

[0134] ;

[0135] in, and Group convolution and point convolution replace k×k convolution to extract high-level features and reduce computational cost; and are input features and output feature maps respectively. In short, the important information feature mapping process uses feature maps The combination of group convolution and point convolution extracts representative features at a lower computational cost .

[0136] In the segmentation-mapping-fusion process of channel feature reconstruction, r determines the compression ratio of the feature channel, which is used to reduce the amount of calculation and highlight important features. When using group convolution and point convolution to extract features, the number of channels is compressed by r from Compress to .

[0137] For non-important features Perform feature mapping, including:

[0138] 1×1 point convolution is used to generate feature maps with shallow hidden details as a supplement to the important information feature maps.

[0139] The generated feature map is concatenated with the initial non-important information features to form the output of non-important information features. :

[0140] ;

[0141] in, is the weight matrix of point convolution; For splicing; and are input features and output features respectively;

[0142] Fusion:

[0143] After the mapping is completed, the two types of features are not directly concatenated or added together. Instead, a simplified residual structure is used to adaptively merge the two types of features. Specifically,

[0144] Use global average pooling to collect global spatial information and perform channel statistics :

[0145]

[0146] in, represents the feature map at spatial position (i, j) in the cth channel;

[0147] The channel features of the upper and lower branches are stacked as and , and use channel attention to generate the corresponding feature importance and :

[0148] is a natural constant used to calculate the exponential part of the Softmax function. and Convert to a positive number and normalize it to get the feature importance and . and It is a channel-level global statistic, which is composed of upper-level features (high information content) and lower-level features (low information content) is calculated by global average pooling, which indicates the global importance of each channel. and Generate feature importance through Softmax normalization and , used for adaptive fusion and , which enables the model to automatically enhance important features and suppress redundant information, thereby improving computational efficiency and feature expression capabilities.

[0149] In feature importance and Under the guidance of and Merge in a channel manner to obtain refined channel features : .

[0150] In short, the channel feature reconstruction submodule, which incorporates a segmentation-mapping-fusion strategy, further reduces the redundancy of spatially optimized features. Furthermore, the channel feature reconstruction submodule extracts rich, representative features through lightweight convolution while simultaneously addressing redundant features through a feature reuse scheme. Overall, the channel feature reconstruction submodule can be used independently or in conjunction with the spatial feature reconstruction submodule.

[0151] Step S2: Feature fusion of feature information of different scales and levels.

[0152] The feature fusion module performs splicing and dimensionality reduction on the channel dimension, and uses two sets of linear calculations and Sigmoid functions to obtain adaptive fusion weights. The present invention captures global context information through the channel space global feature enhancement structure; at the same time, it uses the channel space local feature attention structure to obtain local attention features, thereby achieving the fusion of different features. In order to avoid the limitations of artificially assigned weights, the feature fusion module of the present invention is as follows: Figure 6 shown.

[0153] In this embodiment, preferably, in step S2, feature fusion specifically includes the following steps:

[0154] Step S21: Global enhancement features and local attention features Perform concatenation and dimensionality reduction on the channel dimension and output features :

[0155]

[0156] Represents the feature concatenation operation along the channel dimension C. Features from different sources (global / local) or different levels are concatenated along the channel dimension, preserving their respective feature information and providing richer input for subsequent adaptive weighted fusion.

[0157] Step S22: Using two sets of linear calculations And Sigmoid function to obtain adaptive fusion weights:

[0158]

[0159] in, is the adaptive weight for learning, It includes batch normalization, ReLU activation function and convolution operation.

[0160] Step S23: According to the adaptive weight Realize weighted fusion of global enhancement features and local attention features:

[0161]

[0162] is the adaptive fusion feature, Denotes element-wise multiplication. The designed fusion strategy adaptively learns the importance of global enhancement features and local attention features and performs an autonomous weighted summation between the two, which helps improve the recognition network's ability to resist interference from factors such as occlusion and posture changes.

[0163] In this embodiment, preferably, the adaptive fusion feature Input to the global average pooling layer (GAP layer), fully connected layer and Softmax classifier to obtain its category vector and the final expression category :

[0164]

[0165]

[0166] in, is the number of expression categories that the model can recognize, is the eigenvector Judged as the The probability of expression-like Indicates the final predicted expression category, including Angry, Happy, Disgust, Sadness, Fear, Neutral, Surprise, and Contempt.

[0167] Step S3: Use a global attention compression excitation module to adaptively adjust the feature mapping weights to enhance the model's attention to important features. The global attention compression excitation module includes a dual-branch structure, in which one branch is used to capture global information and the other branch is used to capture local feature detail information.

[0168] The feature capability of convolutional neural networks is crucial for improving model performance. However, a common problem is that as the network depth increases, useful feature information is lost or diluted, making it difficult for the model to capture subtle feature changes, especially in complex facial expression recognition tasks. To solve this problem, existing technologies usually use a compressed excitation module (such as Figure 7 As shown in the figure, it is used as a channel attention mechanism to enhance the feature representation ability of convolutional neural networks. The compression excitation module consists of three stages: compression stage, excitation stage and scaling stage. In the compression stage, global pooling is used to compress the input features of each channel to obtain the global information of each channel. In the excitation stage, the global feature information obtained is first mapped to the feature information of the intermediate dimension using full connection, and then the activation function is used to increase the nonlinear representation ability. Finally, another full connection is used to map the feature information of the intermediate dimension back to the original channel feature information to obtain the activation weight of each channel. In the scaling stage, each channel feature is re-weighted by element-by-element multiplication based on the learned channel activation weight to enhance important features and suppress unimportant features. The input feature is Channel attention The calculation formula is:

[0169]

[0170] represents channel global average pooling, ,in, 、 represents the weights of the two fully connected layers, represents the RELU activation function, represents the Sigmoid function, Represents the spatial position in the cth channel of the input feature map X Feature map at .

[0171] However, the compression-excitation module itself has some limitations. The most significant issue is the compression phase, where dimensionality reduction leads to the loss of useful information, limiting the model's ability to capture complex features. The first full connection reduces the input feature channels from C to C / r, while the second full connection expands the simplified feature channels back to the original feature channels C. Therefore, this change in feature channel size leads to a loss of channel information.

[0172] In this embodiment, in order to avoid information loss and further improve model performance, the enhanced compression excitation module based on global attention in the present invention is preferably Figure 8 As shown, step S3 specifically includes:

[0173] Step S31: Input features Divided into Sub-features , , and and They are respectively used as input for proposing two branches.

[0174] Step S32: Utilize the proposed dual-branch feature capture method to reduce the complexity of the model and prevent information loss.

[0175] Step S33: Capture different semantic information through cross-channel learning without losing generality , and the learned attention weights are used to strengthen the feature representation of the region of interest in each sub-feature.

[0176] Preferably, in this embodiment, in step S3, the global attention compression excitation module uses two parallel branches to extract the attention weights of feature compression, wherein the 1×1 branch is used to capture the global information of the input features and perform cross-channel interactive learning in a global range; the 3×3 branch is used to capture the local feature detail information and perform cross-channel interactive learning in a local range. In the 1×1 branch, the feature values ​​of each channel are first averaged using global average pooling to capture the global information of each channel; then, 1×1 convolution and RELU activation function are used to replace the two fully connected layers in the original compression excitation module to reduce the complexity of the model and prevent information loss. In the 3×3 branch, 3×3 convolution is used to capture multi-scale local feature representations. In this way, not only can the channel features be encoded to adjust the importance of different channels, but also the accurate spatial structure information can be retained.

[0177] In this embodiment, in order to extract more comprehensive features, the present invention preferably adopts a cross-spatial information aggregation method:

[0178] In the first branch, group normalization is first used to normalize each group of features separately to reduce the impact of batch size on the normalization effect, thereby improving model stability; then the feature map of the 1×1 branch is encoded using global average pooling as shown in formula (1):

[0179]

[0180] in, Represents the spatial position in the cth channel of the input feature map X Feature map at ;

[0181] Secondly, softmax is introduced to improve computational efficiency; finally, the average pooling layer is used to fit the linear transformation, and the obtained feature map is multiplied element-by-element with the feature map of the 3×3 branch to obtain the first spatial attention.

[0182] In the second branch, the global information in the 3×3 branch is first encoded using the global average pooling shown in formula (1); then the obtained encoded features are multiplied element-wise with the feature map of the 1×1 branch to obtain the second spatial attention.

[0183] In the final stage of the enhanced compression excitation module, based on the fusion of the two spatial attention features, the RELU activation function is used to obtain nonlinear features, and the weights are reweighted so that the final feature map has the same dimension as the input feature map.

[0184] Preferably, this embodiment further includes a data enhancement step to increase the diversity of samples and fine-tune the pre-trained model using a small amount of sample data to improve the generalization and robustness of the model.

[0185] Data augmentation is the process of generating more diverse training samples by performing a series of transformations on the original training data, thereby increasing the model's adaptability to different inputs. Common data augmentation methods include image rotation, which randomly rotates the input image to enable the model to adapt to faces in different orientations; image flipping, which flips the image horizontally or vertically to increase sample diversity; color adjustment, which simulates images under different lighting conditions by adjusting parameters such as brightness, contrast, and saturation; cropping and scaling, which randomly crops or scales the image to enable the model to handle faces of different sizes; adding noise, which adds Gaussian noise or other types of noise to the image to enhance the model's robustness to noise; and affine transformation, which performs a combination of transformations such as translation, scaling, and rotation on the image to simulate the changes in faces in real scenes.

[0186] The pre-trained model is a model trained on a large-scale dataset and has strong feature extraction capabilities. By fine-tuning the pre-trained model on a small amount of sample data, it can be better adapted to specific facial expression recognition tasks. Specifically, it includes:

[0187] Choose a pre-trained model: Choose a convolutional neural network model that has been pre-trained on a large-scale image dataset (such as ImageNet), such as ResNet and VGG.

[0188] Freeze some layers: During fine-tuning, freeze some layers of the pre-trained model (usually the bottom convolutional layers) to retain its strong feature extraction capabilities.

[0189] Add a custom layer and add a custom fully connected layer on top of the pre-trained model to adapt to the output requirements of the facial expression recognition task.

[0190] Fine-tuning training: The model is trained using a small amount of facial expression sample data. In this invention, a small amount of facial expression sample data refers to a sample data size in the range of [800, 1200]. The model weights are adjusted to achieve better performance on a specific task. In this embodiment, the amount of facial expression sample data for training is 1000.

[0191] Learning rate adjustment: During the fine-tuning process, a lower learning rate is used. In the present invention, a lower learning rate refers to a learning rate in the range of [0.0001, 0.001]. In this embodiment, the learning rate is 0.001 to avoid excessive adjustments to the weights of the pre-trained model, thereby retaining the common features learned in its pre-training stage.

[0192] By combining data augmentation and pre-trained model fine-tuning, the model's generalization and robustness to different facial expressions can be significantly improved, enabling it to accurately identify expression categories even when faced with complex and diverse inputs.

[0193] Preferably, this embodiment also includes an image quality assessment step to reduce the impact of image quality on recognition performance, and combine the predictions of multiple models to reduce recognition bias and improve generalization ability. Specific methods for image quality assessment include image clarity assessment, lighting condition assessment, noise level assessment, face integrity assessment, and image resolution assessment.

[0194] Specifically, image quality assessment involves quantitatively analyzing the quality of input images to determine their suitability for facial expression recognition. The goal of image quality assessment is to select high-quality images for training and testing, thereby minimizing the negative impact of poor image quality on recognition performance. Specific methods for image quality assessment include image clarity assessment, which determines image clarity by calculating metrics such as edge strength and contrast. Low-definition images can blur facial features and affect recognition. Lighting condition assessment involves analyzing the intensity and uniformity of illumination to eliminate images with distorted facial features due to excessive or dim lighting. Noise level assessment involves detecting noise levels in images, such as Gaussian and salt-and-pepper noise. Highly noisy images can interfere with feature extraction and reduce recognition accuracy. Facial integrity assessment involves checking the integrity of the facial region in the image, including any occlusions or truncation. Incomplete facial images may lack key facial features, impacting recognition results. Finally, image resolution assessment ensures that the input image has sufficient resolution to preserve facial details. Low-resolution images can result in feature loss, impacting recognition performance. In addition, multi-model fusion technology refers to combining multiple models with different structures or training methods to improve the overall recognition performance by integrating their prediction results. Specific steps include model selection and training, which involves selecting multiple models with different structures or training strategies, such as convolutional neural networks of different depths, models with different pre-trained datasets, etc. Each model may focus on different features or data distributions during training; feature fusion, which involves fusing the features output by the feature extraction layers of multiple models to generate a more comprehensive feature representation through splicing, weighted summation, or learning-based fusion methods; prediction result fusion, which involves fusing the prediction results of multiple models. Common methods include average fusion, which involves taking the average of the prediction probabilities of multiple models as the final prediction result; weighted fusion, which involves assigning different weights to the prediction results of the model based on the performance or reliability of each model, and then performing a weighted summation; and a voting mechanism, which involves using majority voting or weighted voting to determine the final result based on the prediction categories of multiple models; model collaborative training, which involves using collaborative training to allow multiple models to learn from each other and improve overall performance during training; and model calibration, which involves calibrating the fused model to ensure that its output probability distribution conforms to the actual confidence level and improves the reliability of the model. By combining image quality assessment and multi-model fusion technology, the impact of image quality on recognition performance can be effectively reduced, and the recognition bias can be reduced through the complementarity of multiple models, thereby significantly improving the generalization ability and robustness of the model, enabling it to perform well in complex and changeable practical applications.

[0195] The method model of Example 1 was experimentally validated on multiple public datasets, including FER2013, FERPlus, AffectNet, CK+, and JAFFE, and compared with existing models. The relevant test results are shown in the table below. As can be seen, the facial expression recognition method of the present invention, based on spatial channel convolution and enhanced compressed attention, achieves significant improvements in facial expression recognition accuracy while significantly reducing computational costs and network size compared to existing technologies, resulting in higher accuracy and robustness.

[0196]

[0197]

[0198] Example 2

[0199] Combine Figure 9 , a facial expression recognition system based on spatial channel convolution and enhanced compression-stimulated attention in this embodiment includes:

[0200] The spatial channel convolution module includes a spatial reconstruction unit and a channel reconstruction unit. The spatial reconstruction unit is used to input features and separate features with larger information content from features with smaller information content, and reconstruct them to enhance representative features and suppress redundant features. The channel reconstruction unit adopts a segmentation-mapping-fusion strategy to extract rich representative features through lightweight convolution, and at the same time processes redundant features through a feature reuse scheme.

[0201] The feature fusion module performs splicing and dimensionality reduction in the channel dimension, and uses two sets of linear calculations and Sigmoid functions to obtain adaptive fusion weights, integrating feature information of different scales and levels, and enhancing the perception and representation capabilities of multi-scale, cross-level, and complex environments.

[0202] The compression excitation module, based on global attention, adaptively adjusts the feature mapping weights through a dual-branch structure, enhancing the model's attention to important features and improving recognition accuracy.

[0203] This embodiment preferably also includes a data enhancement and model fine-tuning module, which generates more diverse training samples by performing a series of transformation operations on the original training data, thereby increasing the model's adaptability to different inputs, and fine-tuning the pre-trained model on a small amount of sample data to make it better suited to specific facial expression recognition tasks;

[0204] Preferably, this embodiment further includes an image quality assessment module, which is used to combine the image quality assessment steps to reduce the impact of image quality on recognition performance and improve recognition accuracy and robustness through multi-model fusion technology.

[0205] Example 3

[0206] The electronic device of this embodiment includes:

[0207] at least one processor; and

[0208] at least one memory in communication with the processor;

[0209] The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to enable the electronic device to perform the facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention in Example 1.

[0210] In the above description, many specific details are set forth in order to fully understand the present invention. However, the above description is only a preferred embodiment of the present invention. The present invention can be implemented in many other ways different from those described herein, so the present invention is not limited to the specific implementation disclosed above. At the same time, any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A facial expression recognition method based on spatial channel convolution and enhanced compression-stimulated attention, characterized by: The steps include: Step S1: Use spatial channel convolution to expand the receptive field and capture the global information of the image; Step S2: feature fusion of feature information of different scales and levels; Step S3: Adaptively adjust the feature map weights using a global attention compression excitation module to enhance the model's focus on important features. The global attention compression excitation module includes a dual-branch structure, where one branch is used to capture global information and the other branch is used to capture local feature detail information. In step S1, for the input features of the residual network , first obtain the spatial refinement features through spatial feature reconstruction , and then obtain channel refinement features through channel feature reconstruction ; Spatial feature reconstruction includes: Feature map segmentation: Segment feature maps with more information from those with less information, and use the scale factor of group normalization to evaluate the different information contents of feature maps; Feature map reconstruction: information-rich features Features with less information Sum, by adopting the cross reconstruction strategy to fully combine different information features, the cross reconstruction features and Splicing to obtain optimized spatial feature map ; In step S2, feature fusion specifically includes the following steps: Step S21: Global enhancement features and local attention features Perform concatenation and dimensionality reduction on the channel dimension and output features ; Step S22: Using two sets of linear calculations And Sigmoid function to obtain adaptive fusion weights: ； in, is the adaptive weight for learning, It includes batch normalization, ReLU activation function and convolution operation; Step S23: According to the adaptive weight Realize weighted fusion of global enhancement features and local attention features: ； is the adaptive fusion feature, represents element-wise multiplication; Adaptively fusion features Input to the global average pooling layer, fully connected layer and Softmax classifier to obtain its category vector and the final expression category : ； ； in, is the number of expression categories that the model can recognize, is the eigenvector Judged as the The probability of expression-like Indicates the final predicted expression category.

2. The facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention according to claim 1, characterized in that: The specific method of feature map segmentation is: given feature ,in Indicates batch processing, Indicates the number of channels, and Represents height and width respectively, specifically including the following steps: Step 1: Input features Standardized processing to obtain , ,in, is the mean, is the standard deviation, is a trainable affine transformation, is a constant; Step 2: Obtain the importance of different feature maps : ； Step 3: Use the Sigmod function to map the weighted feature map to the (0, 1) range and gate it by the threshold: set the weight of the feature map greater than the threshold to 1, and construct the important content weight ; At the same time, the weight of the feature map smaller than the threshold is set to 0, and the weight of non-important content is constructed ; Step 4: Input features Multiply by and Get two weighted features: informative and less informative features .

3. The facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention according to claim 1, characterized in that: Channel feature reconstruction includes: segmentation: For spatial optimization features , first divide its channels into Channel and The channel has two parts, , then use 1×1 convolution to compress the feature map, and control the feature channel by introducing the compression ratio r. At this time, the spatial optimization feature Divided into important features and unimportant features ; Mapping: Important features Perform feature mapping, including: Use group convolution and point convolution instead of k×k convolution to extract high-level features; Add the two parts of output features to construct a representative feature map : ； in, and Group convolution and point convolution replace k×k convolution to extract high-level features; and are input features and output feature maps respectively; For non-important features Perform feature mapping, including: Use 1×1 point convolution to generate feature maps with shallow hidden details as a supplement to the important information feature maps; The generated feature map is concatenated with the initial non-important information features to form the output of non-important information features. : ； in, is the weight matrix of point convolution; For splicing; and are input features and output features respectively; Fusion: After the mapping is completed, the two types of features are adaptively merged using a simplified residual structure, including: Use global average pooling to collect global spatial information and perform channel statistics : ； in, Indicates the spatial position in the cth channel Feature map at ; The channel features of the upper and lower branches are stacked as and , and use channel attention to generate the corresponding feature importance and : ； In feature importance and Under the guidance of and Merge in a channel manner to obtain refined channel features : .

4. The facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention according to claim 1, characterized in that: Step S3 specifically includes: Step S31: Input features Divided into Sub-features , , and and They are respectively used as input for proposing two branches; Step S32: using the proposed dual-branch feature capture to reduce the complexity of the model and prevent information loss; Step S33: Capture different semantic information through cross-channel learning without losing generality , and the learned attention weights are used to strengthen the feature representation of the region of interest in each sub-feature.

5. The facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention according to claim 1, characterized in that: In step S3, the global attention compression excitation module uses two parallel branches to extract the attention weights of feature compression, where the 1×1 branch is used to capture the global information of the input features and perform cross-channel interactive learning in a global range; the 3×3 branch is used to capture local feature detail information and perform cross-channel interactive learning in a local range.

6. The facial expression recognition method based on spatial channel convolution and enhanced compression to stimulate attention according to claim 5, characterized in that: In the first branch, each group of features is first normalized separately using group normalization; The feature map of the 1×1 branch is then encoded using the global average pooling shown in formula (1): ； in, Represents the spatial position in the cth channel of the input feature map X Feature map at ; Secondly, softmax is introduced to improve computational efficiency; finally, the average pooling layer is used to fit the linear transformation, and the obtained feature map is multiplied element-wise with the feature map of the 3×3 branch to obtain the first spatial attention; In the second branch, the global information in the 3×3 branch is first encoded using the global average pooling shown in formula (1); then the obtained encoded features are element-wise multiplied with the feature map of the 1×1 branch to obtain the second spatial attention; In the final stage of the enhanced compression excitation module, based on the fusion of the two spatial attention features, the RELU activation function is used to obtain nonlinear features, and the weights are reweighted so that the final feature map has the same dimension as the input feature map.

7. The facial expression recognition method based on spatial channel convolution and enhanced compression-stimulated attention according to any one of claims 1 to 6, characterized in that: It also includes data augmentation steps and fine-tuning the pre-trained model using few-shot data; Data augmentation is to generate more diverse training samples by performing a series of transformation operations on the original training data, thereby increasing the model's adaptability to different inputs; The pre-trained model is a model trained on a large-scale dataset, including: Select a pre-trained model: Choose a convolutional neural network model pre-trained on a large-scale image dataset; Freeze some layers: During fine-tuning, freeze some layers of the pre-trained model to retain its strong feature extraction capabilities; Add a custom layer: Add a custom fully connected layer on top of the pre-trained model to adapt to the output requirements of the facial expression recognition task; Fine-tuning training: Use a small amount of facial expression sample data to train the model and adjust the model weights to achieve better performance on specific tasks; Learning rate adjustment: During fine-tuning, a lower learning rate is used to avoid overly adjusting the weights of the pre-trained model, thereby preserving the common features learned during its pre-training phase.

8. The facial expression recognition method based on spatial channel convolution and enhanced compression-stimulated attention according to any one of claims 1 to 6, characterized in that: It also includes an image quality assessment step, and combines the predictions of multiple models to reduce recognition bias and improve generalization ability. The specific methods of image quality assessment include image clarity assessment, lighting condition assessment, noise level assessment, face integrity assessment, and image resolution assessment.

9. A facial expression recognition system based on spatial channel convolution and enhanced compression-stimulated attention, characterized by: The method for facial expression recognition based on spatial channel convolution and enhanced compression-stimulated attention according to any one of claims 1 to 6 comprises: The spatial channel convolution module includes a spatial reconstruction unit and a channel reconstruction unit. The spatial reconstruction unit is used to input features and separate features with large information content from features with small information content, and reconstruct them to enhance representative features and suppress redundant features. The channel reconstruction unit adopts a split-map-fusion strategy to extract rich representative features through lightweight convolution, while processing redundant features through a feature reuse scheme. The feature fusion module performs splicing and dimensionality reduction in the channel dimension, and uses two sets of linear calculations and Sigmoid functions to obtain adaptive fusion weights. It integrates feature information at different scales and levels, enhancing the perception and representation capabilities of multi-scale, cross-level, and complex environments. The compression excitation module, based on global attention, adaptively adjusts the feature mapping weights through a dual-branch structure, enhancing the model's attention to important features and improving recognition accuracy.

10. An electronic device, characterized in that: include: at least one processor; as well as at least one memory in communication with the processor; The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to enable the electronic device to perform the facial expression recognition method based on spatial channel convolution and enhanced compression-stimulated attention described in any one of claims 1-6.

Images