Eca lightweight facial expression recognition method based on edge-cloud cooperation
By adopting an edge-cloud collaborative architecture and lightweight design, combined with ECA attention mechanism and depthwise separable convolution, the real-time and accuracy issues of facial expression recognition on edge devices are solved, achieving efficient and low-latency expression recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING INFORMATION HIGH-SPEED RAILWAY RES INST OF SCI AND TECH
- Filing Date
- 2025-11-24
- Publication Date
- 2026-07-24
Smart Images
Figure CN121459408B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a lightweight facial expression recognition method based on edge-cloud collaboration (ECA). Background Technology
[0002] Facial expressions are not only important external manifestations of emotions and intentions, but also key factors reflecting an individual's inner state, emotional experience, and communication intentions. Approximately 55% of human emotional information is conveyed through facial expressions, while only 38% and 7% are conveyed through sound and language, respectively. Through experiments and research with students from different countries and cultural backgrounds, scholars have proposed the Facial Action Coding System (FACS). FACS categorizes basic human expressions into six types: Sad, Fear, Disgust, Happy, Angry, and Surprise. Subsequent research has added a neutral state, further enriching the coding system for facial emotion expression.
[0003] These basic facial expressions are considered core elements of human emotional expression, providing a standardized and quantifiable analytical basis for facial expression recognition research, and laying the theoretical foundation for the design and optimization of expression recognition modules in intelligent systems. With the rapid development of artificial intelligence and computer vision technologies, facial expression recognition technology has also entered a stage of rapid development. Traditional facial expression recognition algorithms generally extract facial features manually, such as optical flow, principal component analysis, Gabor wavelet transform, and local binary methods. However, manual methods have significant drawbacks such as low efficiency and low accuracy.
[0004] In recent years, with the emergence of Convolutional Neural Networks (CNNs), facial expression recognition technology has made significant progress. CNNs can automatically learn and extract complex facial features, overcoming the shortcomings of traditional manual feature extraction methods. The emergence of the classic deep learning network model LeNet-5 marked the practical application of CNNs in image recognition. Subsequently, mainstream neural network models such as GoogLeNet, ResNet, VGGNet, MobileNet, and ShuffleNet were also developed. While CNNs have made great strides in image recognition, they have also driven the rapid development of facial expression recognition.
[0005] In existing research, improved methods for facial expression recognition are constantly emerging. One type of method directly implements expression recognition based on convolutional neural networks, achieving good accuracy on multiple public datasets. Another type of method introduces category attention mechanisms and context-aware modules into classic convolutional network structures (such as VGG16), effectively improving the ability to extract expression features by expanding the network's receptive field. Still other methods combine transfer learning and optimization algorithms to achieve efficient feature extraction in specific scenarios such as driver monitoring, significantly improving recognition accuracy. To further reduce model complexity, some methods combine transfer learning with network pruning techniques to propose lightweight expression recognition models, effectively reducing model size and improving deployment efficiency while maintaining recognition accuracy. Some studies improve residual networks and integrate efficient channel attention mechanisms, using image preprocessing to enhance the expression of facial feature channels, improving recognition accuracy while reducing parameter count and increasing inference speed. Furthermore, some methods propose multi-scale fusion attention mechanism networks, combining Inception structures to achieve multi-scale feature extraction and introducing channel attention mechanisms to focus on key facial regions, while utilizing depthwise separable convolutions to significantly reduce model parameter count.
[0006] While the aforementioned methods have made some progress in improving facial expression recognition performance, they still have shortcomings in practical applications. First, the complex network models have a large parameter scale, making real-time inference difficult on resource-constrained edge devices. They are also susceptible to interference from lighting conditions, cluttered backgrounds, and subtle differences in facial expressions, resulting in insufficient robustness. Second, methods relying entirely on cloud-based inference, while leveraging powerful computing capabilities, introduce communication latency and pose risks of user privacy breaches. Third, while existing lightweight models reduce computational overhead to some extent, they often come at the cost of decreased recognition accuracy, making it difficult to balance efficiency and accuracy.
[0007] Edge computing, due to its proximity to the data source, can reduce latency and protect user privacy to some extent, but its computing power is limited, making it difficult to independently support the training and inference of complex models. Edge-cloud collaboration combines the advantages of the powerful computing power of the cloud and the low latency of the edge, providing an effective solution to these problems. However, existing edge-cloud collaboration methods often fail to fully utilize the computing power of the edge, or still present a difficult trade-off between lightweight design and recognition accuracy. Therefore, there is an urgent need for a lightweight edge-cloud collaborative facial expression recognition method that can guarantee recognition accuracy while effectively reducing computational overhead and latency to meet the needs of practical applications. Summary of the Invention
[0008] To address the problems of complex model parameters, high computational load, high recognition latency, and limited computing resources on edge devices in existing facial expression recognition technologies, this invention aims to propose a lightweight facial expression recognition method based on an edge-cloud collaborative architecture using ECA (Electronic Channel Attention). This method deploys a general-purpose model in the cloud using ample storage and computing resources and trains it on a large-scale general dataset. Subsequently, through transfer learning, the shallow feature extraction portion of the cloud-pre-trained model is transferred to the edge for sharing. The edge model then fine-tunes only subsequent layers to adapt to the distribution differences of local data. Simultaneously, an efficient channel attention (ECA) mechanism is introduced into the edge model to enhance the ability to focus on key region features, and a depthwise separable convolutional structure is combined to reduce the number of model parameters, thereby achieving low-latency, high-precision facial expression recognition.
[0009] To achieve the above objectives, the technical solution adopted in this invention is: a lightweight facial expression recognition method based on edge-cloud collaboration (ECA), comprising the following steps:
[0010] S1. Deploy the VGG16 model in the cloud and train a general model using the ImageNet dataset;
[0011] S2. Train a lightweight ECA facial expression recognition model based on edge-cloud collaboration at the edge.
[0012] S21. Divide the facial expression dataset into training and testing sets, and preprocess it using an enhancement strategy combining adaptive histogram equalization (CLAHE) and gamma correction; divide the facial expression dataset into training and testing sets, and preprocess the datasets.
[0013] S22. Load and freeze the first four convolutional layers of the cloud-pretrained model, and fine-tune the mid-to-high-level convolutional layers and classification layers.
[0014] S23. Based on transfer learning, introduce the ECA attention mechanism to highlight facial expression regions and suppress redundancy;
[0015] S24. In subsequent layers, depthwise separable convolution and H-swish activation function are used to reduce the number of parameters and computational cost, and speed up inference.
[0016] S25. The FocalLoss loss function is used instead of the traditional cross-entropy loss function to solve the problem of imbalance in the number of expression categories in the dataset. Finally, expression feature extraction and classification are completed independently at the edge, realizing lightweight and efficient expression recognition.
[0017] In a preferred embodiment of the present invention, S1 specifically comprises:
[0018] A classic VGG16 model is built and deployed in the cloud, and trained using the ImageNet dataset. Let the training dataset be... ,in Indicates the input sample. For the corresponding category labels, D is the input sample dimension, K is the number of categories, and U is the size of the dataset; a convolutional neural network model is deployed on a cloud server to learn the parameters. The optimization objective is:
[0019]
[0020] Indicates cloud model to The output, This indicates the softmax operation. ; Focalloss loss function: ,in, for The true label, For predicting labels, , Indicates the prediction is the first The probability of a class As a category balance factor, is the modulation factor.
[0021] In a preferred embodiment of the present invention, S2 specifically comprises:
[0022] The edge model performs transfer learning and freezes the shallow convolutions of the cloud model, only fine-tuning the remaining parameters. To adapt to the specific data distribution on edge devices. The edge devices are also trained using the cross-entropy loss function, with the optimization objective being:
[0023]
[0024] in For edge datasets, Indicates the input sample. For the corresponding category label, It is the dimension of the input sample. It is the number of categories. It represents the number of samples at the edge. Indicates the edge model pair The output, . , yes The true label, yes Predicted labels.
[0025] In a preferred embodiment of the present invention, S21 specifically comprises:
[0026] An enhancement strategy combining adaptive histogram equalization (CLAHE) and gamma correction is employed for preprocessing. CLAHE enhances local contrast and highlights facial details, but it can easily cause overall brightness shifts. Gamma correction balances image brightness distribution but struggles to enhance local details. The combination of both approaches balances local detail enhancement with global brightness equalization, effectively improving the recognizability of facial expressions in complex environments.
[0027] In a preferred embodiment of the present invention, S22 specifically comprises:
[0028] In building a lightweight facial expression recognition model at the edge, the first four convolutional parameters of the pre-trained model in the cloud are transferred to the edge and frozen. The main function of the shallow convolutional layers is to extract low-level features from facial images. These features have strong universality and stability, maintaining good expressive performance in different datasets and application environments. Based on this, the model is fine-tuned on a local facial expression dataset at the edge, allowing it to adapt to the distribution characteristics of the edge dataset, thereby improving the recognition accuracy and robustness in edge expression recognition scenarios. This transfer learning method fully utilizes the general knowledge gained from training on large-scale data in the cloud while combining it with personalized data from the edge for targeted optimization. Ultimately, this results in a lightweight facial expression recognition model that maintains high efficiency while possessing good generalization ability and practical value.
[0029] In a preferred embodiment of the present invention, S23 specifically comprises:
[0030] In facial expression recognition, background redundancy can interfere with the model's extraction of key facial expression features. To enhance the model's focus on facial expression regions, this paper introduces an ECA attention mechanism based on transfer learning. This mechanism generates channel weights through global average pooling and adaptive one-dimensional convolution, thereby highlighting key regions and suppressing redundancy. Considering factors such as feature map spatial resolution and task difficulty, this invention optimizes the original ECA attention mechanism by using an adaptive kernel. Size calculation method:
[0031]
[0032] in, Input the number of channels. , Here, represents the height and width of the input feature map, τ is the task complexity parameter, and odd(·) ensures that the convolution kernel has an odd number to maintain the symmetry of the convolution. This relates to the effect of spatial resolution on the receptive field range. The convolution kernel size is dynamically adjusted based on task complexity. max(3, This avoids the convolution kernel from becoming too small and failing. By introducing spatial and task factors, the improved... It is more adaptable and can extract key features more effectively in different image sizes and expression recognition scenarios, thereby improving model performance and robustness.
[0033] In a preferred embodiment of the present invention, S24 specifically comprises:
[0034] The depthwise separable convolution includes channel-wise convolution (DW) and pointwise convolution (PW). First, channel-wise convolution (DW) uses a single convolution kernel to act on only a single channel of the input feature map to independently extract spatial features. Then, pointwise convolution (PW) uses a 1×1 convolution kernel to linearly combine each channel and fuse the channel information to generate the output feature map.
[0035] As a preferred embodiment of the present invention, assuming the number of input channels is... The output pass number is The kernel size is k k, the output feature space size is The number of parameters and computational cost of standard convolution are: and The number of parameters and computational cost of depthwise convolution are and The number of parameters and computational cost of pointwise convolution PW are: and The ratio of parameters and computational cost between depthwise separable convolution and standard convolution is: .
[0036] Traditional activation functions for depthwise separable convolutions can lead to the vanishing gradient problem, affecting model training performance. Therefore, this study employs the H-swish function:
[0037]
[0038] H-swish retains some gradient information in the negative range, alleviating the gradient vanishing problem, and possesses strong nonlinear expressive power. Furthermore, its computational form is simple and efficient, making it well-suited for model deployment and acceleration in edge computing scenarios.
[0039] In a preferred embodiment of the present invention, S25 specifically comprises:
[0040] Focal Loss is used: This method replaces the traditional cross-entropy loss function to address the imbalance in the number of facial expression categories in a dataset. Used to adjust the weights of different classes in the overall loss, preventing majority class samples from dominating the training process; Then, the sample loss is dynamically scaled based on the predicted probability: when the sample is easy to classify ( When the sample is relatively large, this factor rapidly reduces its loss; however, when the sample is difficult to classify ( When the class distribution is imbalanced, the loss remains relatively large, thus strengthening the model's learning of minority or difficult samples. Through these two mechanisms, FocalLoss can effectively suppress the dominant role of easily classified samples and highlight the contribution of difficult samples in cases of class imbalance, significantly improving the overall classification balance.
[0041] As a preferred embodiment of the present invention, the lightweight edge-based facial expression recognition model comprises seven convolutional layers. The first four layers are convolutional layers migrated from the cloud, with a kernel size of 3×3 and a stride of 1. Each layer is connected to batch normalization and the ReLU activation function. Subsequently, an ECA-Net channel attention module is introduced after the fourth convolutional layer to enhance the channel feature representation capability. The fifth to seventh layers employ depthwise separable convolutions, with a depthwise convolutional kernel size of 3×3 and a pointwise convolutional kernel size of 1×1, both with a stride of 1. Each layer is connected to batch normalization and the H-swish activation function. For pooling layers, a max pooling layer with a window size of 2 and a stride of 2 is added after the second, fourth, fifth, sixth, and seventh convolutional layers. Finally, an average pooling layer is used to obtain the final feature representation.
[0042] Compared with the prior art, the technical solution adopted in this invention has the following beneficial effects:
[0043] 1. This invention combines cloud-based transfer learning with lightweight edge design to achieve efficient facial expression recognition on edge devices, significantly reducing the number of model parameters and computational cost (only 0.703M and 192.043M respectively), and reducing recognition latency.
[0044] 2. The recognition accuracy of this invention on the FER2013 and CK+ datasets reached 71.93% and 98.76% respectively. While ensuring high accuracy, it also takes into account the lightweight characteristics, which verifies the practicality and superiority of the method.
[0045] 3. This invention leverages the collaborative advantages of cloud and edge computing, combining transfer learning, efficient channel attention (ECA) mechanism, and deep separable convolutional structure to effectively reduce the number of model parameters and computational complexity, thereby achieving lightweight, low-latency, and high-precision facial expression recognition. Attached Figure Description
[0046] Figure 1 This is a schematic diagram of the edge-cloud collaboration framework in this embodiment.
[0047] Figure 2 This is a schematic diagram of the edge-cloud collaborative facial expression recognition process in this embodiment.
[0048] Figure 3 The image in this embodiment is enhanced using the CLAHE+Gamma combination during preprocessing.
[0049] Figure 4 This is the preprocessed grayscale histogram of this embodiment.
[0050] Figure 5 This is a schematic diagram of the VGG16 network structure in the cloud model of this embodiment.
[0051] Figure 6 This is a schematic diagram of the ECA-Net structure in this embodiment.
[0052] Figure 7 This is a schematic diagram of the depth-separable convolution structure in this embodiment.
[0053] Figure 8 This is a comparison chart of the accuracy of different algorithms in this embodiment on the CK+ dataset.
[0054] Figure 9 This is a comparison chart of the accuracy of different algorithms in this embodiment on the FER2013 dataset.
[0055] Figure 10 This is a comparison chart of the model parameters in this embodiment.
[0056] Figure 11 This is a comparison chart of latency at different deployment locations in this embodiment. Detailed Implementation
[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.
[0058] The system model of this invention consists of cloud and edge components, and the edge-cloud collaborative architecture is as follows: Figure 1 As shown, this framework fully leverages the powerful computing and storage capabilities of the cloud and the low latency advantages of the edge to meet the real-time and accuracy requirements in recognition scenarios. The invention is described below in two steps: training the cloud model and the edge model. The specific implementation process is as follows: Figure 2 As shown.
[0059] Step 1: Train a general model in the cloud
[0060] A classic VGG16 model is built and deployed in the cloud, and trained using the ImageNet dataset. Let the training dataset be... ,in Indicates the input sample. Here, D represents the input sample dimension, K is the number of categories, and U is the size of the dataset. A convolutional neural network model is deployed on a cloud server to learn the parameters. The optimization objective is:
[0061]
[0062] Indicates cloud model to The output, This indicates the softmax operation. . Focalloss loss function: ,in, for The true label, For predicting labels, , Indicates the prediction is the first The probability of a class As a category balance factor, is the modulation factor.
[0063] Step 2: Build and train the edge model, such as Figures 3-6 As shown:
[0064] The edge model performs transfer learning and freezes the shallow convolutions of the cloud model, only fine-tuning the remaining parameters. To adapt to the specific data distribution on edge devices. The edge devices are also trained using the cross-entropy loss function, with the optimization objective being:
[0065]
[0066] in For edge datasets, Indicates the input sample. For the corresponding category label, It is the dimension of the input sample. It is the number of categories. It represents the number of samples at the edge. Indicates the edge model pair The output, . , yes The true label, yes Predicted labels.
[0067] Step 2.1: Edge-side transfer learning of cloud model convolutional layers
[0068] In building a lightweight facial expression recognition model at the edge, the shallow convolutional parameters from the cloud-pre-trained model are first transferred to the edge and frozen. The main function of the shallow convolutional layers is to extract low-level features from facial images. These features have strong universality and stability, maintaining good expressive performance across different datasets and application environments. Based on this, the model is fine-tuned on a local facial expression dataset at the edge, allowing it to adapt to the distribution characteristics of the edge dataset, thereby improving the recognition accuracy and robustness in edge-based facial expression recognition scenarios. This transfer learning method fully utilizes the general knowledge gained from training on large-scale data in the cloud while combining it with personalized data from the edge for targeted optimization. Ultimately, this results in a lightweight facial expression recognition model that maintains high efficiency while possessing good generalization ability and practical value.
[0069] Step 2.2: Incorporate ECA attention mechanism
[0070] In facial expression recognition, background redundancy can interfere with the model's extraction of key facial expression features. To enhance the model's focus on facial expression regions, this paper introduces an ECA attention mechanism based on transfer learning. This mechanism generates channel weights through global average pooling and adaptive one-dimensional convolution, thereby highlighting key regions and suppressing redundancy. Considering factors such as feature map spatial resolution and task difficulty, this invention optimizes the original ECA attention mechanism by using an adaptive kernel. Size calculation method:
[0071]
[0072] in, This relates to the effect of spatial resolution on the receptive field range. The convolution kernel size is dynamically adjusted based on task complexity. Meanwhile, odd( To ensure the convolution kernel has an odd number of elements and maintain the symmetry of the convolution, max(3, This avoids the convolution kernel from becoming too small and failing. By introducing spatial and task factors, the improved... It is more adaptable and can extract key features more effectively in different image sizes and expression recognition scenarios, thereby improving model performance and robustness.
[0073] Step 2.3: Depthwise Separable Convolution
[0074] This study uses a Depthwise Separable Convolution (DSC) network to replace standard convolutions to reduce the computational cost and parameter count of the edge model, thereby achieving model lightweighting. The structure of the Depthwise Separable Convolution is as follows: Figure 7 As shown.
[0075] First, depthwise convolution (DW) uses a single kernel to operate on only a single channel of the input feature map, independently extracting spatial features. Then, pointwise convolution (PW) uses a 1×1 kernel to linearly combine the channels, fusing channel information to generate the output feature map. Compared to standard convolution, DSC significantly reduces the number of parameters and computational overhead. Assuming the number of input channels is... The output pass number is The kernel size is k k, the output feature space size is The standard convolution has the following parameter count and computational cost: and The number of parameters and computational cost of depthwise convolution are and The number of parameters and computational cost of pointwise convolution are: and The ratio of parameters and computational cost between depthwise separable convolution and standard convolution is: Traditional activation functions for depthwise separable convolutions can lead to the vanishing gradient problem, affecting the model's training performance. Therefore, this embodiment uses the H-swish activation function, expressed as:
[0076]
[0077] H-swish retains some gradient information in the negative range, alleviating the gradient vanishing problem, and possesses strong nonlinear expressive power. Furthermore, its computational form is simple and efficient, making it well-suited for model deployment and acceleration in edge computing scenarios.
[0078] Step 2.4: Edge Model Structure
[0079] The edge-side model comprises seven convolutional layers. The first four layers are derived from cloud-based convolutional layers, each with a 3×3 kernel and a stride of 1. Each layer is connected to batch normalization and the ReLU activation function. An ECA attention module is then introduced after the fourth convolutional layer to enhance the representation of channel features. Layers five through seven employ depthwise separable convolutions, with 3×3 kernels for depthwise convolutions and 1×1 kernels for pointwise convolutions, all with a stride of 1. Each layer is connected to batch normalization and the H-swish activation function. For pooling, max pooling with a window size of 2 and a stride of 2 is added after the second, fourth, fifth, sixth, and seventh convolutional layers. Finally, an average pooling layer is used to obtain the final feature representation.
[0080] To verify the effectiveness of the method of this invention, it was compared with existing typical lightweight models on two public datasets, CK+ and FER2013. Figure 8 and Figure 9 As shown, on the CK+ dataset, the recognition accuracy of the method of this invention reaches 98.76%, which is 3.36 percentage points higher than the recently proposed SIFT+CNN method (95.40%); on the FER2013 dataset, the recognition accuracy of the method of this invention is 71.93%, which is 1.91 percentage points higher than the Deep-emotion method (70.02%). Furthermore, as... Figure 10 As shown, the method of this invention outperforms the EfficientNet-B0 and MobileNet series models in key indicators such as computational cost (FLOPs), model parameter size, and recognition accuracy, fully demonstrating the advantages of this invention in balancing lightweight design and performance. Figure 11 To verify the effect of latency comparison, the results show that by deploying the facial expression recognition model trained by edge-cloud collaboration to the cloud and edge respectively, and uploading facial expression images to the cloud and edge via 5G and WiFi networks and receiving the results, the latency at the edge was reduced by about 38%.
[0081] Although the present invention has been disclosed above with reference to preferred embodiments, the embodiments and accompanying drawings are not intended to limit the invention. Any person skilled in the art can make various changes or modifications without departing from the spirit and scope of the invention, and these changes will also be within the protection scope of the invention. Therefore, the protection scope of the present invention should be defined by the scope of the claims of this application.
Claims
1. A lightweight facial expression recognition method based on edge-cloud collaboration (ECA), characterized in that, Includes the following steps: S1. Deploy the VGG16 model in the cloud and train a general model using the ImageNet dataset; S2. Train a lightweight ECA facial expression recognition model based on edge-cloud collaboration at the edge. S21. Divide the facial expression dataset into training and testing sets, and preprocess it using an enhancement strategy combining adaptive histogram equalization (CLAHE) and gamma correction; divide the facial expression dataset into training and testing sets, and preprocess the datasets. S22. Load and freeze the first four convolutional layers of the cloud-pretrained model, and fine-tune the mid-to-high-level convolutional layers and classification layers. S23. Based on transfer learning, introduce the ECA attention mechanism to highlight facial expression regions and suppress redundancy; S24. In subsequent layers, depthwise separable convolution and H-swish activation function are used to reduce the number of parameters and computational cost, and speed up inference. S25. Use the FocalLoss loss function instead of the traditional cross-entropy loss function to solve the problem of imbalance between the number of facial expression categories in the dataset; Specifically, S1 is: A classic VGG16 model is built and deployed in the cloud, and trained using the ImageNet dataset; let the training dataset be... ,in Indicates the input sample. For the corresponding category labels, D is the input sample dimension, K is the number of categories, and U is the size of the dataset; a convolutional neural network model is deployed on a cloud server to learn the parameters. The optimization objective is: Indicates cloud model to The output, This indicates the softmax operation. ; Focalloss loss function: ,in, for The true label, For predicting labels, , Indicates the prediction is the first The probability of a class As a class balance factor, Modulation factor; Specifically, S2 is: The edge model performs transfer learning and freezes the shallow convolutions of the cloud model, only fine-tuning the remaining parameters. To adapt to the specific data distribution on edge devices; the edge devices are also trained using the cross-entropy loss function, with the optimization objective being: in For edge datasets, Indicates the input sample. For the corresponding category label, It is the dimension of the input sample. It is the number of categories. It represents the number of samples at the edge. Indicates the edge model pair The output, ; , yes The true label, yes Predicted labels; Specifically, S23 is: The ECA attention mechanism adaptively generates channel weights through global average pooling and one-dimensional convolution, and uses an adaptive convolution kernel size calculation method: in, Input the number of channels. , represents the height and width of the input feature map, respectively; τ is the task complexity parameter; odd(·) ensures that the number of convolution kernels is odd. This relates to the effect of spatial resolution on the receptive field range. The kernel size is dynamically adjusted based on the task complexity.
2. The lightweight facial expression recognition method based on edge-cloud collaboration (ECA) according to claim 1, characterized in that, Specifically, S21 is: An enhancement strategy combining adaptive histogram equalization (CLAHE) and gamma correction is used for preprocessing. CLAHE is used to improve local contrast, while gamma correction is used to equalize the overall brightness of the image.
3. The lightweight facial expression recognition method based on edge-cloud collaboration (ECA) according to claim 1, characterized in that, Specifically, S22 is: In the process of building a lightweight facial expression recognition model at the edge, the first four convolutional parameters of the pre-trained model in the cloud are first transferred to the edge and set to a frozen state. On this basis, the edge fine-tunes the mid-to-high-level convolutional layers and classification layers on the local expression dataset so that the model can adapt to the distribution characteristics of the edge dataset.
4. The lightweight facial expression recognition method based on edge-cloud collaboration (ECA) according to claim 1, characterized in that, Specifically, S24 is: The depthwise separable convolution includes channel-wise convolution (DW) and pointwise convolution (PW). First, channel-wise convolution (DW) uses a single convolution kernel to act on only a single channel of the input feature map to independently extract spatial features. Then, pointwise convolution (PW) uses a 1×1 convolution kernel to linearly combine each channel and fuse the channel information to generate the output feature map.
5. The lightweight facial expression recognition method based on edge-cloud collaboration (ECA) according to claim 1, characterized in that: Assume the number of input channels is The output pass number is The kernel size is k k, the output feature space size is The number of parameters and computational cost of standard convolution are: and The number of parameters and computational cost of depthwise convolution are and The number of parameters and computational cost of pointwise convolution PW are: and The ratio of parameters and computational cost between depthwise separable convolution and standard convolution is: ; The activation function is the H-swish function: H-swish retains some gradient information in the negative range, alleviating the gradient vanishing problem, and has strong nonlinear expression capabilities.
6. The lightweight facial expression recognition method based on edge-cloud collaboration (ECA) according to claim 1, characterized in that, Specifically, S25 is: Focal Loss is used: This replaces the traditional cross-entropy loss function to address the problem of imbalanced expression categories in a dataset; among which... Used to adjust the weights of different classes in the overall loss, preventing majority class samples from dominating the training process; The sample loss is dynamically scaled according to the predicted probability: when the sample is easy to classify, the modulation factor reduces the loss rapidly; while when the sample is difficult to classify, the loss remains large, thereby strengthening the model's learning of minority classes or difficult samples.
7. The lightweight facial expression recognition method based on edge-cloud collaboration according to any one of claims 1 to 6, characterized in that: The lightweight facial expression recognition model at the edge consists of 7 convolutional layers. The first four layers are convolutional layers migrated from the cloud. The kernel size is 3×3 and the stride is 1. Each layer is connected to batch normalization and ReLU activation function. Then, after the fourth convolutional layer, an ECA-Net channel attention module is introduced to improve the channel feature expression capability. Depthwise separable convolutions are used in layers 5 to 7, with a depthwise convolution kernel size of 3×3 and a pointwise convolution kernel size of 1×1, and a stride of 1 for all layers. Each layer is connected to batch normalization and H-swish activation function. For pooling layers, max pooling layers with a window size of 2 and a stride of 2 are added after the convolutions in layers 2, 4, 5, 6, and 7. Finally, an average pooling layer is used to obtain the final feature representation.