A deep learning-based facial expression recognition method for autistic children

Abstract

The present application relates to the cross field of target detection and emotion recognition, and particularly relates to a facial expression recognition method for autistic children based on deep learning. The method comprises the following steps: embedding a three-dimensional attention (Tri-Attention) after a C3k2 module of a YOLOv11 backbone network to model the interaction relationship among height, width and channel dimensions in parallel, and enhancing the response to key expression regions such as eyes and corners of the mouth; introducing the three-dimensional attention (Tri-Attention) again at the multi-scale feature output end of the neck network to realize effective positioning of small-scale faces in a complex background and suppression of background noise; training and verifying the YOLOv11-TA model by using a public data set and a self-built autistic children data set respectively, and outputting the detection frame and expression category of each face; and comparing various indexes of the original YOLOv11 and the YOLOv11-TA on multiple data sets, and actually verifying that the method significantly improves the accuracy and robustness of the expression recognition of autistic children.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a deep learning-based method for facial expression recognition in children with autism. Background Technology

[0002] Children with autism spectrum disorder (ASD) often experience difficulties in both verbal and nonverbal communication, making it challenging for them to express emotions in conventional ways. Accurate facial expression recognition is crucial for providing timely emotional support and intervention. Currently, facial expression recognition for autistic children primarily relies on manual observation or methods based on traditional convolutional neural networks. Manual observation is highly subjective and inefficient; while existing deep learning models are mostly designed for general faces, facing three major challenges in facial expression recognition scenarios for autistic children: first, cluttered backgrounds in experimental or everyday environments interfere with the model's judgment of facial regions; second, the varying distances between children and cameras lead to significant differences in face size in images, making it difficult to extract features from small faces; and third, the expressions of autistic children are often atypical and subtle, making it difficult for conventional models to capture nuanced changes.

[0003] While some studies have attempted to improve performance by introducing attention mechanisms or using lightweight networks, these methods are either computationally complex and difficult to run in real time, or they only focus on a single channel or spatial dimension, failing to comprehensively enhance the response of key facial expression areas (such as around the eyes and corners of the mouth). In addition, most models are not optimized for the data distribution specific to children with autism, resulting in limited generalization ability.

[0004] In summary, existing technologies for facial expression recognition in children with autism suffer from problems such as strong background interference, difficulty in detecting small faces, insufficient capture of facial expression details, and poor real-time performance. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a deep learning-based method for facial expression recognition in children with autism.

[0006] The purpose of this invention is to provide a deep learning-based method for facial expression recognition in children with autism, specifically including the following steps:

[0007] Step 1: Obtain facial expression data of children with autism, process this data to construct a facial expression dataset of children with autism, and divide it into training set and validation set;

[0008] Step 2: Construct the YOLOv11-TA network model. This model is based on the YOLOv11 architecture, and its core innovation lies in introducing the Tri-Attention mechanism and embedding it into two key locations, the backbone and the neck of the network, to solve the problems of difficulty in locating small faces and insufficient capture of facial expression details;

[0009] Step 3: Use the training set and validation set to perform multiple rounds of iterative training and validation on the YOLOv11-TA network, monitor for overfitting, and provide feedback for further model optimization;

[0010] Step 4: Select the best network model based on the model's scores on multiple evaluation metrics on the test set and public dataset, and save the parameters of the best network model;

[0011] Step 5: Use the best network model saved in Step 4 to perform inference on the input real-time image or video frame, and output the face detection box of the autistic child and its corresponding emotion category ("happy", "sad" or "normal") to achieve automatic recognition of emotional state.

[0012] Further, in step 1, a facial expression dataset of autistic children is acquired and preprocessed. This self-built dataset was obtained through a multi-camera facial expression imitation experiment. Specifically, autistic children and their mothers sat facing each other to conduct the facial expression imitation experiment. Three cameras were used to simultaneously record videos from three positions: sideways towards the mother and child, frontally towards the child, and frontally towards the mother. Frames were extracted from the recorded videos, resulting in a total of 3300 images. These images were then manually labeled with one of three emotion tags: "happy," "sad," or "normal." Finally, the self-built dataset was divided into a training set and a validation set in an 8:2 ratio. This dataset features complex backgrounds, a low proportion of faces in the images, and atypical facial expressions, effectively reflecting the challenges in real-world application scenarios.

[0013] As a further improvement of the present invention, in step 2, the network structure of the deep learning-based facial expression recognition method for children with autism is as follows: Figure 2 As shown, the specific description is as follows:

[0014] Step 2.1: For the Tri-Attention module. This module consists of three parallel branches, each focusing on the interaction dependencies between any two dimensions of the input tensor, specifically: height (H) and width (W), channel (C) and height (H), and channel (C) and width (W). The Tri-Attention module receives an input feature map X of shape (C, H, W) and outputs an enhanced feature map of the same shape;

[0015] Furthermore, regarding the design of the Tri-Attention module in step 2.1, such as... Figure 3 As shown, taking the first branch as an example, it can be specifically described as follows:

[0016] Step 2.1.1: Perform adaptive average pooling (AvgPool) with variable-size kernels on the input feature map X along the height (H), width (W), and channel (C) dimensions to extract three feature maps:

[0017] ,

[0018] Where P h (X), P w (X), P c (X) represents adaptive average pooling performed on the height dimension (H), width dimension (W), and channel dimension (C), respectively;

[0019] Step 2.1.2: In the first branch, rotate x2. After rotation, the shape of x2 changes from (C, H, 1) to (C, 1, H). Thus, x1 and x2 can be concatenated in the third dimension to generate a feature map X with the shape (C, 1, H+W). HW；

[0020] Step 2.1.3: Combine the above-mentioned stitched feature map X HW Channel dimensionality reduction is performed using a 1×1 convolutional layer, resulting in C channels after dimensionality reduction. out The calculation formula is:

[0021] ,

[0022] Where r defaults to 16, C in For X HW The number of channels, at this time the output X HW The shape is (C / r, 1, H+W);

[0023] Step 2.1.4: Process the dimensionality-reduced feature map X HW Apply batch normalization (BN);

[0024] Step 2.1.5: Apply the ReLU activation function to the batch-normalized feature map X HW Perform nonlinear activation;

[0025] Step 2.1.6: For feature map X HW Segmenting along the third dimension allows the feature maps to be gradually restored to their initial shapes. The resulting feature maps x1' and x2' have shapes of (C / r, 1, H) and (C / r, 1, W), respectively.

[0026] Step 2.1.7: Pass the feature maps x1' and x2' through a 1×1 convolutional layer to increase the number of channels. At this time, the output feature maps x1' and x2' have the same shape as x1 and x2, which are (C, 1, W) and (C, 1, H) respectively.

[0027] Step 2.1.8: Rotate x2' so that its shape becomes (C, H, 1) in order to perform the subsequent element-wise multiplication operation;

[0028] Step 2.1.9: Apply the Sigmoid activation function to x1' and x2'. The Sigmoid activation function maps the values ​​of the feature maps to the range of 0 to 1, thereby obtaining two attention weight maps, which represent the importance distribution in the width and height directions, respectively.

[0029] Step 2.1.10: Multiply x1' and x2' as attention weight maps element-wise with the initial input feature map X of shape (C, H, W) to obtain the enhanced feature map X of this branch. HW By performing element-wise multiplication, the initial input feature map is adaptively adjusted in both width and height directions according to the attention weight map.

[0030] Step 2.1.11: In the second branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 with shape (1,H,W) and x1 with shape (C,1,W) to generate the corresponding channel-height attention weight maps and output the enhanced feature map X. CH ;

[0031] Step 2.1.12: In the third branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 of shape (1,H,W) and x2 of shape (C,H,1) to generate the corresponding channel-width attention weight maps and output the enhanced feature map X. CW ;

[0032] Step 2.1.13: Enhance the feature map X HW X CH X CW Perform adaptive weighted summation to output the final feature map Y:

[0033] ,

[0034] The final feature map Y has the same shape as the input feature map X, both being (C, H, W). α, β, and γ are learnable parameters that satisfy α + β + γ = 1. By introducing learnable parameters α, β, and γ, the Tri-Attention module can automatically adjust the contribution of each branch feature map to the final output according to the needs of different tasks.

[0035] Step 2.2: Embed the Tri-Attention module into the YOLOv11 network. Specifically, a Tri-Attention module is embedded after each C3k2 module of the YOLOv11 backbone network to enhance the feature extraction capability of key expression regions. At the same time, Tri-Attention modules are introduced at the ends of the three multi-scale feature branches output by the neck network (corresponding to downsampling by 8x, 16x, and 32x) to suppress complex background noise through adaptive weighting and enhance the detection capability of small-sized faces.

[0036] Compared with the prior art, the present invention has the following significant advantages:

[0037] This invention proposes a facial expression recognition method for children with autism that integrates an improved YOLO model with three-dimensional attention (Tri-Attention). By embedding Tri-Attention modules into the backbone network and neck structure of the model, the feature response to key facial regions such as the eyes and corners of the mouth is effectively enhanced, while significantly suppressing interference from complex backgrounds.

[0038] Furthermore, the study used a self-built dataset closely resembling real-world application scenarios for training, significantly improving the model's ability to recognize small and atypical facial expressions. The overall solution maintains the high inference speed advantage of the YOLO model while substantially improving recognition accuracy and robustness, providing a low-cost, high-efficiency technical solution for emotion support systems for children with autism.

[0039] Finally, a systematic comparison of the Tri-Attention module of this invention with other mainstream attention mechanisms was conducted within the YOLOv5 framework. Experimental results show that this method is widely applicable to various image tasks and effectively improves the overall performance of the model, fully demonstrating the superiority and wide applicability of the proposed method in multi-scale feature extraction and adaptability to complex environments. Attached Figure Description

[0040] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0041] Figure 1 This is a flowchart of the deep learning-based facial expression recognition method for children with autism in this application.

[0042] Figure 2 This is a network structure diagram of the deep learning-based facial expression recognition method for children with autism in this application.

[0043] Figure 3 This is a detailed design diagram of the Tri-Attention module in this application.

[0044] Figure 4 The image shows the performance comparison of the Tri-Attention module designed for this application with other mainstream attention mechanisms on the MS COCO dataset under the YOLOv5 framework.

[0045] Figure 5 This figure shows a comparison of the performance of the embodiments of this application and the original YOLOv11 model on a self-built dataset for detecting facial expressions in children with autism, with or without the use of transfer learning. Detailed Implementation

[0046] The present application will now be described in further detail with reference to the embodiments and the accompanying drawings.

[0047] One embodiment of this application is a deep learning-based method for facial expression recognition in children with autism.

[0048] Its overall process is as follows Figure 1 As shown, it includes the following steps:

[0049] Step 1: Obtain facial expression data of children with autism, process this data to construct a facial expression dataset of children with autism, and divide it into training set and validation set;

[0050] Step 2: Construct the YOLOv11-TA network model. This model is based on the YOLOv11 architecture, and its core innovation lies in introducing the Tri-Attention mechanism and embedding it into two key locations, the backbone and the neck of the network, to solve the problems of difficulty in locating small faces and insufficient capture of facial expression details;

[0051] Step 3: Use the training set and validation set to perform multiple rounds of iterative training and validation on the YOLOv11-TA network, monitor for overfitting, and provide feedback for further model optimization;

[0052] Step 4: Select the best network model based on the model's scores on multiple evaluation metrics on the test set and public dataset, and save the parameters of the best network model;

[0053] Step 5: Use the best network model saved in Step 4 to perform inference on the input real-time image or video frame, and output the face detection box of the autistic child and its corresponding emotion category ("happy", "sad" or "normal") to achieve automatic recognition of emotional state.

[0054] Further, in step 1, a facial expression dataset of autistic children is acquired and preprocessed. This self-built dataset was obtained through a multi-camera facial expression imitation experiment. Specifically, autistic children and their mothers sat facing each other to conduct the facial expression imitation experiment. Three cameras were used to simultaneously record videos from three positions: sideways towards the mother and child, frontally towards the child, and frontally towards the mother. Frames were extracted from the recorded videos, resulting in a total of 3300 images. These images were then manually labeled with one of three emotion tags: "happy," "sad," or "normal." Finally, the self-built dataset was divided into a training set and a validation set in an 8:2 ratio. This dataset features complex backgrounds, a low proportion of faces in the images, and atypical facial expressions, effectively reflecting the challenges in real-world application scenarios.

[0055] As a further improvement of the present invention, in step 2, the network structure of the deep learning-based facial expression recognition method for children with autism is as follows: Figure 2 As shown, the specific description is as follows:

[0056] Step 2.1.1: Perform adaptive average pooling (AvgPool) with variable-size kernels on the input feature map X along the height (H), width (W), and channel (C) dimensions to extract three feature maps:

[0057] ,

[0058] Where P h (X), P w (X), P c (X) represents adaptive average pooling performed on the height dimension (H), width dimension (W), and channel dimension (C), respectively;

[0059] Step 2.1.2: In the first branch, rotate x2. After rotation, the shape of x2 changes from (C, H, 1) to (C, 1, H). Thus, x1 and x2 can be concatenated in the third dimension to generate a feature map X with the shape (C, 1, H+W). HW；

[0060] Step 2.1.3: Combine the above-mentioned stitched feature map X HW Channel dimensionality reduction is performed using a 1×1 convolutional layer, resulting in C channels after dimensionality reduction. out The calculation formula is:

[0061] ,

[0062] Where r defaults to 16, C in For X HW The number of channels, at this time the output X HWThe shape is (C / r, 1, H+W);

[0063] Step 2.1.4: Process the dimensionality-reduced feature map X HW Apply batch normalization (BN);

[0064] Step 2.1.5: Apply the ReLU activation function to the batch-normalized feature map X HW Perform nonlinear activation;

[0065] Step 2.1.6: For feature map X HW Segmenting along the third dimension allows the feature maps to be gradually restored to their initial shapes. The resulting feature maps x1' and x2' have shapes of (C / r, 1, H) and (C / r, 1, W), respectively.

[0066] Step 2.1.7: Pass the feature maps x1' and x2' through a 1×1 convolutional layer to increase the number of channels. At this time, the output feature maps x1' and x2' have the same shape as x1 and x2, which are (C, 1, W) and (C, 1, H) respectively.

[0067] Step 2.1.8: Rotate x2' so that its shape becomes (C, H, 1) in order to perform the subsequent element-wise multiplication operation;

[0068] Step 2.1.9: Apply the Sigmoid activation function to x1' and x2'. The Sigmoid activation function maps the values ​​of the feature maps to the range of 0 to 1, thereby obtaining two attention weight maps, which represent the importance distribution in the width and height directions, respectively.

[0069] Step 2.1.10: Multiply x1' and x2' as attention weight maps element-wise with the initial input feature map X of shape (C, H, W) to obtain the enhanced feature map X of this branch. HW By performing element-wise multiplication, the initial input feature map is adaptively adjusted in both width and height directions according to the attention weight map.

[0070] Step 2.1.11: In the second branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 with shape (1,H,W) and x1 with shape (C,1,W) to generate the corresponding channel-height attention weight maps and output the enhanced feature map X. CH ;

[0071] Step 2.1.12: In the third branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 of shape (1,H,W) and x2 of shape (C,H,1) to generate the corresponding channel-width attention weight maps and output the enhanced feature map X.CW ;

[0072] Step 2.1.13: Enhance the feature map X HW X CH X CW Perform adaptive weighted summation to output the final feature map Y:

[0073] ,

[0074] The final feature map Y has the same shape as the input feature map X, both being (C, H, W). α, β, and γ are learnable parameters that satisfy α + β + γ = 1. By introducing learnable parameters α, β, and γ, the Tri-Attention module can automatically adjust the contribution of each branch feature map to the final output according to the needs of different tasks.

[0075] Step 2.2: Embed the Tri-Attention module into the YOLOv11 network. Specifically, a Tri-Attention module is embedded after each C3k2 module of the YOLOv11 backbone network to enhance the feature extraction capability of key expression regions. At the same time, Tri-Attention modules are introduced at the ends of the three multi-scale feature branches output by the neck network (corresponding to downsampling by 8x, 16x, and 32x) to suppress complex background noise through adaptive weighting and enhance the detection capability of small-sized faces.

[0076] Furthermore, in step 3, the training and validation of the model employ the following comprehensive strategy:

[0077] First, stochastic gradient descent (SGD) is used as the optimizer, with an initial learning rate of 0.01, a momentum coefficient of 0.937, and a weight decay coefficient of 0.0005. This configuration not only helps improve the stability of parameter updates but also reduces the over-reliance on training data by constraining model complexity through weight decay.

[0078] Secondly, a linear warmup mechanism is introduced in the early stage of training, gradually increasing the learning rate from low to high in the first 3 iterations to avoid training oscillations caused by unstable initial gradients. After the warmup phase, a cosine annealing scheduling method is adopted to make the learning rate decay smoothly with the training process, thereby accelerating convergence in the early stage and providing more refined parameter fine-tuning capabilities in the later stage.

[0079] In addition, the maximum number of training rounds is set to 300 iterations, the batch size is fixed at 128, and all input images are uniformly scaled to 640×640 pixels to achieve a reasonable trade-off between hardware resource limitations and model expressive power.

[0080] Meanwhile, the most critical metrics on the validation set are continuously tracked during training. If the metrics do not improve within 100 consecutive iterations, an early stopping mechanism is triggered, which automatically stops training in advance to prevent the model from degrading due to continuous fitting of training noise.

[0081] In addition, a model checkpoint mechanism is implemented, which saves the best-performing model parameters on the validation set every 10 iterations for subsequent evaluation and deployment;

[0082] Finally, in step 3, a transfer learning strategy is applied. Specifically, the parameters of the YOLOv11-TA model trained on a public dataset are used as initial parameters, and then the model is further trained using a self-built image dataset of autistic children to adapt to real-world application scenarios.

[0083] In step 4, a systematic performance evaluation is performed on the trained model to select and save the optimal model. The evaluation metrics include accuracy, recall, and mean average precision (mAP), as detailed below:

[0084] ,

[0085] Where TP represents the number of targets detected at that location; FP represents the number of targets that were not detected at that location; FN represents the number of background elements that were falsely detected as targets at that location; AP is the area under the PR (Precision-Recall) curve for a certain category; and c is the number of categories detected.

[0086] The core metrics used in the evaluation include mAP@.5 and mAP@.5:.95. The concept of IoU (Intersection over Union) is utilized. IoU measures the degree of overlap between two bounding boxes, calculated as: IoU = Area of ​​intersection of the two boxes ÷ Area of ​​union of the two boxes. Its value ranges from 0 to 1: IoU = 0 indicates no overlap, and IoU = 1 indicates complete overlap. In object detection, IoU is commonly used to determine the degree of matching between predicted and ground truth bounding boxes.

[0087] In the evaluation metric mAP, mAP@.5 corresponds to the average accuracy when the IoU threshold is 0.5, used to measure the model's recognition performance under relatively lenient localization conditions; mAP@.5:.95 represents the average accuracy calculated at ten different IoU thresholds from 0.5 to 0.95 with a step size of 0.05, comprehensively reflecting the model's detection capability under different localization strictness; the specific calculation formula is as follows:

[0088]

[0089] Furthermore, for the MS COCO dataset, the detection accuracy is further subdivided into different scales of targets, including small targets (AP). S Target area <32², medium target (AP) M (32² ≤ target area ≤ 96²) and large targets (AP) L The average precision is for targets with an area > 96². Here, 32 and 96 are in pixels.

[0090] Based on the comprehensive performance of the above indicators, this solution ultimately selected the YOLOv11-TA network, which scored highest on mAP@.5:.95, as the optimal model. This model was identified as the most suitable architecture for facial expression recognition tasks in children with autism, and its corresponding parameters were subsequently saved in their entirety for use in the subsequent inference stage.

[0091] Furthermore, such as Figure 4 A systematic comparison of the Tri-Attention module of this invention with other mainstream attention mechanisms is conducted within the YOLOv5 framework. Experimental results show that this method is applicable to various image tasks and can effectively improve the overall performance of the model. Figure 5 After using public datasets for transfer learning, the YOLOv11-TA network model exhibits stronger robustness and better transferability. Although the YOLOv11 model has specific advantages without transfer learning, the YOLOv11-TA network model, due to its enhanced structural and feature extraction capabilities in complex environments, demonstrates higher adaptability and effectiveness in a wider range of application scenarios.

[0092] Furthermore, in step 5, the present invention relates to using the optimal network model selected in step 4 to perform inference on new facial images or video frames of autistic children to achieve automatic recognition of emotional states. The specific description is as follows:

[0093] Step 5.1: Load the optimal YOLOv11-TA network model parameters evaluated and saved in Step 4 into the model architecture to ensure that the model can perform inference tasks in the best state.

[0094] Step 5.2: Preprocess the input real-time image or video frames, including scaling the image size to 640×640 and normalization, to ensure that the input data meets the model's format and distribution requirements;

[0095] Step 5.3: Input the preprocessed image into the model, perform the forward propagation process, use the deep structure of the model to perform face detection and expression feature extraction, and output the bounding box of each face and its corresponding emotion category ("happy", "sad" or "normal").

[0096] Step 5.4: Based on the detection results output by the model, and combined with post-processing strategies such as non-maximum suppression (NMS), redundant detection boxes are removed, and the effective predictions with the highest confidence are retained to finally generate structured recognition results.

[0097] Step 5.5: Visualize the generated recognition results or output them as structured data (such as JSON format) for use by rehabilitation teachers, parents or intelligent assistance systems, ensuring that they meet the accuracy and real-time requirements of actual application scenarios;

[0098] Step 5.6: Feed back the emotion recognition results to the clinical or educational intervention system to help professionals perceive the emotional state of children with autism in a timely manner, thereby developing more targeted emotional support and behavioral intervention strategies, improving intervention efficiency and the level of humanistic care;

[0099] Step 5.7: During actual deployment, continuously monitor the inference performance and recognition stability of the model in real-world scenarios, and establish a user feedback mechanism to collect opinions from frontline users, providing a basis for further optimizing the model structure, updating training data, and improving system robustness.

Claims

1. A facial expression recognition method for autistic children based on the YOLO model, characterized in that: Specifically, the steps include the following: Step 1: Obtain facial expression data of children with autism, process this data to construct a facial expression dataset of children with autism, and divide it into training set and validation set; Step 2: Construct the YOLOv11-TA network model. This model is based on the YOLOv11 architecture. Its core innovation lies in introducing the Tri-Attention mechanism and embedding it into the two key positions of the network backbone and neck to solve the problems of difficulty in localizing small faces and insufficient capture of facial expression details. Step 3: Use the training set and validation set to perform multiple rounds of iterative training and validation on the YOLOv11-TA network, monitor for overfitting, and provide feedback for further model optimization; Step 4: Select the best network model based on the model's scores on multiple evaluation metrics on the test set and public dataset, and save the parameters of the best network model; Step 5: Use the best network model saved in Step 4 to perform inference on the input real-time image or video frame, and output the face detection box of the autistic child and its corresponding "happy", "sad" or "normal" emotion category to achieve automatic recognition of emotional state. In step 2, the network structure of the facial expression recognition method for autistic children based on the YOLO model is specifically described as follows: Step 2.1: For the Tri-Attention module, this module consists of three parallel branches, each focusing on the interaction dependency between any two dimensions of the input tensor, specifically including: height H and width W, channel C and height H, and channel C and width W; the Tri-Attention module receives an input feature map X with shape (C, H, W) and outputs an enhanced feature map of the same shape; Furthermore, regarding the design of the Tri-Attention module in step 2.1, taking the first branch as an example, it can be specifically described as follows: Step 2.1.1: Perform adaptive average pooling with variable-size kernels on the input feature map X along the height dimension H, width dimension W, and channel dimension C to extract three feature maps: ， Where P h (X), P w (X), P c (X) represents adaptive average pooling performed on the height dimension (H), width dimension (W), and channel dimension (C), respectively; Step 2.1.2: In the first branch, rotate x2. After rotation, the shape of x2 changes from (C, H, 1) to (C, 1, H). Thus, x1 and x2 can be concatenated in the third dimension to generate a feature map X with the shape (C, 1, H+W). HW ; Step 2.1.3: Combine the above-mentioned stitched feature map X HW Channel dimensionality reduction is performed using a 1×1 convolutional layer, resulting in C channels after dimensionality reduction. out The calculation formula is: ， Where r defaults to 16, C in For X HW The number of channels, at this time the output X HW The shape is (C / r, 1, H+W); Step 2.1.4: Process the dimensionality-reduced feature map X HW Apply batch normalization (BN); Step 2.1.5: Apply the ReLU activation function to the batch-normalized feature map X HW Perform nonlinear activation; Step 2.1.6: For feature map X HW The feature maps are segmented along the third dimension so that they can be gradually restored to their initial shape; the shapes of the segmented feature maps x1' and x2' are (C / r, 1, H) and (C / r, 1, W), respectively. Step 2.1.7: Pass the feature maps x1' and x2' through a 1×1 convolutional layer to increase the number of channels. At this time, the output feature maps x1' and x2' have the same shape as x1 and x2, which are (C, 1, W) and (C, 1, H) respectively. Step 2.1.8: Rotate x2' so that its shape becomes (C, H, 1) in order to perform the subsequent element-wise multiplication operation; Step 2.1.9: Apply the Sigmoid activation function to x1' and x2'. The Sigmoid activation function maps the values ​​of the feature maps to the range of 0 to 1, thereby obtaining two attention weight maps, which represent the importance distribution in the width and height directions, respectively. Step 2.1.10: Multiply x1' and x2' as attention weight maps element-wise with the initial input feature map X of shape (C, H, W) to obtain the enhanced feature map X of this branch. HW By performing element-wise multiplication, the initial input feature map is adaptively adjusted in both width and height directions according to the attention weight map. Step 2.1.11: In the second branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 with shape (1,H,W) and x1 with shape (C,1,W) to generate the corresponding channel-height attention weight maps and output the enhanced feature map X. CH ; Step 2.1.12: In the third branch, perform a similar processing flow as in steps S2.1.2–2.1.10 on x3 of shape (1,H,W) and x2 of shape (C,H,1) to generate the corresponding channel-width attention weight maps and output the enhanced feature map X. CW ; Step 2.1.13: Enhance the feature map X HW X CH X CW Perform adaptive weighted summation to output the final feature map Y: ， The final feature map Y has the same shape as the input feature map X, both being (C, H, W). α, β, and γ are all learnable parameters that satisfy α+β+γ=1. By introducing learnable parameters α, β, and γ, the Tri-Attention module can automatically adjust the contribution of each branch feature map to the final output according to the needs of different tasks. Step 2.2: Embed the Tri-Attention module into the YOLOv11 network; specifically, embed a Tri-Attention module after each C3k2 module of the YOLOv11 backbone network to enhance the feature extraction capability of key expression regions; at the same time, introduce the Tri-Attention module at the end of the three multi-scale feature branches output by the neck network to suppress complex background noise through adaptive weighting and enhance the detection capability of small faces.

2. The method according to claim 1, characterized in that, Step 1: Obtain a facial expression dataset of autistic children and preprocess it; the self-built dataset is obtained by collecting data through multiple cameras in the facial expression imitation experiment; specifically, organize autistic children and their mothers to sit facing each other and carry out facial expression imitation experiments; use three cameras to record videos simultaneously from three positions: the side facing the mother and child, the front facing the child, and the front facing the mother. Frames were extracted from the recorded video, resulting in a total of 3,300 images. These images were then manually labeled with one of three emotion categories: "happy," "sad," or "normal." Finally, the self-built dataset was divided into a training set and a validation set in an 8:2 ratio.

Images