Artificial intelligence automatic recognition system based on facial expressions

Abstract

The present application relates to the technical field of artificial intelligence, in particular to an artificial intelligence automatic identification system based on facial expression, comprising an audio-visual stimulation module, which outputs inducing information for guiding a target object to produce a specific emotion; a head detection module connected with an information acquisition unit, which is used for detecting head reaction information; and a label classification module connected with the head detection module, which has a pre-trained classification model and is used for outputting a classification label according to facial forward proportion data, neutral emotion proportion data and head complexity data.The present application takes facial expression as a specific identification index, can perform unified standardized identification on children in early stage, can objectively evaluate children under non-contact and non-stimulation conditions based on machine learning technology, maximally reduces time cost and labor cost, and realizes large-scale screening of ASD children.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to an AI-based automatic facial expression recognition system. Background Technology

[0002] Specifically, two types of tools are commonly used in the identification of children with autism spectrum disorder (ASD): standardized identification tools and invasive identification tools. Standardized identification tools use questionnaires, interviews, observations, etc., but the differences in different identification standards can easily lead to discrepancies in assessment results. In addition, due to the high degree of subjective experience judgment required by screening tools, gender and racial biases are more likely to occur in the assessment process, resulting in a lack of stability in clinical diagnosis. In the early stages of a child's development, subtle differences shown by children with ASD are difficult to detect and identify, leading to a later age of diagnosis.

[0003] Invasive testing tools based on facial electromyography (FEMG) identify muscle changes in children with ASD and typical developmental individuals by measuring electrical impulses from facial muscle contractions. However, this tool is only applicable to the corrugator supercilii and zygomaticus major muscles and cannot define other facial movements, thus limiting its results. Furthermore, because electromyographic signals are weak, human hair and skin keratin can hinder signal transmission; therefore, before applying the electrode patches, it is necessary to remove the influence of oils and other substances using alcohol. Overall, this process is invasive and unsuitable for highly sensitive individuals with ASD. Summary of the Invention

[0004] The purpose of this invention is to provide an artificial intelligence-based automatic facial expression recognition system to solve the above-mentioned technical problems;

[0005] The technical problem solved by this invention can be achieved by the following technical solutions:

[0006] AI-based automatic facial expression recognition systems include,

[0007] The audiovisual stimulation module outputs evoked information to guide the target audience to generate specific emotions.

[0008] A head detection module is connected to an information acquisition unit for acquiring head response information of a target object based on the evoked information. The head detection module is used to detect the head response information, including...

[0009] A face-facing proportion detection unit, connected to the information acquisition unit, is used to calculate the proportion of face-facing time to total time and output face-facing proportion data.

[0010] The neutral emotion ratio detection module, connected to the information acquisition unit, is used to calculate the proportion of neutral emotion duration to total duration and output neutral emotion ratio data.

[0011] A head complexity detection unit, connected to the information acquisition unit, is used to calculate the head complexity of the target object and output the head complexity data.

[0012] The label classification module is connected to the head detection module. The label classification module has a pre-trained classification model. The classification model is used to output classification labels based on the face-facing proportion data, the neutral emotion proportion data, and the head complexity data.

[0013] Preferably, the audiovisual stimulation module guides the target object to generate specific emotions, including at least happiness, surprise, and anger; the information acquisition unit is a camera positioned facing the target object's face; and the head reaction information is a head reaction video captured by the camera.

[0014] Preferably, the face-facing-forward ratio detection unit has a head pose estimation model based on a convolutional neural network. The head pose estimation model outputs the head pose features corresponding to each video frame by traversing each video frame of the head response information, and marks the corresponding images according to the head pose features as video frames facing forward and video frames not facing forward.

[0015] The proportion of faces facing forward is obtained by statistically analyzing the percentage of video frames facing forward in the total number of frames.

[0016] Preferably, the condition that the forward-looking video frame needs to meet is that the angle between the head pitch angle and the yaw angle of the target object is less than or equal to a preset angle.

[0017] Preferably, the neutral emotion ratio detection module has an emotion detection model based on a convolutional neural network, which is pre-trained using a dataset containing facial expression information.

[0018] The emotion detection model iterates through each video frame of the head reaction information, outputs the emotion label corresponding to each video frame, and marks the corresponding video frames according to the emotion label, marking them as neutral emotion video frames and other emotion video frames; the neutral emotion ratio data is obtained based on the statistical proportion of the neutral emotion video frames in the total number of frames.

[0019] Preferably, the head complexity data output by the head complexity detection unit includes facial complexity, gaze complexity, and head posture complexity.

[0020] Preferably, the head complexity detection unit includes,

[0021] A head pose estimation model, connected to the information acquisition unit, is used to output the head pose of each video frame in the head response information.

[0022] A gaze detection model, connected to the information acquisition unit, is used to detect the gaze vectors of each video frame in the head response information;

[0023] A facial landmark detection model, connected to the information acquisition unit, is used to detect facial landmarks in each video frame of the head reaction information.

[0024] Preferably, the head complexity detection unit further includes,

[0025] The head posture complexity calculation subunit is connected to the head posture estimation model and is used to calculate the angular rate of head swing based on the head posture to obtain angular velocity sequence information as the head posture complexity.

[0026] The gaze complexity calculation subunit is connected to the gaze detection model and is used to calculate the Euclidean distance between the gaze vectors of two adjacent video frames to obtain gaze difference sequence information as the gaze complexity.

[0027] The facial complexity calculation subunit is connected to the facial key point detection model and is used to calculate the Euclidean distance of the changes in the facial key points between two adjacent video frames, and obtain facial Euclidean distance sequence information as the facial complexity.

[0028] The sequence information processing subunit, connected to the head posture complexity calculation subunit, the gaze complexity calculation subunit, and the face complexity calculation subunit, is used to convert the angular velocity sequence information, the gaze difference sequence information, and the face Euclidean distance sequence information into multi-scale entropy.

[0029] Preferably, the classification model adopts a machine learning model based on the gradient boosting framework and is trained using a binary classification task. The classification labels output by the classification model include autism labels and non-autism labels.

[0030] Preferably, it further includes a model optimization unit, connected to the classification model, used to update the classification model based on the error between the prediction result of the classification model and the actual label, and used to construct a new model tree based on residual information and optimize the classification model by adjusting the leaf node values.

[0031] The beneficial effects of the present invention are as follows: By adopting the above technical solutions, the present invention uses facial expressions as a specific identification indicator, which enables standardized identification in early childhood. At the same time, based on machine learning technology, it can objectively assess children under non-contact and non-stimulating conditions, minimizing time and manpower costs, and realizing large-scale screening of children with ASD. Attached Figure Description

[0032] Figure 1 This is an architecture diagram of an AI-based automatic facial expression recognition system according to an embodiment of the present invention.

[0033] Figure 2 This is an architecture diagram of the header complexity detection unit in an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the head posture features of the human model in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram showing the face-forward proportions of a typical developing child and a child with autism in an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram illustrating the difference in the distribution of neutral emotions between typical developmental children and children with autism in an embodiment of the present invention;

[0037] Figure 6a This is a graph showing the difference in MSE values ​​of facial complexity between typical developmental children and children with autism in an embodiment of the present invention;

[0038] Figure 6b This is a graph showing the difference in MSE values ​​at key ocular points between typical developing children and children with autism in an embodiment of the present invention;

[0039] Figure 6c This is a graph showing the difference in MSE values ​​of key nasal points between a typical developing child and a child with autism in an embodiment of the present invention.

[0040] Figure 7 This is a graph showing the difference in the trend of head movement multiscale entropy between typical developing children and children with autism in an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram illustrating the automatic identification of training set data in an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram illustrating the automatic identification of test set data in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0046] AI-based automatic facial expression recognition systems, such as Figure 1 , Figure 2 As shown, including,

[0047] Audiovisual stimulation module 1 outputs inducing information to guide the target object to generate specific emotions;

[0048] Head detection module 3 is connected to information acquisition unit 2, which is used to acquire head response information of the target object based on evoked information. Head detection module 3 is used to detect head response information, including...

[0049] The face-facing proportion detection unit 31 is connected to the information acquisition unit 2 and is used to calculate the proportion of face-facing time to total time and output face-facing proportion data.

[0050] The neutral emotion ratio detection module 32 is connected to the information acquisition unit 2 and is used to calculate the proportion of neutral emotion duration to total duration and output neutral emotion ratio data.

[0051] The head complexity detection unit 33 is connected to the information acquisition unit 2 and is used to calculate the head complexity of the target object and output the head complexity data.

[0052] The label classification module 34 is connected to the head detection module 3. The label classification module 34 has a pre-trained classification model. The classification model is used to output classification labels based on the face-facing proportion data, neutral emotion proportion data, and head complexity data.

[0053] Specifically, the working process of the facial expression-based artificial intelligence automatic recognition system of the present invention mainly consists of two processes: visual and auditory stimulation and automatic detection.

[0054] Firstly, the information acquisition unit 2, i.e., the camera, records the head reaction of the target object under specific audiovisual stimuli. In this invention, the target object is a child, and the stimulus is an animated video that can evoke feelings of happiness, surprise, or anger in the child.

[0055] Secondly, it automatically detects various indicators of an individual's head, mainly including facial emotions, head posture, and head complexity. Among them, facial emotions are estimated by the proportion of neutral emotions, head posture is estimated by the proportion of faces facing forward, and head complexity is estimated by the complexity of the face, gaze, and head posture.

[0056] By developing an AI-powered automatic facial expression recognition system, large-scale screening and early warning of children with ASD can be achieved. This system uses facial expressions as a specific identification indicator, enabling standardized identification in early childhood. Furthermore, based on machine learning technology, it allows for objective assessment of children without contact or stimulation, minimizing time and manpower costs and enabling large-scale screening of children with ASD.

[0057] All code in this invention is written in Python and implemented using the PyTorch framework. The GPU used is an RTX3090.

[0058] In a preferred embodiment, the audiovisual stimulation module 1 guides the target object to generate specific emotions, including at least happiness, surprise, and anger. The information acquisition unit 2 is a camera positioned facing the target object's face, and the head reaction information is a video of the head reaction captured by the camera.

[0059] In a preferred embodiment, the face-facing-forward ratio detection unit 31 has a head pose estimation model 331 based on a convolutional neural network. The head pose estimation model 331 outputs the head pose features corresponding to each video frame by traversing each video frame of the head response information, and marks the corresponding images according to the head pose features as video frames facing forward and video frames not facing forward.

[0060] The proportion of faces facing forward is obtained by statistically analyzing the percentage of video frames that are viewed from the front in the total number of frames.

[0061] Specifically, the face-facing proportion detection unit 31 is used to perform face-facing proportion data processing and calculate the proportion of the time the child's face is facing forward in the video sequence to the total time.

[0062] First, the face-facing proportion detection unit 31 traverses each frame of the video recording and outputs head pose features (yaw, pitch, roll) through the head pose estimation model 3313DDFA based on the convolutional neural network.

[0063] yaw is the yaw angle, the angle of rotation around the vertical axis;

[0064] pitch is the angle of elevation, the angle of rotation around the horizontal axis;

[0065] Roll is the tumble angle, the angle of rotation around the longitudinal axis.

[0066] The angular information of head pose features can accurately describe the orientation of the head in three-dimensional space. The model dataset is 300W-LP head pose data.

[0067] Specifically, a forward-facing posture is defined as a head pitch angle and yaw angle that are less than or equal to 20 degrees. Figure 3 As shown, Figure 3 This is a schematic diagram of head posture features. Figure 3 The character model 5 in the image is for reference only. In actual scenes, glasses are usually removed, and only the head features are retained.

[0068] In a preferred embodiment, the condition for marking a video frame as looking forward is that the angle between the pitch angle and yaw angle of the target object's head posture is less than or equal to a preset angle.

[0069] Specifically, this invention defines "face forward" as the angle between the head's pitch and yaw angles being less than or equal to 20 degrees. Based on this quantification standard, the combination of these two angles is used to determine whether the face is facing forward.

[0070] For example, in a video frame, if the angle between the pitch and yaw angles calculated from the head pose features output by the 3DDFA model is 15 degrees, then this video frame is determined to be a face facing forward; if the angle is 30 degrees, it is determined to be a face not facing forward.

[0071] The proportion of video frames with faces facing forward and those without faces facing forward is counted in the total number of video frames to obtain data on the proportion of faces facing forward.

[0072] In a preferred embodiment, the neutral emotion ratio detection module 32 has an emotion detection model based on a convolutional neural network, which is pre-trained using a dataset containing facial expression information.

[0073] The emotion detection model iterates through each video frame containing head reaction information, outputs the emotion label corresponding to each video frame, and marks the corresponding video frames according to the emotion label, marking them as neutral emotion video frames and other emotion video frames; and obtains neutral emotion ratio data based on the statistical proportion of neutral emotion video frames in the total number of frames.

[0074] Specifically, the neutral emotion ratio detection module 32 is used to perform neutral emotion ratio data processing and calculate the proportion of children's neutral emotion duration to the total duration in the video sequence.

[0075] The neutral emotion ratio detection module iterates through each frame of the video recording and outputs emotion labels (e.g., Neutral, Happy, Sad, Surprise, Fear, Disgust, Anger) based on the Affect Net emotion detection model using a convolutional neural network. Its model dataset is the Affect Net Facial Expression data.

[0076] The neutral emotion ratio detection module 32 classifies the output frame labels into neutral emotions or other emotions, and counts the proportion of video frames of the two emotion categories to the total number of video frames.

[0077] More specifically, for the emotion detection model Affect Net, it is necessary to learn the emotion categories corresponding to different facial expressions through a large amount of data.

[0078] This invention utilizes the Affect Net Facial Expression dataset, which contains facial expression information, to train the Affect Net model. The dataset includes facial images labeled with different emotions (such as Neutral, Happy, Sad, etc.).

[0079] During training, the emotion detection model learns the association between facial features and emotion labels in these images, gradually building a mapping relationship from facial features to emotion judgment. Thus, when outputting emotion labels for each frame of a new image, i.e., a video, the model can determine the emotion label corresponding to the facial expression in that frame based on the previously learned mapping relationship.

[0080] This invention utilizes the Affect Net emotion detection model. The emotion detection model is built upon a convolutional neural network and trained using Affect Net Facial Expression data. In this invention, the emotion detection model analyzes and processes each frame of the video recording and then outputs the corresponding emotion label.

[0081] The emotion labels include (Neutral, Happy, Sad, Surprise, Fear, Disgust, Anger), which means that it can determine the general emotional state of the child in each frame.

[0082] After obtaining the emotion tag for each frame, the next step is to perform classification and statistical analysis. These output frame tags are simply divided into two categories based on emotion: neutral emotions and other emotions. Neutral emotions refer to those with the emotion tag "Neutral," while other emotions include non-neutral emotion types such as "Happy," "Sad," "Surprise," "Fear," "Disgust," and "Anger." Then, the number of video frames belonging to neutral emotions and the number of video frames belonging to other emotions are counted, and their respective proportions in the total number of frames in the video are calculated.

[0083] In a preferred embodiment, the head complexity data output by the head complexity detection unit 33 includes facial complexity, gaze complexity, and head posture complexity.

[0084] In a preferred embodiment, the head complexity detection unit 33 includes,

[0085] The head pose estimation model 331 is connected to the information acquisition unit 2 and is used to output the head pose of each video frame in the head response information.

[0086] The gaze detection model 332 is connected to the information acquisition unit 2 and is used to detect the gaze vectors of each video frame in the head reaction information.

[0087] The facial key point detection model 333 is connected to the information acquisition unit 2 and is used to detect facial key points in each video frame of the head reaction information.

[0088] In a preferred embodiment, the head complexity detection unit 33 further includes,

[0089] The head posture complexity calculation subunit 334 is connected to the head posture estimation model 331 and is used to calculate the angular rate of head swing based on the head posture to obtain the angular velocity sequence information as the head posture complexity.

[0090] The gaze complexity calculation subunit 335 is connected to the gaze detection model 332 and is used to calculate the Euclidean distance between the gaze vectors of two adjacent video frames to obtain gaze difference sequence information as gaze complexity.

[0091] The facial complexity calculation subunit 336 is connected to the facial key point detection model 333 and is used to calculate the Euclidean distance of the facial key point changes between two adjacent video frames to obtain facial Euclidean distance sequence information as facial complexity.

[0092] The sequence information processing subunit 337 is connected to the head posture complexity calculation subunit 334, the gaze complexity calculation subunit 335, and the face complexity calculation subunit 336. It is used to convert angular velocity sequence information, gaze difference sequence information, and face Euclidean distance sequence information into multi-scale entropy.

[0093] Specifically, the head complexity detection unit 33 processes and calculates the complexity of the child's face, gaze, and head posture in the video sequence. In this process, the facial complexity is measured by the Euclidean distance between the changes of key points in two adjacent frames of the face, the gaze complexity is measured by the Euclidean distance between the gaze vectors in two adjacent frames, and the head posture complexity is measured by the angular rate of head movement.

[0094] The head complexity detection unit 33 traverses each frame of the video recording and outputs the head pose, 3D gaze vector and facial key points of each frame of the video through the head pose estimation model 331, gaze detection model 332 and facial key point detection model 333.

[0095] The angular rate of head movement, the difference in gaze vector, and the Euclidean distance of changes in adjacent facial key points were calculated between frames.

[0096] The sequence information processing subunit 337 processes the angular velocity sequence information, line-of-sight difference sequence information, and facial Euclidean distance sequence information into multi-scale entropy (MSE).

[0097] More specifically, for the different data obtained in the previous step, the relevant change indicators between each frame are calculated:

[0098] Head pose complexity metric calculation: The head pose complexity is measured by calculating the angular rate of head movement. Angular rate describes the speed of head rotation, and it is calculated by dividing the change in head pose angle between adjacent frames by the corresponding time interval.

[0099] Since the time interval between video frames is usually fixed, angular velocity can be represented by changes in angle.

[0100] Line-of-sight complexity metric calculation: Calculate the Euclidean distance between the line-of-sight vectors of two adjacent frames. Euclidean distance is used to measure the straight-line distance between two points in space;

[0101] The Euclidean distance between the gaze vectors of two adjacent frames reflects the magnitude of change in gaze direction between adjacent moments. A large Euclidean distance indicates a significant change in gaze direction and complex gaze variations; a small Euclidean distance indicates a relatively stable gaze direction and low complexity.

[0102] Facial complexity index calculation: The Euclidean distance between the changes in key points between two adjacent frames of the face is used as the metric. Since facial key points contain the coordinate information of key facial features, calculating the Euclidean distance between the coordinate changes of these key points between adjacent frames can reflect the degree of change in facial shape at adjacent time points.

[0103] For example, blinking causes changes in the coordinates of key points around the eyes, and smiling causes changes in the coordinates of key points around the mouth. These changes can be quantified using Euclidean distance. The greater the distance, the more frequent and complex the changes in facial shape, and vice versa.

[0104] The previously calculated angular velocity sequence information (reflecting changes in head posture complexity), gaze difference sequence information (reflecting changes in gaze complexity), and facial Euclidean distance sequence information (characterizing changes in facial complexity) are further processed and uniformly converted into multi-scale entropy (MSE).

[0105] Multiscale entropy can comprehensively and quantitatively represent the overall complexity of a child's facial features, gaze, and head posture.

[0106] In a preferred embodiment, the classification model employs a gradient boosting framework-based machine learning model and is trained using a binary classification task. The classification labels output by the classification model include autism labels and non-autism labels.

[0107] This invention employs the LGB model as the classification model, namely, a lightweight gradient booster. The LGB model is a machine learning algorithm based on the gradient boosting framework, used to solve different types of prediction tasks such as classification and regression, and is particularly suitable for the binary classification task in the application scenario of this invention: determining whether someone has autism or not.

[0108] The LGB model follows the basic idea of ​​gradient boosting. It builds multiple weak learners (usually decision trees) in sequence and trains the next learner based on the prediction residual of the previous learner. Through continuous iteration and optimization, the overall predictive ability of the model is gradually improved. Finally, multiple weak learners are combined to form a strong learner to achieve a more accurate fit and prediction of the data.

[0109] Specifically, the present invention first initializes the classification model (LGB model) by inputting variable data such as the proportion of faces facing forward, the proportion of neutral emotions, and complexity into the model. During this process, the model constructs an initial decision tree structure based on the set parameters, such as determining the approximate shape of the tree and the node partitioning rules.

[0110] Simultaneously, optimization algorithms (such as gradient descent algorithms) are used to adjust the parameters in the model, minimizing the loss function value in the initial stage. The loss function measures the difference between the model's predictions and the true labels.

[0111] After the model has made preliminary predictions based on the initial parameters, it is necessary to compare these predictions with the actual labels, that is, the actual situation of whether a child has autism.

[0112] Error can be quantified in several ways, such as by directly calculating the difference between the predicted value and the actual value, or by measuring it based on the results of the loss function calculation.

[0113] Then, based on the calculated error information, the model parameters are updated along the direction of error reduction using the principle of gradient boosting.

[0114] For example, by adjusting parameters such as the splitting threshold of decision tree nodes and the value of leaf nodes, the model can reduce errors in the next round of prediction and more accurately determine whether a child has autism.

[0115] By iteratively calculating errors and updating parameters, the model gradually learns the hidden patterns and feature relationships in the data, thereby improving the accuracy of classification.

[0116] like Figures 4 to 7 As shown, based on different biomarkers, the differences between children with autism and typically developing children can be clearly identified, among which... Figure 4The diagram illustrates the proportions of faces facing forward. Children with autism (ASD) have a higher density when their face-facing proportion is around 0.7, while children with typical development have a higher density when their face-facing proportion is around 0.4.

[0117] Figure 5 The study showed the differences in the distribution of neutral mood ratios between typical developmental children and children with autism. Children with autism (ASD) were most concentrated when the neutral mood ratio was approximately 0.2, while typical developmental children were most concentrated when the neutral mood ratio was approximately 0.6.

[0118] Figure 6a The blue (Typical) and red (ASD) curves represent the overall difference in MSE values ​​for facial complexity. Figure 6b The blue (Typical) and red (ASD) curves represent the difference in MSE values ​​of key eye points in relation to facial complexity. Figure 6c The blue curve (Typical) and the red curve (ASD) represent the difference in MSE values ​​of the nasal key points in terms of facial complexity.

[0119] Figure 7 This study demonstrates the differences in the trends of head motor multiscale entropy (MSE) between the typical developmental and autism spectrum disorder data groups.

[0120] based on Figures 4 to 7 The present invention allocates the training set and test set in an 8:2 ratio. In the training set, the automatic identification accuracy is 92%, sensitivity is 93.8%, specificity is 87.5%, image threshold is 0.7, true positive population / detected positive population is 0.94, the harmonic mean of precision and recall is 0.94, and the area under the ROC curve is 0.97. Figure 8 As shown.

[0121] In the test set, the automatic identification accuracy was 67%, sensitivity was 50%, specificity was 100%, image threshold was 0.7, true positive population / detected positive population ratio was 1.0, the harmonic mean of precision and recall was 0.67, and the area under the ROC curve was 0.88. Figure 9 As shown.

[0122] In a preferred embodiment, the system further includes a model optimization unit connected to the classification model. This unit updates the classification model based on the error between the prediction results of the classification model and the actual labels, and constructs a new model tree based on residual information to optimize the classification model by adjusting the leaf node values.

[0123] Specifically, variables such as the proportion of faces facing forward, the proportion of neutral emotions, and the complexity of the situation are input into the model. The LGB classification model is used for binary classification training, and the autism or non-autism label is output.

[0124] The execution code is as follows:

[0125] params = {'num_leaves':3, #number of leaf nodes}

[0126] 'min_data_in_leaf':3, # Data in each leaf node

[0127] 'objective':'binary', #Task: Binary Classification

[0128] 'max_depth':-1,#-1: No depth limit

[0129] "boosting_type":"gbdt",#

[0130] "metric":'auc', #metric

[0131] 'random_state':66,#random seed}

[0132] This invention first initializes the model, inputs data and minimizes its loss, while dividing the data into different nodes.

[0133] Then, the error between the predicted results and the actual labels is identified, and the model is updated accordingly.

[0134] A new model tree is constructed based on the residuals, and the leaf node values ​​are continuously adjusted to improve the model's ability to fit the data.

[0135] Specifically, this invention constructs a new model tree based on the residuals calculated in the previous step (i.e., the difference between the model prediction results of the previous round and the true labels).

[0136] During the construction of the new tree, the values ​​of the leaf nodes are continuously adjusted to optimize the classification model's ability to fit data samples under different feature combinations. This enables the classification model to more accurately determine the category of a child based on variables such as the proportion of faces facing forward, the proportion of neutral emotions, and the complexity of the input data. It can also reliably output labels for autism or non-autism, thus completing the task of assisting in the assessment of autism in children.

[0137] It should be noted that the autism or non-autism labels output by the system provided by this invention are only auxiliary data for reference and need to be input into other external medical diagnostic systems for use. The output of this invention is not a diagnostic result.

[0138] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. An artificial intelligence automatic recognition system based on facial expressions, characterized in that, The system comprises: an audio-visual stimulation module (1) configured to output evoking information for guiding a target subject to generate a specific emotion; a head detection module (3) connected to an information acquisition unit (2) configured to acquire head reaction information generated by the target subject based on the evoking information, the head detection module (3) being configured to detect the head reaction information, comprising: a face-forward proportion detection unit (31) connected to the information acquisition unit (2) and configured to calculate a proportion of a face-forward time length in a total time length and output face-forward proportion data; a neutral emotion proportion detection module (32) connected to the information acquisition unit (2) and configured to calculate a proportion of a neutral emotion time length in the total time length and output neutral emotion proportion data; a head complexity detection unit (33) connected to the information acquisition unit (2) and configured to calculate a head complexity condition of the target subject and output head complexity data; a label classification module (4) connected to the head detection module (3), the label classification module (4) being provided with a pre-trained classification model, the classification model being configured to output a classification label according to the face-forward proportion data, the neutral emotion proportion data and the head complexity data; the head complexity data output by the head complexity detection unit (33) comprises face complexity, gaze complexity and head posture complexity; the head complexity detection unit (33) comprises: a head posture estimation model (331) connected to the information acquisition unit (2) and configured to output a head posture of each video frame in the head reaction information; a gaze detection model (332) connected to the information acquisition unit (2) and configured to detect a gaze vector of each video frame in the head reaction information; a face key point detection model (333) connected to the information acquisition unit (2) and configured to detect a face key point of each video frame in the head reaction information; the classification label output by the classification model comprises an autism label and a non-autism label. 2.The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The audio-visual stimulation module (1) guides the target subject to generate at least happy emotion, surprise emotion and anger emotion, the information acquisition unit (2) is a camera arranged in a direction of a face of the target subject, and the head reaction information is a head reaction video captured by the camera. 3.The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The face-forward proportion detection unit (31) is provided with a head posture estimation model (331) based on a convolutional neural network, the head posture estimation model (331) outputs a head posture feature corresponding to each video frame by traversing each video frame of the head reaction information, labels a corresponding picture according to the head posture feature, and labels the picture as a forward-looking video frame and a non-forward-looking video frame; The face-forward proportion data is obtained based on a proportion of the forward-looking video frames in a total number of frames.

4. The facial expression based artificial intelligence automatic identification system according to claim 3, wherein, The forward-looking video frame needs to meet a condition that an included angle between a pitch angle and a yaw angle of a head posture of the target subject is less than or equal to a preset angle. 5.The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The neutral emotion proportion detection module (32) is provided with an emotion detection model based on a convolutional neural network, and the emotion detection model is pre-trained based on a data set containing face expression information. The emotion detection model outputs an emotion label corresponding to each video frame by traversing each video frame of the head reaction information, labels the corresponding video frame as a neutral emotion video frame and other emotion video frame according to the emotion label, and obtains the neutral emotion proportion data based on a proportion of the neutral emotion video frame in a total number of frames. 6.The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The head complexity detection unit (33) further comprises, A head posture complexity calculation subunit (334) connected to the head posture estimation model (331) and configured to calculate an angular velocity of head swing according to the head posture and obtain angular velocity sequence information as the head posture complexity; A line-of-sight complexity calculation subunit (335) connected to the line-of-sight detection model (332) and configured to calculate an Euclidean distance of the line-of-sight vectors of adjacent two video frames and obtain line-of-sight difference value sequence information as the line-of-sight complexity; A face complexity calculation subunit (336) connected to the face key point detection model (333) and configured to calculate an Euclidean distance of changes of the face key points of adjacent two video frames and obtain face Euclidean distance sequence information as the face complexity; A sequence information processing subunit (337) connected to the head posture complexity calculation subunit (334), the line-of-sight complexity calculation subunit (335) and the face complexity calculation subunit (336) and configured to convert the angular velocity sequence information, the line-of-sight difference value sequence information and the face Euclidean distance sequence information into multi-scale entropy.

7. The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The classification model adopts a machine learning model based on a gradient boosting framework and is trained by using a binary classification task. 8.The facial expression based artificial intelligence automatic identification system according to claim 1, wherein, The model optimization unit is further connected to the classification model and configured to update the classification model based on an error between a prediction result of the classification model and an actual label and to construct a new round of model tree based on residual information and optimize the classification model by adjusting leaf node values. The model optimization unit is further connected to the classification model and configured to update the classification model based on an error between a prediction result of the classification model and an actual label and to construct a new round of model tree based on residual information and optimize the classification model by adjusting leaf node values.

Images