Autism behavior characteristic prediction method based on head posture analysis

Through head posture analysis and time series LSTM network model based on dlib and OpenCV, the problem of diagnostic uncertainty in early screening of autism is solved, efficient and accurate prediction of behavioral characteristics is achieved, adapting to different individuals and environments, and providing powerful generalization ability and non-invasive prediction.

CN119069115APending Publication Date: 2024-12-03SHANGHAI UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411230028.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing technology lacks accurate and objective methods in early screening of autism. Traditional diagnosis relies on subjective judgment of professional doctors and fails to effectively utilize the characteristics of children's head posture. The existing video analysis technology is in real-time processing and algorithm interpretability. There are shortcomings in terms of aspects.

Method used

Using head pose analysis technology based on dlib and OpenCV, combined with time series LSTM deep learning network model, children's head pose characteristics are extracted from videos, and autism-related behavioral characteristics are identified through preprocessing and model training, and backpropagation is used. and gradient descent optimize model parameters to ensure model generalization capabilities and prediction accuracy.

Benefits of technology

A non-invasive and accurate prediction of autism behavior characteristics is achieved, which improves prediction accuracy and model generalization ability, adapts to different individuals and environments, and reduces potential interference to children.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119069115A_ABST
    Figure CN119069115A_ABST
Patent Text Reader

Abstract

The invention relates to a head posture analysis-based autism behavior characteristic prediction method, which belongs to the field of autism and comprises the following steps of S1, establishing a preprocessing and training model; s2, data preprocessing; s3, classification data set division; s4, performing model training test; and S5, verifying a behavior characteristic result. According to the autism behavior characteristic prediction method based on the head posture analysis, training data is provided for an LSTM deep learning network model by using the head posture characteristics of a child, and a human face is accurately detected and tracked on the level of a time sequence, so that the model can accurately capture the dynamic change of the head posture, the behavior characteristics of the child are effectively predicted, and the prediction accuracy of the autism behavior characteristics is improved. And the prediction accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to artificial intelligence and medical health, and in particular to the application of deep learning and video analysis technology in early screening of autism. By analyzing video data of children in a natural interactive environment and extracting their head posture features, we are committed to developing an accurate and objective early screening method for autism. Background Art

[0002] Although with the continuous advancement of computer vision and image processing technology, video analysis technology has become one of the hot spots of research and application, and the development of video analysis technology provides us with powerful tools to enable us to extract useful information from complex video data and make accurate predictions and analyses. However, with the diversification and complexity of application scenarios, video analysis technology also faces many challenges. Machine learning methods perform poorly in processing feature extraction and pattern recognition of video data. Although deep learning methods have solved this problem to a certain extent, their model training process requires a large amount of data and computing resources, and has high requirements for data quality and annotation accuracy. Although the current solutions have made some progress in improving the accuracy and performance of the algorithms, there are still some problems. The existing technologies often ignore the requirements for the interpretability of the algorithms, and there is a problem of insufficient adaptability in some application scenarios. In addition, the existing technologies are also insufficient in real-time processing and analysis capabilities of video data.

[0003] In the field of healthcare, early diagnosis and intervention are crucial to the treatment of many diseases. Autism is a neurodevelopmental disorder that can be effectively diagnosed and intervened in early childhood, which will help improve the quality of life of children and enhance their social adaptability. However, the diagnosis and treatment of autism face many challenges.

[0004] Traditional diagnostic methods often rely on the subjective judgment and observation of professional doctors, which are subject to diagnostic uncertainty and bias. Diagnosis is made only through the visual eye movement characteristics of children, without considering that the head posture of the child is also different from that of children with autism and normally developing children. There is a lack of an accurate, fast and objective method for early screening of autism. Summary of the invention

[0005] 1. Technical issues to be resolved The purpose of the present invention is to solve the problem that the existing technology can only use the Childhood Autism Rating Scale (CARS) and the Childhood Autism Behavior Checklist (ABC), or only diagnose through the visual eye movement characteristics of the child, without considering that the head posture of the child is also the difference between autistic children and normally developing children. A method for predicting autism behavioral characteristics based on head posture analysis is proposed. It uses dlib and OpenCV technology to recognize faces from videos and extract head posture features, thereby proposing a video analysis and time series LSTM deep learning network model for identifying behavioral characteristics related to autism. It can accurately capture the dynamic changes of head posture while effectively predicting children's behavioral characteristics, thereby improving the accuracy of prediction.

[0006] (II) Technical solution In order to achieve the above-mentioned purpose of improving the accuracy of prediction, the present invention provides the following technical solution: a method for predicting autism behavior characteristics based on head posture analysis, comprising the following steps: S1. Establishing preprocessing and training models: The preprocessing subsystem can automatically identify faces from videos and extract head posture features, including yaw, pitch, and roll angles. The posture features are then used to train a deep learning model based on a long short-term memory (LSTM) network, which is specially designed to process and analyze time series data to identify potential behavioral patterns; S2. Data preprocessing: Standardize the extracted head posture features to make all feature data consistent and comparable before entering the model training, so as to improve the efficiency of model training and the accuracy of prediction; S3, classification data set division, the sample data extracted from the video in step S1 is randomly divided into a training set and a validation set according to a certain ratio; S4, Model training and testing: The preprocessed data is used to train the LSTM model; the model identifies behavioral characteristics related to autism by learning the time series data of head posture; during the training process, algorithms such as back propagation and gradient descent are used to continuously optimize the model parameters, and early stopping technology is used to prevent overfitting; the cross entropy loss function is used in the model training process, focusing on optimizing the model to reduce classification prediction errors; S5. Verification of behavioral feature results: The model is verified on an independent test set. The model is comprehensively evaluated using accuracy, recall, F1 score and other evaluation indicators to ensure that the model has good classification efficiency and generalization ability.

[0007] Preferably, in the step S1, the introduction of the LSTM network provides the model with a powerful ability to process time series data, enabling it to capture the dynamic changes of head posture over time, thereby effectively predicting children's behavioral characteristics.

[0008] Preferably, in the step S2, the number of frames for reading video data is to read a group of data every 25 frames, so that data can be read effectively and the system is highly resistant to environmental interference.

[0009] Preferably, the data preprocessing stage in S2 includes accurate detection and tracking of faces in the video to ensure accurate extraction of head posture features, laying a solid foundation for subsequent model training.

[0010] Preferably, in step S3, the classification data set has 2 categories and 46 samples.

[0011] Preferably, in step S3, the ratio of the classified training set to the validation set is 7:3.

[0012] Preferably, in step S4, a cross entropy loss function is defined as the loss function of the model, and an Adam optimizer is used to optimize the parameters of the model. The learning rate defaults to 0.001, and the initial running loss is 0; for the features and labels in the data loader, they are moved to the specified device; the gradient of the optimizer is reset to zero, and the features are passed to the model to obtain the output of the model; the loss between the model output and the label is calculated, the loss is back-propagated and the model parameters are updated, and the current average loss is calculated.

[0013] Preferably, in step S5, the behavior feature result verification module verifies the performance of the model on an independent test set, and uses evaluation indicators such as accuracy, recall, F1 score, etc. to perform a comprehensive evaluation, wherein the evaluation of the confusion matrix and the area under the ROC curve AUC is used to increase the predictive perspective of the model.

[0014] (III) Beneficial effects Compared with the prior art, the present invention provides a method for predicting autism behavior characteristics based on head posture analysis, which has the following beneficial effects: This method for predicting autism behavioral characteristics based on head posture analysis has the following features: non-invasive prediction: predicting children's behavioral characteristics by analyzing video data, without the need for direct interaction with children, reducing potential interference to children; efficient and accurate behavioral characteristic analysis: using LSTM network to process time series data, it can accurately capture the dynamic changes of head posture and improve prediction accuracy; strong generalization ability: through cross-validation on independent test sets, it ensures that the model can maintain high performance on unseen data and adapt to different individuals and environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a method for predicting behavioral characteristics of autism based on head posture analysis proposed by the present invention; Figure 2 This is a schematic diagram of the network structure of a method for predicting autism behavioral characteristics based on head posture analysis proposed by the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Please see attached Figure 1-2 , a method for predicting autism behavior characteristics based on head posture analysis, comprising the following steps: S1. Establish a preprocessing and training model preprocessing subsystem. This subsystem is based on advanced dlib and OpenCV technologies and can automatically recognize faces from videos and extract head posture features, including yaw, pitch, and roll angles. These key features are then used to train a deep learning model based on a long short-term memory (LSTM) network, which is specially designed to process and analyze time series data to identify potential behavior patterns. In the LSTM network module, units with memory functions are used to capture the dynamic changes of head posture over time, and the back-propagation and gradient descent algorithms are used to continuously optimize the model parameters. At the same time, the early stopping technique is applied to prevent overfitting and improve the generalization ability and prediction accuracy of the model.

[0018] S2. Data preprocessing: This step focuses on thoroughly standardizing the extracted head posture features to ensure that all feature data remain consistent and comparable before entering the model training. This step is crucial to improving the efficiency of model training and the accuracy of prediction.

[0019] S3, classification data set division, the 46 sample data extracted from the video in step S1 are randomly divided into a training set and a validation set according to a certain ratio; S4. Model training and testing: In this stage, the preprocessed data is used to train the LSTM model. The model learns the time series data of head posture to identify behavioral characteristics related to autism. During the training process, algorithms such as back propagation and gradient descent are used to continuously optimize the model parameters, and early stopping techniques are used to prevent overfitting. The cross entropy loss function is used during the model training process, focusing on optimizing the model to reduce classification prediction errors and further improve the learning efficiency of the model.

[0020] S5. Verification of behavioral feature results: Finally, the model is validated on an independent test set. The model is comprehensively evaluated using accuracy, recall, F1 score and other evaluation indicators to ensure that the model has good classification efficiency and generalization ability.

[0021] In step S1, set the specified video file to process one frame every 25 frames, and define the positions of key points including the nose tip, chin, eyes, and mouth corners. For each frame of the video file, convert it to a grayscale image and use a face detector to detect the face. For each detected face, use the face key point detector to obtain the key points. Construct a 3X4 projection matrix and obtain the yaw, pitch, and roll angles from it. Use the camera intrinsic parameter matrix and the PnP algorithm to solve the problem of head posture, and obtain the rotation vector and translation vector. Write the posture angle of each face and a predefined label value (set to 0 here) into a CSV file. Update the frame count, release the video capture object, and end the processing. Create a custom dataset that can process head posture data, define the layers of the LSTM model, and the data is forward propagated in the model.

[0022] In step S2, the angle data in each CSV file is standardized. For the yaw and roll angles, the angles are adjusted between -180° and 180°; for the pitch angle, the angle is adjusted between -90° and 90°. If the length of the data sequence is less than the standard length, it is padded with zeros; if it is greater than the standard length, it is truncated.

[0023] In step S3, after multiple training and evaluations, the video datasets of autistic children and normal children are divided, with 70% used as a training set and 30% as a validation set.

[0024] In step S4, define the cross entropy loss function as the loss function of the model. Use the Adam optimizer to optimize the parameters of the model, and the learning rate defaults to 0.001. Initialize the running loss to 0; for the features and labels in the data loader, move them to the specified device; reset the optimizer's gradient to zero, pass the features to the model, and get the model's output; calculate the loss between the model output and the label, back-propagate the loss and update the model parameters, and calculate the current average loss.

[0025] In step S5, for a given data loader, the features and labels are moved to the specified device. The features are input into the model, the output of the model is obtained, and the prediction value and loss value are calculated based on the output. Using the prediction value and the true label, the accuracy, precision, recall rate, F1 score and other evaluation indicators of the model on the data set are calculated, as well as the comparison between a part of the prediction value and the true label.

[0026] Furthermore, on the existing training set, the autism behavioral characteristics prediction method based on head posture analysis proposed in the present invention achieved 76.67% accuracy, 75% precision, 80% recall, and 77.42% F1 score; on the existing test set, it achieved 75% accuracy, 60% precision, 100% recall, and 75% F1 score.

[0027] Laboratory conditions for algorithm testing: There are 21 video files of autistic children, including 15 training sets and 6 test sets; there are 25 video files of normal children, including 15 training sets and 10 test sets.

[0028] The beneficial effects of the present invention are as follows: the method for predicting behavioral characteristics of autism based on head posture analysis has non-invasive prediction: predicting children's behavioral characteristics by analyzing video data, without direct interaction with children, reducing potential interference to children; efficient and accurate behavioral characteristic analysis: using LSTM network to process time series data, it can accurately capture the dynamic changes of head posture and improve prediction accuracy; strong generalization ability: through cross-validation on independent test sets, it ensures that the model can maintain high performance on unseen data and adapt to different individuals and environments. Typical cases: Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting autism behavior characteristics based on head posture analysis, characterized in that: The following steps are involved: S1. Establish preprocessing and training model: The preprocessing subsystem can automatically recognize faces from videos and extract head posture features, including yaw, pitch, and roll angles. The posture features are then used to train a deep learning model based on a long short-term memory (LSTM) network, which is specially designed to process and analyze time series data to identify potential behavior patterns. S2. Data preprocessing: Standardize the extracted head posture features to make all feature data consistent and comparable before entering the model training, so as to improve the efficiency of model training and the accuracy of prediction; S3, classification data set division, the sample data extracted from the video in step S1 is randomly divided into a training set and a validation set according to a certain ratio; S4. Model training and testing: The preprocessed data is used to train the LSTM model; the model learns the time series data of head posture to identify behavioral characteristics related to autism; During the training process, algorithms such as back propagation and gradient descent are used to continuously optimize model parameters, while early stopping techniques are used to prevent overfitting. During the model training process, the cross entropy loss function is used to focus on optimizing the model to reduce classification prediction errors. S5. Verification of behavioral feature results: The model is verified on an independent test set. The model is comprehensively evaluated using accuracy, recall, F1 score and other evaluation indicators to ensure that the model has good classification efficiency and generalization ability.

2. The method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S1, the preprocessing and training model construction module consists of a face detection module, a head posture feature extraction module, and a long short-term memory network module. The face detection module uses dlib and OpenCV technology to perform face recognition on video frames. The head posture feature extraction module is responsible for extracting the yaw, pitch and roll angles of the head from the recognized face. These features are then fed into the LSTM network module, which is specially designed to analyze time series data to identify potential autistic behavioral characteristics.

3. A method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S1, the head posture feature extraction module reads a set of data every 25 frames to enhance the model's resistance to environmental interference.

4. The method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S2, the data preprocessing stage includes standardizing the head posture features to ensure that all feature data remain consistent and comparable before entering the LSTM model training, thereby improving the efficiency of model training and the accuracy of prediction.

5. A method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: The data preprocessing stage in step S2 includes accurate detection and tracking of faces in the video to ensure accurate extraction of head posture features; standardization of each angle data, adjusting the yaw and roll angles between -180° and 180°; adjusting the pitch angle between -90° and 90°. If the data sequence length is less than the standard length, zero padding is performed; if it is greater than the standard length, truncation is performed.

6. The method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S3, the classification data set division module is used to randomly divide the sample data extracted from the video into a training set and a validation set according to a certain ratio, wherein the ratio of the training set to the validation set is preferably 7:

3.

7. A method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S3, the classification data set contains 46 samples divided into 2 categories to support the model's prediction of autism behavioral characteristics.

8. The method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S1, in the LSTM network module, units with memory functions are used to capture the dynamic changes of head posture over time, and back propagation and gradient descent algorithms are used to continuously optimize model parameters.

9. A method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S4, the cross entropy loss function is defined as the loss function of the model, and the Adam optimizer is used to optimize the parameters of the model. The learning rate defaults to 0.001, and the initial running loss is 0; for the features and labels in the data loader, move them to the specified device; reset the gradient of the optimizer, and pass the features to the model to get the output of the model; calculate the loss between the model output and the label, back-propagate the loss and update the model parameters, and calculate the current average loss.

10. A method for predicting autism behavior characteristics based on head posture analysis according to claim 1, characterized in that: In step S5, the behavior feature result verification module verifies the performance of the model on an independent test set and uses accuracy, recall, F1 score and other evaluation indicators for comprehensive evaluation. The evaluation of the confusion matrix and the area under the ROC curve AUC is used to increase the prediction perspective of the model.