Autism spectrum disorder detection system based on eye movement recognition

Abstract

The invention relates to the technical field of autism detection, in particular to an autism spectrum disorder detection system based on eye movement recognition, which comprises a video acquisition module, a data preprocessing module, a sight line drop point analysis module, a data correction module, a recognition model module and a month age association evaluation module, the data preprocessing module is used for determining a target user, the sight falling point analysis module is used for forming a sight falling point time sequence, the data correction module is used for correcting the sight falling point time sequence, and the recognition model module is used for outputting a result. According to the system and the method, the user is guided to watch various types of interactive videos through the video acquisition module to obtain rich eye movement related data, and visual performance in different scenes is integrated, so that the detection basis is more comprehensive, the autism spectrum disorder detection accuracy is improved, and autism spectrum disorder and typical development conditions are distinguished more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of autism detection, and in particular to an autism spectrum disorder detection system based on eye movement recognition. Background Art

[0002] Autism Spectrum Disorder (ASD), also known as autism, is a type of neurodevelopmental disorder that begins in early childhood, with two major clinical features: social interaction disorders, language communication disorders, and repetitive stereotyped behaviors and narrow interests. Patients often suffer from lifelong impaired social and cognitive functions, which brings great mental stress and economic burden to their families. In recent years, the global incidence of autism in children has shown a significant upward trend. As of 2023, the Centers for Disease Control and Prevention in the United States reported that the prevalence of autism reached 1 / 361; the incidence of autism in my country has also risen rapidly.

[0003] In order to help families with children with autism to detect, intervene and treat the disease early, it is necessary to conduct extensive screening of autistic patients and suspected autistic patients among preschool children. The current difficulty in early screening of ASD is that for patients in the infant and toddler period before 18 months of age, their behavioral manifestations are not typical and their parents are not sensitive to the reporting scale, which may lead to missed screening of ASD risk groups.

[0004] In addition to the organized screening by the hospital, it is also hoped that self-help auxiliary testing can be carried out to expand the scope of screening as much as possible. Therefore, it is necessary to provide a testing system that can not only test and diagnose patients of appropriate age, but also provide risk assessment prompts for younger patients. It is not only suitable for screening with the participation of doctors, but also suitable for self-help testing by users. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides an autism spectrum disorder detection system based on eye movement recognition, which can effectively solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: The present invention provides an autism spectrum disorder detection system based on eye movement recognition, comprising a video acquisition module, a data preprocessing module, a sight point analysis module, a data correction module, a recognition model module and a month-age correlation assessment module; The video acquisition module is used to guide the user to watch the interactive video and synchronously record the viewing process to obtain the user video; The data preprocessing module is used to identify the faces of people in the user video, determine the target user by age judgment, and block the faces of people other than the target user in the user video; The sight point analysis module is used to identify each frame of the user's video, extract facial images, eye images and facial position information, and integrate them to form a sight point time series; The data correction module is used to correct the time series of the sight point according to the device model and orientation information of the recording user video, and the device model and orientation information are automatically obtained when the user video is recorded; The recognition model module is used to input the corrected sight point time series into the recognition model including the third convolutional neural network, and output the classification prediction results about autism spectrum disorder and typical development; The monthly age-related assessment module receives the user's monthly age information, compares the monthly age with a set threshold, performs early screening evaluation and divides the risk level when the monthly age is lower than the threshold, and outputs an autism spectrum disorder detection result when the monthly age is equal to or higher than the threshold.

[0007] Furthermore, the interactive video is a variety of interactive task videos, including blank overlapping videos, free view videos, guardian gaze videos and spatial clue videos; the blank overlapping videos are one or more target stimuli, which are images, graphics or simple symbols. After a preset time interval, one or more blank areas or new interference stimuli are superimposed on the previous target stimuli; The free view video is an image with multiple semantic information, and the content of the image includes people, animals, scenery and objects; The guardian watching video is a photo, video clip or real-time video stream of the user's guardian, including a single photo of the guardian and a photo of the guardian and the user; The spatial clue video is a video in which preset spatial clues are set in visual stimulation. The spatial clues include arrows, color marks, and flashing areas. The target is set at a position that has a spatial relationship with the spatial clues.

[0008] Further, the data preprocessing module includes a facial recognition unit, a target user determination unit and a facial shielding unit; The facial recognition unit adopts a face detection algorithm based on deep learning to analyze the acquired user video frame by frame, locate the facial area of ​​the person in the video, mark the corresponding coordinate information, distinguish the faces of different individuals, and obtain the facial features of the person; The target user determination unit determines the target user, i.e., the subject of the autism spectrum disorder test, by analyzing the age-related features presented on the face based on the identified facial features of the person; The face shielding unit shields the faces of people other than the target user in the user video by using image blurring processing.

[0009] Furthermore, the sight point analysis module includes an image recognition unit and a time series integration unit; The image recognition unit recognizes and segments each frame of the user video through a first convolutional neural network, and extracts facial images, eye images and facial position information; The time series integration unit predicts the sight point of each frame of image through a second convolutional neural network, and integrates to form a sight point time series.

[0010] Further, the data correction module includes a device information acquisition unit and a correction execution unit; The device information acquisition unit automatically acquires the device model and orientation information of the device used to record the user video while the user video is being recorded, and accurately records the information; The correction execution unit obtains the device model and orientation information and uses a correction algorithm set for different device characteristics to perform correction operations on the formed sight point time series to eliminate the sight point data deviation.

[0011] Furthermore, the recognition model module includes a data input unit and a model operation unit; The data input unit is used to organize and prepare the corrected sight point time series data according to the format required by the recognition model; The model operation unit inputs the prepared data into the recognition model including the third convolutional neural network, starts the model's operation process, extracts features and makes classification judgments on the input data through the convolutional layer, pooling layer, and fully connected layer within the model, and uses the pre-trained weight parameters and classification logic of the model to output classification prediction results about autism spectrum disorder and typical development.

[0012] Further, the moon age association evaluation module includes a moon age receiving unit and a comparison evaluation unit; The age-in-month receiving unit is used to receive the age-in-month information about the target user to ensure the accuracy and completeness of the information; The comparison and evaluation unit is used to compare and analyze the received user age information with a pre-set threshold. If the age is lower than the threshold, the user's autism spectrum disorder risk is evaluated according to the standards and logic of early screening evaluation, and the corresponding risk levels are divided, including low risk, medium risk and high risk; if the age is equal to or higher than the threshold, the autism spectrum disorder detection result obtained based on the recognition model module is directly output.

[0013] Furthermore, the construction of the recognition model includes: Acquire a data set, the data set including a plurality of training samples, each of the training samples including a time series of a target user's sight point obtained by collecting the user's sight point from a user video of the target user and a training user classification label; Inputting the time series of the user's gaze points in the training samples into a convolutional neural network model with initial model parameters for training, and obtaining classification prediction results on autism spectrum disorder and typical development; Comparing the obtained classification prediction results of the training samples regarding autism spectrum disorder and typical development with the target user classification labels of the training samples, thereby updating model parameters; The convolutional neural network model is trained successively using all training samples, and the model parameters are updated successively, so as to finally obtain the autism spectrum disorder recognition model.

[0014] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements an autism spectrum disorder detection system based on eye movement recognition.

[0015] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects: (1) The present invention guides users to watch various types of interactive videos through the video acquisition module to obtain rich eye movement related data, and integrates the visual performance in different scenes to make the detection basis more comprehensive, which helps to improve the accuracy of autism spectrum disorder detection and more accurately distinguish autism spectrum disorder from typical development.

[0016] (2) The present invention performs early screening evaluation and divides the risk level for children at a relatively low age, and outputs corresponding test results when the age is higher, fully considering the differences in developmental characteristics of children of different age groups, so that the test is more in line with the actual individual situation and more targeted.

[0017] (3) The present invention has a high degree of automation. The system can automatically obtain device-related information for data correction during video recording. During the operation of each module, it can also automatically complete a series of operations such as facial recognition, gaze point analysis, model calculation, etc. according to the established process, reducing the tediousness of manual intervention, improving detection efficiency, and facilitating practical application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is a system flow chart of the present invention; Figure 2 This is a schematic diagram of the association constructed by the recognition model of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The present invention will be further described below in conjunction with the embodiments.

[0022] Example: Reference Figure 1 to Figure 2 , an autism spectrum disorder detection system based on eye movement recognition, including a video acquisition module, a data preprocessing module, a gaze point analysis module, a data correction module, a recognition model module and a month-age correlation assessment module; The video acquisition module is used to guide users to watch interactive videos and simultaneously record the viewing process to obtain user videos; The data preprocessing module is used to identify the faces of people in the user video, determine the target user by age, and block the faces of people other than the target user in the user video; The gaze point analysis module is used to identify each frame of the user's video, extract facial images, eye images and facial position information, and integrate them to form a gaze point time series; The data correction module is used to correct the time series of the sight point according to the device model and orientation information of the user video. The device model and orientation information are automatically obtained when the user video is recorded; The recognition model module is used to input the corrected gaze point time series into the recognition model including the third convolutional neural network, and output the classification prediction results about autism spectrum disorder and typical development; The monthly age-related assessment module receives the user's monthly age information, compares the monthly age with the set threshold, conducts early screening evaluation and divides the risk level when the monthly age is lower than the threshold, and outputs the autism spectrum disorder detection results when the monthly age is equal to or higher than the threshold.

[0023] Furthermore, the interactive video is a variety of interactive task videos, including blank overlapping videos, free view videos, guardian gaze videos and spatial clue videos; the blank overlapping videos are one or more target stimuli, the stimuli are images, graphics or simple symbols, and after a preset time interval, one or more blank areas or new interference stimuli are superimposed on the previous target stimuli; Free-view videos are images with a variety of semantic information, including people, animals, scenery, and objects; The guardian watching video is a photo, video clip or real-time video stream of the user's guardian, including a single photo of the guardian or a photo of the guardian and the user; Spatial cue videos are videos that set preset spatial cues in visual stimulation. The spatial cues include arrows, color marks, and flashing areas. The target is set at a position that has a spatial relationship with the spatial cues.

[0024] Further, the data preprocessing module includes a facial recognition unit, a target user determination unit, and a facial occlusion unit; The facial recognition unit uses a deep learning-based face detection algorithm to analyze the acquired user video frame by frame, locate the facial area of ​​the person in the video, mark the corresponding coordinate information, distinguish the faces of different individuals, and obtain the facial features of the person; The target user determination unit determines the target user, i.e., the subject of the autism spectrum disorder test, by analyzing the age-related features presented on the face based on the recognized facial features of the person; The face occlusion unit occludes the faces of people other than the target user in the user video by using image blurring processing.

[0025] Furthermore, the sight point analysis module includes an image recognition unit and a time series integration unit; The image recognition unit recognizes and segments each frame of the user video through the first convolutional neural network, and extracts the facial image, eye image and facial position information; The time series integration unit predicts the gaze point of each frame of the image through the second convolutional neural network and integrates them to form a time series of the gaze point.

[0026] Furthermore, for different types of interactive videos, the specific method of obtaining the time series of the gaze points is as follows: Blank overlapping video: When each frame of the blank overlapping video is input into the first convolutional neural network, the network first extracts the overall features of the image, identifies the target stimulus (image, graphic or simple symbol) in the image and the rough outline and position information of the blank area or interference stimulus that is subsequently superimposed; using the segmentation ability of the convolutional neural network, the target stimulus, blank area and interference stimulus are segmented from the image background, and the pixel range and coordinate information of each in the image are determined. At the same time, in terms of facial image extraction, the blank overlapping video mainly focuses on the target stimulus and user reaction. Facial image extraction ensures that the user's face can still be accurately locked in the presence of interference elements. The first convolutional neural network can accurately extract clear facial images by learning facial features in a large number of different scenarios, and mark the position of the face in the image, including the relative position relationship relative to the target stimulus and interference elements, and the precise position of the eye image in the facial image.

[0027] Free-view video: For images containing people, animals, scenery, and objects in free-view videos, the first convolutional neural network performs multi-level feature extraction. At the lower level, the network captures the basic visual features of the image's edges and textures to distinguish the boundaries and shapes of different objects. At a higher level, through semantic understanding capabilities, it identifies different semantic regions of people, animals, scenery, and objects in the image, and segments and annotates each region.

[0028] Guardian watching video: When the guardian watching video is input, including the guardian's single photo and the guardian and the user's photo, the first convolutional neural network focuses on analyzing the characteristics of the characters in the picture. First, the guardian's facial features and body posture are identified, distinguished from the user's facial features, and the guardian's area in the picture is accurately segmented. For the extraction of the user's facial image, it is necessary to ensure accurate recognition when it coexists with the guardian's image, extract the eye image at the same time, and record the facial position information in detail, including the relative position to the guardian's face, the coordinate position in the picture, and the distance from the picture boundary.

[0029] Spatial clue video: Arrows, color marks, flickering area spatial clues and target settings in spatial clue videos are the focus of the first convolutional neural network recognition. These spatial clues and targets are quickly located in the image, and their feature information is accurately segmented and extracted, including the direction of the arrow, the color value and area range of the color mark, and the frequency and position of the flickering area. When extracting facial images and eye images, the distribution of spatial clues and targets in the picture is analyzed to determine the relative relationship between the facial position and these elements.

[0030] Furthermore, different types of interactive videos are passed through the first convolutional neural network and then through the second convolutional neural network to predict the sight point of each frame image, and the sight point time series is integrated to obtain the sight point time series of the blank overlapping video. , the time series of gaze points in free-view video , the time series of the guardian's gaze points in the video , the time series of gaze points in spatial cue videos , splicing different types of interactive videos to obtain the final gaze point time series .

[0031] Further, the data correction module includes a device information acquisition unit and a correction execution unit; The device information acquisition unit automatically acquires the model of the device used to record the user video and the orientation information of the device while the user video is being recorded, and accurately records the information; The correction execution unit uses the device model and orientation information obtained, and uses the set correction algorithm for different device characteristics to correct the formed sight point time series, eliminate the sight point data deviation, and obtain the corrected sight point time series data. .

[0032] Further, the recognition model module includes a data input unit and a model operation unit; The data input unit is used to organize and prepare the corrected sight point time series data according to the format required by the recognition model; The model operation unit inputs the prepared data into the recognition model containing the third convolutional neural network, starts the model's operation process, extracts features and classifies the input data through the convolutional layer, pooling layer, and fully connected layer within the model, and uses the model's pre-trained weight parameters and classification logic to output classification prediction results about autism spectrum disorder and typical development.

[0033] Furthermore, the third convolutional neural network structure and processing flow are as follows: First, the convolution layer operation is performed to convert the corrected sight point time series data The data is input into the convolution layer of the third convolutional neural network. The convolution kernel in the convolution layer extracts the associations and patterns between different local features by sliding on the feature vector. The smaller convolution kernel can capture the changing rules of the sight point features in a short time range, while the larger convolution kernel can learn the overall sight trend features in a longer time span.

[0034] The result of sliding calculation of each convolution kernel on the feature vector constitutes a feature map. Through the operation of multiple convolution kernels, multiple different feature maps are generated. These feature maps together represent the multi-level feature representation of the time series of the gaze point. After the convolution operation, the ReLU activation function is applied to introduce nonlinear factors, so that the network can learn the relationship between more complex gaze point features and autism spectrum disorders.

[0035] The pooling layer is then used, and the feature map obtained by the convolution layer then enters the pooling layer. The pooling layer performs dimensionality reduction on the feature map, reducing the amount of data while retaining the main feature information.

[0036] Finally, the fully connected layer and classification output are performed. After multiple convolution and pooling layers are processed alternately, the feature map is flattened into a one-dimensional vector and then input into the fully connected layer. In the last layer of the fully connected layer, the Softmax activation function is used to convert the output into a probability distribution about autism spectrum disorder and typical development. Let the output vector be ,in Corresponding to the score of autism spectrum disorder, The probability output of the score corresponding to typical development after the Softmax function , the calculation formula is:

[0037]

[0038] in, represents the probability of autism spectrum disorder, Represents the probability of typical development. According to the size of the probability value, the final classification prediction result is determined, that is, if , then autism spectrum disorder is predicted; if , typical development is predicted.

[0039] Further, the moon age association evaluation module includes a moon age receiving unit and a comparison evaluation unit; The month age receiving unit is used to receive the month age information about the target user to ensure the accuracy and completeness of the information; The comparison and evaluation unit is used to compare and analyze the received user age information with a pre-set threshold. If the age is lower than the threshold, the user's autism spectrum disorder risk is evaluated according to the standards and logic of early screening evaluation, and the corresponding risk levels are divided, including low risk, medium risk and high risk; if the age is equal to or higher than the threshold, the autism spectrum disorder detection result obtained based on the recognition model module is directly output.

[0040] Furthermore, the construction of the recognition model includes: Obtain a data set, the data set including multiple training samples, each training sample including a time series of a target user's gaze point obtained by collecting the user's gaze point from a user video of the target user and a training user classification label; The time series of the user's gaze points in the training samples are input into a convolutional neural network model with initial model parameters for training, and classification prediction results on autism spectrum disorder and typical development are obtained; Comparing the obtained classification prediction results of the training samples regarding autism spectrum disorder and typical development with the target user classification labels of the training samples, thereby updating the model parameters; All training samples are used to train the convolutional neural network model one by one, and the model parameters are updated one by one, and finally an autism spectrum disorder recognition model is obtained.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements an autism spectrum disorder detection system based on eye movement recognition.

[0042] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. Autism spectrum disorder detection system based on eye movement recognition, characterized in that: It includes video acquisition module, data preprocessing module, sight point analysis module, data correction module, recognition model module and moon age correlation assessment module; The video acquisition module is used to guide the user to watch the interactive video and synchronously record the viewing process to obtain the user video; The data preprocessing module is used to identify the faces of people in the user video, determine the target user by age judgment, and block the faces of people other than the target user in the user video; The sight point analysis module is used to identify each frame of the user's video, extract facial images, eye images and facial position information, and integrate them to form a sight point time series; The data correction module is used to correct the time series of the sight point according to the device model and orientation information of the user video, and the device model and orientation information are automatically obtained when the user video is recorded; The recognition model module is used to input the corrected sight point time series into the recognition model including the third convolutional neural network, and output the classification prediction results about autism spectrum disorder and typical development; The age-related assessment module receives the user's age information, compares the age with a set threshold, performs early screening evaluation and divides the risk level when the age is below the threshold, and outputs an autism spectrum disorder detection result when the age is equal to or above the threshold.

2. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The interactive video is a variety of interactive task videos, including blank overlapping videos, free view videos, guardian gaze videos and spatial clue videos; the blank overlapping videos are one or more target stimuli, which are images, graphics or simple symbols. After a preset time interval, one or more blank areas or new interference stimuli are superimposed on the previous target stimuli; The free view video is an image with multiple semantic information, and the content of the image includes people, animals, scenery and objects; The guardian watching video is a photo, video clip or real-time video stream of the user's guardian, and the user's guardian photo includes a single photo of the guardian and a photo of the guardian and the user; The spatial clue video is a preset spatial clue set in the visual stimulation, and the spatial clue includes an arrow, a color mark, and a flashing area. The target is set at a position that has a spatial relationship with the spatial clue.

3. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The data preprocessing module includes a facial recognition unit, a target user judgment unit and a facial shielding unit; The facial recognition unit adopts a face detection algorithm based on deep learning to analyze the acquired user video frame by frame, locate the facial area of ​​the person in the video, mark the corresponding coordinate information, distinguish the faces of different individuals, and obtain the facial features of the person; The target user determination unit determines the target user, i.e., the subject of the autism spectrum disorder test, by analyzing the age-related features presented on the face based on the identified facial features of the person; The face shielding unit shields the faces of people other than the target user in the user video by using image blurring processing.

4. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The sight point analysis module includes an image recognition unit and a time series integration unit; The image recognition unit recognizes and segments each frame of the user video through a first convolutional neural network, and extracts facial images, eye images and facial position information; The time series integration unit predicts the sight point of each frame of image through a second convolutional neural network, and integrates to form a sight point time series.

5. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The data correction module includes a device information acquisition unit and a correction execution unit; The device information acquisition unit automatically acquires the device model and orientation information of the device used to record the user video while the user video is being recorded, and accurately records the information; The correction execution unit obtains the device model and orientation information and uses a correction algorithm set for different device characteristics to perform correction operations on the formed sight point time series to eliminate the sight point data deviation.

6. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The recognition model module includes a data input unit and a model operation unit; The data input unit is used to organize and prepare the corrected sight point time series data according to the format required by the recognition model; The model operation unit inputs the prepared data into the recognition model including the third convolutional neural network, starts the model's operation process, extracts features and makes classification judgments on the input data through the convolutional layer, pooling layer, and fully connected layer within the model, and uses the pre-trained weight parameters and classification logic of the model to output classification prediction results about autism spectrum disorder and typical development.

7. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The moon age association evaluation module includes a moon age receiving unit and a comparison evaluation unit; The age-in-month receiving unit is used to receive information about the age-in-month of the target user, and ensure the accuracy and completeness of the information; The comparison and evaluation unit is used to compare and analyze the received user age information with a preset threshold. If the age is lower than the threshold, the user's autism spectrum disorder risk is evaluated according to the standards and logic of early screening evaluation, and the corresponding risk level is divided into low risk, medium risk and high risk; If the age in months is equal to or higher than the threshold, the autism spectrum disorder detection result obtained based on the recognition model module is directly output.

8. The autism spectrum disorder detection system based on eye movement recognition according to claim 1, characterized in that: The construction of the recognition model includes: Acquire a data set, the data set including a plurality of training samples, each of the training samples including a time series of a target user's sight point obtained by collecting the user's sight point from a user video of the target user and a training user classification label; Inputting the time series of the user's gaze points in the training samples into a convolutional neural network model with initial model parameters for training, and obtaining classification prediction results on autism spectrum disorder and typical development; Comparing the obtained classification prediction results of the training samples regarding autism spectrum disorder and typical development with the target user classification labels of the training samples, thereby updating model parameters; The convolutional neural network model is trained successively using all training samples, and the model parameters are updated successively, so as to finally obtain the autism spectrum disorder recognition model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the autism spectrum disorder detection system based on eye movement recognition as described in any one of claims 1 to 8 is implemented.

Images