Children autism and hyperactivity assessment system added with feedback mechanism

Abstract

The invention discloses a child autism and hyperactivity assessment system added with a feedback mechanism, which comprises the following steps of: constructing a feedback-based eye movement fixation dependence normal form which is divided into a static picture preparation stage and a dynamic interaction task stage, collecting eye movement signals of a testee when the testee performs a sight following task through an eye tracker in the normal form, and determining the eye movement signals of the testee according to the eye movement signals of the testee when the testee performs the sight following task; changing screen display content in real time according to the eye movement signal, and obtaining eye movement performance under feedback; a subject performs a sight following task according to requirements, eye movement signal data are segmented after being preprocessed, and eye movement features and task features of two stages are calculated; screening out a plurality of key features to train a classification model; and after the testee completes the eye following task, calculating key features of the two stages, and inputting the key features into the classification model to obtain a classification result of autism and hyperactivity of children. According to the method, a feedback mechanism is added into the gaze dependence normal form, the gaze following ability of the child is evaluated, and then clinical diagnosis is assisted to a certain extent.

Description

Technical Field

[0001] The invention relates to the field of eye movement research, and in particular to a children's autism and ADHD assessment system with an added feedback mechanism. Background Art

[0002] Autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) are the most common neurodevelopmental disorders in children. Children with ASD face challenges in social communication, narrow interests, and repetitive behaviors, while ADHD mainly manifests as inattention, hyperactivity, and impulsivity. The diagnosis of ASD and ADHD relies on the physician's expertise and clinical assessment tools, such as the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and the Autism Diagnostic Observation Scale (ADOS-2). However, these methods are often affected by subjective factors and are time-consuming and labor-intensive, so researchers are trying to explore the possibility of diagnosis through objective physiological indicators such as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS).

[0003] Eye tracking technology has become an important tool for studying abnormal eye movement patterns in children with ASD and ADHD due to its convenience and low cost. Studies have found that the eye movement patterns of children with ASD and ADHD are significantly different from those of typically developing (TD) children. For example, children with ASD show higher individual differences when viewing emotional faces, while children with ADHD are more easily distracted when faced with high semantic load distractors during tasks. In addition, children with ADHD spend less time looking at the eyes during emotion recognition tasks and their attention is easily distracted.

[0004] In addition to passive viewing of images and videos, the gaze dependency paradigm is widely used in eye movement research, especially to quantify gaze following ability. This paradigm can change the screen content in real time according to the subject's gaze position, thereby better simulating social interaction. Children with ASD may have underdeveloped gaze following ability, and even if they can follow the gaze of others to find the target, they may not be able to extract meaningful social information. Although children with ADHD do not have social cognitive disorders, their distracted attention affects their response to gaze cues.

[0005] In terms of automated diagnosis, researchers use machine learning technology to classify ASD and ADHD through eye movement features. For example, the Chinese patent document with publication number CN118285795 A discloses an ADHD auxiliary assessment method based on eye tracking. In the present invention, eye tracking technology provides objective measurement indicators, such as gaze time, movement of gaze point, pupil diameter, etc., which can help clinicians more accurately assess patients' distraction and hyperactivity symptoms. Eye tracking can assess attention function in early childhood, which is helpful for early identification and intervention of ADHD, thereby improving long-term prognosis. By analyzing eye movement data, this method can better understand individual attention patterns and symptom manifestations, and provide patients with personalized treatment plans.

[0006] Existing studies have achieved high classification accuracy by analyzing children's gaze patterns, eye movement characteristics, and head position. However, most existing studies focus on a single disorder and mainly use pictures and videos as stimuli. Summary of the invention

[0007] The present invention provides a child autism and ADHD assessment system with an added feedback mechanism, which assesses children's gaze following ability by adding a novel feedback mechanism to the gaze dependence paradigm, thereby assisting clinical diagnosis to a certain extent.

[0008] A child autism and ADHD assessment system with an added feedback mechanism includes a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, the following steps are implemented:

[0009] (1) Construct a feedback-based eye movement dependency paradigm, which is divided into a static picture preparation phase and a dynamic interactive task phase, to jointly evaluate the subject's eye-following ability; this paradigm uses an eye tracker to collect the subject's eye movement signals when performing the eye-following task, and changes the screen display content in real time according to the eye movement signals to obtain the eye movement performance under feedback;

[0010] (2) The subjects performed the gaze following task as required, and the eye movement signal data collected by the eye tracker was preprocessed and divided into preparation phase data and task phase data, and the eye movement characteristics and task characteristics of the two phases were further calculated; among them, the eye movement characteristics were calculated based on the gaze events; the task characteristics were based on the completion of the gaze following task;

[0011] (3) Select multiple key features from eye movement features and task features for machine learning and train the classification model;

[0012] (4) During the application process, the subjects completed the gaze following task and calculated the key features of the two stages to input into the classification model, and obtained the binary / ternary classification results of children with autism and ADHD compared with normal children, which were used to assist clinical diagnosis.

[0013] In step (1), during the preparation phase, the screen display content remains static, and the picture does not change due to the subject's gaze position, so the subject can browse freely;

[0014] At this time, the content displayed on the screen is mainly divided into two categories: cartoon faces representing strong sociality and surrounding objects representing weak sociality; among them, the eyes and mouth of the cartoon faces are the most social, and autistic children tend to look at the mouth more than the eyes; while the surrounding objects serve as distractions, children with autism and ADHD are more easily distracted.

[0015] During the task phase, when it is detected that the subject is looking at the target object, dual visual and auditory feedback will be given to prompt the subject that he is looking at the target object correctly; when the subject no longer looks at the target object, the feedback effect disappears immediately until the subject looks at the target object again; under the dual visual and auditory feedback mechanism, different subjects show different eye movement differences.

[0016] The preparation phase is fixed at 0 to 2 seconds, and the length of the task phase varies with the subjects' task completion. When it is detected that the subject has been looking at the target object for 1 second, the current trial ends early, otherwise the task phase ends after 5 seconds. If the subject has not looked at the target once within 5 seconds, it enters an additional 2-second extra phase. In this phase, all distractions except the target object disappear, and the subject is further guided to look at the target object to complete eye tracking.

[0017] In step (2), the eye movement signal data collected by the eye tracker is preprocessed, specifically:

[0018] First, the binocular data are averaged, and the horizontal and vertical coordinate values ​​of both eyes on the screen are merged into a set of horizontal and vertical coordinate values. For the time period in which the eye movement signal is not captured, the value is set to NaN;

[0019] Secondly, the NaN data is interpolated using the data at both ends of NaN. The brief signal-free part caused by the subject's blinking is supplemented by interpolation, and the long signal-free part is not processed.

[0020] Finally, the data is smoothed through the sliding window, and the data in the sliding window is smoothed to its median value.

[0021] In step (2), the judgment rule for gaze events is:

[0022] First, the speed is calculated. Based on the continuous eye movement data points, the speed of the eye movement at each moment is calculated; the speed threshold is set to 30-100 degrees / second. When the eye movement speed is lower than this threshold, the data point is classified as a gaze, and when the eye movement speed exceeds this threshold, the data point is classified as a saccade; multiple continuous low-speed points are aggregated together and defined as a gaze event, and the position of the gaze event is calculated by the average position of these points.

[0023] Eye movement features are calculated based on gaze events, including gaze count, gaze duration, first gaze time, and gaze distribution;

[0024] Among them, gaze distribution refers to calculating the percentage of the subject's gaze time in different interest areas to the total gaze time in the corresponding stage by artificially setting interest areas.

[0025] In step (3), multiple key features are screened out, including:

[0026] Preparation phase: duration of gaze at the background, number of glances outside the screen; task phase: duration of gaze at the eyes, number of gazes at the eyes, duration of gaze at the target, number of gazes at the background, duration of gazes outside the screen, number of glances outside the screen, number of glances at distractors before looking at the target, and proportion of trials completed in one gaze.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention further triggers abnormal eye movement performance of children with autism / ADHD by adding a feedback mechanism to the eye movement dependency paradigm, and combines the characteristics of both static scenes and dynamic interactions to diagnose the disease. This method uses infrared eye movement signals as data, has a simple paradigm flow, and a short evaluation time. It assists clinical diagnosis to a certain extent, improves the accuracy of clinical diagnosis, and reduces the time required for diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of an eye movement fixation dependency paradigm based on feedback and a corresponding interest area according to an embodiment of the present invention;

[0030] Figure 2 The gaze distribution of the three groups of subjects on different interest areas in the preparation stage;

[0031] Figure 3 The differences in task-related characteristics among the three groups of subjects during the task phase. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0033] The embodiment of the present invention tested the feedback-based eye movement dependent paradigm of the present invention on 24 ASD children, 29 ADHD children and 26 TD children, asking them to perform an eye-following task. All the subjects had an IQ of over 70 and did not suffer from other neurodevelopmental diseases. After data preprocessing, there was no significant difference in the ages of the three groups of subjects. The ages of the three groups of children were all between 6 and 9 years old. All subjects were told to perform an eye-following task before the experiment, and there was a small amount of experimental training. However, the subjects were not told that the experiment would end early after looking at the target continuously for 1 second. The paradigm process is as follows Figure 1 As shown in (a).

[0034] The eye tracker is used to collect data with a sampling rate of 300Hz, and the gaze position of the subjects' eyes on the screen is collected. The data processing flow is as follows:

[0035] (1) Preprocessing of eye movement data: First, average the data of both eyes and merge the horizontal and vertical coordinate values ​​of both eyes on the screen into a set of horizontal and vertical coordinate values. For the time period when the eye movement signal is not captured, set the value to NaN (Not a Number). Secondly, interpolate the NaN data. Use the linear interpolation method to perform linear interpolation using the data at both ends of NaN. Interpolation is used to supplement short signal-free parts such as the subject's blinking to increase the effective data rate. For long signal-free parts, no processing is performed. Third, data smoothing. Smooth the data using a sliding window and smooth the data in the sliding window to its median.

[0036] (2) Segmentation of eye movement data: According to the design of the paradigm, the present invention performs eye movement analysis on the preparation phase and the task phase, and extracts features therein for machine learning classification. The eye movement data is segmented by paradigm timestamp, where the preparation phase is fixed at 2 seconds, and the length of the task phase varies with the completion of the task by the subject. After segmentation, the data of the trials with too high NaN ratio in each segment are eliminated.

[0037] (3) Eye movement and gaze calculation: Use the IVT algorithm to calculate the gaze situation at each stage of each trial to obtain the position and duration of each gaze. First, perform speed calculation. Based on the continuous eye movement data points, calculate the speed of the eye movement at each moment (angular velocity in the visual field, expressed in degrees / second). Set a speed threshold (30-100 degrees / second). When the eye movement speed is lower than this threshold, the data point is classified as a gaze. When the eye movement speed exceeds this threshold, the data point is classified as a saccade. Multiple continuous low-speed points will be aggregated together and defined as a gaze event. The position of the gaze event can be calculated by the average position of these points.

[0038] The data were segmented according to the task trials, and the eye movement features and task features were calculated and averaged. The features included:

[0039] 3.1 Number of gazes: Gaze refers to the moment when a person's eyes are relatively still when looking at an object or area. The number of gazes can reflect the subject's concentration to a certain extent.

[0040] 3.2 Gaze duration: Calculate the average gaze duration and total gaze duration of the subjects, as well as the duration of the first gaze at the target.

[0041] The duration of gaze can reflect the subject's attention level to a certain extent. In the preparation stage, the shorter the gaze duration and the more times the subject stares, the less focused the subject is.

[0042] 3.3 First gaze time: Calculate the time it takes for the subject to first gaze at the target after the task begins. This value is related to the ability to follow the gaze. The shorter the value, the stronger the ability, and the faster the ability to follow the gaze of others. Studies have shown that children with autism have weaker ability to follow the gaze and are slower to look at the target, while children with ADHD perform more consistently with normal children.

[0043] 3.4 Gaze distribution: By setting the interest area, calculate the percentage of the subject's gaze duration in different interest areas. Figure 1 As shown in (b), the area of ​​interest is an artificially defined area on a specific visual stimulus (such as a picture, video, website, interface design, etc.), which is considered to play an important role in the experimental task. The definition of the area of ​​interest is to focus the researcher's analysis goals, so that the individual's attention distribution to specific visual elements in different tasks or situations can be focused. By analyzing the eye movement data in the area of ​​interest, the cognitive state, emotional response and information processing strategy of the participants can be revealed.

[0044] Studies have shown that autistic children are insensitive to eye contact, and the setting of interest areas can separate the elements in the scene for analysis. At the same time, the gaze distribution of the two stages is compared and analyzed.

[0045] Gaze distribution = total gaze duration of a certain area of ​​interest / total gaze duration of this stage × 100%

[0046] 3.5 Task characteristics: task-related characteristics such as the completion of the eye-following task and the speed of completing the task. In the paradigm design, when it is detected that the subject has been looking at the target object for 1 second, the current trial ends in advance, otherwise the task phase ends after 5 seconds; if the subject has not looked at the target once within 5 seconds, it enters an additional 2-second extra phase. In this phase, distractors other than the target object disappear, and the subject is further guided to look at the target object to complete eye-following. Therefore, the average duration of the task phase experiment is also used as a feature. Proportion of trials completed in one gaze: Calculate the percentage of trials in which the subject's first gaze reaches 1 second among all valid trials.

[0047] (4) Analysis of eye movement characteristics: In the preparation stage, ASD children show insensitivity to eyes and faces, and their attention is more dispersed to surrounding stimuli or even outside the screen, while TD children focus on the location of faces. ADHD children are somewhere in between, such as Figure 2 During the task phase, the three groups also had differences in eye movement distribution and gaze, as shown in Figure 2. Figure 3 As shown in Figure 2, some task-related features also showed significant differences. In particular, the addition of the feedback mechanism had different effects on the three groups, and ADHD was more affected, such as Figure 3 In addition, TD children completed the task faster and performed better.

[0048] (5) Machine learning: In order to evaluate the contribution of the two static and dynamic stage features to auxiliary diagnosis, the present invention uses the features of the two stages to perform machine learning respectively. After parameter selection, the top 10 features with high mutual information values ​​(mutual info) are selected. In the three-category selection, the features under the static scene and dynamic interaction are combined to obtain the following 10 best features: static: gaze background duration, number of times looking outside the screen; dynamic: gaze eye duration, number of times gaze eye, gaze target duration, number of times gaze background, length of time looking outside the screen, number of times looking outside the screen, number of times looking at distractors before looking at the target, and the proportion of trials completed in one gaze. It can be seen from the results after selection that the number of features in the static environment is less, and the number of features in the dynamic environment is more, but the selected features in the two static scenes have higher scores.

[0049] Then, the classic support vector machine (SVM) model was used, and the linear kernel (linear) and radial basis function kernel (rbf) were selected as the core (C: 0.001~1000, gamma: 1~0.0001). 5-fold cross validation was used for verification. After the grid parameters were tuned, the classification accuracy was shown in Table 1.

[0050] Table 1

[0051]

[0052] First, for both stages, the features under dynamic interaction in the task stage can better distinguish the three groups of subjects, indicating that dynamic interaction under the feedback mechanism can induce abnormal eye movement behavior in ASD / ADHD children more than static scenes. Secondly, combining the features of the two stages can further improve the accuracy, indicating that the features under static environments and dynamic interactions can promote each other. In the classification task of TD and ASD, an accuracy rate of 88.89% was obtained, the classification accuracy rate of TD and ADHD reached 77.08%, and the accuracy rate of the three classifications reached 65.22%. In distinguishing ASD from ADHD, an accuracy rate of 80.00% was also achieved.

[0053] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A child autism and ADHD assessment system with an added feedback mechanism, characterized in that: The method comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, the following steps are implemented: (1) Construct a feedback-based eye movement dependency paradigm, which is divided into a static picture preparation phase and a dynamic interactive task phase, to jointly evaluate the subject's eye-following ability; this paradigm uses an eye tracker to collect the subject's eye movement signals when performing the eye-following task, and changes the screen display content in real time according to the eye movement signals to obtain the eye movement performance under feedback; (2) The subjects performed the gaze following task as required, and the eye movement signal data collected by the eye tracker was preprocessed and divided into preparation phase data and task phase data, and the eye movement characteristics and task characteristics of the two phases were further calculated; among them, the eye movement characteristics were calculated based on the gaze events; the task characteristics were based on the completion of the gaze following task; (3) Select multiple key features from eye movement features and task features for machine learning and train the classification model; (4) During the application process, the subjects completed the gaze following task and calculated the key features of the two stages to input into the classification model, and obtained the binary / ternary classification results of children with autism and ADHD compared with normal children, which were used to assist clinical diagnosis.

2. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (1), during the preparation phase, the screen display content remains static, and the picture does not change due to the subject's gaze position, so the subject can browse freely; At this time, the content displayed on the screen is mainly divided into two categories: cartoon faces representing strong sociality and surrounding objects representing weak sociality; among them, the eyes and mouth of the cartoon faces are the most social, and autistic children tend to look at the mouth more than the eyes; while the surrounding objects serve as distractions, children with autism and ADHD are more easily distracted.

3. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (1), during the task phase, when it is detected that the subject is looking at the target object, visual and auditory dual feedback will be given to prompt the subject to look at the target object correctly; when the subject no longer looks at the target object, the feedback effect disappears immediately until the subject looks at the target object again; under the dual feedback mechanism of vision and hearing, different subjects show different eye movement differences.

4. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (1), the preparation phase is fixed at 0 to 2 seconds, and the length of the task phase varies with the subject's task completion situation. When it is detected that the subject has been looking at the target object for 1 second, the current trial ends early, otherwise the task phase ends after 5 seconds. If the subject does not look at the target once within 5 seconds, it enters an additional 2-second extra phase. In this phase, distractors other than the target object disappear, and the subject is further guided to look at the target object to complete eye tracking.

5. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (2), the eye movement signal data collected by the eye tracker is preprocessed, specifically: First, the binocular data are averaged, and the horizontal and vertical coordinate values ​​of both eyes on the screen are merged into a set of horizontal and vertical coordinate values. For the time period in which the eye movement signal is not captured, the value is set to NaN; Secondly, the NaN data is interpolated using the data at both ends of NaN. The brief signal-free part caused by the subject's blinking is supplemented by interpolation, and the long signal-free part is not processed. Finally, the data is smoothed through the sliding window, and the data in the sliding window is smoothed to its median value.

6. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (2), the judgment rule for gaze events is: First, the speed is calculated. Based on the continuous eye movement data points, the speed of the eye movement at each moment is calculated; the speed threshold is set to 30-100 degrees / second. When the eye movement speed is lower than this threshold, the data point is classified as a gaze, and when the eye movement speed exceeds this threshold, the data point is classified as a saccade; multiple continuous low-speed points are aggregated together and defined as a gaze event, and the position of the gaze event is calculated by the average position of these points.

7. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (2), the eye movement features are calculated based on the gaze events, including the number of gazes, gaze duration, first gaze time and gaze distribution; Among them, gaze distribution refers to calculating the percentage of the subject's gaze time in different interest areas to the total gaze time in the corresponding stage by artificially setting interest areas.

8. The child autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that: In step (3), multiple key features are screened out, including: Preparation phase: duration of gaze at the background, number of glances outside the screen; task phase: duration of gaze at the eyes, number of gazes at the eyes, duration of gaze at the target, number of gazes at the background, duration of gazes outside the screen, number of glances outside the screen, number of glances at distractors before looking at the target, and proportion of trials completed in one gaze.

Images