A child autism and ADHD assessment system with added feedback mechanism

Abstract

This invention discloses a feedback-based assessment system for childhood autism and ADHD, comprising: constructing a feedback-based eye-tracking fixation-dependent paradigm, divided into a static image preparation stage and a dynamic interactive task stage; collecting eye-tracking signals from subjects during eye-tracking tasks using an eye tracker, and changing the screen display content in real time based on the eye-tracking signals to obtain eye-tracking performance under feedback; subjects performing eye-tracking tasks as required, preprocessing and segmenting the eye-tracking signal data, and calculating eye-tracking features and task features for both stages; selecting multiple key features to train a classification model; subjects completing the eye-tracking task, calculating the key features for both stages and inputting them into the classification model to obtain classification results for childhood autism and ADHD. This invention incorporates a feedback mechanism into the fixation-dependent paradigm to assess children's eye-tracking ability, thereby assisting clinical diagnosis to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of eye movement research, and in particular to an assessment system for childhood autism and ADHD that incorporates a feedback mechanism. Background Technology

[0002] Autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) are the most common neurodevelopmental disorders in children. Children with ASD face challenges in social communication, narrow interests, and repetitive behaviors, while ADHD is primarily characterized by inattention, hyperactivity, and impulsivity. The diagnosis of ASD and ADHD relies on physician expertise and clinical assessment tools such as the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and the Diagnostic Observational Scale for Autism Spectrum Disorder (ADOS-2). However, these methods are often influenced by subjective factors and are time-consuming and labor-intensive. Therefore, researchers are exploring the possibility of diagnosis using objective physiological indicators such as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS).

[0003] Eye-tracking technology has become an important tool for studying abnormal eye movement patterns in children with ASD and ADHD due to its convenience and low cost. Studies have found that the eye movement patterns of children with ASD and ADHD differ significantly from those of children with typical developmental delays (TD). For example, children with ASD exhibit greater individual variability when viewing emotional faces, while children with ADHD are more easily distracted when faced with highly semantically burdensome distractions in tasks. Furthermore, children with ADHD spend less time fixing their eyes on emotion recognition tasks and are more easily distracted.

[0004] Beyond passively viewing images and videos, the gaze-dependent paradigm is widely used in eye-tracking research, particularly to quantify eye-following abilities. This paradigm allows for real-time changes to screen content based on the subject's gaze location, thus better simulating social interactions. Children with ASD may have underdeveloped eye-following abilities; even if they can follow others' gaze to find a target, they may be unable to extract meaningful social information. While children with ADHD do not have social cognitive impairments, their inattention affects their responses to eye cues.

[0005] In the area of ​​automated diagnosis, researchers are utilizing machine learning techniques to classify ASD and ADHD based on eye-tracking features. For example, Chinese patent document CN118285795 A discloses an eye-tracking-based method for assisting in the assessment of ADHD. In this invention, eye-tracking technology provides objective measurement indicators, such as fixation time, fixation point movement, and pupil diameter. This data can help clinicians more accurately assess patients' attention deficit and hyperactivity symptoms. Eye-tracking can assess attentional function in early childhood, contributing to the early identification and intervention of ADHD, thereby improving long-term prognosis. By analyzing eye-tracking data, this method can better understand individual attentional patterns and symptom presentations, providing patients with personalized treatment plans.

[0006] Existing studies have achieved high classification accuracy by analyzing children's gaze, eye movement characteristics, and head position. However, most existing studies focus on single disorders and primarily use pictures and videos as stimuli. Summary of the Invention

[0007] This invention provides a novel feedback mechanism for assessing autism and ADHD in children. By incorporating a novel feedback mechanism into the gaze-dependent paradigm, it assesses children’s eye-following ability and thus assists in clinical diagnosis to some extent.

[0008] A childhood autism and ADHD assessment system with added feedback mechanisms includes a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they perform the following steps:

[0009] (1) Construct a feedback-based eye-tracking fixation-dependent paradigm, which is divided into a static image preparation stage and a dynamic interactive task stage, to jointly assess the subject's eye-tracking ability. This paradigm collects eye-tracking signals from the subject when performing the eye-tracking task using an eye tracker, and changes the screen display content in real time according to the eye-tracking signals to obtain the eye-tracking performance under feedback.

[0010] (2) The subjects performed eye-following tasks as required. The eye-movement signal data collected by the eye tracker was preprocessed and divided into preparation stage data and task stage data. The eye-movement characteristics and task characteristics of the two stages were further calculated. Among them, the eye-movement characteristics were calculated based on the gaze events; the task characteristics were based on the completion of the eye-following task.

[0011] (3) Select multiple key features from eye-tracking features and task features for machine learning and train the classification model;

[0012] (4) During the application, the subjects completed the eye-following task, calculated the key features of the two stages and input them into the classification model to obtain the binary / tri-classification results of children with autism and ADHD compared with normal children, which were used to assist in clinical diagnosis.

[0013] In step (1), during the preparation phase, the content displayed on the screen remains still, and the image does not change due to the subject's gaze position, allowing the subject to browse freely;

[0014] At this point, the content displayed on the screen is mainly divided into two categories: cartoon faces representing strong social interaction and surrounding objects representing weak social interaction. Among them, the eyes and mouth of the cartoon faces are the most social, and children with autism tend to look more at the mouth than at the eyes. As for the surrounding objects, they are more likely to be disturbed by children with autism and ADHD.

[0015] During the task phase, when it is detected that the subject is looking at the target object, both visual and auditory feedback is given to indicate that the subject is looking at the target object correctly. When the subject stops looking at the target object, the feedback effect disappears until the subject looks at the target object again. Under the dual feedback mechanism of visual and auditory feedback, different subjects showed different differences in eye movement.

[0016] The preparation phase is fixed at 0-2 seconds. The length of the task phase varies depending on the subject's performance. If the subject is detected to be looking at the target for 1 second, the current trial ends early; otherwise, the task phase ends after 5 seconds. If the subject does not look at the target even once within 5 seconds, an additional 2-second phase begins. During this phase, all distractions except the target disappear, further guiding the subject to look at the target and complete eye-tracking.

[0017] In step (2), the eye-tracking signal data collected by the eye tracker is preprocessed, specifically as follows:

[0018] First, the data from both eyes are averaged, and the horizontal and vertical coordinate values ​​of both eyes on the screen are combined into a set of horizontal and vertical coordinate values. For the time period in which eye movement signals were not captured, the value is set to NaN.

[0019] Secondly, the NaN data is interpolated using the data at both ends of NaN; the interpolation is used to supplement the brief signal-free portion caused by the subject's blinking, while the long signal-free portion is not processed.

[0020] Finally, the data is smoothed using a sliding window to smooth the data within the sliding window to its median value.

[0021] In step (2), the judgment rule for the gaze event is:

[0022] First, velocity calculation is performed. Based on continuous eye movement data points, the velocity of eye movement at each moment is calculated. A velocity threshold of 30-100 degrees / second is set. When the eye movement velocity is lower than this threshold, the data point is classified as a gaze. When the eye movement velocity exceeds this threshold, the data point is classified as a saccade. Multiple consecutive low-velocity points are aggregated together and defined as a gaze event. The location of the gaze event is calculated by averaging the positions of these points.

[0023] Eye movement features are calculated based on gaze events and include the number of gazes, gaze duration, first gaze time, and gaze distribution.

[0024] Among them, gaze distribution refers to calculating the percentage of the total gaze time in different interest areas relative to the total gaze time in the corresponding phase by artificially setting interest areas.

[0025] In step (3), several key features are selected, including:

[0026] Preparation phase: duration of gazing at the background, number of times looking off the screen; Task phase: duration of gazing at the eyes, number of times gazing at the eyes, duration of gazing at the target, number of times gazing at the background, duration of looking off the screen, number of times looking off the screen, number of times looking at distracting objects before looking at the target, percentage of the test completed in one gaze.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] This invention adds a feedback mechanism to the eye-tracking fixation-dependent paradigm to further trigger abnormal eye movement behaviors in children with autism / ADHD, and combines features from both static and dynamic interactive environments to diagnose the condition. The method uses infrared eye-tracking signals as data, has a simple paradigm process, and a short evaluation time, which can assist clinical diagnosis to a certain extent, improve the accuracy of clinical diagnosis, and reduce the time required for diagnosis. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the feedback-based eye-tracking fixation-dependent paradigm and the corresponding region of interest in an embodiment of the present invention;

[0030] Figure 2 The gaze distribution of the three groups of subjects in different areas of interest during the preparation phase;

[0031] Figure 3 The differences in task-related characteristics among the three groups of participants during the task phase. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0033] This invention tested its feedback-based eye-tracking fixation paradigm on 24 children with ASD, 29 children with ADHD, and 26 children with TD, having them perform an eye-tracking task. All participants had an IQ above 70 and no other neurodevelopmental disorders. After data preprocessing, the ages of the three groups showed no significant difference, with all children aged 6–9 years. All participants were informed before the experiment that they would perform an eye-tracking task and received minimal training. However, participants were not informed that the experiment would end prematurely after maintaining eye contact for 1 second. The paradigm procedure is as follows: Figure 1 As shown.

[0034] Data was collected using an eye tracker with a sampling rate of 300Hz, capturing the subjects' gaze positions on the screen. The data processing flow is as follows:

[0035] (1) Preprocessing of eye-tracking data: First, the data from both eyes were averaged, and the horizontal and vertical coordinate values ​​of both eyes on the screen were combined into a single set of horizontal and vertical coordinate values. For time periods when eye-tracking signals were not captured, the value was set to NaN (Not a Number). Second, the NaN data was interpolated. Linear interpolation was used, with the data at both ends of the NaN value being interpolated. Interpolation supplemented the brief periods of no signal, such as when the subject blinked, increasing the effective data rate. Long periods of no signal were not processed. Third, data smoothing was performed. The data was smoothed using a sliding window, smoothing the data within the sliding window to its median value.

[0036] (2) Eye-tracking data segmentation: Based on the paradigm design, this invention performs eye-tracking analysis on the preparation and task phases and extracts features for machine learning classification. The eye-tracking data is segmented using paradigm timestamps, with the preparation phase fixed at 2 seconds and the task phase length varying depending on the subject's task completion performance. After segmentation, data from trials with excessively high NaN ratios are removed from each segment.

[0037] (3) Eye Movement Gaze Calculation: The IVT algorithm was used to calculate the gaze patterns at each stage of each trial, obtaining the location and duration of each gaze. First, velocity calculation was performed. Based on continuous eye movement data points, the velocity of the eye movement at each moment (angular velocity within the visual field, expressed in degrees per second) was calculated. A velocity threshold was set (30-100 degrees / second). When the eye movement velocity was below this threshold, the data point was classified as a gaze. When the eye movement velocity exceeded this threshold, the data point was classified as a saccade. Multiple consecutive low-velocity points were aggregated together and defined as a gaze event. The location of the gaze event could be calculated by averaging the positions of these points.

[0038] The data were segmented based on the task experiment, and eye-tracking features and task features were calculated and averaged. The features included:

[0039] 3.1 Number of gazes: A gaze refers to the moment when a person's eyes are relatively still while focusing on an object or area. The number of gazes can reflect the subject's level of concentration to some extent.

[0040] 3.2 Gaze duration: The average and total gaze duration of the participants, as well as the duration of the first gaze at the target, were calculated.

[0041] The duration of a gaze can reflect the subject's level of attention to some extent. During the preparation phase, shorter gaze durations and more gazes indicate that the subject is less focused.

[0042] 3.3 First gaze time: The time it takes for the subject to first gaze at the target after the task begins is calculated. This value is related to eye-following ability; the shorter the time, the stronger the ability and the faster the ability to follow another person's gaze. Studies have shown that children with autism have weaker eye-following ability and slower reaction time when looking at a target, while children with ADHD show similar behavior to typically developing children.

[0043] 3.4 Gaze Distribution: By setting up zones of interest, the percentage of participants' gaze duration in different zones of interest was calculated. For example... Figure 1 As shown, the zone of interest (ROI) is an artificially defined area on a specific visual stimulus (such as an image, video, website, or interface design) that is believed to play a significant role in the experimental task. The definition of the ROI aims to focus researchers' analytical goals, enabling them to concentrate on the distribution of an individual's attention to a specific visual element across different tasks or situations. By analyzing eye-tracking data within the ROI, researchers can reveal participants' cognitive states, emotional responses, and information processing strategies.

[0044] Research has shown that children with autism exhibit insensitivity to eye contact, and the establishment of interest zones allows for the separate analysis of elements within a scene. Furthermore, a comparative analysis of gaze distribution across the two phases reveals differences.

[0045] Gaze distribution = (Total gaze duration in a specific area of ​​interest / Total gaze duration during that phase) × 100%

[0046] 3.5 Task Characteristics: Task-related characteristics such as the completion status and speed of the eye-following task. In the paradigm design, if the participant is detected to have continuously looked at the target object for 1 second, the current trial ends early; otherwise, the task phase ends after 5 seconds. If the participant does not look at the target even once within 5 seconds, an additional 2-second phase is entered. In this phase, distractions other than the target object disappear, further guiding the participant to look at the target object and complete eye-following. Therefore, the average duration of the task phase trial is also considered a characteristic. Percentage of trials completed with a single gaze: Calculate the percentage of trials in which the participant's first gaze reaches 1 second out of all valid trials.

[0047] (4) Eye movement characteristics analysis: During the preparation phase, children with ASD exhibit insensitivity to the eyes and face, and their attention is more scattered to surrounding stimuli or even off-screen, while children with TD focus on the location of faces. Children with ADHD fall somewhere in between, such as... Figure 2 As shown. During the task phase, the eye movement distribution and gaze patterns of the three groups also differed, such as... Figure 3 As shown, some task-related characteristics also exhibited significant differences. In particular, with the addition of a feedback mechanism, the degree of influence of interfering factors differed among the three groups, with ADHD being more significantly affected, such as... Figure 3 As shown in (c). Furthermore, TD children completed tasks faster and performed better.

[0048] (5) Machine Learning: To evaluate the contribution of features from both static and dynamic stages to assisted diagnosis, this invention performed machine learning on features from both stages. After parameter selection, the top 10 features with high mutual information values ​​were chosen. Examining the selected features for three-class classification, combining features from both static and dynamic interactive environments, the following 10 optimal features were obtained: Static: duration of gazing at the background, number of times looking away from the screen; Dynamic: duration of gazing at the eyes, number of times gazing at the eyes, duration of gazing at the target, number of times gazing at the background, duration of looking away from the screen, number of times looking away from the screen, number of times looking at distractors before looking at the target, and the proportion of tests completed in one gaze. The results show that fewer features are available in the static environment, while more features are available in the dynamic environment; however, the selected features from the two static scenarios scored higher.

[0049] Subsequently, the classic Support Vector Machine (SVM) model was used, with a linear kernel and a radial basis function kernel (rbf) as the core (C: 0.001–1000, gamma: 1–0.0001). Validation was performed using 5-fold cross-validation. After grid parameter optimization, the classification accuracy is shown in Table 1.

[0050] Table 1

[0051]

[0052] First, for both phases, the features under dynamic interaction in the task phase better distinguished the three groups of subjects, indicating that dynamic interaction under a feedback mechanism is more likely to induce abnormal eye-movement behaviors in children with ASD / ADHD than in a static setting. Second, combining the features from both phases further improved the accuracy, demonstrating that features from static environments and dynamic interactions can mutually reinforce each other. In the classification task of TD and ASD, an accuracy rate of 88.89% was achieved; the classification accuracy of TD and ADHD reached 77.08%; and the tri-class accuracy rate reached 65.22%. The accuracy rate in distinguishing between ASD and ADHD also reached 80.00%.

[0053] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A childhood autism and ADHD assessment system with added feedback mechanism, characterized in that, The system includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to perform the following steps: (1) Construct a feedback-based eye-tracking fixation-dependent paradigm, which is divided into a static image preparation stage and a dynamic interactive task stage, to jointly assess the subject's eye-tracking ability. This paradigm collects eye-tracking signals from the subject when performing the eye-tracking task using an eye tracker, and changes the screen display content in real time according to the eye-tracking signals to obtain the eye-tracking performance under feedback. During the preparation phase, the content displayed on the screen remained still, and the image did not change depending on the subject's gaze position, allowing the subject to browse freely. At this time, the content displayed on the screen was mainly divided into two categories: cartoon faces representing strong social interaction and surrounding objects representing weak social interaction. During the task phase, when it is detected that the subject is looking at the target object, both visual and auditory feedback is given to indicate that the subject is looking at the target object correctly; when the subject stops looking at the target object, the feedback effect disappears until the subject looks at the target object again. The preparation phase is fixed at 0-2 seconds. The length of the task phase varies depending on the subject's performance. If the subject is detected to be looking at the target for 1 second, the current trial ends early; otherwise, the task phase ends after 5 seconds. If the subject does not look at the target even once within 5 seconds, an additional 2-second phase begins. During this phase, all distractions except the target disappear, further guiding the subject to look at the target and complete eye-tracking. (2) The subjects performed eye-tracking tasks as required. The eye-tracking signal data collected by the eye tracker was preprocessed and divided into preparation stage data and task stage data. The eye-tracking characteristics and task characteristics of the two stages were further calculated. Among them, the eye-tracking characteristics were calculated based on the gaze events; the task characteristics were based on the completion of the eye-tracking task. (3) Select multiple key features from eye-tracking features and task features for machine learning and train the classification model; (4) During the application, the subjects completed the eye-following task, calculated the key features of the two stages and input them into the classification model to obtain the binary / tri-classification results of children with autism and ADHD compared with normal children, which were used to assist in clinical diagnosis.

2. The childhood autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that, In step (2), the eye-tracking signal data collected by the eye tracker is preprocessed, specifically as follows: First, the data from both eyes are averaged, and the horizontal and vertical coordinate values ​​of both eyes on the screen are combined into a set of horizontal and vertical coordinate values. For the time period in which eye movement signals were not captured, the value is set to NaN. Secondly, the NaN data is interpolated using the data at both ends of NaN; the interpolation is used to supplement the brief signal-free portion caused by the subject's blinking, while the long signal-free portion is not processed; Finally, the data is smoothed using a sliding window to smooth the data within the sliding window to its median value.

3. The childhood autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that, In step (2), the judgment rule for the gaze event is: First, velocity calculation is performed. Based on continuous eye movement data points, the velocity of eye movement at each moment is calculated. A velocity threshold of 30-100 degrees / second is set. When the eye movement velocity is lower than this threshold, the data point is classified as a gaze. When the eye movement velocity exceeds this threshold, the data point is classified as a saccade. Multiple consecutive low-velocity points are aggregated together and defined as a gaze event. The location of the gaze event is calculated by averaging the positions of these points.

4. The childhood autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that, In step (2), the eye movement features are calculated based on the gaze events and include the number of gazes, gaze duration, first gaze time, and gaze distribution. Among them, gaze distribution refers to calculating the percentage of the total gaze time in different interest areas relative to the total gaze time in the corresponding phase by artificially setting interest areas.

5. The childhood autism and ADHD assessment system with added feedback mechanism according to claim 1, characterized in that, In step (3), several key features are selected, including: Preparation phase: duration of gazing at the background, number of times looking off the screen; Task phase: duration of gazing at the eyes, number of times gazing at the eyes, duration of gazing at the target, number of times gazing at the background, duration of looking off the screen, number of times looking off the screen, number of times looking at distracting objects before looking at the target, percentage of the test completed in one gaze.

Images