Systems and methods for using portable computer device with eye tracking capability

Abstract

Embodiments described herein include a portable device having a user detection device, such as an eye tracker device or other sensor, and data including such a portable device or data collected from such a device, such as eye tracking data and / or other multi-modal data, some examples of the system may provide useful assessments for developmental disorders and may generate normative treatment plans that have separate lengths of time for different treatment-specific skill fields over a period of time.

Description

Cross-referencing of related patent applications

[0001] This application claims the benefits of U.S. Provisional Application No. 63 / 506,211, filed June 5, 2023, and U.S. Provisional Application No. 63 / 604,647, filed November 30, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to portable devices having user detection devices such as eye-tracking sensors, and interconnect devices for using eye-tracking data and / or other multimodal data such as facial expressions, verbal expressions, and / or body movements. Background Technology

[0003] Computer systems have been used to collect eye-tracking data from users, such as young patients in clinical settings, to gather objective data on user responses to stimuli. In some cases, this objective data can indicate developmental disorders, such as autism spectrum disorder (ASD). Various attempts by treatment providers (e.g., pediatricians or other healthcare professionals) to assess the severity of a patient's ASD can vary considerably in terms of objective assessment tools and the specific experience of the treatment provider. In some cases, treatment providers using traditional "best practice" tools demonstrate poor sensitivity and specificity to the condition, especially with young children or other young patients. Furthermore, treatment providers may often lack sufficient tools to objectively measure the progression of these conditions over time, particularly early in a patient's life. Summary of the Invention

[0004] This disclosure describes a portable device (such as an eye tracker device or other sensor) having a user detection device and a computer system that includes such a portable device or data collected from such a device (such as eye tracking data and / or other multimodal data, such as facial expressions, verbal expressions and / or body movements).

[0005] For example, some of the systems described herein can optionally be implemented using improved portable devices that enable improved objective measurements and increase convenience for both the treatment provider and the patient (such as a young child or other younger patient). In some embodiments, the system includes at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye tracker device (or eye tracker device or eye tracker). In specific examples described below, these portable devices may be configured differently, but all wirelessly interact with a network-connected server (hereinafter also referred to as a networked server) platform to advantageously collect session data from the patient in a comfortable and less invasive manner, while increasing the control flexibility for the treatment provider. Optionally, session data collected via the patient-side portable device (e.g., using analysis of eye-tracking data generated in response to the display of predetermined, age-appropriate visual stimuli) can be analyzed in real time to output a results interface on the operator-side portable device, which displays at least one index based on objective factors. In some versions described herein, the system can be used to provide objective and comparative assessments of developmental, cognitive, social, or mental abilities or deficits, including autism spectrum disorder (ASD).

[0006] Some of the examples described herein enable the monitoring of specific skill domains (and / or skills) for developmental assessment, including but not limited to: annotating visual stimuli (e.g., films or videos) to obtain immediate skill relevance; customizing data collection playlists (e.g., based on target skill domains selected by the user); implementing annotated skill visualization and analysis sections in diagnostic / monitoring reports; customizing monitoring reports to target skill domains, which may be automatically selected or selected by the user when starting a session or viewing diagnostic results; providing interactive dashboards for users (e.g., treatment providers, clinicians, or patient guardians such as parents) to explore arbitrary skill domains and visualizations of patient behavior relative to reference groups; and / or capturing multimodal data (e.g., audio / video showing social interactions such as facial expressions, verbal expressions, and / or body movements) during data collection sessions, wherein the multimodal data may be used in conjunction with eye-tracking data for developmental assessment.

[0007] One aspect of this disclosure features a system using at least one portable computing device with eye-tracking capabilities. The system includes: a portable eye-tracking console comprising a display screen and an eye-tracking device mounted near the display screen such that both the display screen and the eye-tracking device face a patient, wherein the eye-tracking device is configured to acquire eye-tracking coordinate data of the patient during a session when a predetermined sequence of stimulus videos is presented on the display screen; a portable computing device having a touchscreen display interface and spaced apart from and portable to different locations relative to the portable eye-tracking console; and a network server wirelessly receiving session data from the portable eye-tracking console, and including a portal that exports assessment results including a numerical defect index score and a graphical correlation between reference assessment metrics. The network server is configured to wirelessly connect to both the portable eye tracker console and the portable computing device, such that after the portable computing device wirelessly communicates with the portable eye tracker console via the network server to control the activation of the session and present the predetermined sequence of stimulus videos on the display screen of the portable eye tracker console, the portable eye tracker console wirelessly transmits the session data to the network server, the session data including the eye tracking coordinate data, the eye tracking coordinate data having a timestamp relationship with the information of the predetermined sequence of stimulus videos displayed by the portable eye tracker console during the session.

[0008] In some implementations, the system includes multiple portable eye-tracking consoles that simultaneously communicate wirelessly with the networked server.

[0009] In some embodiments, the eye tracker device includes one or more eye-tracking sensors mechanically assembled near the periphery of the display screen.

[0010] In some embodiments, each of the one or more eye-tracking sensors includes: an illumination source configured to emit detection light; and a camera configured to capture eye movement data, the eye movement data including at least one of: reflection or reflex of the detection light from the illumination source by the pupil or cornea. The eye-tracking sensor is configured to convert the eye movement data into a data stream containing information on at least one of pupil position, gaze vector, or gaze point for each eye, wherein the patient's eye-tracking data includes the patient's corresponding data stream.

[0011] In some embodiments, the detection light includes infrared light, and the camera includes an infrared-sensitive camera. While the predetermined sequence of stimulus videos is presented on the display screen facing the patient during the session, the caregiver carrying the patient wears glasses having a filter configured to filter the infrared light, such that the camera captures only the patient's eye movement data.

[0012] In some embodiments, the eye tracker device includes at least one image acquisition device configured to capture an image of at least one eye of the patient when a predetermined sequence of stimulus videos is presented on the display screen facing the patient during the session, and the eye tracker device is configured to generate corresponding eye tracking data for the patient based on the captured images of at least one eye of the patient.

[0013] In some embodiments, the system further includes at least one recording device assembled on the portable eye tracker console and configured to collect at least one of image data, audio data, or video data associated with the patient when a predetermined sequence of stimulation videos is presented on the display screen facing the patient during the session, the session data including at least one of the image data, audio data, or video data.

[0014] In some embodiments, the portable eye tracker console includes: a housing configured to hold the display screen and the eye tracker device; and a base connected to the housing via one or more joints. The base is rotatable about the one or more joints to adjust the relative position or angle between the display screen and the patient during the session.

[0015] In some implementations, the network server provides the portal website accessible by the portable computing device, and the network server is configured to output a developmental analysis report, including the patient's developmental analysis data, to the portable computing device via the user interface of the portal website.

[0016] In some implementations, the network server is configured to: receive treatment data associated with the patient, the treatment data including at least one of the following: the patient's previous developmental analysis data, the patient's previous treatment plan, or reference treatment data of other patients; and generate a guided treatment plan for the patient using artificial intelligence based on the treatment data associated with the patient and the patient's developmental analysis data.

[0017] Another aspect of this disclosure is a computer-implemented method comprising: at a networked server, acquiring a treatment plan for a patient's developmental disorder, the treatment plan having individual time lengths for different treatment-specific skill areas over a period of time, the treatment plan having a specific treatment plan format, wherein the networked server is configured to process data associated with a default treatment plan format; and at the networked server, parsing the treatment plan having the specific treatment plan format to determine treatment data for the patient, the treatment data conforming to the default treatment plan format.

[0018] In some embodiments, the computer-implemented method further includes, at the network server, receiving input for selecting a treatment plan format from a plurality of treatment plan formats presented on a user interface, wherein the plurality of treatment plan formats are different from each other.

[0019] In some implementations, the multiple treatment plan formats differ from each other in at least one of the following: skill domain name, prompting method, treatment or training materials, reinforcement method, or data acquisition method.

[0020] In some implementations, parsing the treatment plan having the specific treatment plan format includes: parsing the treatment plan having the specific treatment plan format based on a selected treatment plan format and the default treatment plan format.

[0021] In some implementations, the multiple treatment plan formats include two or more of the following: EarliPoint, Early Intervention Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Joint Attention Symbolic Play Participation and Regulation (JASPER), and the Improvisation of Parents as Communication Teachers Program (ImPACT Program).

[0022] In some implementations, obtaining a treatment plan for a patient's developmental disorder includes uploading the treatment plan, which has the specific treatment plan format, from a repository or storage medium on the networked server.

[0023] In some implementations, the treatment data includes at least one of the following: the corresponding time length of the different treatment-specific skill areas within the time period, the percentage of the corresponding time length of the different treatment-specific skill areas within the time period, the percentage of attention to the different treatment-specific skill areas in a series of sessions, the change in the percentage of attention to the different treatment-specific skill areas between at least two recent sessions, or the relationship between the percentage of the corresponding time length of the different treatment-specific skill areas and the change in the percentage of attention.

[0024] In some embodiments, the computer-implemented method further includes: at the network server, receiving input for selecting a treatment plan format from a plurality of treatment plan formats presented on a user interface, wherein the plurality of treatment plan formats are different from each other; and at the network server, generating a new treatment plan based on treatment data for the patient and the selected treatment plan format.

[0025] In some embodiments, the computer-implemented method further includes: transmitting the new treatment plan, having the selected treatment plan format, to a computing device or an external server via the networked server.

[0026] In some embodiments, the computer-implemented method further includes: at the networked server, generating assessment data of the patient's developmental disorder based on the patient's eye-tracking session data. The new treatment plan is generated based on the assessment data of the patient's developmental disorder.

[0027] In some embodiments, the computer-implemented method further includes: at the networked server, determining a specific treatment plan format for a new treatment plan for the patient among a plurality of treatment plan formats; and presenting a visual indication on the specific treatment plan format among the plurality of treatment plan formats in the user interface, the visual indication indicating that the specific treatment plan format is recommended for the new treatment plan for the patient.

[0028] In some embodiments, the computer-implemented method further includes: receiving, at the network server, a selection of a target session, the target session being selected from a session list on a user interface of a portal on the network server; in response to receiving the selection of the target session, popping up a window for selecting a target skill domain from a plurality of skill domains listed in the window; automatically selecting one or more target skill domains from the plurality of skill domains based on the treatment data, wherein the different treatment-specific skill domains include the one or more target skill domains; and running the target session based on the selected one or more target skill domains.

[0029] In some embodiments, the computer-implemented method further includes: at the network server, presenting the input fields of the treatment plan on a user interface of a portal on the network server; at the network server, receiving input for one of the input fields of the treatment plan on the user interface; and at the network server, updating the treatment plan based on the input for one of the input fields of the treatment plan.

[0030] In some implementations, the different therapeutic-specific skill areas include one or more of the following: manding, listener response, turn-taking, shared attention, social tact, and play.

[0031] In some embodiments, the computer-implemented method further includes: receiving input at the network server for selecting a third-party system from a plurality of third-party systems presented on a user interface; and after a connection is established between the network server and the selected third-party system, retrieving data related to the patient from the selected third-party system by the network server, wherein the data related to the patient includes at least one of the following: the patient's previous clinical data, the patient's previous treatment data, or reference data of other patients.

[0032] In some embodiments, the computer-implemented method further includes generating a new treatment plan for the patient on the networked server, based on the treatment data and the patient-related data.

[0033] Another aspect of this disclosure features a computer-implemented method comprising: receiving, at a networked server, a request for patient assessment results based on patient session data, said session data being collected during a data collection playlist of visual stimuli presented to the patient in a session assessing the patient's developmental disorder; and outputting the patient assessment results by said networked server. The assessment results include: corresponding scores for developmental disorder indices associated with the patient's developmental disorder, and, for each developmental disorder indice, a result showing the correlation between the corresponding score of the developmental disorder indice and a corresponding reference assessment indicator.

[0034] In some implementations, the results of the correlation include at least one of the following: a summary describing the correlation, or a graphical representation of the correlation.

[0035] In some embodiments, the assessment results may further include at least one of the following: an assessment result indicating whether the patient has the developmental disorder, or indicative information for each of the corresponding scores of the developmental disorder index.

[0036] In some implementations, the developmental disability index includes at least one of the following: a social deficit index, a language proficiency index, or a non-verbal learning index.

[0037] In some implementations, the corresponding reference assessment indicator for the social deficit index score includes the ADOS-2 indicator, the corresponding reference assessment indicator for the language proficiency index score includes the Mullen language age equivalent, and the corresponding reference assessment indicator for the non-language learning index score includes the Mullen language age equivalent.

[0038] In some implementations, at least one visual scene of the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene of the data collection playlist. The assessment results include: behavioral data of the patient at moments in the session related to the specific skill domain, for each specific skill domain among one or more of the plurality of skill domains, each moment corresponding to a corresponding visual scene in the visual scene of the data collection playlist.

[0039] In some implementations, the behavioral data includes attention percentage, defined as the ratio between the number of times the patient pays attention to relevant scene content in the visual stimulus and the total number of times the patient is watching the visual stimulus.

[0040] In some implementations, the assessment results include the outline of a distribution map of behavioral data of a reference group, the behavioral data of which is based on reference session data collected during a data collection playlist of the visual stimuli presented to each member of the reference group. The assessment results may also include: representative visual scenes for each of the one or more specific skill domains, the representative visual scenes highlighting one or more areas of attention of the reference group within a predetermined area, or the representative visual scenes highlighting the patient's area of ​​attention during the session.

[0041] In some implementations, the assessment results include at least one of the following: a first graphical representation of the patient’s looking behavior during the session on a time-by-time basis, or a second graphical representation of the attention funnel of a reference group and the patient’s attention during the session.

[0042] Another aspect of this disclosure features a computer-implemented method comprising: initiating a session for a patient by establishing communication with an operator-side computing device and a patient-side portable tablet computing device, the patient-side portable tablet computing device integrating an eye tracker device; sequentially presenting a visual scene of a data collection playlist of visual stimuli to the patient on the screen of the patient-side portable tablet computing device, while simultaneously acquiring eye-tracking data of the patient using the eye tracker device; and transmitting session data of the session to a networked server, the session data including the eye-tracking data of the patient acquired during the session. Acquiring the patient's eye-tracking data using the eye tracker device includes: capturing at least one of an image of the patient's eyes or the patient's eye position, and the eye-tracking data is determined based on at least one of the captured image of the patient's eyes or the eye position.

[0043] In some embodiments, the eye tracker device is configured to: determine eye movement data based on at least one of captured images of the patient's eyes or eye positions, and convert the patient's eye movement data into eye tracking data, the eye tracking data including information associated with at least one of pupil position, gaze vector of each eye, or gaze point.

[0044] In some embodiments, using the eye tracker device to collect the patient's eye tracking data further includes capturing first eye movement data of the patient's eyes by measuring reflected light from the patient's eyes.

[0045] In some embodiments, the eye tracker device includes: at least one eye tracking unit configured to capture first eye movement data of the patient's eye; and at least one image acquisition unit configured to capture at least one of an image of the patient's eye or the position of the patient's eye.

[0046] In some embodiments, the eye tracker device is configured to: determine second eye movement data based on at least one of captured images of the patient's eyes or eye positions, and determine the eye tracking data based on the first eye movement data and the second eye movement data.

[0047] In some embodiments, the eye tracker device is configured to: convert first eye movement data into first eye tracking data, determine second eye movement data based on at least one of an image of the captured eye or the patient's eye position, convert the second eye movement data into second eye tracking data, and determine eye tracking data based on the first eye tracking data and the second eye tracking data.

[0048] In some embodiments, the computer-implemented method further includes: collecting at least one of image data, audio data, or video data collected by one or more recording devices while the visual scenes of the data collection playlist of the visual stimuli are presented sequentially, wherein at least one of the one or more recording devices is assembled inside or outside the patient-side computing device. The session data includes at least one of image data, audio data, or video data.

[0049] Another aspect of this disclosure features an apparatus comprising: at least one processor; and one or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform any of the computer-implemented methods disclosed herein.

[0050] Another aspect of this disclosure is characterized by one or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform any of the computer-implemented methods disclosed herein.

[0051] Another aspect of this disclosure is a computer-implemented method comprising: receiving, on a networked server, a request for an assessment result of a patient based on patient session data, the session data being acquired during a session while a data collection playlist of visual stimuli is presented to the patient; wherein at least one visual scene of the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene of the data collection playlist; and outputting, by the networked server, the assessment result of the patient comprising: behavioral data of the patient at moments in the session related to each specific skill domain from one or more of the plurality of skill domains, each moment corresponding to a corresponding visual scene in the visual scene of the data collection playlist.

[0052] In some embodiments, the behavioral data includes attention percentage, which is defined as the ratio between the number of times the patient pays attention to relevant scene content in the visual stimulus and the total number of times the patient is watching the visual stimulus.

[0053] In some embodiments, the session data includes the patient's eye-tracking data. The computer-implemented method further includes: determining, based on the patient's eye-tracking data, the total number of moments during which the patient is viewing the visual stimulus, and determining, based on the patient's eye-tracking data, the number of moments during which the patient is focused on relevant scene content.

[0054] In some embodiments, the computer-implemented method further includes: determining, at a certain moment in the session, that the patient's area of ​​interest is within a predetermined area; and determining that the moment is one of a plurality of moments when the patient is interested in relevant scene content.

[0055] In some embodiments, the predetermined region corresponds to the outline of a distribution map of behavioral data of a reference group, the behavioral data of which is based on reference session data collected during a data collection playlist of the visual stimuli presented to each member of the reference group.

[0056] In some embodiments, the value of the contour of the distribution map corresponds to a cutoff threshold.

[0057] In some embodiments, the evaluation results may also include a distribution map of the behavioral data of the reference group.

[0058] In some embodiments, the evaluation results may further include at least one of the following: a representative visual scene for each of the one or more specific skill areas, the representative visual scene highlighting one or more areas of attention within a predetermined area for the reference group, or the representative visual scene highlighting the patient's area of ​​attention during the session.

[0059] In some embodiments, the evaluation results may further include at least one of the following: behavioral data from one or more previous sessions of the patient for each of the one or more specific skill domains, or a comparison between the behavioral data from the sessions and the behavioral data from one or more previous sessions of the patient.

[0060] In some embodiments, the assessment results include a graph showing behavioral data of the session for each of the one or more specific skill domains and behavioral data of one or more previous sessions of the patient.

[0061] In some embodiments, the computer-implemented method further includes: selecting one or more specific skill areas from the plurality of skill areas for the patient's assessment results.

[0062] In some embodiments, selecting one or more specific skill areas from the plurality of skill areas includes at least one of the following: selecting a specific skill area with reliable data from the plurality of skill areas; selecting a frequently requested skill area from the plurality of skill areas; selecting a skill area with particularly high, low, or representative scores from the plurality of skill areas, wherein the scores represent the patient's percentage of attention; selecting a skill area previously selected as a target skill area in the session; selecting a skill area selected to customize the assessment results; or selecting a skill area selected in the patient's previous session or the patient's previous assessment results.

[0063] In some embodiments, the computer-implemented method includes: receiving a session request to initiate the session via a portal website on the network server; presenting a session list on a user interface of the portal website; and receiving a selection of the session from the session list on the user interface.

[0064] In some embodiments, the computer-implemented method further includes: in response to receiving a selection of the session, popping up a window for selecting a target skill domain from a plurality of skill domains listed in the window; receiving user input to select one or more target skill domains in the window; and running the session based on the selected one or more target skill domains, wherein the selected one or more target skill domains include the one or more specific skill domains.

[0065] In some embodiments, the computer-implemented method includes: adjusting a data collection playlist of visual stimuli based on one or more selected target skill domains.

[0066] In some embodiments, adjusting the data collection playlist of the visual stimuli includes at least one of the following: prioritizing visual scenes related to one or more selected target skill areas in the data collection playlist; enriching the data collection playlist with additional visual scenes related to one or more selected target skill areas; or reducing or removing visual scenes unrelated to the selected target skill areas in the data collection playlist.

[0067] In some embodiments, prioritizing visual scenes related to one or more selected target skill areas includes at least one of the following: arranging visual scenes related to one or more selected target skill areas at the beginning of the data collection playlist; arranging visual scenes related to one or more selected target skill areas in order of their weighted correlation values ​​with the one or more selected target skill areas; or selecting only visual scenes related to one or more selected target skill areas in the data collection playlist.

[0068] In some embodiments, receiving user input includes receiving the user input from an operator-side computing device, the operator-side computing device communicating with the web server via the portal. In some embodiments, the computer-implemented method further includes establishing communication between the operator-side computing device and the patient-side computing device via the network server, and transmitting information of a modified visual stimulus data collection playlist to the patient-side computing device, such that the modified visual stimulus data collection playlist is presented to the patient on the display screen of the patient-side computing device during a session.

[0069] In some embodiments, the computer-implemented method further includes: receiving patient session data from a patient-side computing device once the session is complete, wherein the patient session data is collected by the patient-side computing device during the session; and processing the patient session data by means of reference data based on a reference group and the one or more specific skill domains to generate behavioral data of the patient.

[0070] In some embodiments, the computer-implemented method further includes: loading reference data for a reference group, the reference data being based on behavioral data of the reference group, the behavioral data of the reference group being based on reference session data collected during the presentation of the visual stimulus in a data collection playlist and the one or more specific skill domains.

[0071] In some embodiments, the reference data for the reference group includes at least one of the following: for each of the one or more specific skill domains, a specific visual scene related to the specific skill domain, each specific visual scene highlighting one or more areas of interest of the reference group; or a distribution map of the behavioral data of the reference group for each specific visual scene.

[0072] In some embodiments, the reference data includes: contours in the distribution map for each of the one or more specific skill domains and for each specific visual scene, the contours representing thresholds, used to determine whether the patient is paying attention to relevant scene content of the specific visual scene. The computer-implemented method further includes at least one of the following: if the patient's area of ​​attention is within a predetermined area corresponding to the contour, then determining that the patient is paying attention to relevant scene content of the specific visual scene; or if the patient's area of ​​attention is outside the predetermined area, then determining that the patient is not paying attention to relevant scene content of the specific visual scene.

[0073] In some embodiments, the patient's behavioral data includes an attention percentage, defined as the ratio between the number of times the patient pays attention to relevant scene content and the total number of times the patient is viewing the visual stimulus. The computer-implemented method may further include: determining, at a certain moment in the session, that the patient's attention area is within a predetermined region; and determining that moment is one of a plurality of moments when the patient pays attention to relevant scene content.

[0074] In some embodiments, the patient's behavioral data includes a comparison between the patient's percentage of attention and a threshold percentage of attention for the reference group. The comparison may include at least one of the following: the ratio between the patient's percentage of attention and the threshold percentage of attention for the reference group, or the relationship between the patient's percentage of attention and the threshold percentage of attention for the reference group.

[0075] In some embodiments, receiving the request includes: receiving user input from a user interface on a portal of a networked server, the user input indicating the request, and the user interface being displayed on a screen of the computing device.

[0076] In some embodiments, the user interface includes at least one of the following: a first user interface element for viewing a default assessment report; a second user interface element for customizing an assessment report; or a third user interface element for launching an interactive dashboard with the assessment results.

[0077] In some embodiments, the computer-implemented method further includes: popping up a window on the user interface in response to the selection of a second user interface element, the window being used to select a target skill area in the assessment report; receiving second user input in the window for selecting one or more target skill areas; and generating the assessment report based on the selected one or more target skill areas, wherein the one or more target skill areas include the one or more specific skill areas in the assessment results.

[0078] In some embodiments, the computer-implemented method further includes: in response to the selection of the third user interface element, presenting the interactive dashboard in the user interface, wherein the interactive dashboard includes a sub-window for selecting an item from a list of skill domains for interaction.

[0079] In some embodiments, the computer-implemented method further includes, in response to receiving a selection of a specific target skill domain from the list of skill domains, presenting at least one of the following: a change in the patient's attention percentage over a series of consecutive sessions for the specific target skill domain; a change in the ratio between the patient's attention percentage and a threshold attention percentage of the reference group; a change in the relationship between the patient's attention percentage and the threshold attention percentage of the reference group; or, for each of a plurality of visual scenes relating to the specific target skill domain, a first scene highlighting one or more areas of attention of the reference group in the visual scene and a second scene highlighting the patient's areas of attention in the visual scene.

[0080] In some embodiments, the plurality of visual scenes overlap each other in the user interface, and the interactive dashboard includes a sliding user interface element for selecting each of the plurality of visual scenes.

[0081] In some embodiments, the computer-implemented method further includes: storing, on the networked server, annotation data of visual scenes in a data collection playlist of visual stimuli, the annotation data specifying a corresponding specific skill domain associated with the visual scene; and storing, on the networked server, reference data of reference groups, the reference data being based on behavioral data, the behavioral data being based on reference session data collected during the presentation of the data collection playlist of visual stimuli.

[0082] In some embodiments, the session data includes at least one of the following: eye-tracking data collected by an eye tracker device assembled in a patient-side computing device communicating with the network server, or at least one of the following collected by one or more recording devices: image data, audio data, or video data, wherein the one or more recording devices are assembled inside or outside of at least one patient-side computing device.

[0083] Another aspect of this disclosure is a computer-implemented method comprising: accessing a portal on a networked server by a computing device; receiving, by the computing device, user input on a user interface of the portal, the user input being for requesting an assessment result of the patient based on patient session data, the session data being collected during a session during which a data collection playlist of visual stimuli is presented to the patient, wherein at least one visual scene of the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene of the data collection playlist; and presenting, by the computing device, the assessment result on a display screen of the computing device, the assessment result comprising: behavioral data of the patient at moments in the session related to each specific skill domain from one or more of the plurality of skill domains, each moment corresponding to a corresponding visual scene in the visual scene of the data collection playlist.

[0084] In some embodiments, the computer-implemented method further includes: establishing a wireless connection with a patient-side computing device integrated with an eye tracker device; and presenting the user interface to communicate with the patient-side computing device to collect the patient's session data.

[0085] Another aspect of this disclosure features a computer-implemented method comprising: initiating a session for a patient by establishing communication with an operator-side computing device and a patient-side portable tablet computing device, the patient-side portable tablet computing device integrating an eye-tracking device; sequentially presenting visual scenes of a data collection playlist of visual stimuli to the patient on the screen of the patient-side portable tablet computing device, while simultaneously acquiring eye-tracking data of the patient using the eye-tracking device, wherein at least one visual scene of the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene of the data collection playlist; and transmitting session data of the session to a networked server, the session data including the patient's eye-tracking data acquired during the session.

[0086] In some embodiments, the data collection playlist includes visual scenes related to one or more specific skill domains from the plurality of skill domains, and the visual scenes are prioritized in the data collection playlist.

[0087] In some embodiments, the computer-implemented method further includes: when presenting visual scenes of a data collection playlist of visual stimuli in sequence, acquiring at least one of the following: image data, audio data, or video data acquired by one or more recording devices, wherein the one or more recording devices are assembled inside or outside at least one patient-side computing device.

[0088] Another aspect of this disclosure features a system for developmental assessment via eye tracking, comprising: a patient-side mobile computing device including a screen for presenting visual stimuli to a patient; an eye tracker device mounted with the patient-side mobile computing device and oriented at a fixed position relative to the screen of the patient-side mobile computing device to acquire eye-tracking data of the patient when visual stimuli are presented to the patient on the screen of the patient-side mobile computing device; and an operator-side mobile computing device configured to present a user interface that controls the activation of visual stimuli presented to the patient on the screen of the patient-side mobile computing device.

[0089] In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection. In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other wirelessly via a network server. In some embodiments, the network server includes a cloud computing system or cloud server implemented in a cloud environment.

[0090] In some embodiments, the patient-side mobile computing device is configured to transmit data to the networked server, the data including eye-tracking data of the patient collected during a session when visual stimuli from a predetermined list of visual stimuli are presented to the patient.

[0091] In some embodiments, the patient-side mobile computing device is configured to automatically transmit the data in response to the completion of presentation of all visual stimuli in the predetermined list of visual stimuli during the session. In some embodiments, the patient-side mobile computing device is configured to transmit the data in response to receiving a completion indication from the operator-side mobile computing device or the network server. In some embodiments, the operator-side mobile computing device or the network server is configured to generate the completion indication in response to determining that the session has ended or receiving input indicating that the session has ended.

[0092] In some embodiments, the data includes eye-tracking data of the patient acquired during the session and information about the predetermined list of visual stimuli. In some embodiments, the patient-side computing mobile device is configured to transmit the data in two files, including a first file and a second file. The first file includes the patient's eye-tracking data and associated timestamp information, and the second file includes information about the list of visual stimuli. In some embodiments, the associated timestamp information of the eye-tracking data includes a timestamp when the eye-tracking data was generated or acquired, and the information about the predetermined list of visual stimuli includes timestamps when individual visual stimuli in the list were presented.

[0093] In some embodiments, the operator-side mobile computing device is configured to access a portal website in the network server. In some embodiments, the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices via the portal website in the network server. In some embodiments, the user interface of the operator-side mobile computing device includes at least one of the following: information about one or more patients associated with the operator, information about one or more patient-side computing devices associated with the operator, or information about the operator.

[0094] In some embodiments, the patient-side mobile computing device is configured to display connection information, including an access code, on its screen in response to connecting to the networking server. In some embodiments, the operator-side mobile computing device is configured to establish a connection with the patient-side computing device by receiving input of the connection information, including the access code, into the user interface. In some embodiments, the user interface of the operator-side mobile computing device, in response to receiving a selection of a patient-side mobile computing device from among one or more patient-side computing devices presented in the user interface, presents a request for connection information.

[0095] In some embodiments, the operator-side mobile computing device is configured to: present a user interface of an operator application running on either the operator-side computing device or the networked server. In some embodiments, the operator application is configured to: present user interface elements in the user interface for initiating desensitization; and, in response to selection of the user interface elements, transmit a command to the patient-side computing device to play visual desensitization information.

[0096] In some embodiments, the patient-side mobile computing device is configured to: in response to receiving the command, play the visual desensitization information to the patient on the screen of the patient-side mobile computing device, and control the eye tracker device not to collect the patient's eye tracking data when the visual desensitization information is displayed on the screen.

[0097] In some embodiments, the operator application is configured to present the user interface when the visual desensitization information is displayed on the screen of the mobile computing device on the patient side, so that the operator can set up a session for the patient by selecting the patient from the patient list or creating a profile for the patient.

[0098] In some embodiments, the operator application is configured to display instructions in the user interface to adjust the position of the eye tracker device relative to the patient or the position of the patient relative to the patient-side motion computing device based on the position sensed by the eye tracker device relative to at least one eye of the patient. In some embodiments, the sensed position is determined based on image data of the patient's at least one eye captured by an image acquisition device included in or near the eye tracker device. In some embodiments, the sensed position is determined by the patient-side motion computing device or the operator application.

[0099] In some embodiments, the operator application is configured to transmit a command for calibration between the patient and the eye tracker device to the patient-side mobile computing device in response to one of the following: selecting a user interface element for calibration in the user interface, or determining that a session for the patient is set.

[0100] In some embodiments, the patient-side mobile computing device is configured to, in response to receiving the command, sequentially present one or more calibration targets at one or more predetermined locations on the screen of the patient-side mobile computing device when eye-tracking calibration data of the patient is captured using the eye tracker device.

[0101] In some embodiments, the patient-side mobile computing device is configured to: process captured eye-tracking calibration data of the patient for each of the one or more calibration targets to determine the position of the patient's corresponding visual fixation for the calibration target; compare the position of the patient's corresponding visual fixation with a corresponding predetermined position of the calibration target presented on the screen; and determine, based on the result of the comparison, whether the calibration target is calibrated for the patient.

[0102] In some embodiments, the patient-side mobile computing device is configured to: determine that the calibration is complete in response to determining that the one or more calibration targets have been calibrated. In some embodiments, the patient-side mobile computing device is configured to: recalibrate the calibration targets in response to determining that the calibration targets have failed to be calibrated.

[0103] In some embodiments, the patient-side mobile computing device is configured to play desensitized information between the presentation of two adjacent calibration targets. In some embodiments, the operator application is configured to initiate calibration verification in response to receiving an indication that calibration is complete. In some embodiments, the patient-side mobile computing device is configured to present at least one additional calibration target on the screen in response to receiving a request to verify the calibration, while capturing the patient's additional eye-tracking calibration data using the eye-tracking device; and to process the captured patient's additional eye-tracking calibration data to determine the position of the patient's corresponding visual fixation for the at least one additional calibration target.

[0104] In some embodiments, the patient-side mobile computing device is configured to: compare the position of the patient's corresponding visual fixation for the at least one additional calibration target with a corresponding predetermined position on the screen where the at least one additional calibration target is presented; and determine whether the calibration has been verified based on the result of the comparison.

[0105] In some embodiments, the operator application is configured to: simultaneously present in the user interface the at least one additional calibration target and a representation of at least one corresponding visual fixation of the patient for the at least one additional calibration target, the at least one additional calibration target being presented at the corresponding predetermined location, and the representation of at least one corresponding visual fixation of the patient being presented at the determined location of the corresponding visual fixation of the patient; and present in the user interface a first user interface element for verifying calibration and a second user interface element for recalibrating.

[0106] In some embodiments, the operator application is configured to transmit a command for data acquisition to the patient-side computing device in response to one of the following: selecting a user interface element for starting data acquisition, or confirming that the calibration is complete or verified.

[0107] In some embodiments, the patient-side mobile computing device is configured to: in response to receiving the command, sequentially present a predetermined list of visual stimuli to the patient on the screen of the patient-side mobile computing device, while simultaneously capturing the patient's eye-tracking data using the eye-tracking device.

[0108] In some embodiments, the patient-side mobile computing device is configured to present a centering target to the patient on the screen of the patient-side mobile computing device before presenting each predetermined visual stimulus in the predetermined list of visual stimuli. In some embodiments, the patient-side mobile computing device is configured to perform calibration of the patient relative to the eye tracker device between the presentation of two adjacent visual stimuli in the predetermined list of visual stimuli. In some embodiments, the eye-tracking data acquired during the calibration process is used for at least one of: recalibrating the patient's eye-tracking data or determining calibration accuracy.

[0109] In some embodiments, the operator application is configured to present at least one of the following in the user interface: a progress indicator that remains updated during the presentation of the predetermined list of visual stimuli, a user interface element for skipping visual stimuli in the predetermined list of visual stimuli, information about visual stimuli that have been presented or are being presented, or information about visual stimuli to be presented.

[0110] In some embodiments, the networked server is configured to provide a diagnostic result for the patient based on the patient's eye-tracking data, the diagnostic result including at least one index value associated with a developmental disorder. In some embodiments, the operator-side mobile computing device is configured to present the diagnostic result in the user interface.

[0111] In some embodiments, the visual stimulus is predetermined based at least on the patient's age or the patient's condition. In some embodiments, each visual stimulus includes at least one of the following: static visual stimulus, dynamic visual stimulus, pre-recorded visual stimulus, pre-recorded audiovisual stimulus, real-time visual stimulus, real-time audiovisual stimulus, two-dimensional stimulus, or three-dimensional stimulus. In some embodiments, each visual stimulus is standardized to elicit a specific eye movement response with a statistical confidence level greater than 95%. In some embodiments, each visual stimulus is configured to elicit an eye movement response with a statistical confidence level greater than 95% at a discrete spatiotemporal location.

[0112] In some embodiments, the eye-tracking device is connected to the patient-side mobile computing device via a wired connection. In some embodiments, the eye-tracking device and the screen are housed together in a mounting bracket. In some embodiments, the patient-side mobile computing device includes a screen holder that holds the screen and the eye-tracking device in a fixed position relative to the screen.

[0113] In some embodiments, the eye-tracking device includes one or more eye-tracking units disposed at one or more locations near the periphery of the screen. In some embodiments, at least one of the following is a tablet computing device: the patient-side mobile computing device or the operator-side mobile computing device. In some embodiments, the operator-side computing device is configured to communicate with the patient-side mobile computing device via bidirectional communication.

[0114] Another aspect of this disclosure features an apparatus comprising: a patient-side computing device including a screen for presenting visual stimuli to a patient; and an eye-tracking device integrated with the patient-side computing device and configured to acquire eye-tracking data of the patient when the visual stimuli are presented to the patient on the screen of the patient-side computing device. The patient-side computing device includes the patient-side computing device described above.

[0115] Another aspect of this disclosure is a device that includes an operator-side computing device as described above.

[0116] Another aspect of this disclosure features an apparatus comprising: at least one processor; and at least one non-transitory memory coupled to the at least one processor and storing programming instructions executed by the at least one processor to perform operations including: establishing a wireless connection with a patient-side computing device integrated with an eye tracker device; and presenting a user interface to communicate with the patient-side computing device to acquire eye-tracking data from a patient.

[0117] In some embodiments, the operation further includes: accessing a portal website of a network server, wherein the wireless connection is established through the portal website. In some embodiments, the operation further includes: presenting diagnostic results based on the patient's eye-tracking data in the user interface.

[0118] Another aspect of this disclosure is a computer-implemented method for developmental assessment via eye tracking. The computer-implemented method includes: initiating a session for a patient by establishing communication with an operator-side computing device and a patient-side computing device, the patient-side computing device integrating an eye tracker device; sequentially presenting a predetermined list of visual stimuli to the patient on the screen of the patient-side computing device while simultaneously acquiring the patient's eye-tracking data using the eye tracker device; and transmitting session data of the session to a networked server, the session data including the patient's eye-tracking data acquired during the session.

[0119] In some embodiments, at least one of the operator-side computing device or the patient-side computing device is a portable device. In some embodiments, establishing the communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device.

[0120] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes: the operator-side computing device accessing a portal website of the network server; and, in response to receiving a selection for the patient-side computing device in the portal website, wirelessly connecting the operator-side computing device to the patient-side computing device. In some embodiments, establishing the wireless connection between the operator-side computing device and the patient-side computing device includes: the patient-side computing device displaying connection information on its screen; and, in response to receiving input of the connection information from the operator-side computing device, establishing the wireless connection between the operator-side computing device and the patient-side computing device.

[0121] In some embodiments, the computer-implemented method further includes: displaying visual desensitization information to the patient on the screen of the patient-side computing device after establishing the communication. In some embodiments, the computer-implemented method further includes: controlling the eye tracker device not to collect the patient's eye-tracking data when the visual desensitization information is displayed.

[0122] In some embodiments, the computer-implemented method further includes: when the visual desensitization information is displayed, the operator-side computing device accesses a portal on the networked server to set up the session for the patient. In some embodiments, setting up the session includes at least one of: selecting the patient in a patient list, or creating a profile for the patient on the networked server. In some embodiments, the computer-implemented method further includes: determining the relative position between the eye tracker device and at least one eye of the patient; and displaying instructions on the user interface of the operator-side computing device for adjusting the position of the eye tracker device or the position of the patient.

[0123] In some embodiments, the computer-implemented method further includes: determining that the patient is aligned with the eye tracker device in response to determining that the relative position of at least one eye of the patient is a predetermined position in the detection area of ​​the eye tracker device.

[0124] In some embodiments, the computer-implemented method further includes calibrating the patient relative to the eye tracker device by displaying one or more calibration targets to the patient on the screen of the patient-side computing device.

[0125] In some embodiments, calibrating the patient relative to the eye tracker device includes: sequentially presenting each of the one or more calibration targets at a corresponding predetermined position on the screen of the patient-side computing device while capturing eye-tracking calibration data of the patient using the eye tracker device; processing the captured eye-tracking calibration data of the patient for each of the one or more calibration targets to determine the position of the patient's corresponding visual fixation relative to the calibration target; comparing the position of the patient's corresponding visual fixation with the corresponding predetermined position where the calibration target is presented; and determining, based on the result of the comparison, whether the calibration target has been calibrated relative to the eye tracker device.

[0126] In some embodiments, calibrating the patient relative to the eye tracker device further includes: determining that the calibration target has been calibrated and displaying the next calibration target in response to determining that the deviation between the patient's corresponding visual fixation position and the corresponding predetermined position is less than or equal to a predetermined threshold; or determining that the calibration target has failed to be calibrated and re-displaying the calibration target for calibration in response to determining that the deviation is greater than the predetermined threshold.

[0127] In some embodiments, the computer-implemented method further includes: verifying the calibration with one or more new calibration targets after calibrating the patient relative to the eye tracker device. In some embodiments, verifying the calibration includes: sequentially presenting each of the one or more new calibration targets at a corresponding predetermined position on the screen of the patient-side computing device while capturing eye-tracking calibration data of the patient using the eye tracker device; and processing the captured eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual fixation for each of the one or more new calibration targets.

[0128] In some embodiments, verifying the calibration includes: simultaneously presenting on the user interface of the operator-side computing device the one or more new calibration targets and representations of one or more corresponding visual fixations of the patient, the one or more new calibration targets being presented at one or more corresponding predetermined locations, and representations of one or more corresponding visual fixations of the patient being presented at the determined one or more locations; and determining that the calibration has been verified in response to receiving an instruction to verify the calibration result, or in response to receiving an instruction to invalidate the calibration result, initiating recalibration of the patient relative to the eye tracker device.

[0129] In some embodiments, verifying the calibration includes: determining the number of new calibration targets that pass each calibration based on the patient’s corresponding visual fixation location and the corresponding predetermined location; and determining that the calibration is verified if the number or associated percentage is greater than or equal to a predetermined threshold, or determining that the calibration fails and initiating recalibration of the patient relative to the eye tracker device if the number or associated percentage is less than the predetermined threshold.

[0130] In some embodiments, presenting the predetermined list of visual stimuli sequentially to the patient on the screen of the patient-side mobile computing device includes: presenting a centering target to the patient on the screen of the patient-side mobile computing device before presenting each predetermined visual stimulus in the predetermined list of visual stimuli.

[0131] In some embodiments, sequentially presenting the predetermined list of visual stimuli to the patient on the screen of the patient-side computing device includes: performing calibration of the patient relative to an eye tracker device between two adjacent visual stimuli presented in the predetermined list of visual stimuli, wherein eye-tracking data acquired during the calibration is used to calibrate the patient's eye-tracking data or to determine at least one of calibration accuracy.

[0132] In some embodiments, the computer-implemented method further includes presenting on the user interface of the operator-side computing device at least one of the following: a progress indicator that remains updated during the presentation of the predetermined list of visual stimuli, information on visual stimuli that have been presented or are being presented, information on visual stimuli to be presented, or user interface elements for skipping visual stimuli in the predetermined list of visual stimuli.

[0133] In some embodiments, transmitting session data of the session to the network server includes: automatically transmitting session data of the session to the network server by the patient-side computing device in response to one of the following: determining that a predetermined list of visual stimuli has been presented on the screen, or receiving a completion indication of the session from the operator-side computing device.

[0134] In some embodiments, the session data includes information relating to the presented list of predetermined visual stimuli. In some embodiments, the information relating to the presented list of predetermined visual stimuli includes: the name of the presented predetermined visual stimulus and the timestamp associated with the presentation of the predetermined visual stimulus.

[0135] In some embodiments, the session data includes the eye-tracking data and a timestamp associated with the generation or acquisition of the eye-tracking data. In some embodiments, the session data is stored in a first file and a second file, the first file storing the patient's eye-tracking data and the second file storing information related to the presented predetermined list of visual stimuli.

[0136] Another aspect of this disclosure is a computer-implemented method for performing developmental assessments using eye-tracking data by a networked server, the computer-implemented method comprising: receiving session data from a plurality of sessions, each session including eye-tracking data of a corresponding patient in the session; processing the session data from the plurality of sessions in parallel to generate processed session data for the plurality of sessions; and, for each of the plurality of sessions, analyzing the processed session data of the session based on corresponding reference data to generate an assessment result for the corresponding patient in the session.

[0137] In some embodiments, the computer-implemented method further includes: loading corresponding reference data for the plurality of sessions in parallel with the session data for processing the plurality of sessions.

[0138] In some embodiments, the network server includes multiple processing cores. Parallel processing of session data for the multiple sessions includes: using a first plurality of processing cores to process the session data for the multiple sessions in parallel, and using different second plurality of processing cores to load corresponding reference data for the multiple sessions, wherein the number of the first plurality of processing cores is greater than the number of the second plurality of processing cores.

[0139] In some embodiments, analyzing the processed session data of the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions includes using the plurality of processing cores, which include the first plurality of processing cores and the second plurality of processing cores.

[0140] In some embodiments, analyzing the processed session data of the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions includes at least one of the following: comparing the processed session data of the sessions with the corresponding reference data, using the corresponding reference data to infer the assessment result of the corresponding patient from the processed session data, or using at least one of a statistical model or an artificial intelligence (AI) model. In some embodiments, the corresponding reference data includes historical eye-tracking data or results from patients with substantially the same age or condition as the corresponding patient.

[0141] In some embodiments, the computer-implemented method further includes: generating the assessment result based on the corresponding patient's previous session data. In some embodiments, the computer-implemented method includes: assigning a corresponding container to each of the plurality of sessions; processing session data of the session in the corresponding container; and analyzing the processed session data of the session based on corresponding model data to generate an assessment result for the corresponding patient in the session.

[0142] In some embodiments, the eye-tracking data is associated with a predetermined list of visual stimuli, which is presented to the patient when the eye-tracking data is collected during the session, wherein the session data includes information associated with the predetermined list of visual stimuli during the session.

[0143] In some embodiments, the computer-implemented method further includes: linking eye-tracking data of the session to a predetermined list of visual stimuli in the session. In some embodiments, linking eye-tracking data of the session to the predetermined list of visual stimuli in the session includes: in a respective container, decomposing the eye-tracking data into multiple parts based on the information associated with the predetermined list of visual stimuli, each part of the eye-tracking data being associated with at least one of: a corresponding predetermined visual stimulus or a corresponding calibration.

[0144] In some embodiments, processing the session data of the session includes a portion of processing eye-tracking data associated with the corresponding predetermined visual stimulus based on information processing of the corresponding predetermined visual stimulus.

[0145] In some embodiments, the computer-implemented method further includes: in the respective container, recalibrating a portion of the eye-tracking data associated with a corresponding predetermined visual stimulus based on at least one portion of the eye-tracking data associated with the corresponding calibration.

[0146] In some embodiments, the computer-implemented method further includes: determining calibration accuracy in the corresponding container using at least a portion of eye-tracking data associated with the corresponding calibration and multiple predetermined locations presenting multiple calibration targets in the corresponding calibration.

[0147] In some embodiments, receiving session data of the plurality of sessions includes: receiving session data of the plurality of sessions from a plurality of computing devices associated with a corresponding entity via a portal website. In some embodiments, the computer-implemented method further includes: in response to receiving session data of a session, adding a file pointer for the session data of the session to a processing queue to be processed. In some embodiments, the computer-implemented method further includes: storing the session data of the session in a database using the file pointer for the session; and retrieving the session data of the session from the database using the file pointer for the session.

[0148] In some embodiments, the computer-implemented method further includes, for each entity, storing session data from one or more computing devices associated with the entity in a corresponding repository. In some embodiments, the corresponding repository for the entity includes at least one of the following: information about the entity, information about one or more operator or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted in the entity, information about one or more patients associated with the entity, or historical information from the corresponding repository.

[0149] In some embodiments, the corresponding repository is included in a NoSQL database. In some embodiments, the corresponding repository is isolated from one or more other repositories and is not accessible to one or more other entities.

[0150] In some embodiments, the computer-implemented method further includes: dynamically adjusting the resources of the network server based on the number of computing devices accessing the network server.

[0151] In some embodiments, the computer-implemented method further includes: copying data from a first data center to a second data center; and automatically redirecting traffic to the second data center in response to determining that the first data center is inaccessible. In some embodiments, both the first data center and the second data center include at least one of the following: a portal accessible to operator-side computing devices, an operator application, or an application layer for data processing and data analysis.

[0152] In some embodiments, the computer-implemented method further includes storing the same data in multiple data centers, wherein the data includes application data for entities and information associated with eye-tracking data.

[0153] In some embodiments, the computer-implemented method further includes: associating the generated assessment results with the corresponding patient in the session; and generating a corresponding patient assessment report.

[0154] In some embodiments, the computer-implemented method further includes: outputting an evaluation result or evaluation report to be presented in the user interface of the operator-side computing device. In some embodiments, the evaluation report includes at least one of the following: information about the corresponding patient, information about the entity performing the session for the corresponding patient, information about the calibration accuracy in the session, information about session data acquisition, or the evaluation result of the corresponding patient.

[0155] In some embodiments, the assessment results are used to indicate the likelihood that the corresponding patient has developmental, cognitive, social, or mental deficits or abilities. In some embodiments, the assessment results are used to indicate the likelihood that the corresponding patient has autism spectrum disorder (ASD) or not ASD. In some embodiments, the assessment results include corresponding scores for one or more of the following: social deficit index, language ability index, non-verbal ability index, social adaptability index, and social communication index.

[0156] In some embodiments, the evaluation results include at least one of the following: a visualization of eye-tracking data on still images of visual stimuli overlapping at socially relevant moments; an animated visualization of eye-tracking data on still images of visual stimuli overlapping at socially relevant moments; a visualization of aggregated reference data of multiple reference patients matched with the corresponding patient on one or more patient attributes; or annotations describing at least one of the visual stimulus content or eye gaze patterns.

[0157] In some embodiments, the corresponding patient's age is in the range of 5 months to 7 years, including: age in the range of 5 months to 43 months or 48 months, age in the range of 16 months to 30 months, age in the range of 18 months to 36 months, age in the range of 16 months to 48 months, or age in the range of 16 months to 7 years.

[0158] Another aspect of this disclosure features a system comprising: at least one processor; and one or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a computer-implemented method as described herein.

[0159] Another aspect of this disclosure is a non-transitory computer-readable medium containing one or more stored instructions that, when executed by at least one processor, cause the at least one processor to perform a computer-implemented method as described herein.

[0160] One or more embodiments described herein achieve numerous technical effects and advantages. In a first example, some embodiments provide a convenient, miniaturized, and efficient computing system for advantageously collecting eye-tracking data and subsequently transmitting such data for analysis and diagnostic results. The computing system may include at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye tracker. These portable devices may be differently equipped (different peripherals or devices, different user interfaces, etc.) and may be wirelessly connected to each other (without a physical connection) or wirelessly connected to a networked server platform (which in turn provides communication between the operator-side and patient-side portable devices). For example, an operator may use the operator-side portable device to calibrate the eye tracker device with the patient and control the playback of predetermined visual stimuli on the patient-side portable device to acquire the patient's eye-tracking data using the eye tracker device. Once the session is complete, the patient-side portable device may transmit session data (e.g., detected eye-tracking data and information about the played visual stimuli) to a cloud server for data storage, processing, and analysis. The cloud server may be remotely connected to these portable devices (e.g., via a network).

[0161] In another example, the technology implemented in this paper provides more detailed and interactive reporting output, allowing users to deeply analyze behaviors and metrics related to specific scenarios or groups of scenarios concerning developmental skills. Annotations made by expert clinicians based on behaviors in reference groups can: accurately identify specific skill areas / skills for patient diagnosis and / or treatment; effectively tailor data collection playlists for patients in selected skill areas / skills; monitor patient improvement or treatment outcomes in selected skill areas / skills; and / or provide patients with automated, accurate, consistent, rapid, manual, and / or cost-effective assessments of developmental disorders. This technology enables operators / users to manage and / or explore session outcomes at multiple customizable levels with detail. Visualizations and metrics of skill-specific behaviors can be configured to: provide users with an objective quantification of the patient's generalization of the target skill outside the treatment setting, and indicate which aspects of treatment are consistent with patient progress. Users (e.g., treatment providers, clinicians, or patient guardians) can view: whether the patient has shown any improvement in one or more target skill areas, whether the treatment for the patient is effective, and / or whether new or modified treatments can be used to replace the current treatment.

[0162] The techniques described herein can be used to provide early identification and assessment of risks to a patient’s developmental, cognitive, social, language or non-verbal abilities or mental capacity or deficits, for example, measuring visual attention to social information in the environment relative to a normative, age-specific benchmark. Patients can be between 5 months and 7 years old, for example, 16 months to 7 years, 12 months to 48 months, 16 months to 30 months, or 18 months to 36 months.

[0163] According to certain aspects, changes in a patient's visual fixation over time relative to certain dynamic stimuli provide a marker of potential developmental, cognitive, social, or mental abilities or impairments, such as ASD. Visual fixation is a type of eye movement used to stabilize visual information on the retina and typically occurs when a person gazes at or "fixates" on a point or area on the display plane. In some embodiments, a patient's visual fixation is identified, monitored, and tracked over time through repeated eye-tracking sessions and / or by comparison with model data based on a large number of patients of similar age and / or background. The data related to visual fixation are then compared to relative standards to determine the patient's potentially increased risk of this condition. Changes in visual fixation (especially a decrease or increase in visual fixation on images of the eyes, body, or other areas of interest of a person or object displayed on a visual stimulus) provide an indication of developmental, cognitive, or mental impairments by comparing visual fixation data with similar data from patients of typical development or with the patient's own, previous visual fixation data. The techniques can be applied to quantitatively measure and monitor symptom presentation of corresponding abilities or deficits, and in some cases, to provide more accurate and relevant guidance to patients, families, and service providers. According to another aspect, the technology can be used to predict outcomes for individuals with autism (thus providing guidance), while also providing similar diagnostic and guidance indicators for holistic developmental, cognitive, social, or mental abilities or deficits.

[0164] As described below, the techniques described herein for detecting developmental, cognitive, social, or mental deficits are applicable to the detection of conditions including, but not limited to: expressive and receptive language delay, nonverbal developmental delay, intellectual disability, intellectual disability of known or unknown genetic origin, traumatic brain injury, unclassified infantile disorder (DOI-NOS), social communication impairment, autism spectrum disorder (ASD), and conditions such as attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), concussion, sports injury, and dementia.

[0165] It is understood that the methods according to this disclosure may include any combination of the embodiments described herein. That is, the methods according to this disclosure are not limited to the combinations of embodiments specifically described herein, but also include any combination of the provided embodiments.

[0166] Details of one or more embodiments of this disclosure are set forth in the accompanying drawings and the following description. Other embodiments and advantages of this disclosure will be apparent from the specification, drawings, and claims. Attached Figure Description

[0167] FIG. 1A This is a block diagram of an example environment for assessing developmental disorders according to one or more embodiments of the present disclosure.

[0168] FIG. 1B Examples of illustrative visual stimuli presented on the screen of a patient device according to one or more embodiments of the present disclosure are shown.

[0169] FIG. 1C Examples of patient devices according to one or more embodiments of the present disclosure are shown.

[0170] FIG. 1D Examples of illustrative user interfaces presented on an operator device according to one or more embodiments of the present disclosure are shown.

[0171] FIG. 2A This is a block diagram of an example system for assessing developmental disorders via eye tracking, according to one or more embodiments of the present disclosure.

[0172] FIG. 2B The illustration shows one or more embodiments according to this disclosure. FIG. 2A An example of managing session data in a system.

[0173] FIG. 2C The illustration shows one or more embodiments according to this disclosure. FIG. 2A An example of managing multiple session data in parallel within a system.

[0174] FIG. 2D The illustration shows one or more embodiments according to this disclosure. FIG. 2A The system stores different types of documents as sample databases for application data.

[0175] FIG. 2E The illustration shows one or more embodiments according to this disclosure. FIG. 2A An example of a multi-tenant architecture in a system.

[0176] FIG. 2F to FIG. 2G One or more embodiments of the present disclosure are shown for use with FIG. 2A This is an example of system data backup.

[0177] FIG. 3 This is a flowchart of an example process for session data acquisition according to one or more embodiments of the present disclosure.

[0178] FIG. 4A to FIG. 4J A series of illustrative user interfaces presented on an operator device (graphic a) and a participant device (graphic b) during session data acquisition, according to one or more embodiments of the present disclosure.

[0179] FIG. 5Example session data according to one or more embodiments of the present disclosure is shown, including (a) video playlist data; and (b) eye-tracking sensor data.

[0180] FIG. 6 This is a flowchart of an example process for managing session data according to one or more embodiments of the present disclosure.

[0181] FIG. 7A to FIG. 7B A flowchart is shown as another example process for managing session data according to one or more embodiments of this disclosure.

[0182] FIG. 8A An example results interface according to one or more embodiments of the present disclosure is shown, which displays at least one index value based on eye-tracking data.

[0183] FIG. 8B to FIG. 8C Another example results interface according to one or more embodiments of the present disclosure is shown, which displays performance-based measurements of developmental assessment based on eye-tracking data, with reference to the following examples: FIG. 8B Nonverbal communication and gestures (A) and shared attention & eye contact (B) in the context of communication, and FIG. 8C Facial expressions (C) and pointing and social monitoring (D) in the image.

[0184] FIG. 9 This is a flowchart of an example process for session data acquisition according to one or more embodiments of the present disclosure.

[0185] FIG. 10 This is a flowchart of an example process for managing session data according to one or more embodiments of the present disclosure.

[0186] FIG. 11 Examples of comparisons between annotated video scenes for different specific skill domains, information on typical gaze behavior groups, and information on patient gaze behavior are shown according to one or more embodiments of this disclosure.

[0187] FIG. 12A An example illustrative user interface for session initiation presented on an operator device according to one or more embodiments of the present disclosure is shown.

[0188] FIG. 12B An example illustrative window is shown on an operator device, according to one or more embodiments of the present disclosure, for selecting a target skill domain for a target monitoring session.

[0189] FIG. 13A An example illustrative user interface for reviewing session information on a user device, according to one or more embodiments of this disclosure, is shown.

[0190] FIG. 13B-1 An example portion of an evaluation report according to one or more embodiments of the present disclosure is shown, illustrating a comparison between annotated video scenes for different specific skill domains, information on typical gaze behavior groups, and information on patient gaze behavior.

[0191] FIG. 13B-2 Another example portion of an assessment report according to one or more embodiments of the present disclosure is shown, illustrating monitoring of a specific skill and information about the characteristic skill.

[0192] FIG. 13C An example illustrative window is shown on a user device, according to one or more embodiments of the present disclosure, for selecting a target skill area to generate a customized report.

[0193] FIG. 13D An example interactive results dashboard presented on a user device according to one or more embodiments of this disclosure is shown.

[0194] FIG. 14 This is a flowchart of an example process for managing specific skills for developmental disability assessment, according to one or more embodiments of this disclosure.

[0195] FIG. 15A An example illustrative user interface is shown on a computing device when a cloud server is running a data aggregator application, according to one or more embodiments of this disclosure.

[0196] FIG. 15B An example illustrative user interface is shown on a computing device when a cloud server runs a data aggregator application to aggregate data from external tools, according to one or more embodiments of the present disclosure.

[0197] FIG. 15C An example illustrative user interface is shown on a computing device when a cloud server is running a data aggregator application for an operator to manually enter patient information, according to one or more embodiments of the present disclosure.

[0198] FIG. 15D An example illustrative user interface for session initiation presented on a computing device according to one or more embodiments of the present disclosure is shown.

[0199] FIG. 15E An exploded diagram illustrating an example effort in treating a specific skill area within a patient's treatment plan, according to one or more embodiments of this disclosure.

[0200] FIG. 15F A graph illustrating a patient’s attention to a relevant scene of a characteristic skill over a period of time is shown according to one or more embodiments of the present disclosure.

[0201] FIG. 15G A graph illustrating the relationship between inputs and effects in different skill areas is shown according to one or more embodiments of this disclosure.

[0202] FIG. 15H An example illustrative user interface is shown that is presented on a computing device when a cloud server outputs a treatment plan, according to one or more embodiments of the present disclosure.

[0203] FIG. 16A to FIG. 16F An example result interface of an example evaluation report of an evaluation system according to one or more embodiments of the present disclosure is shown.

[0204] FIG. 17A This is a flowchart of an example process for managing a treatment plan for developmental disorder assessment, according to one or more embodiments of this disclosure.

[0205] FIG. 17B This is a flowchart of an example process for managing assessment reports according to one or more embodiments of this disclosure.

[0206] FIG. 18 An architecture of a cloud computing system according to one or more embodiments of the present disclosure is shown.

[0207] FIG. 19 An architecture of a computing device according to one or more embodiments of the present disclosure is shown.

[0208] The same reference numerals and names in each figure indicate the same elements. It is also understood that the various exemplary embodiments shown in the figures are merely illustrative representations and are not necessarily drawn to scale. Detailed Implementation

[0209] This disclosure describes portable devices with user detection capabilities, such as eye-tracking devices or other sensors, and computer systems and methods that include such portable devices or data collected from such devices (such as eye-tracking data and / or other multimodal data, such as facial expressions, verbal expressions, and / or body movements). In some examples, the systems, devices, methods, and techniques described herein are capable of using eye-tracking data and / or multimodal data (e.g., facial expressions, verbal expressions, and / or body movements) generated in response to the display of specific predetermined visual stimuli (e.g., one or more films or videos) and providing patients with objective assessments of improvements in a patient's developmental, cognitive, social, or mental abilities or deficits (including autism spectrum disorder (ASD)). These techniques are capable of providing detailed and interactive reporting outputs, allowing users to deeply analyze behaviors and indicators related to specific skill domains (and / or skills), such as developmentally relevant skill domains / skills (or therapeutically specific skills). In this disclosure, the term "skill domain" refers to a set of skills related to each other. The term "skill domain" is used interchangeably with the terms "developmental concept" or "skill category." Example skill areas may include: manding, listener response, turn-taking, shared attention, social tact, and / or play. Skill areas may be associated with developmental assessments and / or therapy. For illustrative purposes, therapy for a specific skill area (or skill) is used as an example of that specific skill area (or skill).

[0210] In some implementations, for example, such as FIG. 11 The text further details that pre-annotated visual stimuli can be pre-annotated moment-by-moment for skill relevance, for example, by associating specific skill domains and / or skills with scenarios of visual stimuli that relate to those skill domains and / or skills. These skill domains or skills can be targeted as therapeutic goals, which is important, for example, for Certified Behavioral Analysts (BCBA®). Example specific skill domains may include: autonomous requests, listener responses, turn-taking, co-attention, social decorum, and / or play. Annotations can be made by one or more expert clinicians by observing a scenario of visual stimuli and optionally observing the behavior (e.g., gaze behavior, facial expressions, verbal expressions, and / or body movements) of a reference group (e.g., typical children of similar age) when viewing the same visual stimuli. Scenario annotations can be used for any developmental skill domain (or concept), therapeutic cue / measure, severity index, or any other skill present or associated with the scenario content. Visual representations of behaviors in example scenarios can be considered representative of skill domains or skills. Given a reference group, behavioral convergence can be quantified for scenarios annotated for specific skill domains or skills, which can be used as additional skill-specific indicators.

[0211] Annotations made by expert clinicians based on the behavior of a reference group can: accurately identify specific skill areas / skills used for patient diagnosis and / or treatment; effectively tailor the data collection playlist for patients in selected skill areas / skills; monitor patient improvement or treatment outcomes in selected skill areas / skills; and / or provide patients with automated, accurate, consistent, rapid, manual, and / or cost-effective developmental disorder assessments. These technologies enable operators / users to manage and / or explore session outcomes and detailed information at multiple customizable levels. Configurable behavioral visualizations and metrics for specific skills provide users with objective quantification of the patient's generalization of the target skill outside the treatment setting and indicate which aspects of treatment are consistent with patient progress.

[0212] For example, these technologies can customize the list of visual stimulus data to be collected. From the portal of the networked server, when a session is initiated for a patient, the operator (e.g., a treatment provider or clinician) is prompted on the user interface displayed on the screen of the operator-side computing device to select the session type (e.g., diagnostic session, monitoring session, or target monitoring session), for example, as... FIG. 12A to FIG. 12B This will be discussed in further detail below. If a target monitoring session is selected, a prompt window will appear allowing the operator to select a set of skill areas that the operator wishes to target. The default selection can be any skill area selected in a previous target monitoring session. The data collection playlist presented on the patient-side computing device, which communicates with the operator-side computing device and / or the networked server, can prioritize data collection of videos with moments relevant to the selected skill areas, for example, by: reordering the standard playlist; adding new videos specifically enriched for the selected skill areas; and / or reducing or removing videos irrelevant to the selected skill areas.

[0213] In some implementations, for example, such as FIG. 13A to FIG. 13CAs discussed further in detail, these technologies enable users (e.g., treatment providers, clinicians, or patient caregivers) to see information about a patient in one or more specific skill areas in a patient's diagnostic report. Patient information can be displayed, for example, in a manner comparing it to information from a reference group, such as a distribution plot (e.g., a saliency plot) or frames / moments showing typical gaze behavior in the area. Patient information may include the patient's convergent gaze percentage (or attention percentage) at moments related to a specific skill area. One or more specific skill areas for a patient can be automatically selected, such as those with the largest amount of reliable data, the most frequently requested skills, those with particularly high, low, or representative scores, or combinations thereof. One or more specific skill areas can be pre-selected as target skill areas when initiating a targeted monitoring session or customizing a patient's diagnostic results. If a monitoring session is performed and one or more previous sessions have been performed on the patient, the monitoring report can indicate changes in the patient's convergent gaze percentage compared to previous sessions. In this way, users (e.g., treatment providers, clinicians, or patient caregivers) can see whether a patient has made any progress in one or more target skill areas, whether treatment for the patient is effective or efficient, and / or whether new or modified treatments can be used to replace the current treatment.

[0214] These technologies also allow users to select interactive results dashboards from patient session pages on the portal, for example, in FIG. 13D This will be discussed in further detail. Users can interactively explore outcomes in any skill domain, such as a patient's rating of a specific skill over a period of time or across multiple consecutive sessions, and / or a moment-by-moment (or frame-by-frame) comparison of the patient's behavior (e.g., fixation behavior) with a reference group. For example, users can view possible skills grouped by skill domain or developmental concept, age, or treatment type. The interactive outcomes dashboard allows users to select a subset of target skill domains of interest and view combined metrics for the selected subset. The interactive outcomes dashboard may also allow users to watch video or view moments / frames of patient behavior, each moment with metrics contributing to a specific skill, and / or still images of the behavior and / or scene content of a reference group.

[0215] In some implementations, in addition to the eye-tracking device, the patient-side computing device may also be equipped with a recording device (e.g., such as...). FIG. 1A to FIG. 1B(As further detailed below), one or more external recording devices may be configured to record video and / or audio of the patient during viewing sessions, during unstructured social interactions, and / or during therapeutic sessions. This video and / or audio may be processed by an artificial intelligence (AI) model to generate multimodal data, such as a machine learning (ML) model, a single-layer neural network model, a multi-layer neural network model, or another trained AI model that has been trained during the same session with reference group video / audio under the guidance / annotation of an expert clinician on a list of specific therapeutic skills. Multimodal data can replace, supplement, validate, or provide additional context for the monitoring of specific therapeutic skills or the diagnosis, assessment, and / or severity measurement of developmental disorders.

[0216] In some embodiments, the eye tracker device includes one or more eye tracking units configured to directly capture / track a patient's eye movements (e.g., by detecting reflected and / or scattered illumination light, such as infrared light). In some embodiments, the eye tracker device includes one or more image acquisition devices (e.g., cameras) configured to determine a patient's eye movements based on images of the patient's eyes captured while the patient is viewing a visual stimulus, or images of the patient's eye position and / or captured head movement / facial data. In some embodiments, the eye tracker device includes one or more eye tracking units and one or more image acquisition devices, and the eye movement data may include at least one of the following: direct eye movements of the patient, captured images and / or positions of the patient's eyes, or eye movements derived from captured images and / or positions.

[0217] In some embodiments, the eye tracker device is configured to convert a patient's eye movement data into eye-tracking data, which may include information such as pupil position, gaze vectors for each eye, and / or gaze points. In some embodiments, the eye tracker device is configured to: determine first eye-tracking data based on the patient's direct eye movements, and determine second eye-tracking data based on the patient's eye movements derived from captured images and / or positions of the patient's eyes. The first and second eye-tracking data may be interchangeable, complementary, validated, or provide additional background information. The eye tracker device may be configured to generate final eye-tracking data for the patient based on the first and second eye-tracking data.

[0218] Eye-tracking devices can transmit a patient's eye-tracking data to an assessment system (such as EarliPoint) for processing, for example, generating a patient assessment report or rating report. The assessment system may include, for example, a cloud server as described in this disclosure, such as... FIG. 1A Cloud server 110 or relatedFIG. 2A to FIG. 2G The cloud server. In some cases, the eye tracker device transmits first eye-tracking data to the assessment system for processing. In some cases, the eye tracker device transmits second eye-tracking data to the assessment system for processing. In some cases, the eye tracker device transmits both first and second eye-tracking data to the assessment system for processing, wherein the processing results based on the first and second eye-tracking data can replace, supplement, verify, or provide additional background information. In some cases, the eye tracker device transmits final eye-tracking data to the assessment system for processing. In some cases, the eye tracker device transmits first, second, and final eye-tracking data to the assessment system for processing.

[0219] In some implementations, the assessment system for developmental disorders provides a data aggregator, for example, such as FIG. 15A to FIG. 15B Further details are provided below. The data aggregator can be configured to connect with one or more third-party tools to ingest (e.g., by parsing) patient treatment data, including data from EHR (Electronic Health Record) / EMR (Electronic Medical Record) and ABA (Applied Behavior Analysis) practice management tools, and optionally, reference patient data to ingest (e.g., by parsing) the patient's (and / or other patients') treatment plans, goals, behavioral stress, patient responses over time, and other relevant clinical or treatment data. The data aggregator can be configured to combine assessment data made by the assessment system to build a large-scale, unique data repository of clinical treatment and patient trajectories, thereby enabling a comprehensive understanding of the patient. These technologies enable the data aggregator to retrieve reference data based on patient information (e.g., based on similar groups, age, background, developmental stage, demographics, or region), allowing the assessment system to generate specific treatment plans based on all relevant data, including the patient's own data and reference data.

[0220] In some implementations, the assessment system is configured to provide practice management tools (e.g., ... FIG. 15C to FIG. 15D and FIG. 15H (As further detailed below), this tool provides clinicians (e.g., those not using third-party systems to track treatment plans and data) with their own direct-entry practice management tools. The practice management tool can be configured to allow clinicians to manually enter treatment plan information or upload existing treatment plan documents via the assessment system's web browser application. Clinicians can also annotate treatment goals or enter treatment data in real time during treatment, and it can also be used to track treatment billing. The assessment system can automatically parse and process the input to extract and tabulate relevant data, including the time spent on each skill.

[0221] The assessment system can also be configured to perform target monitoring sessions or customizable eye-tracking sessions using playlists configured to quantify progress in skill areas most relevant to the patient's treatment plan, strengths, weaknesses, and / or developmental stage (e.g., such as...). FIG. 12A to FIG. 12B or FIG. 15D (Further details are provided below). The assessment system can be configured for monitoring treatment progress, such as automatically comparing a patient's current treatment plan with objective skill level progress during an eye-tracking session to identify patient treatment areas associated with measurable skill improvement. The assessment system can be configured to generate guided treatment plans based on models and aggregated treatment data, such as through artificial intelligence (AI) models or algorithms. The assessment system can recommend optimal treatment methods (e.g., EarliPoint, ESDM (Early Intervention Denver Model), ESI (Early Social Interaction), DTT (Discrete Trial Training), Jasper (Co-Focus Symbolic Play Participation and Modulation), or the ImPACT project (Elevating Parents to Communication Teachers)) and generate specific plans based on the patient's unique presentation. Treatment plans can be customized and formatted for easy import into and from third-party tools.

[0222] In some embodiments, the assessment system may provide treatment providers with specific tutorials (e.g., video / audio / text) related to a selected treatment plan, enabling them to understand and learn how to implement it. These specific tutorials are content-based and can be selected from multiple tutorials based on the chosen treatment plan, allowing treatment providers to understand the plan using only the selected tutorials (e.g., fewer than 10 tutorials) without having to review a large number of tutorials (e.g., approximately 100). In this way, the assessment system enables inexperienced or less experienced treatment providers (e.g., providers in rural areas) to understand, interpret, and / or perform the selected treatment plan. It also enables experienced providers to use the selected tutorials as evidence, reference, or support to understand, interpret, and / or perform the selected treatment plan.

[0223] In some embodiments, the assessment system generates an assessment report of the patient's developmental disorder, for example, such as FIG. 8A to FIG. 8C or FIG. 16A to FIG. 16FFurther details are provided below. The assessment report may include patient information, conversation information, and a summary of the assessment results (e.g., ASD or non-ASD). The assessment results may also include evaluation results, which may include one or more index scores, such as a social deficit index score, a language proficiency index score, and a non-language learning index score. These scores may be obtained from an artificial intelligence (AI) model, such as a machine learning (ML) model, a single-layer neural network model, a multi-layer neural network model, or another trained AI model, in response to inputs of processed conversation data and corresponding model data for a specific conversation (as described above). The assessment report may include correlations (e.g., side-by-side graphical correlations) that present one or more individual index scores of the assessment system (as described above, obtained from the AI ​​model) related to a “reference assessment metric” (such as the ADOS-2 metric or the Mullen Early Learning Scale metric), thereby providing the healthcare provider reviewing the assessment report with greater understanding (even in cases where the healthcare provider has limited experience). As used herein, the term “reference assessment measure” refers to a measurement derived from an assessment scale, tool, or system that has been professionally adopted, implemented, and / or peer-reviewed by those medically trained in diagnosing one or more developmental disorders, such as ASD. Assessment scales, tools, or systems may include at least one of the following: ADOS-2 (Autism Diagnosis Observation Scale - Version 2), MSEL (Mullen Early Learning Scale), ADI-R (Autism Diagnosis Interview Scale - Revised), CARS (Childhood Autism Rating Scale), VABS (Vineland Adaptive Behaviors Scale), DAS-II (Differential Abilities Scale - Version 2), WISC (Wechsler Intelligence Scale for Children), WASI (Wechsler Short Form Intelligence Scale), or VB-MAPP (Language and Behavioral Milestones Assessment and Placement Program). The term “reference assessment measure” may also be referred to as “clinical reference assessment measure,” “developmental assessment measure,” “developmental reference measure,” “standard clinical assessment,” “standard developmental reference measure,” or any other suitable term. In some examples, reference assessment measures are used to assess one or more developmental disorders or one or more developmental skills. In some examples, reference assessment measures are used for treatment. The assessment report may also include visualized individual test results (e.g., including the patient’s gaze behavior when viewing visual stimuli) and / or attention funnel (e.g., time-by-time gaze behavior across multiple visual scenes).

[0224] To provide a general understanding of the systems, devices, and methods described herein, certain illustrative embodiments will be described. It should be understood that such data, even if not indicative of an impairment, can provide an indicator of the typicality of canonical development, thereby providing an indication of the variability of canonical development. Furthermore, all the components and other features outlined below can be combined with each other in any suitable manner and can be adapted and applied to systems outside of medical diagnostics. For example, the interactive visual stimuli of this disclosure can be used as a tool for therapy. In addition, the collected data can generate indicators of certain types of visual stimuli that patients preferentially seek. These indicators of preference have applications both within and outside the fields of medical diagnostics and therapy, including, for example, advertising or other industries interested in data related to visual stimulus preferences.

[0225] Example Environment and System FIG. 1A This is a block diagram of an example environment 100 for assessing developmental disorders via eye tracking, according to one or more embodiments of this disclosure. Environment 100 involves a cloud server 110 and multiple computing systems 120-1, ..., 120-n (collectively referred to as computing system 120 or individually referred to as computing system 120) communicating via a network 102. Cloud server 110 is capable of providing developmental disorder assessment or diagnostic services to multiple users (e.g., treatment providers). Treatment providers can use the corresponding computing system 120 to conveniently and reliably collect patient data in sessions (e.g., processes related to data collection) of any age (from newborns to the elderly), with the example embodiments described below being particularly suitable for young children or other young patients. Data collected in a session (or session data) may include eye-tracking data generated in response to displaying a specific predetermined visual stimulus (e.g., one or more videos) to the patient. The computing system 120 can securely transmit session data to the cloud server 110, which can store, process and analyze the session data to diagnose ASD or other cognitive, developmental, social or mental abilities or impairments of the patient, and provide diagnostic results or reports to the treatment provider in a highly secure, robust, fast and accurate manner, as detailed below.

[0226] Treatment providers can be individual healthcare organizations, including but not limited to autism treatment centers, healthcare facilities, specialists, physicians, or clinical research institutions. Healthcare organizations may provide patients with developmental assessments and diagnoses, clinical care, and / or treatment services. FIG. 1AAs shown, a patient (e.g., an infant or child) may be brought to a healthcare facility by a caregiver (e.g., a parent). An operator (e.g., a specialist, physician, medical assistant, technician, or other healthcare professional in the facility) may use computing system 120 to collect, for example, non-invasive eye-tracking data from the patient while he or she views visual stimuli (e.g., dynamic visual stimuli, such as videos) depicting common social interactions (e.g., interactions between two or three people). The stimuli displayed to the patient for data collection purposes may be patient-specific, for example, based on the patient's age and condition. The stimuli can be any suitable visual image (whether static or dynamic), including videos or films, as well as still images or any other visual stimuli. It should be understood that videos or films are cited only by way of example, and any such discussion also applies to other forms of visual stimuli.

[0227] In some embodiments, such as FIG. 1A As shown, the computing system 120 includes at least two separate computing devices 130 and 140, such as an operator-side computing device 140 and at least one patient-side computing device 130. Optionally, the two computing devices 130 and 140 can be wirelessly connected, for example, via wireless connection 131, without a physical connection. Wireless connection 131 can be implemented via a cellular network, wireless network, Bluetooth, near field communication (NFC), or other standard wireless network protocols. In some embodiments, the patient-side computing device 130 is configured to connect to the operator-side computing device 140 via a wired connection such as Universal Serial Bus (USB), for example, when wireless connection 131 fails.

[0228] In some cases, the two computing devices 130 and 140 communicate with each other via network 102, each communicating with cloud server 110, which in turn provides communication between operator-side computing device 140 and patient-side computing device 130. For example, as in FIG. 4A to FIG. 4B As further discussed, operators can log in to a portal running on cloud server 110, for example, via a web-based operator application, to manage devices, patients, and data. Operators can use operator-side computing devices 140 (e.g., tablets) to communicate with multiple patient-side portable devices 130, for example, within the same healthcare facility, to collect eye-tracking data from multiple patients in multiple sessions, thereby greatly simplifying computing system 120, reducing system costs, improving work efficiency, and reducing operator workload.

[0229] Computing device 130 or 140 may include any suitable type of device, such as a tablet computing device, camera, handheld computer, portable device, mobile device, personal digital assistant (PDA), cellular phone, network device, smartphone, enhanced general packet radio service (EGPRS) mobile phone, or any suitable combination of any two or more of these data processing devices or other data processing devices. As an example, FIG. 19 The architecture of the computing device is shown, which can be implemented as computing device 130 or 140.

[0230] At least one of computing devices 130 or 140 can be a portable device, such as a tablet device. In some cases, both computing devices 130 and 140 are portable and wirelessly connected to each other. In this way, computing system 120 can be moved and repositioned more easily, and allows the operator more flexibility in choosing his or her position relative to the patient. For example, the operator (carrying operator-side computing device 140) is not physically bound to patient-side computing device 130 and can easily position himself or her in an optimal position (e.g., away from the patient's direct line of sight) during setup and data collection. Furthermore, the patient (e.g., a toddler or other child) can be carried by caregivers (e.g., parents) in a more suitable position and in a more comfortable manner, allowing the patient to focus more on the visual stimuli being played, thus enabling effective and accurate eye-tracking data acquisition. Patient-side computing device 130 can be carried or positioned (e.g., adjustable) in front of the patient and caregiver.

[0231] like FIG. 1A As shown, the patient-side computing device 130 includes a screen 132 (e.g., a display screen) for displaying or presenting visual stimuli to the patient. The patient-side computing device 130 may also include an eye tracker device 134 or be integrated with the eye tracker device 134 into the same housing 136. In some embodiments, the patient-side computing device 130 with the integrated eye tracker device 134 may be referred to as an eye-tracking console or an eye-tracking system.

[0232] The eye tracker device 134 may be connected to the patient-side computing device 130 via a wired connection (e.g., using a USB cable or wire or using an electrical pin). In some cases, the eye tracker device 134 is configured to connect to the patient-side computing device 130 via a wireless connection (e.g., Bluetooth or NFC). The eye tracker device 134 may be positioned appropriately relative to the screen 132 and / or the patient, wherein the eye tracker device 134 can capture the patient's eye movements while viewing visual stimuli, while also minimizing visual distractions from the patient's field of vision.

[0233] like FIG. 1AAs shown, the eye tracker device 134 may include one or more eye tracking units (or sensors) 135 disposed below the bottom of the screen 132. The one or more eye tracking units 135 may be disposed on one or more sides, the top of, and / or around the screen 132. The one or more eye tracking units 135 may be mechanically mounted to the patient-side computing device 130 at a location adjacent to the periphery of the screen 132. For example, the patient-side computing device 130 may include the screen 132 and a screen mounting structure that holds the eye tracker device 134 in a fixed, predetermined position relative to the screen 132. In some embodiments, the eye tracker device 134 includes a first eye tracking unit configured to capture or collect eye movements of the patient's left eye and a second eye tracking unit configured to capture or collect eye movements of the patient's right eye. The eye tracker device 134 may also include a third eye tracking unit configured to capture the patient's eye position or an image acquisition unit (e.g., a camera) configured to capture images of the patient's eyes. In some embodiments, the eye tracker device 134 is configured to determine eye movements based on the position and / or images of the patient's eyes captured by a third eye tracking unit. In some embodiments, the patient's eye movement data includes at least one of eye movements of the patient's eyes collected (e.g., by a first eye tracking unit and a second eye tracking unit), eye movements derived from a position and / or image captured (e.g., by a third eye tracking unit), the captured position, or the captured image. As described below, the eye movement data may be converted into eye tracking data for processing in the cloud server 110.

[0234] An eye-tracking unit includes sensors that detect a person's presence in real time and track what he / she is looking at, or measure where the person is looking or how the eyes respond to stimuli. The sensors can convert the person's eyes into a data stream containing information such as pupil position, gaze vectors for each eye, and / or gaze points. In some embodiments, the eye-tracking unit includes a camera (e.g., an infrared-sensitive camera), an illumination source (e.g., infrared (IR) illumination), and algorithms for data collection and / or processing. The eye-tracking unit may be configured to track pupillary or corneal reflexes or reflections (CR). The algorithm may be configured for pupil center and / or corneal detection and / or artifact suppression. In some embodiments, the eye-tracking unit includes an image acquisition unit (e.g., a camera) configured to capture images of the patient's eyes when the patient views a visual stimulus. The eye-tracking unit may be configured to process the captured images of the patient's eyes to determine eye movements as the patient views the visual stimulus. Eye tracker device 134 (e.g., eye tracking unit) may be configured to convert eye movement data into eye tracking data, which may also include information such as pupil position, gaze vector for each eye, and / or gaze point. In some embodiments, eye tracker device 134 may transmit eye tracking data based on captured eye images to cloud server 110 for further processing.

[0235] In some embodiments, the eye tracker device 134 determines first eye tracking data based on the tracked pupil or corneal reflex or CR, and determines second eye tracking data based on captured eye images. The first and second eye tracking data can be processed by the eye tracker device 134 to replace, supplement, validate, or provide additional background information. The eye tracker device 134 can be configured to generate final eye tracking data for a patient based on the first and second eye tracking data. The eye tracker device 134 can transmit the final eye tracking data based on the captured eye images to a cloud server 110 for further processing. In some embodiments, the eye tracker device 134 transmits the first and second eye tracking data to the cloud server 110 for further processing, wherein a first processing result based on the first eye tracking data and a second processing result based on the second eye tracking data can replace, supplement, validate, or provide additional background information. An evaluation result can be determined based on the first and second processing results.

[0236] Because individual eye size, foveal position, and general physiological characteristics can vary, the eye-tracking unit can be calibrated before using it to collect eye-tracking data from participants (e.g., patients). During calibration, the physical position of the eye is algorithmically associated with a point in the space the participant is gazing at (e.g., a gaze point). The gaze position can be a function of the participant's perception. In some embodiments, calibration involves a fixed, known calibration target (e.g., a point) within the participant's gaze field. Calibration can include a single centered target, or 2, 5, 9, or even 13 targets. The algorithm can create a mathematical transformation between eye position (minus CR) and gaze position for each target, and then create a matrix covering the entire calibration area, for example, by interpolation between each target. The more targets used, the higher and more uniform the accuracy across the entire visual field. The calibration area defines the portion of the eye-tracking unit's range with the highest accuracy; accuracy decreases if the eye moves at an angle greater than the target used.

[0237] In some embodiments, the eye-tracking unit is capable of self-calibration, for example, by creating a model of the eye and passively measuring the characteristics of each individual. Calibration can also be achieved without the active cooperation of participants by making assumptions about the gaze position based on content, thereby effectively "hiding" the calibration target within other visual information. In some embodiments, if useful data can be obtained from the original pupil position, for example using a medical vestibular-ocular reflex (VOR) system or a fatigue monitoring system, calibration is not performed on the eye-tracking unit.

[0238] In some cases, validation can be performed to measure the success of the calibration, for example, by presenting a new target and measuring the calculated accuracy of the gaze. The tolerance for calibration accuracy can vary depending on the application of the eye-tracking unit. For example, a visual error between 0.25 and 0.5 degrees may be considered acceptable. For some applications, an error exceeding 1 degree is considered a failed calibration and requires another attempt. Participants can improve on their second or third attempt. Participants who consistently exhibit high validation errors may have visual or physiological problems that prevent them from participating in the experiment. Validation results can be expressed in degrees of visual angle and displayed graphically.

[0239] like FIG. 2A As shown, the patient-side computing device 130 may include an eye-tracking application (or software) configured to retrieve or receive raw eye-tracking data collected by the eye-tracker device 134. The patient-side computing device 130 may generate session data based on the raw eye-tracking data, for example, by storing the raw eye-tracking data along with associated information (timestamp information) in a data file (e.g., in .tsv, .idf, or any suitable format). FIG. 5As shown in (b). Session data may also include information about visual stimuli played or presented in another data file (e.g., in .tsv format or any suitable format), such as FIG. 5 As shown in (a). The information may include timestamp information for each visual stimulus played.

[0240] In some embodiments, the patient-side computing device 130 stores a plurality of predetermined visual stimuli (e.g., movies or video files) grouped to correspond to patients in specific age groups and / or disease groups. For example, a first list of predetermined visual stimuli may be configured to assess ASD in patients within a first age range (e.g., 5 to 16 months of age), and a second list of predetermined visual stimuli may be configured to assess ASD in patients within a second age range (e.g., 16 to 30 months of age), different from the first age range. In some embodiments, an operator may use the operator-side computing device 140 to control which list of predetermined visual stimuli is played to a specific patient based on information about that patient. In some embodiments, the operator application sends age information to an eye-tracking application upon patient selection, and the eye-tracking application then dynamically selects an appropriate preset playlist based on the age information without operator intervention or selection. In some embodiments, the number of predetermined visual stimuli may also be stored in the operator-side computing device 140.

[0241] The testing method depends on the patient being awake and looking at the screen 132 of the patient-side computing device 130. During calibration and data collection, predetermined videos and / or other visual stimuli are presented to the patient via the patient-side computing device 130. These videos and / or other visual stimuli may include human or animated actors with hand / face / body movements. FIG. 5 As discussed in further detail, during the data collection period, the computing system 120 can periodically display calibration or fixation targets (which may be animated) to the patient. This data can then be used to verify accuracy.

[0242] The visual stimuli displayed to the patient (e.g., a film or video scene) may be age-dependent. That is, the visual stimuli can be age-specific. In some embodiments, processing session data includes measuring the amount of fixation time the patient spends looking at an actor's eyes, mouth, or body or other predetermined regions of interest, and the amount of time the patient spends looking at background areas in the video. FIG. 1B As shown, the video scene displayed to the patient via the screen 132 of the patient-side computing device 130 can depict a scene of social interaction 133 (e.g., an actor looking directly at the camera, attempting to interact with the watching patient, or, for example, a scene of children playing). In some embodiments, the video scene may include other suitable stimuli, such as animation and a priority viewing task, for example, as...FIG. 4C to FIG. 4F As further detailed in the diagram, the measure of fixation time relative to a specific spatial location in a video can be correlated with the patient's level of social and / or cognitive development. For example, children aged 12–15 months may exhibit increased oral fixation, alternating between ocular and oral fixation, as a result of a developmental stage in their language development. As another example, a decrease in a patient's visual fixation on an actor's eyes in a video over time can be an indicator of ASD or other developmental conditions in the patient. Analysis of a patient's viewing patterns (spanning multiple viewing sessions during film display, or compared with historical data from patients of essentially the same age and / or with similar conditions) can be performed to diagnose and monitor developmental, cognitive, social, or psychological abilities or impairments, including ASD.

[0243] When both the patient and caregiver are facing the eye-tracking device 134 on the patient-side computing device, detection light (e.g., infrared light) emitted from the eye-tracking device 134 can propagate towards both the patient's and caregiver's eyes. In some embodiments, the patient's caregiver (e.g., a parent) is given a pair of glasses 122 to wear while holding the patient to view visual stimuli displayed on the patient-side computing device 130, such as... FIG. 1A to FIG. 1B As shown. The glasses 122 can be configured to filter or block detection light from the eye tracker device 134, so that the eye tracker device 134 can only collect reflected or scattered light from the patient's eyes to track / capture the patient's eye movements while the patient (and caregiver) are viewing visual stimuli. In this way, the detection accuracy of the eye tracker device 134 can be improved without being interfered with by the caregiver's eye movement data.

[0244] In some embodiments, the patient-side computing device 130 includes a recording device 138 configured to record images, audio, and / or video of the patient during viewing sessions, during unstructured social interactions, and / or during therapeutic sessions (e.g., with a therapy provider), while the patient is viewing visual stimuli presented on a screen 132 of the patient-side computing device 130. The recording device 138 may be a camera, an audio recorder, or a video recorder. In some embodiments, such as FIG. 1A As shown, compared to the eye tracker device 134 located below the bottom of the screen 132, the recording device 138 can be arranged in the housing 136, for example, positioned at the top of the screen 132.

[0245] FIG. 1C An example of a patient-side computing device 130 is shown, including an eye tracker device 134 and a recording device 138. The housing 136 may have a recess at the center of the top of the screen 132, and the recording device 138 may be configured to be disposed within the recess. FIG. 1CAs shown, the patient-side computing device 130 can include a foldable base (or support) 137 via one or more articulated members 139 between a housing 136 and a base 137. The base 137 can be rotated to approach a cover serving as a screen 132, or opened to serve as a support. The patient-side computing device 130 can be adjustable to accommodate the operation of the recording device 138, for example, based on the patient's height and / or the patient's viewing angle. The patient-side computing device 130 can be carried by the patient's guardian (e.g., a parent, etc.). FIG. 1A (as shown), or placed on a table, for example, to play videos during a session.

[0246] In some embodiments, the patient-side computing device 130 includes one or more recording devices 138, for example, located at the top of a screen 132. In some examples, one recording device is located in the center of the top, and two other recording devices are located on either side of the top. In some examples, two recording devices are distributed at the top. In some embodiments, alternatively or additionally, one or more external recording devices are arranged (e.g., on the ceiling and / or in corners and / or on walls) in the patient's healthcare facility and are configured to record images, audio, and / or video related to patient information.

[0247] Compared to the eye tracker device 134 configured to capture the patient's eye-tracking data, the recording device 138 and / or one or more external recording devices may be configured to capture other information about the patient, such as facial information (e.g., facial expressions), verbal information, and / or physical behavior. For example, while the patient is watching a visual stimulus, the patient may repeat the words of a character in the visual stimulus, talk to others, smile, raise a hand, point, stand up or sit down, or remain quiet; this can be captured by the recording device 138 and / or one or more external recording devices. This other information, referred to as multimodal data, is used in conjunction with the eye-tracking data to expand the data input of the developmental disorder assessment system (e.g., cloud server 110). In addition to or in combination with eye-tracking data, multimodal data may replace, supplement, validate, and / or provide additional context for the developmental disorder assessment.

[0248] In some implementations, as discussed further in detail below, similar to eye-tracking data, multimodal data can be used to monitor one or more specific (e.g., treatment-specific) skill areas, such as autonomous requests, listener responses, co-attention, social etiquette, play, turn-taking, and / or any other skill area. In some cases, clinicians or treatment providers review one or more videos of an individual patient to assess developmental impairments in these skill areas, which can be subjective, time-consuming, unreliable, and / or inconsistent. In contrast, the techniques implemented in this disclosure can use one or more artificial intelligence (AI) models, such as machine learning (ML) models, to automatically analyze multimodal data from an individual patient to identify one or more specific skill areas for assessing the patient's developmental impairments, thereby significantly improving processing speed, consistency, accuracy, and / or reducing the time / cost for clinicians or treatment providers.

[0249] In some implementations, for example, multimodal data (e.g., in the form of image, audio, and / or video data) of a reference group (e.g., typical children of similar age, sex, and / or condition) is acquired during, before, and / or after a session of viewing visual stimuli. One or more expert clinicians may analyze the multimodal data of the reference group and annotate one or more specific skill domains (and skills) to the multimodal data. The annotated multimodal data of the reference group may be provided to one or more artificial intelligence (AI) or machine learning (ML) models for training, for example, in conjunction with eye-tracking data acquired for the reference group. When multimodal data of an individual patient is fed into one or more trained AI or ML models, the one or more AI or ML models may automatically analyze the individual patient's multimodal data, optionally incorporating eye-tracking data, to identify one or more specific skills for assessing developmental disorders in the individual patient.

[0250] Operator-side computing device 140 is configured to run an operator application (or software). In some embodiments, the operator application is installed and runs on operator-side computing device 140. In some embodiments, the operator application runs on cloud server 110, and the operator can log in to a portal to interact with the operator application through a user interface presented on screen 142 of operator-side computing device 140, for example, as... FIG. 1D As shown. In some implementations, such as FIG. 4A to FIG. 4J , FIG. 12A to FIG. 12B and FIG. 13A to FIG. 13D As discussed in further detail, the operator application can be configured to supervise or control the steps of the eye-tracking application or software in the patient-side computing device 130, such as selecting and playing specific visual stimuli for the patient and collecting raw eye-tracking data, and / or viewing results or reports.

[0251] In some examples, such as FIG. 1D As shown and as FIG. 12A to FIG. 12B As discussed in further detail below, the operator application may present different sessions (e.g., diagnostic sessions, monitoring sessions, target monitoring sessions) in user interface 150 for the operator to select. For example, in the same healthcare facility as the patient, when the operator selects to initiate a target monitoring session, the operator application may pop up a new window 160 for the operator to select target skill areas (e.g., self-request, listener response, co-attention, and play) to monitor the patient's behavior in these target skill areas during the session. As discussed in further detail below (e.g., in...), FIG. 11 In the visual stimulation playlist, individual moments or frames can be annotated by an expert clinician to specify one or more specific skill domains (and / or skills), for example, based on gaze behavior of a reference group. If the operator selects a target skill domain during the session, the operator can adjust the visual stimulation to be presented to the patient on the patient-side computing device 130 based on the selected target skill domain, for example, prioritizing annotated / known videos to monitor the selected target skill domain, and / or enriching additional videos related to the selected target skill domain, and / or removing frames irrelevant to the selected target skill domain, and / or optimizing the playlist to maximize the target skill domain. When the operator selects user interface element 162 to run the session in a new window 160, the adjusted visual stimulation can be presented to the patient on the patient-side computing device 130.

[0252] In some examples, such as FIG. 13A and FIG. 13C As discussed in further detail, the operator can use the operator-side computing device 140 (or any other computing device associated with the operator) to view diagnostic results / reports. The operator application can display a user interface on the screen 142 of the operator-side computing device 140. The user interface may include options for the operator to select, for example, different patients or different sessions or histories of patients. Through the user interface, the operator can also view default results (e.g., such as...). FIG. 8A to FIG. 8C as well as FIG. 13B-1 and FIG. 13B-2 As shown), customized reports (e.g., such as...) FIG. 13C (as shown) and / or launch an interactive results dashboard (e.g., as shown) FIG. 13D(As shown). For example, when an operator chooses to view a customized report, the operator application can pop up a new window for the operator to select a target skill area (e.g., autonomous requests, listener responses, co-focus, and play) to customize the diagnostic or monitoring report. In some examples, if the operator selects a target monitoring session, the operator application can automatically customize the report for that target monitoring session to select the same target skill area as the target skill area selected in the playlist for that target monitoring session. The new window 160 can overlay on, be side-by-side with, or overlap with the user interface 250. The user interface 150 can be changed to the new window 160.

[0253] In some embodiments, the operator application interfaces with the eye-tracking software via a software development kit (SDK). In some embodiments, communication between the patient-side computing device 130 and the operator-side computing device 140, or between the operator application and the eye-tracking application, can be implemented using WebSocket communication. WebSocket communication allows bidirectional communication between the two devices. This bidirectional communication allows the operator to control the patient-side computing device 130 while simultaneously receiving information from it. WebSocket communication can be implemented using a secure implementation of WebSocket known as WebSocket Secure (WSS). As described above, communication between the patient-side computing device 130 and the operator-side computing device 140 (e.g., communication between the operator application and the eye-tracking application) can be implemented via a cloud server 110. For example, the operator can use the operator-side computing device 140 to log in to a portal running on the cloud server 110 and establish a wireless connection with the patient-side computing device 130 to collect patient eye-tracking data.

[0254] The operator application can also be used to perform other functions, such as presenting an interface to the operator (showing the patient's name, date of birth, etc.) and information related to stimuli shown to the patient (e.g., videos). The operator can also use the operator-side computing device 140 to log in to the portal of the cloud server 110 for device management, patient management, and data management. In some embodiments, the operator application runs on the cloud server 110 and is controlled by the operator using the operator-side computing device through the portal. The operator can operate the computing system 120 with minimal training.

[0255] like FIG. 3 , FIG. 4A to FIG. 4J and FIG. 5As discussed in further detail, the computing system 120 can be configured for session data acquisition. In some embodiments, a session is initialized by establishing a connection between the operator-side computing device 140 and the patient-side computing device 130. After patient information is entered into an operator application (e.g., custom software) running on the operator-side computing device 140, the operator application can control an eye-tracking application running on the patient-side computing device 130 to select age-specific stimuli and instruct the operator or the patient's caregiver to position the patient-side computing device 130 in front of the patient with an appropriate orientation and / or location. The operator can use the operator-side computing device 140 to control the operator application and / or the eye-tracking application or software to (a) calibrate the eye tracker device 134 relative to the patient, (b) verify that the calibration is accurate, and (c) collect eye-tracking data from the patient, such as from the patient moving their eyes in response to a predetermined video or other visual stimulus, while the patient is watching dynamic video or other visual stimuli during the session. After the session ends, eye-tracking data and stimulus-related information (e.g., a list of stimuli viewed by the patient) can be stored as session data in two separate data files. The session data can then be automatically transmitted, for example, by the patient-side computing device 130 via network 102 to a secure database in cloud server 110. The database can be located remotely from computing system 120 and configured to accommodate and aggregate data collected from multiple computing systems 120.

[0256] Network 102 may include a large computer network connecting any number of mobile computing devices, fixed computing devices, and server systems, such as a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, or a combination thereof. Each computing device 130 and 140 in computing system 120 can communicate with cloud server 110 via network 102.

[0257] In some embodiments, communication between local computing devices 130 and 140 and cloud server 110 may be implemented using the Hypertext Transfer Protocol (HTTP). HTTP follows a request-response model, in which a client (e.g., through a browser or desktop application) sends a request to the server, and the server sends a response. Responses sent from the server may contain various types of information, such as documents, structured data, or authentication information. HTTP communication may be implemented using a secure implementation of HTTP known as Hypertext Transfer Protocol Security (HTTPS). Information transmitted over HTTP is encrypted to protect the privacy and integrity of the information.

[0258] Cloud server 110 may be a computing system hosted in a cloud environment. Cloud server 110 may include one or more computing devices and one or more machine-readable repositories or databases. In some embodiments, cloud server 110 may be a cloud computing system comprising one or more server computers in a local or distributed network, each server computer having one or more processing cores. Cloud server 110 may be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device with one or more processors. As an example, FIG. 18 It is the architecture of a cloud computing system that can be implemented as a cloud server 110.

[0259] like FIG. 1A As shown, the cloud server 110 includes a cloud platform 112 and a data pipeline system 114. For example... FIG. 2A to FIG. 2G As discussed in further detail, cloud platform 112 can be configured to provide a portal, store application data associated with treatment providers or tenants, and store data such as raw eye-tracking data, processed data, and analytical and / or diagnostic results. Data pipeline system 114 is configured to perform data processing and data analysis.

[0260] In some embodiments, such as FIG. 6 and FIG. 7A to FIG. 7B As discussed in further detail, cloud server 110 is configured to automatically receive, process, and analyze session data from multiple computing systems. Furthermore, the cloud server can process and analyze session data from multiple sessions across a large number of computing systems in parallel, thereby significantly improving session processing speed and providing diagnostic results within a short timeframe (e.g., within a 24-hour window). For example, receiving session data by cloud server 110 (e.g., by cloud platform 112) can initiate an automated software-implemented processing and analysis process (e.g., by data pipeline system 114). During this process, individual patient data can be compared with models of eye-tracking data previously generated from historical eye-tracking data of patients with substantially the same age, background, and / or condition. The results of these comparisons can be a diagnosis of a neurodevelopmental disorder, including but not limited to ASD, indicators of the patient's developmental / cognitive function, and / or guidance for treatment planning. Alternatively or additionally, collected data can be compared and / or examined for a given patient across multiple sessions (and within a predetermined time period) to identify potential changes in visual fixation (e.g., a decline in visual fixation). These results can be condensed into a diagnostic report for use by the patient's physician. In some embodiments, once the diagnostic results are ready, the cloud server 110 can transmit the diagnostic results to the operator-side computing device 140, and the diagnostic results can be displayed on the user interface of the operator-side computing device 140, for example, in FIG. 8A to FIG. 8C orFIG. 16A to FIG. 16F This will be discussed in further detail later.

[0261] In some embodiments, a large amount of model data (including data relating to patients of similar age, similar background and / or similar condition) can be used with the patient’s processed session data to generate a diagnosis for the patient, for example, by using comparisons or inferences via statistical models, algorithms, artificial intelligence (AI) models such as machine learning or artificial neural network models, thereby greatly improving the accuracy of the diagnosis.

[0262] Environment 100 involves corresponding to FIG. 1A The three main steps of the three parts of the environment 100 shown (e.g., computing system 120 for data acquisition, cloud platform 112, and data pipeline system 114). FIG. 2A to FIG. 2G As discussed in further detail, these three components can be configured together to reliably collect patient data and effectively process and analyze the collected data to diagnose ASD or other cognitive, developmental, social, or mental abilities or deficits.

[0263] FIG. 2A This is a block diagram of an example system 200 for assessing developmental disorders via eye tracking, according to one or more embodiments of the present disclosure. System 200 can be implemented in environment 100 of FIG1. ​​System 200 can be considered as an assessment system for developmental disorders. In some examples, the assessment system is referred to as EarliPoint. System 200 includes three subsystems according to three steps of data processing: a data acquisition subsystem 210, a platform subsystem 220, and a data pipeline subsystem 230. Each subsystem may consist of corresponding hardware and software components. Platform subsystem 220 and data pipeline subsystem 230 may form a cloud server, for example, FIG. 1A Cloud server 110.

[0264] The data acquisition subsystem 210 is configured to collect eye-tracking data from the patient. The data acquisition subsystem 210 may be the computing system 120 shown in Figure 1. FIG. 2AAs shown, the data acquisition subsystem 210 includes an eye-tracking console 212 running an eye-tracking application 214 and an operator-side computing device (e.g., 140 in FIG. 1) running an operator application 216. In some embodiments, the operator application 216 is deployed in the operator-side computing device. In some embodiments, the operator application 216 is deployed in the platform subsystem 220, and the operator can use the operator-side computing device to log in to the platform subsystem 220 via a portal 222 to run the operator application 216 on the platform subsystem 220. Deploying the operator application 216 in the platform subsystem 220 avoids deploying the operator application 216 on one or more operator-side computing devices, thereby reducing the software and hardware requirements of the operator-side computing devices and facilitating the maintenance or updating of the operator application 216, for example, eliminating the need to maintain or update operator information on each of the one or more operator-side computing devices.

[0265] The eye-tracking console 212 may be an integrated device including the patient-side computing device 130 (e.g., a tablet computer) and the eye-tracking device 134 of FIG1. ​​As described above, the data acquisition subsystem 210 may include multiple video files 218 stored in the eye-tracking console 212 and optionally in the operator-side computing device. The video files 218 may be age-specific visual stimuli for patients of different ages and / or with different conditions.

[0266] like FIG. 1A As described, platform subsystem 220 and data pipeline subsystem 230 may be included in a networked server such as a cloud server (e.g., cloud server 110 of Figure 1) and implemented in a centralized cloud-hosted environment provided by a cloud provider (e.g., Microsoft Azure). In some embodiments, platform subsystem 220 is configured for the management and orchestration of resources in the cloud-hosted environment. Platform subsystem 220 may be cloud platform 112 of Figure 1. FIG. 2A As shown, the platform subsystem 220 includes a portal website 222, a database 224 for storing application data, and a database 226.

[0267] Portal 222 can be a web-based interface. Through portal 222, operators (e.g., medical professionals) can log in to platform subsystem 220, for example, using operator-side computing devices, to manage (view and / or query) application data stored in database 224 and / or data in database 226. For example, portal 222 allows operators to view diagnostic results. Pre-written action procedures (e.g., seeking further evaluation) can be provided based on the diagnostic results.

[0268] As an example of database 224, FIG. 2DDatabase 240 is shown, storing different types of documents. Database 240 can be a NoSQL database such as Azure Cosmos DB. Different types of documents can be stored in database 240 as application data. Unlike relational databases, NoSQL databases do not have strong relationships between documents. FIG. 2D The dashed lines in the text indicate references and information embedding between documents.

[0269] In some embodiments, database 240 stores corresponding application data for treatment providers (or tenants). Treatment providers can be healthcare organizations, including but not limited to autism centers, healthcare facilities, specialists, physicians, or clinical research institutions. Organizations may vary in structure, number of patients, and duration of operation. FIG. 2D As shown, the corresponding application data may include organization documents 242, user documents 244, device documents 246, patient documents 248, session documents 250, and history documents 252. Users can be operators associated with a healthcare organization, such as medical assistants, specialists, physicians, or any other medical professionals.

[0270] Organization document 242 contains the organization's settings and customizations. User document 244 contains identifier information as well as user roles and permissions. User roles indicate whether a user is an administrator or an operator associated with a different security level or permission. Device document 246 contains information about each eye-tracking console associated with the organization (e.g., FIG. 2A The patient document 248 contains information about the patient, such as an infant or child considered for developmental assessment. The session document 250 contains session-related information, which may consist of a session identifier (session ID), an introduction to the patient, an introduction to the user performing the session, a pointer to eye-tracking data, and the results of data processing and analysis. The history document 252 may be used to maintain a version history of changes to the document. The document mirrors the structure of its parent document and includes additional audit information. In some embodiments, the database 224 allows URL-based queries (e.g., queries performed with an administrator role) to enable queries across multiple variables. For example, variables may include patient / device / session, adverse events, etc.

[0271] In some embodiments, the cloud server, including platform subsystem 220 and data pipeline subsystem 230, can be implemented in a centralized cloud environment, providing greater flexibility to scale the cloud server's capabilities. For example, the cloud server can leverage a multi-tenant architecture to deliver Software-as-a-Service (SaaS) subscription-based diagnostic services to treatment providers. In a multi-tenant architecture, treatment providers share a single version of the software across various geographic locations. The term "tenant" in a multi-tenant architecture describes a single treatment provider within the system. Cloud server resources can be dynamically managed based on the total number of tenants and the expected average workload (e.g., how many tenants are accessing the cloud server at a given point in time). The cloud server can employ horizontal scaling techniques such as autoscaling to handle spikes in resource workload.

[0272] In a multi-tenant architecture for shared applications, it is important to isolate tenant data and prevent other tenants from accessing it. This is known as isolation. There are three different isolation strategies that can be implemented: shared database, per-tenant database, and per-tenant application. In a shared database strategy, tenants share a single instance of the application, and all data is stored in a single database. In a per-tenant database strategy, for example, in... FIG. 2E In strategy 260 shown in Figure (a), tenants share a single instance of the application in application layer 262, but have their own database 264 (e.g., FIG. 2A Database 224 or FIG. 2D Database 240). In per-tenant application policies, for example, in FIG. 2E In strategy 270 shown in Figure (b), each tenant obtains its own application instance and its own database 274 in the corresponding application layer 272 (e.g., FIG. 2A Database 224 or FIG. 2D (Database 240). The cloud server can deploy a one-database-per-tenant policy 260 or a one-application-per-tenant policy 270 to the treatment provider.

[0273] Continue to refer to FIG. 2ADatabase 226 is configured to store raw eye-tracking data or session data, processed session data, analysis results, and / or diagnostic results or reports. Database 226 can be a storage platform (e.g., Azure Blob) and can be paired with tools written in any suitable programming language (e.g., Python, Matlab), supporting URL-based interfaces and queries to database 226. Furthermore, database 226 can be compatible with programming languages ​​(e.g., Python, Matlab) used to transfer data from data acquisition subsystem 210 to database 226 and from database 226 to data pipeline subsystem 230. For example, in the case where the patient-side computing device (e.g., 130 in Figure 1) is located in a medical facility, data collection occurs within that facility, and data is transferred between database 226 and the patient-side computing device. Database 226 can be secure, HIPAA-compliant, and protected by a redundant backup system.

[0274] In some embodiments, platform subsystem 220 is configured to enable one or more of the following operations, including: (a) entry of new patient information, (b) storage of raw data files (e.g., including eye-tracking data), and (c) storage of files on a data collection device (e.g., FIG. 2A (a) Automatic and secure transfer between the eye-tracking console 212, the data processing computer, and the database; (d) data tabulation and querying for assessing device utilization and other data quality metrics; and (e) access to the processing results by the physician. One or more of operations (a) through (c) may be performed by the upload function module 221 in the platform subsystem 220.

[0275] Continue to refer to FIG. 2A The data pipeline subsystem 230 is configured to process and analyze patient eye-tracking data and generate diagnostic results. In some embodiments, the data pipeline subsystem 230 includes a data processing module 232, a data analysis module 234, and model data 236. (See below) FIG. 7A to FIG. 7B As discussed in further detail, the data processing module 232 is configured to process session data including eye-tracking data to obtain processed session data, and the data analysis module 234 is configured to analyze the processed session data using model data 236 to generate diagnostic results.

[0276] In some embodiments, system 200 includes interfaces for devices and subsystems. The interfaces can be inter-subsystem. For example, system 200 may also include an interface between data acquisition subsystem 210 and cloud platform subsystem 220, and an interface from cloud platform subsystem 220 to data pipeline subsystem 230. The interfaces can be intra-subsystem. For example, system 200 may include an interface between eye-tracking console hardware (e.g., tablet computer and eye-tracking device) and eye-tracking application software.

[0277] FIG. 2B The illustration shows one or more embodiments according to this disclosure. FIG. 2A This is an example of processing single session data in system 200. As described above, after a data collection session is completed, the eye-tracking console 212 can automatically transfer the session data to platform subsystem 220. The session data may include two files: one containing raw eye-tracking data (e.g., gaze position coordinates, blink data, pupil size data, or combinations thereof), and the other containing information related to the stimuli (e.g., a list or playlist of videos viewed by the patient). Through the upload function module 221 implemented in platform subsystem 220, the session data can be stored in database 226 and in application data in database 224. The stored session data can then be automatically transferred from platform subsystem 220 to data pipeline subsystem 230 for data processing and analysis without human intervention. For example, software scripts written in any suitable programming language (e.g., Python, Matlab) can be used to transfer raw, unprocessed data files from database 226 to data pipeline subsystem 230 for processing. The session data is first processed by the data processing module 232 and then analyzed by the data analysis module 234 to generate diagnostic information about the patient.

[0278] In some embodiments, three files are generated: one containing processed eye-tracking data, one containing a summary of eye-tracking statistics, and one containing diagnostic information. The file containing the diagnostic information can be uploaded to database 224 to be associated with the patient in application data, such as... FIG. 2D As shown. These three files can then be uploaded to database 226 for storage. In some cases, the processed eye-tracking data is tabulated to a session table. A summary of eye-tracking information (e.g., fixation samples / videos, etc.) can be read from the processed summary file and tabulated in database 226 for subsequent queries. Summary values ​​(e.g., fixation / video percentage, etc.) can then be calculated within database 226.

[0279] FIG. 2C The illustration shows one or more embodiments according to this disclosure. FIG. 2AAn example of parallel processing of multiple session data in System 200. For example... FIG. 2C As shown, multiple eye-tracking consoles 212a, 212b can send multiple session data 213a, 213b, 213c (collectively referred to as session data 213 or individually referred to as session data 213) to platform subsystem 220. In data pipeline subsystem 230, data processing module 232 and data analysis module 234 can be written in a suitable programming language (e.g., Python), and can be deployed in containers 231a, 231b, 231c (collectively referred to as container 231 or individually referred to as container 231). Each session can be processed using its own instance of data processing and analysis. The use of containers allows data processing and analysis to be performed while session data is being uploaded from data acquisition subsystem 210, which can facilitate session retransmission within a short period of time, such as within a 24-hour window.

[0280] like FIG. 7A to FIG. 7B As discussed further in detail, the cloud server can process and analyze session data from multiple sessions of a large number of computing systems in parallel. First, the cloud server can deploy a corresponding container (e.g., 231) for each session, and this corresponding container can include a corresponding data processing module 232 and a corresponding data analysis module 234. Thus, once session data (e.g., 213) is uploaded by the corresponding eye-tracking console 212, the session data can be processed and analyzed using its own container (e.g., 231, an instance with its own data processing and data analysis). Second, while session data from multiple sessions is being processed in the corresponding container, for example using most of the processing units (or cores) in the cloud server, model data for analyzing the processed session data can be preloaded in parallel into the corresponding container, for example using the remaining or a few processing units in the cloud server. Furthermore, all processed session data and loaded model data can be analyzed in parallel in the corresponding container, for example using all the processing units in the cloud server. Parallelization, used in various ways, can significantly improve the speed of session data processing and analysis, and provide rapid diagnostic results within a short timeframe (e.g., within a 24-hour window). For example, once diagnostic results are available, the cloud server can transmit them to the corresponding operator-side computing device (e.g., 140 in Figure 1), and then display the results in the results interface of operator application 216. Parallelization also allows the cloud server to be more efficient in resource utilization, thereby further improving system performance.

[0281] FIG. 2F One or more embodiments of the present disclosure are shown for use with FIG. 2AExample configuration 280 for data backup of system 200. Configuration 280 enables high availability of services to treatment providers, ensuring that treatment providers can still access their services regardless of any outages occurring in one or more specific regions of the cloud server (e.g., platform subsystem 220 and data pipeline subsystem 230).

[0282] High availability refers to the ability of a treatment provider to access its services regardless of whether the cloud service provider experiences an outage. Availability can be achieved by replicating resources across different physical locations. The cloud servers implemented in this paper can be provided by a cloud service provider that offers Platform as a Service (PaaS) resources with built-in or configurable high availability. Resources hosted in a cloud environment can achieve high availability through the use of high-availability service level agreements or through the use of geographical redundancy.

[0283] FIG. 2F An example of achieving high availability through geo-redundancy is shown. FIG. 2F As shown in (a), the cloud server's resources can be hosted in a first data center 282 with a portal 222a. The resources are replicated in a second data center 284. When the first data center 282 is functioning normally, treatment provider traffic is directed to the first data center 282, and the second data center 282b is a mirror. However, as... FIG. 2F As shown in (b), when the first data center 282 fails, the treatment provider's traffic is redirected to the replication resources in the second data center 284, which runs the replication portal 222b. The switchover process can be seamless, and the treatment provider may not be aware that it has switched to a different resource in the replication data center.

[0284] FIG. 2G An example data backup of system 200 (e.g., platform subsystem 220 and data pipeline subsystem 230) is shown. Database 224 storing application data and database 226 storing raw and processed eye-tracking data, as well as analysis or diagnostic results, may be stored in multiple data centers. The portal 222 in platform subsystem 220, the data processing module 232 and data analysis module 234 in data pipeline subsystem 230, and optionally the operator application 216 (running on platform subsystem 220) may be included in active data center 282 and may be replicated in backup data center 284.

[0285] Example process for session data collection FIG. 3 This is a flowchart of an example process 300 for session data acquisition according to one or more embodiments of this disclosure. Process 300 may be provided by a system (e.g., FIG. 1A The computing system 120 or FIG. 2AThe data acquisition subsystem 210 performs this operation. This system includes an operator-side computing device (e.g., 140 in Figure 1) and one or more patient-side computing devices (e.g., 130 in Figure 1) integrating an associated eye tracker device (e.g., 134 in Figure 1). Each of the operator-side computing devices and the one or more patient-side computing devices can be connected via a network (e.g., network 102 in Figure 1) to a network-based server or cloud server (e.g., [website name missing]). FIG. 1A cloud server 110 or such FIG. 2A to FIG. 2G The system communicates with the cloud server described herein. It can associate with treatment providers, such as providing developmental disorder assessments and / or treatment services to patients. The cloud server can also associate with service providers used to provide services to treatment providers (e.g., data processing, analysis, and diagnostic results). For illustration, FIG. 4A to FIG. 4J The illustration shows one or more embodiments of the present disclosure during session data acquisition on an operator-side computing device (a) and a patient-side computing device (b), for example, in... FIG. 3 In the process 300), a series of illustrative displays (or user interfaces) are presented.

[0286] In step 302, a session is initiated, for example, by establishing a connection or communication between the operator-side computing device and the patient-side computing device. In some embodiments, the two computing devices 130 and 140 may be wirelessly connected (e.g., via a wireless connection) without a physical connection. The wireless connection may be implemented via a cellular network, wireless network, Bluetooth, near field communication (NFC), or other standard wireless network protocols. In some cases, the patient-side computing device may also be configured to connect to the operator-side computing device via a wired connection, such as a Universal Serial Bus (USB), for example, when the wireless connection fails.

[0287] In some embodiments, the connection between the operator-side computing device and the patient-side computing device is established by two computing devices communicating with a cloud server, which in turn provides communication between the operator-side computing device and the patient-side computing device. For example, such as FIG. 4A As shown, operators (e.g., medical assistants, medical professionals, or any other representative of a treatment provider) can log in to a portal running on a cloud server (e.g., FIG. 2A (222) This allows for equipment management, patient management, and data management. Operators can have corresponding user roles and permissions, for example, such as... FIG. 2D As described in [the text]. FIG. 4A Figure (a) illustrates the user interface (UI) displayed on the operator-side computing device's screen after the operator logs into the portal using the operator-side computing device. This UI can be an operator application running on a cloud server or on the operator-side computing device (e.g., FIG. 2A The user interface of (216).

[0288] like FIG. 4A As shown in Figure (a), the UI includes a menu displaying buttons for "Home," "Patient," "Device," and "User." Clicking these buttons displays corresponding information (e.g., patient information, device information, or user information) within the UI. For example, when the "Device" button is clicked, the UI displays a list of names of patient-side computing devices (e.g., Device 1, Device 2, Device 3, Device 4, Device 5) that can be controlled by the operator. If a patient-side computing device (e.g., Device 4, Device 5) is connected to a cloud server, an indication (e.g., a string indicating "Connect") is displayed near the name of that patient-side computing device. The operator can select one of the names, such as Device 4, to connect the corresponding patient-side computing device to the operator-side computing device. Once a name is selected, the UI displays a request to enter an access code for connecting the corresponding patient-side computing device, such as... FIG. 4B As shown in Figure (a).

[0289] FIG. 4A Figure (b) illustrates the user interface presented on the screen (e.g., 132 in Figure 1) of a patient-side computing device (e.g., device 4). For example, this UI may appear after the patient-side computing device is turned on and logged in by an operator. The UI may display a "Start" button, which can be clicked by the operator, for example, to begin a session. After clicking the "Start" button, the patient-side computing device is connected to a cloud server, for example, to a portal. The cloud server may associate the patient-side computing device with the operator based on the device's identifier, for example, as... FIG. 2D As shown. Once the patient-side computing device is successfully connected to the cloud server, the UI presented on the patient-side computing device can display access code (e.g., "5678") information generated by the portal for connecting to the operator-side computing device, such as... FIG. 4B As shown in Figure (b), the operator obtains an access code from the UI displayed on the patient-side computing device, enters it into the UI displayed on the operator-side computing device, and then submits the access code to the portal. After the portal verifies that the access code entered on the operator-side computing device matches the access code generator on the patient-side computing device, the portal can establish a wireless connection between the operator-side computing device and the patient-side computing device.

[0290] Once a connection is established between the operator-side computing device and the patient-side computing device (e.g., device 4), connection information (e.g., "Connected to device 4") can be displayed on the operator-side computing device's UI, for example, as shown below. FIG. 4CAs shown in Figure (a), the UI may also display a button for initiating the display of visual information (e.g., a video) to the patient on the screen of the patient-side computing device. The patient's (e.g., a parent's) human caregiver may watch the video presented on the screen of the patient-side computing device with the patient (or carry the patient). In some embodiments, the patient's human caregiver may wear glasses configured to filter or block light (e.g., IR light) from the eye-tracking device, such that the eye-tracking device only collects reflected or scattered light from the patient's eyes and not the human caregiver's eyes, in order to track / capture the patient's eye movements while the patient (and human caregiver) are viewing visual stimuli on the patient-side computing device.

[0291] In step 304, desensitization begins, for example, by the operator clicking the "Start Video" button on the UI of the operator-side computing device, thereby displaying visual desensitization information (e.g., a video) to the patient on the screen of the patient-side computing device. FIG. 4C As shown in Figure (b).

[0292] Data is typically not recorded during the display of desensitized videos. Instead, the videos are displayed to attract the patient's attention. The videos can reflexively elicit exogenous cueing from the patient without requiring verbal mediation or instruction from the operator. For example, the operator does not need to give instructions to look at the screen of the computing device on the patient's side, because the video itself captures the patient's attention.

[0293] When the desensitized video is displayed on the screen of the patient's computing device, such as FIG. 4D As shown in Figure (b), the operator can select patient information through the UI of the operator-side computing device, such as... FIG. 4D As shown in Figure (a). The operator can select patients from the existing patient list associated with the operator in the cloud server, for example, as... FIG. 2D As shown, a patient profile may be created for a new patient. After patient confirmation, the process begins by setting up the eye tracker device (or patient-side computing device) relative to the patient by displaying setup information on the operator-side computing device's UI, as shown. FIG. 4E As shown in Figure (a). The operator can also select "Pause Movie" or "Skip Movie" on the UI of the operator-side computing device.

[0294] During setup, the desensitized video can be continuously played on the screen of the patient's computing device, such as... FIG. 4E Figure (b) and FIG. 4F As shown in Figure (b). FIG. 4FAs shown in Figure (a), the relative position between the eye tracker device and the patient's eyes is displayed on the UI of the operator-side computing device, for example, in text or graphical form. The relative position can be determined by capturing image data of the patient's eyes using an image acquisition device (e.g., a camera) included in or adjacent to the eye tracker device. In some embodiments, after the operator clicks the "Start Setup" button in the UI of the operator-side computing device, as shown... FIG. 4E As shown in Figure (a), an operator application running on a cloud server can send commands to a patient-side computing device to capture images of the patient's eyes using an image acquisition device. The patient-side computing device can then transmit the captured images to the cloud server, and the operator application can process the images to determine the relative position between the eye tracker device and the patient's eyes. The relative position can include the distance between the eye tracker device and the patient's eyes, and the horizontal and / or vertical deviation between the center of the eye and the center of the eye tracker device's field of view (or detection area). Based on the relative position, the operator application can display instructions on the operator-side computing device's UI for adjusting the position of the eye tracker device, such as "Move the console down," etc. FIG. 4F As shown in Figure (a). Once the relative position of the patient's eyes and the eye tracker device is acceptable, the operator can confirm the settings, for example, by clicking the "Confirm Settings" button in the UI. In some embodiments, in response to determining that the relative position of the patient's eyes and the eye tracker device is less than a predetermined threshold (e.g., horizontal / vertical deviation less than 0.1 cm), the operator application can confirm that the settings are complete and show an instruction to the operator.

[0295] In step 306, regarding the calibration of the eye tracker device on the patient. After setup is complete, the operator application can display a "Start Calibration" button on the operator-side computing device's UI, such as... FIG. 4G As shown in Figure (a). In some embodiments, calibration involves the patient viewing one or more fixed, known calibration targets (e.g., dots or icons) in their visual field. The calibration or fixation target reflexively attracts the patient's attention and causes them to saccade and fixate on the known target location. The target reliably induces fixation on a finite location; for example, a radially symmetrical target with a spectral span of less than 0.5 degrees. Other examples include concentric patterns, shapes, or contractile stimuli that reliably induce fixation on a fixed target location, even if initially larger in size.

[0296] For example, once the operator clicks the button to begin calibration, multiple calibration targets can be sequentially presented at predetermined (or target) locations (e.g., center, top left, or bottom right) on the patient-side computing device screen, such as... FIG. 4GAs shown in Figure (b). When multiple calibration targets are presented on the screen of the patient-side computing device, the eye tracker device can be activated to capture the patient's eye-tracking calibration data, for example, in response to a command received from an operator application. The eye-tracking application (e.g., FIG. 2A (214) can run on the patient-side computing device to collect the patient's eye-tracking calibration data.

[0297] In some embodiments, the patient-side computing device (e.g., an eye-tracking application) is configured to: determine the position of the corresponding visual fixation of the calibration target, and then compare the determined position of the patient's corresponding visual fixation with a predetermined position where the calibration target is presented. Based on the comparison result, the eye-tracking application may determine whether the calibration target has been calibrated. If the distance between the patient's corresponding visual fixation position and the predetermined position of the calibration target is within a predetermined threshold, the eye-tracking application may determine that the patient's corresponding visual fixation matches the predetermined position of the calibration target, or that the calibration target has been calibrated. If the distance is greater than or equal to the predetermined threshold, the eye-tracking application may determine that the patient's corresponding visual fixation does not match the predetermined position, or that the calibration target has not been calibrated.

[0298] In some embodiments, the patient-side computing device transmits information about the patient's captured eye-tracking calibration data and / or predetermined positions to the operator-side computing device or cloud server. The operator application can determine the patient's corresponding visual fixation position and compare the determined position with multiple predetermined positions, and / or determine whether the calibration target has been calibrated based on the comparison results.

[0299] In some embodiments, a first calibration target may be presented initially at the center of the screen, and calibration may continue, with four additional calibration targets presented at each corner of the screen along the direction of rotation. The operator application may alert the operator to the calibration activity status, such as calibration point 1, calibration point 2, calibration point 3, or 4 / 5 targets calibrated. Between each calibration target, a desensitized video is played for a set time period before a new calibration target is presented. Each calibration target may be cycled a set number of times, then determined to have failed calibration, and the process continues to the next calibration target. If a calibration target fails calibration, it may be retried after all remaining calibration targets are presented and a gaze collection attempt is made.

[0300] In step 308, the calibration is verified. Verification can be performed to measure the success of the calibration, for example, by showing a new target and measuring the accuracy of the calculated gaze. Verification may show a smaller number of calibration targets (e.g., 3) compared to the calibration targets in calibration step 306 (e.g., 5). A desensitization video may be played between showing two adjacent calibration targets.

[0301] In some embodiments, based on the comparison between the determined position of the patient’s corresponding visual fixation and the predetermined position of the calibration target, the initial verification of different success levels (e.g., the number of calibration targets that are calibrated or verified) can automatically instruct the operator to: (1) recalibrate the eye tracker device with respect to the patient, (2) reverify those calibration targets that were not verified, or (3) accept the calibration and continue with the data acquisition in step 310.

[0302] In some embodiments, the operator may decide whether or not to accept calibration. For example... FIG. 4H As shown, on the operator-side computing device display, calibration targets are simultaneously presented at multiple predetermined locations, and a representation of the corresponding visual fixation (e.g., a dot) is shown at a specific location corresponding to the patient's visual fixation. The UI may also display a first button for "Accept Verification" and a second button for "Recalibrate." The operator can view the matching between multiple calibration targets and the representation of the patient's corresponding visual fixation and determine whether to accept verification (by clicking the first button) or recalibrate the patient to the eye tracker device (by clicking the second button).

[0303] In step 310, eye-tracking data of the patient is collected by presenting a playlist of predetermined visual stimuli (e.g., stimulus videos) to the patient on the screen of the patient-side computing device, for example, after calibration is verified or the operator accepts verification. FIG. 5 As shown in (a), the list of predetermined visual stimuli may include, for example, multiple patient-specific social stimulus videos (e.g., 0075PEER, 0076PEER, 0079PEER) based on the patient's age and / or condition. Between each social stimulus video or before presenting each social stimulus video, a centering video (e.g., a centerstim video) may be shown to briefly centralize the patient's gaze. In some embodiments, as... FIG. 5 As shown in (a), a calibration check (e.g., similar to the calibration check at step 306) is performed during the data acquisition step (e.g., between the center-positioning videos shown). For example, the calibration check may include five calibration targets: CCTL for the upper left calibration check, CCTR for the upper right calibration check, CCBL for the lower left calibration check, CCCC for the center-to-center calibration check, and CCBR for the lower right calibration check. Data related to the calibration check can be used for post-processing, e.g., for recalibrating eye-tracking data and / or for determining calibration accuracy.

[0304] In a specific example, the sequence of data collection in step 310 may be as follows: 1. Centering stimulus 2. Stimulating movies 3. Central localization stimulation 4. Stimulus videos or calibration checks (e.g., displaying 5 calibration targets, such as playing randomly between 2 to 4 stimulus videos). 5. Repeat steps 1-4 until the scheduled list of exciting videos is complete. In some embodiments, such as FIG. 4I As shown, the UI on the operator-side computing device displays a button for "Start Acquisition". After the operator clicks the "Start Acquisition" button, a predetermined playlist of visual stimuli can be presented sequentially on the screen of the patient-side computing device according to a predetermined order. On the screen of the operator-side computing device, as shown... FIG. 4I As shown, the UI can display the status of the running playlist in text (e.g., playing a video: central localization stimulation) or display the same content as presented on the screen of the patient-side computing device (e.g., showing a central localization stimulation video).

[0305] In some embodiments, such as FIG. 4J As shown, the UI can display a running playlist of videos that are already playing or currently playing, such as Centering Stim, PEER1234, Centering Stim, and PEER5678. The UI can also display the video being presented on the patient-side computing device screen. The UI can also display a progress bar indicating the percentage of stimulus videos already played in the pre-defined stimulus video playlist. The UI can also display a button for the operator to skip videos.

[0306] In some embodiments, the collected eye-tracking data may be evaluated, for example, by presenting visual stimuli (e.g., FIG. 8A The calibration accuracy of the 812) is such that the visual stimulus reflexively captures attention and causes a saccade toward and fixation on a known target location. The target reliably induces fixation on a finite location; for example, a radially symmetrical target spanning a visual angle of less than 0.5 degrees. Other examples include concentric patterns, shapes, or contractile stimuli that reliably induce fixation on a fixation target location, even if initially larger. Such stimuli can be tested under data collection using head protection devices to ensure they reliably induce fixation under ideal testing conditions; their use can then be extended to include data collection without head restraint.

[0307] In some embodiments, a numerical evaluation of the accuracy of the collected eye-tracking data may include the following steps: (1) presenting a fixation target that reliably induces fixation on a small area of ​​the visual display unit; (2) recording eye-tracking data throughout the target presentation; (3) identifying fixations in the collected eye-tracking data; (4) calculating the difference between the fixation position coordinates and the target position coordinates; and (5) storing the calculated difference between the fixation position coordinates and the target position coordinates as vector data (direction and magnitude) for as few targets as possible or for as many targets as possible (e.g., five or nine, but more). In some embodiments, a recalibration or post-processing step may be performed, for example, by applying a spatial transformation to align the fixation position coordinates with the actual target position coordinates, through methods including but not limited to (a) trilinear interpolation, (b) linear interpolation in the barycentric coordinates, (c) affine transformation, and (d) piecewise polynomial transformation.

[0308] The session ends once the playlist of pre-selected visual stimuli has finished playing. The patient-side computing device can generate session data based on raw eye-tracking data collected by the eye-tracking device, for example, by storing the raw eye-tracking data along with associated information (timestamp information) in a data file (e.g., in .tsv, .idf, or any suitable format). FIG. 5 As shown in (b). The raw eye-tracking data may include the values ​​of multiple eye-tracking parameters at different timestamps. The eye-tracking parameters may include gaze coordinate information for the left eye, right eye, left pupil, and / or right pupil.

[0309] Session data may also include information about visual stimuli that are played or presented in another data file (e.g., in .tsv format or any suitable format), such as FIG. 5 As shown in (a). This information may include timestamp information and the name of each visual stimulus played. The timestamp information of the visual stimulus can be associated with the timestamp information of the eye-tracking data, so that the eye-tracking data for each visual stimulus (and / or calibration check) can be determined individually based on the timestamp information in these two data files.

[0310] In step 312, the session data is sent to the cloud server. Once the session data is generated by the patient-side computing device, the patient-side computing device can transfer the session data to the cloud server. For example... FIG. 2A to FIG. 2G China and FIG. 6 , FIG. 7A to FIG. 7B As discussed in further detail in Figure 8, cloud servers can first store session data in a centralized database (e.g., FIG. 2A to FIG. 2B In the database 226), the session data is then processed, the processed data is analyzed, and a diagnosis result for the patient is generated, which can be accessed or viewed by the operator or medical professional.

[0311] Example data processing and data analysis FIG. 6 It is according to one or more embodiments of this disclosure for use by a cloud server (e.g., FIG. 1A cloud server 110 or such FIG. 2A to FIG. 2G The flowchart of an example process 600 for managing session data (e.g., data processing and data analysis) using a cloud server as described herein. FIG. 7A to FIG. 7B A flowchart of an example process 700 for managing session data by a cloud server according to one or more embodiments of the present disclosure is shown, which has a higher... FIG. 6 More details.

[0312] In step 702, once the session is complete, the corresponding patient-side computing device (e.g., 130 in Figure 1) or eye-tracking console (e.g., FIG. 2A-2G (212) For example, a cloud platform that transmits session data to a cloud server via a portal website. The cloud platform can be... FIG. 1A Platform 112 or FIG. 2A-2G The platform subsystem 220. In response to receiving session data, the cloud server's cloud platform stores the session data in a database (e.g., in the cloud platform's...). FIG. 2A to FIG. 2G The cloud platform then automatically transmits the session data to the data pipeline system (e.g., database 226). FIG. 1A 114 or FIG. 2A to FIG. 2G (230) to perform data processing and data analysis.

[0313] In step 704, a file pointer for the session data is added to the processing queue (step 704). Session data for all completed sessions await processing according to the processing queue. Once the session data is uploaded and stored in the cloud server, a corresponding file pointer can be assigned to the session data and added to the processing queue. The file pointer can be an identifier for the session data of the corresponding session. Based on the file pointer, the session data of the corresponding session can be retrieved from the database in the cloud platform.

[0314] In step 706, a corresponding container is created for the session data of each session, for example, based on auto-scaling technology to achieve session parallelization. For example, a new container can be created for a new session in response to adding a file pointer to the processing queue. Each container (e.g., FIG. 2C 231) can have instances with their own data processing modules and data analysis modules, for example, such as FIG. 2C As shown.

[0315] In each container, for example, by FIG. 2A to FIG. 2GThe data processing module 232 executes steps 708 to 714 for the session data of the corresponding session. Note that steps 708 to 714 can be executed in parallel for the session data of multiple sessions in multiple containers.

[0316] In step 708 (corresponding to FIG. 6 (Step 602) uses the corresponding file pointer to obtain session data from the database in the cloud platform. As mentioned above, session data may include two files: an eye-tracking data file (e.g., ... FIG. 5 (b) as shown) and playlist files (e.g., such as FIG. 5 (as shown in (a)).

[0317] refer to FIG. 6 Step 602 may correspond to step 708. In step 604, session data is prepared for processing. Step 604 may include one or more steps as described in steps 710 to 714 of FIG7.

[0318] Step 604 may include linking eye-tracking data from an eye-tracking data file to a video played in a playlist file. In some embodiments, such as FIG. 7A As shown, in step 710, eye-tracking data is decomposed into independent running segments, for example, based on timestamp information from the two files. Each running segment may correspond to the playback of a corresponding video (e.g., a center-localization target, a predetermined visual stimulus, or one or more calibration targets). For example, eye-tracking data corresponding to timestamps within a range defined by the timestamps of two adjacent videos is included in the running segment. In step 712, eye-tracking data in each running segment is linked to the corresponding video from the playlist based on the timestamp information from the two files. In some embodiments, eye-tracking data is not decomposed into independent running segments, but is processed as a continuous stream of data samples linked to corresponding videos in the playlist.

[0319] In step 714, the eye-tracking data is recalibrated to compensate for drift or bias. In some embodiments, the eye-tracking data collected during the calibration step during playlist presentation (e.g., as shown in the image) FIG. 5 Figure (a) shown can be used to calibrate or align eye-tracking data collected during playback of each film in different runs. For example, data from times adjacent to the time showing the additional calibration target can be used to correct for any differences in gaze position. Some larger differences may exclude certain data from subsequent analyses.

[0320] In step 606, the prepared session data is processed. In some embodiments, the data processing module extracts relevant information from the prepared session data, such as the patient's visual fixation and / or visual fixation on objects or regions of interest in the video. In some embodiments, the data is resampled to account for any temporal variations between samples. Any suitable interpolation and / or smoothing techniques can be used to resample the data. The data can be converted from the specified raw resolution and / or coordinate system of the collected eye-tracking data to an appropriate resolution and / or coordinate system for analysis. For example, the raw data can be collected at a higher resolution (e.g., 1024 × 768 pixels) than the resolution of the presented stimulus (e.g., rescaled to 640 × 480 pixels). In some embodiments, the data processing module can automatically identify basic oculomotor events (unwanted fixation, saccades, blinks, off-screen or missing data, etc.) and can automatically identify the duration of the subject's fixation (in an undesirable manner), saccades, blinks, or the time the subject is not looking at the screen. The data processing module can adjust for outliers in the gaze position estimation output by the eye tracker device.

[0321] In some embodiments, such as FIG. 7B As shown in step 716, the patient's corresponding model data is preloaded into multiple session containers in parallel, and session data for multiple sessions of the patient is processed in these multiple session containers. In some examples, the majority of the processing units (e.g., N processing cores) in the cloud server are used to process the session data for multiple sessions in the multiple session containers, while a minority of the processing units (e.g., M processing cores) in the cloud server are used to preload the corresponding model data into these multiple session containers in parallel. The processing units or cores can be central processing units (CPUs). Parallelization avoids the additional time spent waiting for model data to be uploaded.

[0322] Cloud servers can pre-store model data in a database, for example, FIG. 2A to FIG. 2G 226. Model data may include data on a large number of instances of significant differences in gaze positions among patients (e.g., infants, toddlers, or children) across different levels of social, cognitive, or developmental function. Patient-corresponding model data may include data relating to patients of similar age, similar background, and / or similar condition, which may be used in conjunction with the patient's processed session data to generate a patient diagnosis. Patient-corresponding model data may be identified and retrieved from a database, for example, based on the patient's age, patient background, and / or patient condition. Step 608 of process 600 (in which step 608, the processed data is prepared for analysis) may include obtaining the processed data in multiple session containers and preloading the corresponding model data in multiple session containers.

[0323] In step 610, the processed data is analyzed to generate analysis results. In some embodiments, for a given session, the processed data is compared with corresponding model data in the corresponding session container to obtain comparison results. In some embodiments, the data analysis module uses the processed data and corresponding model data to generate results, for example, using comparisons or inferences via statistical models, algorithms, or artificial intelligence (AI) models (such as machine learning or artificial neural network models). In some embodiments, such as FIG. 7B As shown in step 718, the processed session data and preloaded model data are analyzed in parallel using the full number of processing units (e.g., N+M cores) across multiple session containers.

[0324] In some embodiments, processed session data is compared to a corresponding data model to determine the level of developmental, cognitive, social, or mental condition. The generated score is then compared to a predetermined cutoff value or other value to determine the patient's ASD diagnosis and the severity level of the condition. In some other embodiments, the patient's gaze data (e.g., visual fixation data) is analyzed over a predetermined time period (e.g., over multiple sessions spanning several months) to identify a decrease, increase, or other significant change in visual fixation (e.g., gaze data initially corresponding to a child with typical development becomes more unstable, corresponding to a child exhibiting ASD, or gaze data becoming more similar to that of a child with typical development in response to targeted (targeted) therapy).

[0325] In step 720 (corresponding to step 612), a summary of the results is calculated. As described above, the analysis results can be used to determine scores for at least one index, such as the social deficit index, verbal ability index, nonverbal ability index, social adaptability index, and social communication index. Based on the comparison of the scores with at least one predetermined cutoff value, the patient's ASD diagnosis and the severity level of the condition can be calculated. For example, as... FIG. 8A As shown, based on the analysis results and / or any other suitable information (e.g., from other relevant analyses of patients), a social deficit index score of 6.12 is presented in the range of -50 (social deficit) to 50 (social competence), indicating no concern about social deficit; a language competence index score of 85.89 is presented in the range of 0 to 100, indicating above-average language competence; and a non-language competence index score of 85.89 is presented in the range of 0 to 100, indicating above-average non-language competence. Furthermore, a diagnosis of non-ASD can also be calculated based on the analyzed data.

[0326] In some embodiments, the summary of the results includes: a visualization of individual eye-tracking data (e.g., gaze point data) on still images of the film overlapping socially relevant moments, allowing clinicians and parents to better understand how the patient visually focuses on social information. For example, at step 610, still images of the patient with available data may be cross-referenced with a pre-determined list of still images of the film that elicit eye-gazing behavior, along with information about the diagnostic status (including symptom severity). The visualization may also include visualizations of aggregated reference data from typical developing children, such as matching patient attributes like age, sex, etc. These visualizations may be side-by-side, allowing clinicians and / or parents to compare individual patient data with reference data and see how gaze patterns align or diverge. These visualizations may include annotations explaining the film content, eye-gazing patterns, etc.

[0327] In some embodiments, the summary of results includes an animated visualization of the patient's eye-tracking data overlaid on still images of videos from socially relevant moments. For example, a portal may include a dashboard that allows clinicians to view stimuli shown to patients, overlaid with the patient's eye gaze data. The dashboard may be configured to allow users to select which videos to visualize and whether to visualize frames that capture information about social deficit indices, verbal ability indices, nonverbal indices, or any other indices calculated in the report.

[0328] Continue to refer to FIG. 7B In step 722, the data pipeline subsystem sends the output back to the portal website, for example, such as... FIG. 2B As shown. In some embodiments, the output includes three files: one containing processed eye-tracking data, one containing a summary of eye-tracking statistics, and one containing diagnostic information (e.g., a summary of results). These three files can then be uploaded to a database (e.g., FIG. 2A to FIG. 2G (226) is then stored. In some cases, the processed eye-tracking data is tabulated to a session table. A summary of eye-tracking information (e.g., fixation samples / videos, etc.) can be read from the processed summary file and tabulated in the database for subsequent queries. Summary values ​​(e.g., fixation / video percentage, etc.) can then be calculated within database 226.

[0329] In step 724, the output is reconnected with the patient information to generate a diagnostic report or result for the patient. For example, a file containing diagnostic information may be uploaded to an application data database (e.g., FIG. 2A to FIG. 2G (224) to be associated with patients in the application data, for example, such as FIG. 2DAs shown. Diagnostic reports or results can be presented in the application data database in any suitable manner to users (operators or medical professionals, such as physicians) associated with the patient or caregivers associated with the patient.

[0330] In some embodiments, once a patient's diagnostic report or results are generated, the user may be notified (e.g., via email or message) to log in to view the report or results through a portal. The diagnostic report or results may be presented on a user interface, for example, as... FIG. 8A or FIG. 8B to FIG. 8C or FIG. 16A to FIG. 16F As shown. In some embodiments, once a patient's diagnostic report or results are generated, they can be sent to an operator-side computing device for presentation to the user. Diagnostic reports or results can also be sent to the operator via secure email or messages. Diagnostic reports or results can be stored in an application data database (e.g., [database name missing]). FIG. 2A to FIG. 2G 224) and / or databases (e.g., FIG. 2A to FIG. 2G In 226).

[0331] FIG. 8A An example results interface 800 according to one or more embodiments of the present disclosure is shown, which displays an assessment report (or diagnostic report or result) including at least one index value based on eye-tracking data. The results interface 800 shows patient information 802, requesting physician / institution information 804, device ID of the patient-side computing device 806, processing date 807 (indicating the time when session data used for processing was acquired), and report release date 808.

[0332] The results interface 800 also displays collected information 810, including calibration accuracy 812, oculomotor function 814, and data acquisition summary 816. Calibration accuracy 812 and oculomotor function 814 may be presented graphically. The data acquisition summary 816 may include at least one of the following: the number of videos viewed, the number of videos excluded, the duration of data collection, the time spent watching videos, the time spent not watching videos, calibration accuracy, oculomotor indicators, or quality control indicators.

[0333] The results interface 800 also displays neurodevelopmental test results 820, which may include diagnostic results 822 (e.g., ASD or non-ASD), social deficit index information 824, language ability index information 826, and non-language ability index information 828. The results interface 800 may display these index information 824, 826, 828 and their corresponding descriptions in graphical form.

[0334] FIG. 8B to FIG. 8C Another example results interface 850 according to one or more embodiments of the present disclosure is shown, which displays performance-based metrics of developmental assessment, with reference to the following examples:FIG. 8B Nonverbal communication and gestures (A) and shared attention & eye contact (B) in the context of communication, and FIG. 8C Facial emotions (C) and pointing and social monitoring (D) in the context.

[0335] Results interface 850 shows performance-based indicators of children's individual disadvantages and skill development opportunities. Neurodevelopmental assessment via eye-tracking measures how children engage with continuous social and non-social cues within a natural environmental context (left column 852 shows still frames from the test video). In those contexts, prescriptive reference indicators provide an objective quantification of non-ASD, age-expected visual engagement (middle column 854 shows density distribution in pseudo-color format, middle column 856 shows color-to-grayscale gradients overlapping the corresponding still frames). Age-expected reference indicators can be used to measure and visualize patient comparisons, revealing individual strengths, weaknesses, and skill-building opportunities (right column 858, example patient data shown as overlapping annular diaphragms covering portions of the video each patient gazes at; for example, each diaphragm spans approximately 5.2 degrees from the center of the patient's visual field). Individual patients with ASD may exhibit non-gazing in the following instances: (A) verbal and nonverbal interactions and gestures (860); (B) shared attention and mutual eye contact cues (870); (C) dynamic facial expressions (880); and (D) shared attention and social monitoring (890). FIG. 8B to FIG. 8C As shown, children with ASD exhibited interaction with toys of interest (1, 3, 5, 7); color and contrast cues (2, 6, 8); objects (10, 11, 12); background elements not directly related to the social environment (4, 9, 13); and recurring visual features (14, 15, 16, 17, 18). The elapsed times in the lower right corner of the still frames highlight the rapidly changing nature of social interaction: hundreds of verbal and nonverbal communication cues were presented within a viewing time of approximately 12 minutes, each evoking age-predicted patterns of engagement and providing an opportunity for objective quantitative comparison of patient behavior.

[0336] Example process FIG. 9 This is a flowchart of an example process 900 for session data acquisition according to one or more embodiments of this disclosure. Process 900 can be executed by a system, for example, FIG. 1A The computing system 120 or FIG. 2A-2G The data acquisition subsystem 210. Process 900 can be similar to... FIG. 3 The process is 300, and can be referenced. FIG. 4A to 4J To describe.

[0337] The system includes an operator-side computing device (e.g., 140 in Figure 1) and one or more patient-side computing devices (e.g., 130 in Figure 1), the patient-side computing devices integrating an associated eye-tracking device (e.g., 134 in Figure 1). At least one of the operator-side computing device or patient-side computing device may be a portable device. Each of the operator-side computing device and one or more patient-side computing devices can be connected via a network (e.g., network 102 in Figure 1) to a network-based server or cloud server (e.g., FIG. 1A cloud server 110 or such FIG. 2A to FIG. 2G The system communicates with the cloud server described herein. The system can be associated with a treatment provider, for example, to provide developmental disorder assessment and / or treatment services to a patient. The cloud server can be associated with a service provider that provides services to the treatment provider (e.g., data processing, analysis, and diagnostic results). Process 900 may include multiple steps, wherein some steps are performed by an operator-side computing device, some steps are performed by a patient-side computing device and / or an eye-tracking device, and some steps are performed by a combination of operator-side and patient-side computing devices.

[0338] In step 902, a session is initiated for the patient by establishing communication with both the operator-side computing device and the patient-side computing device. In some embodiments, establishing communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device, such as wireless connection 131 in FIG1.

[0339] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes: the operator-side computing device accessing a portal on a networked server (e.g., ...). FIG. 2A to FIG. 2G (222), and in response to receiving a selection of the patient-side computing device in the portal, wirelessly connect the operator-side computing device to the patient-side computing device.

[0340] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes, for example, as... FIG. 4B As shown, the patient-side computing device displays connection information on its screen, and in response to the operator-side computing device receiving the connection information input, a wireless connection is established between the operator-side computing device and the patient-side computing device.

[0341] In some embodiments, process 900 further includes: after establishing communication, displaying visual desensitization information to the patient on the screen of the patient-side computing device, for example, such as... FIG. 4C As shown. The eye tracker device can be configured not to collect the patient's eye-tracking data when visual desensitization information is displayed.

[0342] In some embodiments, process 900 further includes: when visual desensitization information is displayed, the operator-side computing device accesses a portal on a networked server to set up a session for the patient, for example, such as... FIG. 4D and FIG. 4E As shown. In some cases, setting up a session includes at least one of the following: selecting a patient from a patient list, or creating a patient profile on a networked server.

[0343] In some embodiments, process 900 further includes: determining the relative position between the eye-tracking device and at least one eye of the patient; and displaying instructions on the user interface of the operator-side computing device for adjusting the position of the eye-tracking device or the patient's position, for example, such as... FIG. 4F As shown. In some cases, process 900 further includes: determining that the relative position of at least one eye of the patient is at a predetermined position in the detection area of ​​the eye tracker device, in response to determining that the patient is aligned with the eye tracker device.

[0344] In step 904, the patient is calibrated relative to the eye tracker device by displaying one or more calibration targets to the patient on the screen of the patient-side computing device, for example, as... FIG. 4G As shown. Each of one or more calibration targets may be sequentially presented at a corresponding predetermined position on the screen of a patient-side computing device, while eye-tracking calibration data of the patient is captured using an eye-tracking device. Process 900 may include: for each of the one or more calibration targets, processing the captured eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual fixation for the calibration target; comparing the position of the patient's corresponding visual fixation with the corresponding predetermined position where the calibration target is presented; and determining, based on the result of the comparison, whether the calibration target has been calibrated to the eye-tracking device.

[0345] In some embodiments, calibrating a patient to an eye tracker device further includes: determining that a calibration target has been calibrated and displaying the next calibration target in response to determining that the deviation between the patient’s corresponding visual fixation position and the corresponding predetermined position is less than or equal to a predetermined threshold; or determining that a calibration target has failed to be calibrated and redisplaying the calibration target for calibration in response to determining that the deviation is greater than a predetermined threshold.

[0346] In some embodiments, process 900 further includes: verifying the calibration with one or more new calibration targets after calibrating the patient to the eye tracker device. Similar to the calibration described in step 904, verifying the calibration includes: sequentially presenting each of the one or more new calibration targets at a corresponding predetermined position on the screen of the patient-side computing device while capturing the patient's eye-tracking calibration data using the eye tracker device; and processing the captured eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual fixation for each of the one or more new calibration targets.

[0347] In some embodiments, such as FIG. 4H As shown, the verification calibration includes: simultaneously presenting one or more new calibration targets and one or more corresponding visual fixations of the patient on the user interface of the operator-side computing device, with the one or more new calibration targets presented at one or more corresponding predetermined locations, and the one or more corresponding visual fixations of the patient presented at one or more determined locations; and in response to receiving an indication that the calibration result is valid, determining that the calibration has been verified, or in response to receiving an indication that the calibration result is invalid, initiating the recalibration of the patient to the eye tracker device.

[0348] In some embodiments, verifying calibration includes: determining the number of new calibration targets that pass calibration based on the patient’s corresponding visual fixation position and corresponding predetermined position; and determining that the calibration is verified if the number or associated percentage is greater than or equal to a predetermined threshold, or determining that the calibration is invalid and initiating recalibration of the patient relative to the eye tracker device if the number or associated percentage is less than the predetermined threshold.

[0349] In step 906, after confirming that the calibration has been verified, a predetermined list of visual stimuli is sequentially presented to the patient on the screen of the patient-side computing device, while eye-tracking data of the patient is collected using an eye-tracking device.

[0350] In some embodiments, such as FIG. 4I , FIG. 4J or FIG. 5 As shown, before each predetermined visual stimulus from the predetermined list of visual stimuli is presented, a central positioning target is presented to the patient on the screen of the patient-side mobile computing device for central positioning of the patient's gaze.

[0351] In some embodiments, such as FIG. 5 As shown, patient calibration relative to an eye tracker device is performed between two adjacent visual stimuli in a playlist presenting predetermined visual stimuli. Eye-tracking data collected during calibration can be used for at least one of the following: calibrating the patient's eye-tracking data, or for determining calibration accuracy by a networked server.

[0352] In some embodiments, such as FIG. 4J As shown, process 900 further includes presenting at least one of the following on the user interface of the operator-side computing device: a progress indicator that remains updated during the presentation of a predetermined list of visual stimuli, information on visual stimuli that have been presented or are being presented, information on visual stimuli to be presented, or a user interface element for skipping visual stimuli in the predetermined list of visual stimuli.

[0353] In step 908, session data, including eye-tracking data of the patient collected during the session, is transmitted from the patient-side computing device to the network server. The patient-side computing device may automatically transmit session data to the network server in response to one of the following: determining completion of a playlist of predetermined visual stimuli displayed on the screen, or receiving a session completion indication from the operator-side computing device, for example, via a portal on the network server.

[0354] In some embodiments, the session data includes information relating to a predetermined list of presented visual stimuli, which may include the name of the presented predetermined visual stimuli and a timestamp associated with the presentation of the predetermined visual stimuli, such as... FIG. 5 As shown in Figure (a). Session data may include: eye-tracking data and a timestamp associated with the generation or collection of the eye-tracking data, for example, as FIG. 5 As shown in Figure (b). In some embodiments, transmitting session data includes transmitting a first file and a second file, the first file storing the patient's eye-tracking data and the second file storing information related to a predetermined list of visual stimuli presented.

[0355] FIG. 10 This is a flowchart of an example process 1000 for data processing and data analysis according to one or more embodiments of this disclosure. Process 1000 can be executed by a network-connected server, which may be a cloud server in a cloud environment, for example... FIG. 1A cloud server 110 or such FIG. 2A to FIG. 2G The cloud server described herein. For example, a network-connected server may include a platform (e.g., FIG. 1A 112 or FIG. 2A to 2G 220) and data pipeline systems (e.g., FIG. 1A 114 or Figures 2A to 2G (230). The platform may include a portal website (e.g., Figures 2A to 2G 222), application data database (e.g., Figures 2A to 2G 224) and databases (e.g., Figures 2A to 2G (226). A data pipeline system may include one or more data processing modules (e.g., Figures 2A to 2G (232) and one or more data analysis modules (e.g., Figures 2A to 2G (234). Process 1000 can be similar to Figure 6 Process 600 or Figures 7A to 7B The process is 700.

[0356] In step 1002, session data from multiple sessions is received, for example, such as... Figure 2B As shown, the session data for each session includes eye-tracking data of the corresponding patient in the session. In step 1004, the session data of multiple sessions are processed in parallel to generate processed session data for multiple sessions. In step 1006, for each session in the multiple sessions, the processed session data of the session is analyzed based on the corresponding reference data to generate an assessment result for the corresponding patient in the session.

[0357] In some embodiments, process 1000 further includes: loading corresponding reference data for multiple sessions in parallel with processing session data for multiple sessions.

[0358] In some embodiments, the networking server includes multiple processing cores. Parallel processing of session data for multiple sessions may include, for example,... Figure 7B As shown in step 716, a first plurality of processing cores are used to process session data of multiple sessions in parallel, and different second plurality of processing cores are used to load corresponding reference data of multiple sessions. The number of the first plurality of processing cores may be greater than the number of the second plurality of processing cores. In some embodiments, analyzing the processed session data of multiple sessions based on the loaded corresponding reference data of multiple sessions includes, for example, as... Figure 7B As shown in step 718, multiple processing cores, including a first plurality of processing cores and a second plurality of processing cores, are used.

[0359] In some embodiments, analyzing the processed session data of multiple sessions based on corresponding reference data of multiple loaded sessions includes at least one of the following: comparing the processed session data of a session with corresponding reference data, inferring the assessment results of a corresponding patient from the processed session data using the corresponding reference data, or using at least one of a statistical model or an artificial intelligence (AI) model.

[0360] In some embodiments, the corresponding reference data includes historical eye-tracking data or results from patients with substantially the same age or condition as the corresponding patient. In some embodiments, process 1000 includes generating an assessment result based on the corresponding patient's previous session data.

[0361] In some embodiments, for each of the multiple sessions, a corresponding container is assigned to that session, for example, such as Figure 2C or Figure 7AAs shown. Process 1000 may include: processing session data of the session in the corresponding container, and analyzing the processed session data of the session based on the corresponding model data to generate the assessment results of the corresponding patient in the session.

[0362] In some embodiments, eye-tracking data is associated with a predetermined list of visual stimuli presented to the patient when eye-tracking data is collected during a session, and the session data includes information associated with the predetermined list of visual stimuli during the session.

[0363] In some embodiments, process 1000 further includes, for example, as Figure 7A As shown, in the corresponding container, eye-tracking data is decomposed into multiple parts based on information associated with a list of predetermined visual stimuli, each part of the eye-tracking data being associated with at least one of the following: the corresponding predetermined visual stimulus or the corresponding calibration.

[0364] In some embodiments, processing session data includes processing a portion of eye-tracking data associated with a corresponding predetermined visual stimulus based on information about that stimulus. In some embodiments, process 1000 further includes recalibrating a portion of eye-tracking data associated with a corresponding predetermined visual stimulus in a corresponding container, based on at least one portion of the eye-tracking data associated with a corresponding calibration.

[0365] In some embodiments, process 1000 further includes: determining calibration accuracy in a corresponding container using at least a portion of eye-tracking data associated with the corresponding calibration and multiple predetermined locations presenting multiple calibration targets in the corresponding calibration.

[0366] In some embodiments, receiving session data for multiple sessions includes: receiving session data for multiple sessions from multiple computing devices associated with a corresponding entity via a portal website, for example, such as... Figure 2C As shown.

[0367] In some embodiments, process 1000 further includes, for example, as Figure 7A As shown, in response to receiving session data, a file pointer is added to the processing queue for the session data. Process 1000 may further include: storing the session data in a database using the session's file pointer; and retrieving the session data from the database using the session's file pointer.

[0368] In some embodiments, process 1000 further includes: for each entity, storing session data from one or more computing devices associated with the entity in a corresponding repository, for example, such as... Figure 2E As shown. The corresponding repository can be isolated from one or more other repositories and cannot be accessed by one or more other entities. The application data database can be a NoSQL database.

[0369] In some examples, such as Figure 2D As shown, the corresponding repository of an entity includes at least one of the following: information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted within the entity, information about one or more patients associated with the entity, or historical information of the corresponding repository.

[0370] In some embodiments, process 1000 further includes: dynamically adjusting the resources of the network server based on the number of computing devices accessing the network server, for example, such as... Figure 2F As shown. Process 1000 may also include: copying data from the first data center to the second data center; and automatically redirecting traffic to the second data center in response to determining that the first data center is inaccessible.

[0371] In some embodiments, each of the first and second data centers includes at least one instance of a portal accessible to operator-side computing devices, an operator application, or an application layer for data processing and data analysis, such as... Figure 2G As shown. The process may also include storing the same data in multiple data centers. The data may include application data of the entity and information associated with eye-tracking data.

[0372] In some embodiments, process 1000 further includes: associating the generated assessment results with the corresponding patient in the session, and generating an assessment report for the corresponding patient, for example, such as Figure 7B Step 724 is shown.

[0373] In some embodiments, process 1000 further includes: outputting evaluation results or evaluation reports to be presented, for example, via a portal on the user interface of the operator-side computing device.

[0374] In some embodiments, such as Figure 8A As shown, the assessment report includes at least one of the following: information about the corresponding patient, information about the entity performing the session for the corresponding patient, information about the calibration accuracy during the session, information about the session data collection, or the assessment results for the corresponding patient. In some embodiments, the assessment results indicate the likelihood that the corresponding patient has developmental, cognitive, social, or mental deficits or abilities. For example, the assessment results are used to indicate the likelihood that the corresponding patient has autism spectrum disorder (ASD) or not ASD. In some embodiments, the assessment results include a social deficit index, a language ability index, and a corresponding score for each of one or more of non-language abilities, for example, such as... Figure 8A As shown.

[0375] In some embodiments, the corresponding patient's age is in the range of 5 months to 7 years, including: age in the range of 5 months to 43 months or 48 months, age in the range of 16 months to 30 months, age in the range of 18 months to 36 months, age in the range of 16 months to 48 months, or age in the range of 16 months to 7 years.

[0376] Example of specific skill monitoring As mentioned above, for example, regarding Figure 8A The diagnostic report can provide an overall diagnostic result (e.g., ASD or non-ASD), as well as scores and information on three severity indices (e.g., social deficits, language abilities, and non-language learning). Embodiments of this disclosure can provide more detailed and interactive report output, allowing users to delve into behaviors and indicators related to specific scenarios or groups of scenarios concerning developmental skills (such as treating specific skill areas / skills), for example, such as... Figures 11 to 14 This will be discussed in further detail later.

[0377] Figure 11 An example 1100 is illustrated, according to one or more embodiments of the present disclosure and for different specific skill domains 1110, comparing information between annotated video scene 1120, typical gaze behavior groups 1130, and patient gaze behavior 1140. Information on typical gaze behavior groups 1130 may include a distribution map 1132 and an example highlighted video scene 1134. The distribution map 1132 may be a saliency map. Information on patient gaze behavior 1140 may include representative video scene 1142 (which may be a highlighted video scene) and specific skill indicators (e.g., convergent gaze percentage or attention percentage) 1144.

[0378] A patient's developmental assessment may be associated with one or more specific skill areas (or developmental concepts or skill categories). A skill area may include one or more skills that can be related to each other. A skill may be associated with one or more skill areas. A specific skill area may be self-request, listener response, turn-taking, shared attention, social decorum, or play. A specific skill area may correspond to one or more therapies, and a therapy may be associated with one or more specific skill areas.

[0379] As an example, the skill domain “shared attention” can include multiple skills, such as pointing to something, following someone else’s pointing, and / or looking at (looking at) someone’s pointing. As another example, the skill domain “autonomous request” indicates a request for something, which can include, for example, pointing to something (accompanied by gestures) and / or verbally requesting something. As yet another example, pointing to something can be associated with the skill domains “shared attention” and “autonomous request”.

[0380] For a given session (e.g., a diagnostic session, a monitoring session, or a target monitoring session), the data collection playlist for visual stimuli can include multiple videos (or films), for example, about Figures 4A to 4J The video may include multiple video scenes (moments or frames), for example, such as... Figure 11 The example video scene shown is 1120. A video scene can be associated with one or more skill areas or skills.

[0381] As an example, in example video scene 1120a, boy A (on the right) reaches his hand towards boy B (on the left) and asks for a toy. Boy B takes the toy and says "no." When watching video scene 1120a, a typical (normal) child might look at boy A's hand and / or boy B's toy; in contrast, a child with a developmental disorder might look around, such as at a picture on the wall, or at the feet of boy A or boy B. As another example, in example video scene 1120b, girl C turns her head and listens to the person speaking. When watching video scene 1120b, a typical (normal) child might look at girl C's head or eyes; in contrast, a child with a developmental disorder might look elsewhere, such as at a nearby table.

[0382] Information on a typical gaze behavior group 1130 may include a distribution map 1132 and a highlighted video scene 1134. The highlighted video scene 1134 may be a representative video scene from multiple video scenes within a reference group. The distribution map 1132 (e.g., map 1132a of video scene 1120a or map 1132b of video scene 1120b) shows the distribution of gaze regions (or gaze behaviors) among multiple individuals (e.g., children) in the reference group, obtained, for example, based on analysis of eye-tracking data as described above. The distribution map 1132 may be a saliency map. The distribution map 1132 may be shown in color or grayscale. A higher map value indicates a higher level of normality for the child. The reference group may include children with normal development and children with abnormal development (or developmental disorders). Children in the reference group may have similar ages, sexes, and / or other similar characteristics. Compared to the original video scene 1120a, the highlighted video scene 1134 (e.g., the highlighted video scene 1134a of video scene 1120a or the highlighted video scene 1134b of video scene 1120b) highlights two areas corresponding to the two areas that most children in the reference group are looking at.

[0383] One or more specialist clinicians can review each video scene in the video (e.g., frame-by-frame or time-by-time) and annotate which skill domain and / or specific skill the video scene is associated with. For example, a specialist clinician can annotate video scene 1120a as associated with the skill domain “voluntary request” and / or the skill “requesting something”, and video scene 1120b as associated with the skill domain “listener response” and / or the skill “responding to someone speaking”. In some cases, the specialist clinician may also annotate video scenes with reference to information from typical gaze behavior groups 1130 (e.g., distribution map 1132 and modified video scene 1134). In some examples, a specialist clinician can annotate videos associated with a specific skill domain, a first series of video scenes associated with a first skill associated with the specific skill domain, and a second series of video scenes associated with a second skill associated with the specific skill domain. In some examples, a specialist clinician can annotate a first series of video scenes in a video to be associated with a first specific skill domain, and annotate a second series of video scenes in a video to be associated with a second specific skill domain. The expert clinician can also annotate each video scene in the first series of video scenes as a corresponding skill associated with a first specific skill domain, and annotate each video scene in the second series of video scenes as a corresponding skill associated with a second specific skill domain. The expert clinician's annotations can be associated with video scenes individually and stored along with information on typical gaze behaviors 1130 of the video scene 1120 and the reference group, and can be stored on a networked server, for example, Figure 1A The network server 110, such as Figure 1A Platform subsystem 112 or Figure 2A The platform subsystem 220, or the library or cloud storage device.

[0384] If a patient's gaze area is closer to (or converges to) a highlighted area in Figure 1132 or the highlighted video scene 1134 for a specific skill domain, it may indicate that the patient is more normal in that specific skill domain. If, for a specific skill, the patient's gaze area is farther from (or different from) a highlighted area in Figure 1132 or the highlighted video scene 1134, it may indicate that the patient is abnormal in that specific skill. In some implementations, for video scenes annotated with specific skill domains, a cutoff threshold is determined for assessing the patient's gaze behavior, such as a contour around one or more highlighted areas in Figure 1132, for example, contour 1133a in Figure 1132a or contour 1133b in Figure 1132b. If the patient's gaze area is within the contour, for example, above the cutoff threshold value, it may be determined that the patient is normal in the specific skill domain at that moment. If the patient's gaze area is outside the contour, for example, below the cutoff threshold value, it may be determined that the patient is abnormal at that moment or has a developmental disorder in the specific skill domain (e.g., if abnormal at multiple moments). It is important to note that a patient's gaze area being within or outside the contour corresponding to a cutoff threshold at a given moment only indicates that their behavior is normal / abnormal at that moment (or frame), not that their overall development or behavior is normal / abnormal. For example, if the abnormality exceeds the threshold number of moments, it may indicate an overall abnormality / disorder from the perspective of the entire session. The cutoff threshold and / or contour can be determined by analyzing gaze behavior in a typical group using AI or ML models, for example, based on gaze behavior in known normal children, or through analysis of expert clinician reference models.

[0385] In some implementations, a skill-specific indicator used to assess a patient's developmental impairment in a particular skill domain is defined as the percentage of moments when the patient is identified as normal while watching, relative to the total number of moments in the data collection playlist that relate to the specific skill domain. The skill-specific indicator may also be referred to as convergent gaze percentage or attention percentage. As an example, during a data collection playlist in a session with patient Ben, there are 200 video scenes annotated with the skill domain "autonomous request." Of these 200 video scenes, for example, based on eye-collected data captured during the session, there are 100 moments (corresponding to 100 video scenes) when the patient is looking at the screen of the patient-side computing device. That is, there may be another 100 moments when the patient is not looking at the screen of the patient-side computing device. Of these 100 moments, there are 29 moments where the patient's gaze area is within the outline or has a map value above a cutoff threshold; that is, the patient is normal at 29 moments. There are another 71 moments where the patient is watching the video scene but is not within the outline or has a map value below the cutoff threshold. Therefore, for this session, the skill-specific indicator of the patient's "autonomous requests" was 29%. For example, as Figure 11As shown, the information on Ben's gaze behavior 1140 includes representative video scenes 1142a and a "self-request" specific indicator 1144a, which indicates that Ben focused on 29% of the moments in the session related to "self-request." Similarly, the information on Ben's gaze behavior 1140 includes representative video scenes 1142b and a skill-specific indicator 1144b, which indicates that Ben focused on 37% of the moments related to listener response, thus indicating that Ben was identified as normal in listener response in the skill domain at 37% of the time while watching video scenes related to listener response. In some implementations, alternatively or additionally, the information on Ben's gaze behavior 1140 is compared with reference data from a reference group. For example, the attention percentage of the reference group can be used as a baseline for determining what percentage of attention is normal, expected, or typical. The patient's attention percentage can be compared with the attention percentage of the reference group, and the information on the patient's gaze behavior can include the results of the comparison as an indicator of the expected / target / typical attention percentage and / or canonical percentile. As an example, the reference group's percentage of attention in listener responses is 40%, while the patient's (e.g., Ben) percentage is 37%. Patient behavior information could state: "37% is at the 92.5 percentile in the 'listener response' skills domain," or "37% is within the expected 90-94% range of typical peers' 'listener response' skills domain."

[0386] As discussed in further detail below, video scenes annotated by expert clinicians based on the gaze behavior of reference groups can accurately identify specific skill areas / skills used for patient diagnosis and / or treatment, effectively tailoring patient data collection playlists regarding selected skill areas / skills, and monitoring patient improvement or treatment outcomes in selected skill areas / skills.

[0387] Figure 12A An example illustrative user interface 1200 for session initiation presented on an operator device according to one or more embodiments of the present disclosure is shown. The operator device may be an operator-side computing device, such as... Figures 1A to 1B The computing device 140, or about Figures 4A to 4J The operator-side computing device described.

[0388] For example, an operator (e.g., a medical assistant, medical professional, or any other representative of a treatment provider) can log in to a networked server (e.g., Figure 1A cloud server 110 or Figure 2A The portal website running on the platform subsystem 220 (e.g., Figure 2A The portal website (222) is used for equipment management, patient management, and data management. Operators can have corresponding user roles and permissions, for example, such as... Figure 2D As described above. After the operator logs into the portal using the operator-side computing device, the user interface (UI) 1200 can be displayed on the screen of the operator-side computing device. The UI 1200 can be an operator application running on a network server or on the operator-side computing device (e.g., Figure 2A The user interface of the operator application (216).

[0389] like Figure 12A As shown, UI 1200 includes a menu 1210 displaying buttons for "Home," "Patient," and "Appointment." Clicking these buttons displays corresponding information (e.g., patient information, device information, or appointment information) within UI 1200. Figures 4A to 4B As discussed herein, the operator-side computing device can, for example, connect to the patient-side computing device via a networked server (e.g., Figure 1A Patient-side computing device 130 or about Figures 4A to 4J The patient-side computing device discussed establishes communication. After communication is established, the operator can select a patient (or create a new patient) to begin a patient session. When the operator chooses to start the session from the portal, a UI 1200 for session initiation 1220 is presented to the operator.

[0390] like Figure 12A As shown, under session initiation 1220, the operator can select session type 1230, which may include diagnostic session 1232, monitoring session 1234, and target monitoring session 1236. Diagnostic session 1232 is configured as a running session, for example, as... Figures 4C to 4J As shown, and generates a diagnostic report, for example, such as Figure 8A As shown. The diagnostic report may include diagnostic results, such as whether the patient has ASD or not and / or scores on the three indices (social deficit index, language ability index, and non-language ability index). Monitoring sessions 1234 are configured to monitor changes in the behavior (or performance) of existing patients across a series of sessions by running sessions, for example, as Figures 4C to 4J As described in [the document], and generates monitoring reports, such as... Figures 13A to 13D This is further elaborated upon. In some examples, for instance, such as... Figure 12A As shown, UI 1200 can display the time (e.g., date) when the last run was performed for monitoring session 1234 and / or diagnostic session 1232.

[0391] In some implementations, diagnostic session 1232 and monitoring session 1234 have the same visual stimulus data collection playlist. In some implementations, monitoring session 1234 may have a different visual stimulus data collection playlist than diagnostic session 1232. Monitoring session 1234 may run a default playlist, which may be the same as the playlist executed in the patient's last session. The playlist executed in the last session may be tailored for one or more specific skill domains, for example, the last session is a targeted monitoring session.

[0392] If target monitoring session 1236 is selected, when the operator clicks the "Continue" button 1238 in UI 1200, a prompt window 1250 will appear on UI 1200 for the operator to select a set of skill areas that the operator wants to target, for example, such as Figure 12B As shown. Window 1250 can be displayed side-by-side with user interface 1200, overlap with user interface 1200, or partially overlap with user interface 1200. User interface 1200 can be changed into a new window 1250.

[0393] The skill set may include, but is not limited to, self-initiated requests, listener responses, turn-taking, shared attention, social etiquette, and play. The default selection can be any skill set chosen in a previous target monitoring session. The network server may recommend consistent skill set selections across multiple sequential monitoring sessions. In some cases, the network server stores previously selected target skill sets and / or personalized playlists, allowing the operator to repeat the same target monitoring session in the next session without needing to reselect target skill sets as needed. Window 1250 may also display the annotation "Automatically select from previous target monitoring sessions," which the operator can click to automatically select the same target skill sets as in previous target monitoring sessions. The operator can choose to keep the same selected target skill sets or change one or more target skill sets. In some implementations, there is a maximum number of target skill sets that can be set for selection, for example, up to four. The maximum number may be determined, for example, by the duration of the videos in the data collection playlist. After selecting or confirming a target skill set, the operator can click button 1260 in window 1250 to run a session for that target skill set.

[0394] Based on the selected target skill area, personalized playlists of videos can be built and enriched for that target skill area, for example, by including and prioritizing videos that have been identified or are known to be best for monitoring that target skill area. The networked server can optimize the playlists to maximize video content relevant to the target skill area. In some implementations, the video playlists are reordered to arrange videos (or video scenes) with annotations relevant to the target skill area at the start of a session where the patient is more focused on watching the videos. In some implementations, new videos that are already particularly rich for one or more selected skill areas are added to the playlist. In some implementations, videos irrelevant to the selected skill area are reduced or removed from the playlist to maintain a reasonable video length for the session. In some implementations, visual scenes relevant to the selected one or more target skill areas are ordered according to their weighted relevance values ​​to the selected one or more target skill areas. In some implementations, only visual scenes relevant to the selected one or more target skill areas are selected in the data collection playlist.

[0395] For example, when a target skill area is selected in window 1250 or when button 1260 for running a session in the target skill area is clicked, a video playlist can be personalized by a network server, an operator-side computing device, or a patient-side computing device. In some implementations, the network server receives input from the operator via a portal, personalizes the video playlist, and transmits information about the personalized playlist (e.g., the video sequences in the playlist) to the patient-side computing device, which can be configured to adjust the playlist based on the personalized playlist information. In some cases, if a new video is not previously installed on the patient-side computing device, the patient-side computing device can download the new video from the network server. In some implementations, a command can be transmitted from the network server to the patient-side computing device, the command being configured to personalize the playlist on the patient-side computing device based on the command.

[0396] Figures 13A to 13D It shows a method for users to connect to a network server (e.g., Figure 1A cloud server 110 or Figure 2A The portal website of the platform subsystem 220 (e.g., Figure 2A The portal website (222) reviews the user interface for session information on the user's device. The user can be, for example, a treatment provider, clinician, or patient caregiver, or any other suitable person with permission or authorization to access patient information. The user device can be any suitable computing device and is capable of interacting with... Figures 1A to 1B Operator-side computing device 140 or Figures 4A to 4J The operator-side computing devices described herein may be the same or different.

[0397] Figure 13A An example illustrative user interface 1300 for reviewing session information on a user device is shown according to one or more embodiments of the present disclosure. The user interface 1300 includes a menu 1302 displaying buttons “Home,” “Patient,” and “Appointment” (e.g., Figure 12A The user interface 1300 includes a menu 1210. Clicking one of the buttons displays corresponding information (e.g., patient information, device information, or appointment information) in the user interface 1300. Users can select a patient to review their information. The user interface 1300 also includes another menu 1304 displaying buttons for "Information," "Session," and "History." Clicking one of the buttons displays corresponding information (e.g., patient or appointment information, session information, or history information) in the user interface 1300.

[0398] For example, by selecting “Session” in menu 1304, patient-associated session information 1306 is presented in user interface 1300. Session information 1306 includes session background information 1308, which may include session date, patient name, session age (in months) indicating how long the session has been performed, session status (uploaded or not), quality checks passed (yes or no), device (which patient-side computing device was used to capture session data), and / or operator (who used an operator-side computing device to run the session).

[0399] In addition to the conversation background information 1308, the conversation information 1306 also includes a button 1310 for reviewing the conversation results (e.g., such as...). Figure 8A , Figures 8B to 8C , Figure 13B-1 and / or Figure 13B-2 (As detailed above), button 1312 for customizing diagnostic / monitoring reports (e.g., as... Figure 13C (as detailed above), and button 1314 for launching the interactive results dashboard (e.g., as described above). Figure 13D (As detailed above).

[0400] Figure 13B-1 An example portion 1320 of an evaluation report according to one or more embodiments of the present disclosure is shown, which illustrates a comparison between annotated video scenes, information on typical gaze behavior groups, and information on patient gaze behavior in different specific skill domains. Figure 13B-2 Another example section 1330 of an evaluation report according to one or more embodiments of the present disclosure is shown, illustrating information on monitoring treatment-specific skills in section 1332 and characteristic skills in section 1334.

[0401] In some implementations, the evaluation report may include example section 1320 and / or Figure 8A , Figure 8B and / or Figure 8CThe diagnostic report presents information as described above. This can be achieved by running, for example... Figure 12A The diagnostic session is used to generate a diagnostic report. In some implementations, the assessment report may include example section 1320, example section 1330, and / or... Figure 8A , Figure 8B and / or Figure 8C The monitoring report presents the information. As mentioned above, this can be achieved by running, for example... Figures 12A to 12B The monitoring session or target monitoring session described herein generates a monitoring report.

[0402] In some implementations, for example, such as Figure 13A As shown, the example section 1330 of the assessment report can include an introductory section 1322 and a graph section 1324. The introductory section 1322 describes the information in the graph section 1324, such as, "Quantifying the degree to which an individual's gaze behavior indicators are consistent with expected age-related gaze behavior during a period when an expert clinician has identified them as relevant to a specific skill domain..." Chart section 1324 can be similar to Figure 11 Example 1100 includes relevant skill domains, example video scenes, information on typical group annotation behaviors (distribution plots and highlighted video scenes), and information on patient gaze behaviors (highlighted video scenes and statistical scores or percentages of convergent gaze). Relevant skill domains may be automatically selected by the networked server, or selected for those with the largest amount of reliable data or the most frequently requested skills / skill domains, or for those with particularly high, low, or representative scores, or selected from the patient's previous assessment reports, or selected when initiating a target monitoring session, or a combination thereof. For example, as... Figure 13B-1 As shown, Chart 1324 includes information on the skill areas of “autonomous requests,” “listener responses,” and “co-concerns,” which can be used by expert clinicians to annotate video scenes. This shows that patient Ben was involved in 29% of the moments related to “autonomous requests,” 37% of the moments related to “listener responses,” and 32% of the moments related to “co-concerns.”

[0403] If a previous monitoring session for the patient existed, sample section 1330 of the assessment report can indicate changes relative to the previous session. For example... Figure 13B-2As shown, Example Section 1330 includes a monitoring section 1332 that illustrates changes in relevant skill areas, including a graph showing the percentage of convergent fixation (e.g., voluntary requests, listener responses, and co-focus) changing across a series of sessions in each relevant skill area. Monitoring Section 1332 can record comparisons of the percentage of convergent fixation between the current and previous sessions; for example, Ben focused on 29% of the time related to “voluntary requests,” up from 21% in the previous session; Ben focused on 37% of the time related to “listener responses,” unchanged from the previous session; and Ben focused on 14% of the time related to “co-focus,” up from 13% in the previous session. In this way, the user can easily determine whether the patient has made any progress in each relevant skill area and / or whether the current treatment is effective or helpful. Example Section 1330 may also include information 1334 about important skills / skill areas, which may include definitions of relevant skill areas (e.g., voluntary requests, listener responses, and co-focus).

[0404] If the user selects the button 1310 for reviewing results in the user interface 1300, a default report with automatically selected skill areas can be provided to the user, for example, Figures 8A to 8C , Figure 13B-1 and / or Figure 13B-2 The aforementioned assessment report. In contrast, if the user selects button 1312 to view a customized report, the network server can display a new window 1340, for example, as... Figure 13C As shown, instead of generating a default report, users can select target skill areas (e.g., self-request, listener response, shared attention, and play) to customize the assessment report. A maximum number of target skill areas (e.g., up to four) can be selected for customization. After selecting target skill areas, users can click the "View Customized Report" button to generate a customized report. The customized report can resemble the assessment report, for example, as... Figure 13B-1 and / or Figure 13B-2 As shown. The relevant skill areas in the customized report are the target skill areas selected via window 1340. In some examples, if the user selects a target monitoring session, the network server (e.g., an operator application running on the network server) can automatically customize the report for the target monitoring session, selecting the same target skill areas as those selected in the playlist for that target monitoring session. The new window 1340 can be displayed side-by-side with, overlap with, or partially overlap with the user interface 1300. The user interface 1300 can become the new window 1340.

[0405] Return to reference Figure 13AIf the user selects button 1314 to launch the interactive results dashboard, an example interactive results dashboard 1360 can be displayed in the user interface 1350 presented on the user device's screen, for example, as shown in the example. Figure 13D As shown. The dashboard 1360 may include a section 1362 for the user to select a target skill area to interact with (e.g., a self-request), a section 1364 for the user to select a specific session from a series of sequential sessions, and a section 1366 that displays the patient's gaze behavior (e.g., time-by-time (or frame-by-frame)) compared to the gaze behavior of a reference group. For example, section 1366 may display a highlighted video scene 1370 of the reference group (e.g., Figure 11 The highlighted video scene 1134) and the highlighted video scene 1372 of the patient (e.g., Figure 11 The comparison of highlighted video scenes (1142). Section 1366 may also include a playback slider (1368) that allows the user to watch the video or pause or select a specific moment / frame (still image) for comparison or viewing. Section 1366 may also include convergent gaze percentage information for the current session and previous sessions.

[0406] Figure 14 This is a flowchart of an example process 1400 for managing a specific skill in an assessment of a developmental disability, according to one or more embodiments of this disclosure. Process 1400 may be executed by a network-connected server, which may be a cloud server in a cloud environment, for example... Figure 1A cloud server 110 or such Figures 2A to 2G The cloud server described herein. For example, a network-connected server may include a platform (e.g., Figure 1A 112 or Figures 2A to 2G 220) and data pipeline systems (e.g., Figure 1A 114 or Figures 2A to 2G (230). The platform may include a portal website (e.g., Figures 2A to 2G 222), application data database (e.g., Figures 2A to 2G 224) and databases (e.g., Figures 2A to 2G (226). A data pipeline system may include one or more data processing modules (e.g., Figures 2A to 2G (232) and one or more data analysis modules (e.g., Figures 2A to 2G (234).

[0407] In step 1402, a request is received at the network server. This request is for the patient's assessment results based on the patient's session data. Session data is collected during a data collection playlist of visual stimuli presented to the patient in the session. At least one visual scene in the data collection playlist is annotated with at least one of several skill areas associated with the visual scene in the data collection playlist.

[0408] In step 1404, the networked server outputs the patient's assessment results. For each specific skill domain among multiple skill domains, the assessment results include the patient's behavioral data relative to moments in the session associated with that specific skill domain, each moment corresponding to a corresponding visual scene in the visual scenes of the data collection playlist.

[0409] In some embodiments, behavioral data includes attention percentage, defined as the ratio between the number of times a patient pays attention to relevant scene content and the total number of times a patient is viewing visual stimuli, for example, such as... Figure 11 As described in [the text].

[0410] In some embodiments, session data includes the patient's eye-tracking data. The network server can determine the total number of times the patient is viewing visual stimuli based on the patient's eye-tracking data, and the number of times the patient is focused on relevant scene content based on the patient's eye-tracking data.

[0411] In some embodiments, process 1400 may further include: determining, at a certain moment in the session, that the patient's area of ​​attention is within a predetermined region, and determining that the moment is one of a plurality of moments when the patient is focused on relevant scene content. The predetermined region corresponds to a distribution map of behavioral data of a reference group (e.g., Figure 11 The outline of Figure 1132 (e.g., Figure 11 (The contours are 1133a or 1133b). Behavioral data for the reference group can be based on reference session data collected during a data collection playlist presented to each member of the reference group. The values ​​of the contours in the distribution map can correspond to cutoff thresholds.

[0412] In some embodiments, such as Figure 13B-1 As shown, the assessment results also include a distribution map of the behavioral data of the reference group. The assessment results may also include at least one of the following: a representative visual scene for each specific skill domain within one or more specific skill domains, the representative visual scene highlighting one or more areas of attention of the reference group in a predetermined region, or a representative visual scene highlighting the patient's areas of attention during the session.

[0413] In some embodiments, such as Figure 13B-2 As shown, the assessment results may also include at least one of the following: behavioral data from one or more previous sessions of the patient for each of the one or more specific skill areas, or a comparison between the behavioral data from the session and the behavioral data from one or more previous sessions of the patient. The assessment results may include a graph showing the behavioral data from the patient's session and the behavioral data from one or more previous sessions for each of the one or more specific skill areas.

[0414] In some embodiments, process 1400 further includes: selecting one or more specific skill areas from a plurality of skill areas for the patient's assessment outcome. Selecting one or more specific skill areas from a plurality of skill areas includes at least one of the following: selecting a specific skill area with reliable data from a plurality of skill areas; selecting a frequently requested skill area from a plurality of skill areas; selecting a skill area with particularly high, low, or representative scores from a plurality of skill areas, wherein the scores represent the patient's percentage of attention; selecting a skill area previously selected as a target skill area in a session; selecting a skill area selected for customizing the assessment outcome; or selecting a skill area selected in a previous session of the patient or a previous assessment outcome of the patient.

[0415] In some embodiments, such as Figure 12A As shown, process 1400 includes: receiving a session request to initiate a session via a portal website on a network server; and displaying the request on the user interface of the portal website (e.g., ...). Figure 12A The user interface 1200 presents a list of sessions; and the user interface also presents a list of sessions and receives selections of sessions from the list of sessions on the user interface.

[0416] In some embodiments, such as Figure 12B As shown, process 1400 further includes: in response to receiving a selection of the session, popping up a window for selecting a target skill domain from a list of multiple skill domains in the window (e.g., Figure 12B The window (1250) receives user input by selecting one or more target skill areas, and runs a session based on the selected target skill areas. The selected target skill areas may include one or more specific skill areas from the evaluation results.

[0417] In some embodiments, the network server adjusts the data collection playlist of visual stimuli based on one or more selected target skill domains, for example by at least one of the following: prioritizing visual scenes annotated to monitor one or more selected target skill domains in the data collection playlist; enriching the data collection playlist with additional visual scenes related to one or more selected target skill domains; or reducing or removing visual scenes unrelated to the selected target skill domains in the data collection playlist.

[0418] In some embodiments, prioritizing visual scenes annotated to monitor one or more selected target skill domains includes at least one of the following: arranging the visual scenes annotated to monitor one or more selected target skill domains at the beginning of the data collection playlist; arranging visual scenes related to the one or more selected target skill domains in order of their weighted correlation values ​​with the one or more selected target skill domains; or selecting only the visual scenes in the data collection playlist related to the one or more selected target skill domains. User input can be received from an operator-side computing device, which communicates with a web server via a portal. A network server can establish communication between the operator-side computing device and the patient-side computing device, and transmit information about the adjusted data collection playlist of visual stimuli to the patient-side computing device, such that the adjusted data collection playlist of visual stimuli is presented to the patient on the display screen of the patient-side computing device during the session. The operator-side computing device may be... Figures 1A to 1B Computing device 140 or about Figures 4A to 4J and / or Figures 12A to 12B and Figures 13A to 13D The operator-side computing device. The patient-side computing device can be... Figure 1A Computing device 130 or about Figures 4A to 4J The patient-side computing device.

[0419] In some embodiments, process 1400 further includes: receiving patient session data from the patient's patient-side computing device once the session has ended. The patient's session data is collected by the patient-side computing device during the session, and the patient's session data is processed using reference data based on reference groups and one or more specific skill domains to generate the patient's behavioral data.

[0420] In some embodiments, process 1400 further includes, for example, loading reference data from a library or cloud storage. The reference data may be based on behavioral data of the reference group, which is based on reference session data collected during the presentation of visual stimuli in a data collection playlist and one or more specific skill domains.

[0421] In some embodiments, the reference data includes at least one of the following: for each specific skill domain in one or more specific skill domains, a specific visual scene associated with that specific skill domain, each specific visual scene highlighting one or more areas of interest of the reference group; or for each specific visual scene, a distribution map of behavioral data of the reference group, for example, such as Figure 11 and Figure 13B-1 As shown.

[0422] In some embodiments, the reference data includes: contours in a distribution map representing thresholds for each specific skill domain in one or more specific skill domains and for each specific visual scene, used to determine whether a patient is paying attention to relevant scene content of a specific visual scene. Process 1400 may further include at least one of the following: determining that the patient is paying attention to relevant scene content of a specific visual scene if the patient's area of ​​attention is within a predetermined area corresponding to the contour, or determining that the patient is not paying attention to a specific visual scene if the patient's area of ​​attention is outside the predetermined area.

[0423] In some embodiments, the patient's behavioral data includes an attention percentage, defined as the ratio between the number of times the patient pays attention to relevant scene content and the total number of times the patient is viewing visual stimuli. Process 1400 may further include: determining, at a certain moment in the session, that the patient's attention area is within a predetermined region, and determining that moment as one of a plurality of moments when the patient is paying attention to relevant scene content. In some embodiments, the patient's behavioral data includes a comparison between the patient's attention percentage and a threshold attention percentage of a reference group. For example, the comparison result may include at least one of the following: a ratio between the patient's attention percentage and a threshold attention percentage of the reference group, or a relationship between the patient's attention percentage and a threshold attention percentage of the reference group.

[0424] In some embodiments, receiving a request includes: receiving the user interface of a portal website of an internet server from a user equipment (e.g., ...). Figure 13A User input on the user interface (1300) indicates a request, and the user interface is presented on the display screen of the user device.

[0425] In some embodiments, such as Figure 13A As shown, the user interface includes a first user interface element for viewing the default rating report (e.g., Figure 13A Button 1310), and a second user interface element for customizing the assessment report (e.g., Figure 13A Button 1312) or used to launch an interactive dashboard with evaluation results (e.g., Figure 13D The third user interface element of the dashboard (e.g., 360) Figure 13A At least one of the buttons (1314).

[0426] In some embodiments, such as Figure 13CAs shown, process 1400 further includes: in response to the selection of a second user interface element, popping up a window on the user interface for selecting a target skill area in the assessment report; receiving second user input in the window for selecting one or more target skill areas; and generating an assessment report based on the selected one or more target skill areas, wherein the one or more target skill areas include one or more specific skill areas in the assessment results.

[0427] In some embodiments, such as Figure 13D As shown, process 1400 further includes: in response to the selection of a third user interface element, presenting an interactive dashboard in the user interface. The interactive dashboard may include a sub-window for interacting with an item selected from the list of skill domains. In response to receiving a selection of a specific target skill domain from the list of skill domains, process 1400 may present at least one of the following: a change in the patient's percentage of attention for the specific target skill domain over a series of consecutive sessions; a change in the ratio between the patient's percentage of attention and a threshold percentage of attention for a reference group; a change in the relationship between the patient's percentage of attention and the threshold percentage of attention for a reference group; or for each of multiple visual scenes related to the specific target skill domain, a first scene (highlighting one or more areas of attention of the reference group in that visual scene) and a second scene (highlighting the patient's area of ​​attention in that visual scene).

[0428] In some embodiments, multiple visual scenes overlap in the user interface, and the interactive dashboard includes a sliding user interface element for selecting each of the multiple visual scenes (e.g., ...). Figure 13D (Slider 1368).

[0429] In some embodiments, the network server is configured to: store annotation data of visual scenes from a data collection playlist of visual stimuli, the annotation data specifying a corresponding specific skill domain associated with the visual scene; and store reference data of reference groups, the reference data being based on behavioral data collected during the presentation of the data collection playlist of visual stimuli. The annotation data and reference data may be stored in a library or cloud storage.

[0430] In some embodiments, session data includes at least one of the following: an eye tracker device (e.g., an eye tracker device assembled in a patient-side computing device communicating with a networked server). Figure 1A Eye-tracking data collected by an eye-tracking device 134, or by one or more recording devices (e.g., Figure 1A The recording device 138 collects at least one of image data, audio data, or video data. One or more recording devices may be integrated into at least one patient-side computing device (e.g., such as...). Figure 1A(As shown) Inside or outside the computing device located on the patient side.

[0431] In some embodiments, the operator-side computing device is configured to: access a portal at a networked server, receive user input on a user interface of the portal for requesting assessment results from the patient based on the patient's session data, which is collected during a session while a data collection playlist of visual stimuli is presented to the patient. Each visual scene in the data collection playlist is annotated with at least one of a plurality of skill domains associated with the visual scene in the data collection playlist. The operator-side computing device may be configured to: present the assessment results on a display screen of the operator-side computing device. For each specific skill domain among one or more of the plurality of skill domains, the assessment results may include behavioral data of the patient relative to moments in the session associated with the specific skill domain, each moment corresponding to a corresponding visual scene in the visual scene of the data collection playlist.

[0432] In some embodiments, the patient-side computing device is configured to initiate a session for the patient by establishing communication with an operator-side computing device and a patient-side portable tablet computing device, wherein the patient-side portable tablet computing device communicates with an eye-tracking device (e.g., ...). Figure 1A The device integrates an eye tracker device 134; sequentially presents visual scenes of a data collection playlist of visual stimuli to the patient on the screen of a portable tablet computing device on the patient's side, while simultaneously collecting the patient's eye-tracking data using the eye tracker device. Each visual scene in the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scenes in the data collection playlist; and transmits session data, including the patient's eye-tracking data collected during the session, to a networked server. The data collection playlist may include visual scenes associated with one or more specific skill domains from a plurality of skill domains prioritized in the data collection playlist.

[0433] In some embodiments, the patient-side computing device is configured to collect at least one of image data, audio data, or video data collected by one or more recording devices as visual scenes in a visual stimulus data collection playlist are presented sequentially. The one or more recording devices may be integrated within or located outside the patient-side computing device. Session data may include at least one of image data, audio data, or video data.

[0434] Example Management of Treatment Plans The assessment system disclosed herein can be configured to assess developmental disorders, such as autism spectrum disorder (ASD). The assessment system can be... Figures 2A to 2G System 200, and can Figure 1AThis is implemented in environment 100. The evaluation system may include a cloud server (or a network-connected server), for example, Figures 1A to 1D cloud server 110 or about Figures 2A to 2G The cloud server. In some examples, the assessment system disclosed herein is referred to as EarliPoint.

[0435] As discussed in detail above (for example, Figures 1A to 14 The assessment system can perform one or more assessment sessions (e.g., diagnostic sessions or monitoring sessions) to assess a patient's developmental disorder and generate corresponding assessment reports (e.g., such as...). Figures 8A to 8C , Figures 13A to 13D or Figures 16A to 16F As shown). Figures 15A to 15H As discussed in further detail, the assessment system can also generate specific guiding treatment plans for patients based on at least one of treatment plan data, patient data, or reference data.

[0436] In some implementations, the cloud server includes a cloud platform (e.g., Figure 1A cloud platform 112 or Figures 2A to 2G Platform subsystem 220) and data pipeline system (e.g., Figure 1A Data pipeline system 114 or Figures 2A to 2G The data pipeline subsystem 230). The cloud platform can be configured to provide a portal, store application data associated with the treatment provider, and store data such as raw eye-tracking data, processed data, analytical and / or diagnostic results, and / or treatment plan data. The data pipeline system can be configured to perform data processing and data analysis. Figure 6 and Figures 7A to 7B As discussed in detail, cloud servers can automatically receive, process, and analyze session data from multiple computing systems, and process and analyze session data from multiple sessions from a large number of computing systems in parallel.

[0437] In some implementations, during processing and analysis (e.g., via a data pipeline system), individual patient data can be compared with reference data generated from historical eye-tracking data of patients previously belonging to the same or similar groups and / or having similar age, background, and / or condition (e.g., using electronic intelligent algorithms or models). The results of the comparison can be a diagnosis of a neurodevelopmental disorder, including but not limited to ASD, indicators of the patient's developmental / cognitive function, and / or guidance for treatment planning. Alternatively or additionally, the collected data can be compared and / or reviewed for a given patient in multiple sessions (and over a predetermined time period) to identify potential changes in visual fixation (e.g., a decline in visual fixation). These results can be compressed into a diagnostic report for use by the patient's physician. Once the diagnostic results are ready, the assessment system can transmit the diagnostic results to a computing device (e.g., operator-side computing device 140 of Figure 1) for viewing, and the diagnostic results can be presented on the user interface of operator-side computing device 140, for example, as shown in the image. Figures 8A to 8C and Figure 13B-1 , Figure 13B-2 , Figure 13C and Figure 13D as well as Figures 16A to 16F As discussed in the document. In some specific implementations, large amounts of model data (including data relating to patients of similar age, background and / or condition) can be used with processed patient session data to generate patient diagnoses and / or patient-guided treatment plans, for example, by using comparisons or inferences via statistical models, algorithms, artificial intelligence (AI) models (such as machine learning or artificial neural network models), which can greatly improve the accuracy of diagnostic results.

[0438] For patients with developmental disorders (e.g., ASD), different treatment plans may exist based on the patient's age, background, condition, assessment report, developmental stage, demographics, geography, and / or available resources / practitioners. Furthermore, the same patient may have already undergone different treatment plans. Example treatment plans are described below, including but not limited to at least one of the following: EarliPoint, the Early Intervention Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Jointly Focused Symbolic Play Participation and Regulation (JASPER), and the ImPACT Project (Parent-to-Communication Teachers). Note that the above treatment plans are merely examples, and any other suitable treatment plans may be implemented within this disclosure.

[0439] The EarliPoint assessment system uses three developmental disability indices to assess a patient's developmental disabilities: a social deficit index, a language impairment index, and a non-language learning index (e.g., ...). Figure 8A and Figures 16A to 16E (as shown in the image) and specific skill areas, which include autonomous requests, listener responses, turn-taking, co-attention or responsive co-attention (RJA), social decorum and / or play (e.g., as shown in the image) Figures 11 to 14 and Figure 16F As shown in the diagram. As mentioned above, skill domains can include multiple specific skills (e.g., more than 10, 20, 50, or 100). The EarliPoint assessment system can generate treatment plans based on the developmental disability index and the scores of skill domains / skills.

[0440] The Early Intervention Denver Model (ESDM) is a behavioral therapy for children with autism aged 12-48 months. It is based on the applied Behavior Analysis (ABA) approach. Parents and therapists use play to build positive and engaging relationships. Through play and collaborative activities, children are encouraged to improve their language, social, and cognitive skills. ESDM treatment is based on an understanding of typical early childhood learning and development, focusing on building positive relationships. Teaching occurs during natural play and daily activities, using play to encourage interaction and communication. ESDM helps children progress in their social, language, and cognitive skills.

[0441] Early Social Interaction (ESI) provides early intervention for young children with autism spectrum disorder (ASD) and their families. ESI teaches parents how to support their children’s social communication, emotional regulation, and play in daily activities and environments.

[0442] Discrete Trial Training (DTT) is an intensive therapeutic program used to help children with developmental disabilities such as autism. DTT involves training children with autism who may not be able to acquire various skills independently. The method focuses on teaching skills through a step-by-step process, rather than teaching the desired skill all at once. DTT uses a basic process to teach a new skill or behavior and repeats it until the child learns it. This process involves giving instructions such as picking up a cup. If needed, the child follows the instructions using physical or verbal cues (such as pointing to the cup).

[0443] JASPER is a treatment approach based on a combination of developmental and behavioral principles. This intervention model targets the foundations of social communication (co-attention, imitation, play), uses natural strategies to increase the rate and complexity of social communication, and includes parents and teachers as implementers of the intervention to promote generalization across settings and activities and ensure maintenance over time.

[0444] Project ImPACT is a parent-mediated intervention for young children with autism spectrum disorder (ASD) and associated social communication delays. Parent mediation means that Project ImPACT coaches teach parents techniques to use with their children. Project ImPACT instruction uses systematic instructions to increase parental responsiveness to children's behavior and teaches parents how to use cueing and reinforcement to teach children to use new communication, imitation, and play skills in child-led interactions. It is a Natural Developmental Behavioral Intervention (NDBI). NDBI is a relatively new category of interventions already notified by the fields of developmental and communication science and applied behavior analysis (ABA). Project ImPACT can be implemented in individual or group instruction models and is applicable to remote health.

[0445] Different treatment programs may have different skill domain names, different cuing methods, different treatment / training materials, different reinforcement methods, and / or different data collection methods. For example, Discrete Trial Training (DTT) is more rigid in terms of structure, materials, location, and implementation methods. Natural Developmental Behavioral Interventions (NDBI) (such as ESDM, JASPER, etc.) maintain the core principles of ABA but are less rigid, more natural, and more flexible in terms of location and materials. There are also different subgroups within these two ABA categories that use their own specific vocabulary, cue structures, data collection forms, etc. Some key differences between treatment methods may include: a) the exact name of the skill (e.g., “request” vs. “autonomous request,” “listener response” vs. “follow direction”), b) cuing methods (e.g., from more to less, from less to more, how and when to adjust cues, and the level of verbal or nonverbal communication in the cues), c) materials (e.g., pre-arranged and intentionally used in the therapeutic environment or for a sense of naturalness in the child's environment), and d) reinforcement methods that may be physical rewards, intrinsic rewards, screen time, editable, etc.

[0446] For illustration, Table 1 shows two example treatment plans (ESDM and DTT) for the same skill area. Note that clinicians may have flexibility in creating specific treatment plans for patients.

[0447] Table 1 Comparison of ESDM and DTT treatment program plans

[0448] like Figures 15A to 15H As further detailed below, embodiments of this disclosure provide techniques for integrating treatment data (e.g., in different treatment plan formats), patient data, and reference data to generate a patient-specific treatment plan.

[0449] Figures 15A to 15HAn example illustrative user interface presented on a computing device according to one or more embodiments of the present disclosure is shown. The computing device may be an operator device or an operator-side computing device, such as... Figures 1A to 1B The computing device 140, or about Figures 4A to 4J The operator-side computing device described. For example, an operator (e.g., a medical assistant, medical professional, or any other representative of a treatment provider) can log in to a portal running on a cloud server (e.g., Figure 2A The portal website (222) is used for equipment management, patient management, and data management. Operators can have corresponding user roles and permissions, for example, such as... Figure 2D As discussed in [the document]. After the operator logs into the portal using the operator-side computing device, a user interface (UI) can be displayed on the operator-side computing device's screen. The UI can be an operator application running on a cloud server or on the operator-side computing device (e.g., [example application]). Figure 2A The user interface of the operator application (216).

[0450] Figure 15A An example illustrative user interface 1500 is shown presented on a computing device when a data aggregator application is running on a cloud server. The data aggregator application can be configured to: connect to one or more third-party tools to ingest (e.g., by parsing) patient treatment data, including data from EHR (Electronic Health Record) / EMR (Electronic Medical Record) and ABA (Applied Behavior Analysis) practice management tools, and optionally, referencing patient data, ingest (e.g., by parsing) the patient's (and / or other patients') treatment plans, goals, behavioral stressors, patient responses over time, and other relevant clinical or treatment data. The data aggregator application can be configured to build a large and unique data repository of clinical treatment and patient trajectories by combining assessment data with rating systems.

[0451] like Figure 15A As shown, the data aggregator application allows operators to upload patients' latest treatment plans. Treatment plans can be in a format different from the default treatment plan format associated with the assessment system (e.g., EarliPoint). Figure 15A As shown, the user interface 1500 includes a selection element 1502 for the operator to select the plan format of the treatment plan to be uploaded, a drop-down list 1504 displaying a list of treatment plan formats (e.g., EarliPoint, ESDM, ESI, DTT, JASPER, and Project ImPACT), and an upload document element 1506 for uploading the document containing the treatment plan. The data aggregator application can access patient data from a repository stored on a cloud server (e.g., in...). Figure 2AThe treatment plan document is uploaded from a computing device or from a storage medium coupled to the computing device (e.g., USB, disk, or storage disk) in the platform 220.

[0452] After the treatment plan document is uploaded, the data aggregator application can automatically parse the document to retrieve relevant information based on the treatment plan's plan format. As mentioned above, different treatment plans can have different names for the same skill area. The data aggregator application can convert different skill area names into the same skill area names used in the assessment system (e.g., default skill area names). For example, the skill area name "Request" in ESDM can be converted to "Autonomous Request" in EarliPoint. The data aggregator application can also consider skill areas, such as how many hours per week are trained for each treatment-specific skill area? What is the impact of the treatment plan on the treatment-specific skill area? In some examples, the impact of the treatment plan can be determined based on the patient's convergent viewing percentage.

[0453] Figure 15E A graph 1540 illustrates the breakdown of effort in example treatment-specific skill areas within a patient's treatment plan. Example treatment-specific skill areas include autonomous requests, play, social tacting, and RJA. The treatment plan may include the number of hours (e.g., weekly) in the example treatment-specific skill areas since the last assessment report was generated for the patient. Effort can be determined based on the percentage of corresponding hours in a treatment-specific skill area relative to the total hours in the treatment plan. For example, graph 1540 shows the breakdown percentages of different treatment-specific skill areas in the treatment plan, including 41% for "play," 25% for "social tacting," 17% for "autonomous requests," and 17% for RJA.

[0454] As mentioned above, a patient's gaze behavior reflects their proficiency in a specific skill area being treated. Therefore, monitoring changes in the percentage of converging gazes over a series of subsequent sessions (or over a period of time) can demonstrate the impact of the treatment plan on the skill area. Figure 15F Figure 1550 illustrates a patient's attention (represented by the percentage of converging gaze) to scenes related to important skills across four sessions over a period of time, for example, as Figure 13B-2 As shown in Figure 1150, a comparison of the convergent gaze percentage between the current session and one or more previous sessions can be determined. For example, as... Figure 15FAs shown, Ben focused on 29% of the moments related to "autonomous requests," up from 21% in the previous session; Ben focused on 37% of the moments related to "listener responses," unchanged from the previous session; and Ben focused on 14% of the moments related to "co-focus," up from 13% in the previous session. Changes in the percentage of converging gaze based on skill domains can be used to understand the impact of treatment plans across adjacent sessions or multiple sessions on these specific skill domains.

[0455] Figure 15G Figure 1560 illustrates the relationship between effort and effect across different skill domains. It shows that: i) for the skill domain of RJA, high effort results in low effect, thus indicating that treatment plans have no or minimal effect on RJA and require modification (e.g., no treatment for RJA for a period of time); ii) for the skill domain of Gait, high effort results in high effect, thus indicating that treatment plans can have an effect on Gait through high effort; iii) for the skill domain of Social Decency, moderate effort results in high effect, thus indicating that treatment plans can increase Social Decency effort to achieve a higher effect; and iv) for the skill domain of Self-Demand, low effort results in moderate effect, thus indicating that treatment plans can increase Self-Demand effort to achieve a higher effect.

[0456] Return to reference Figure 15A By parsing treatment plans, cloud servers (e.g., data aggregator applications) can retrieve and / or determine information about the treatment plans (or treatment data). Data aggregator applications can also analyze multiple previous treatment plans for a patient to determine the relationship between the treatment plans and the effort and impact on specific skill areas of treatment, for example, such as... Figure 15E , 15F As shown in 15G, it can be presented on the user interface of the computing device for the operator to view. For example, based on this relationship, the operator can consider when selecting a playlist of visual stimuli for the patient, such as selecting more visual scenes in skill domains with high impact and low or moderate effort, such as social decorum and autonomous requests, like... Figure 15G As shown.

[0457] In some implementations, in addition to aggregating data from patients’ treatment plans, the data aggregator application is configured to, for example, aggregate data from treatment plans of other patients identified as having similar age, background, developmental stage, demographics, geography, and / or condition based on patient information.

[0458] In some implementations, data aggregator applications are configured to aggregate patient data and / or reference data from one or more third-party tools for use. For example, a data aggregator may connect to one or ...

Claims

1. A system using at least one portable computing device with eye-tracking capability, the system comprising: A portable eye tracker console includes a display screen and an eye tracker device mounted near the display screen such that both the display screen and the eye tracker device face the patient, wherein the eye tracker device is configured to collect the patient's eye tracking coordinate data during a session when a predetermined sequence of stimulus videos is presented on the display screen; A portable computing device having a touchscreen display interface, spaced apart from the portable eye tracker console, and portable to different locations relative to the portable eye tracker console; and A network server wirelessly receives session data from the portable eye tracker console and includes a portal that exports assessment results, including a graphical correlation between a numerical defect index score and a reference assessment index. The network server is configured to be wirelessly connected to both the portable eye tracker console and the portable computing device, such that after the portable computing device wirelessly communicates with the portable eye tracker console via the network server to control the initiation of the session and presents the predetermined sequence of stimulus videos on the display screen of the portable eye tracker console, the portable eye tracker console wirelessly transmits the session data to the network server. The session data includes eye-tracking coordinate data, which has a timestamp relationship with information about the predetermined sequence of stimulus videos displayed by the portable eye tracker console during the session.

2. The system according to claim 1, comprising a plurality of the portable eye tracker consoles, the plurality of portable eye tracker consoles simultaneously communicating wirelessly with the network server.

3. The system according to claim 1 or 2, wherein, The eye tracker device includes one or more eye-tracking sensors mechanically assembled near the periphery of the display screen.

4. The system according to claim 3, wherein, Each of the one or more eye-tracking sensors includes: The illumination source is configured to emit detection light; and A camera is configured to capture eye movement data, the eye movement data including at least one of the following: reflection or reflection of detection light from the illumination source by the pupil or cornea; The eye-tracking sensor is configured to convert the eye movement data into a data stream, the data stream containing information on at least one of pupil position, gaze vector or gaze point for each eye, wherein the patient's eye-tracking data includes the patient's corresponding data stream.

5. The system according to claim 4, wherein, The detection light includes infrared light, and the camera includes an infrared-sensitive camera. When the predetermined sequence of stimulus videos is presented on the display screen facing the patient during the session, the caregiver carrying the patient wears glasses with a filter configured to filter the infrared light, such that the camera captures only the patient's eye movement data.

6. The system according to any one of claims 1 to 5, wherein, The eye tracker device includes at least one image acquisition device configured to capture an image of at least one eye of the patient when a predetermined sequence of stimulus videos is presented on the display screen facing the patient during the session. The eye tracker device is configured to generate corresponding eye tracking data for the patient based on captured images of at least one of the patient's eyes.

7. The system according to any one of claims 1 to 6, further comprising at least one recording device, said at least one recording device assembled on the portable eye tracker console and configured to: collect at least one of image data, audio data, or video data associated with the patient when the predetermined sequence of stimulation videos is presented on the display screen facing the patient during the session. in, The session data includes at least one of the image data, audio data, or video data.

8. The system according to any one of claims 1 to 7, wherein, The portable eye tracker console includes: A housing, configured to hold the display screen and the eye tracker device; and The base is connected to the housing via one or more joints; The base is rotatable about one or more joints to adjust the relative position or angle between the display screen and the patient during the session.

9. The system according to any one of claims 1 to 8, wherein, The network server provides the portal website accessible by the portable computing device, and The network server is configured to output a developmental analysis report to the portable computing device via the user interface of the portal website. The developmental analysis report includes the patient's developmental analysis data.

10. The system according to any one of claims 1 to 9, wherein, The network server is configured as follows: Receive treatment data associated with the patient, the treatment data including at least one of: the patient's previous developmental analysis data, the patient's previous treatment plan, or reference treatment data from other patients; and Based on the treatment data associated with the patient and the patient's developmental analysis data, artificial intelligence is used to generate a guided treatment plan for the patient.

11. A computer-implemented method, comprising: At a networked server, a treatment plan for the patient's developmental disorder is obtained. This treatment plan has individual time lengths for different specific skill areas within a given time period. The treatment plan has a specific treatment plan format, and the networked server is configured to process data associated with a default treatment plan format. At the network server, the treatment plan with the specific treatment plan format is parsed to determine the patient's treatment data, which conforms to the default treatment plan format.

12. The computer-implemented method according to claim 11, further comprising: At the network server, input is received for selecting a treatment plan format from multiple treatment plan formats presented on the user interface, wherein the multiple treatment plan formats are different from each other.

13. The computer-implemented method according to claim 12, wherein, The multiple treatment plan formats differ from each other in at least one of the following: skill domain name, prompting method, treatment or training materials, reinforcement method, or data collection method.

14. The computer-implemented method according to claim 12 or 13, wherein, Parsing the treatment plan having the specific treatment plan format includes: Based on the selected treatment plan format and the default treatment plan format, the treatment plan with the specific treatment plan format is parsed.

15. The computer-implemented method according to any one of claims 11 to 14, wherein, The multiple therapy program formats include two or more of the following: EarliPoint, Early Intervention Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Joint Attention Symbolic Play Participation and Regulation (JASPER), and the Improving Parents as Communication Teachers Program (ImPACT Program).

16. The computer-implemented method according to any one of claims 11 to 15, wherein, Treatment plans for patients with developmental disorders include: The treatment plan, having the specific treatment plan format, is uploaded from the repository or storage medium of the networked server.

17. The computer-implemented method according to any one of claims 11 to 16, wherein, The treatment data includes at least one of the following: The corresponding time length for each of the different treatment-specific skill areas within the stated time period. The percentage of time spent in each of the different treatment-specific skill areas within the stated time period. The percentage of attention given to different treatment-specific skill areas across a series of sessions. The change in the corresponding percentage of attention for the different treatment-specific skill areas between at least two recent sessions, or The relationship between the percentage of time spent in different therapeutic skill areas and the change in the percentage of attention spent in those areas.

18. The computer-implemented method according to any one of claims 11 to 17, further comprising: At the network server, input is received for selecting a treatment plan format from multiple treatment plan formats presented on a user interface, wherein the multiple treatment plan formats are different from each other; and At the networked server, a new treatment plan is generated based on the patient's treatment data and the selected treatment plan format.

19. The computer-implemented method according to claim 18, further comprising: The new treatment plan, with the selected treatment plan format, is transmitted from the network server to a computing device or an external server.

20. The computer-implemented method according to claim 18 or 19, further comprising: At the networked server, assessment data for the patient's developmental disorder is generated based on the patient's eye-tracking session data. The new treatment plan is generated based on the assessment data of the patient's developmental disorder.

21. The computer-implemented method according to any one of claims 18 to 20, further comprising: At the network server, a specific treatment plan format for the new treatment plan for the patient is determined from among the plurality of treatment plan formats; as well as The user interface presents a visual indication on the specific treatment plan format among the multiple treatment plan formats, the visual indication being used to indicate the new treatment plan recommended for the patient using the specific treatment plan format.

22. The computer-implemented method according to any one of claims 11 to 21, further comprising: At the network server, a selection of a target session is received, the target session being selected from a session list on the user interface of the portal website on the network server; In response to receiving a selection of the target session, a pop-up window appears for selecting a target skill area from a list of multiple skill areas. Based on the treatment data, one or more target skill areas are automatically selected from the plurality of skill areas, wherein the different treatment-specific skill areas include the one or more target skill areas; as well as The target session is run based on one or more selected target skill areas.

23. The computer-implemented method according to any one of claims 11 to 22, further comprising: At the network server, the input fields for the treatment plan are presented on the user interface of the portal website on the network server; At the network server, input for one of the input fields of the treatment plan is received on the user interface; as well as At the networked server, the treatment plan is updated based on the input to one of the input fields.

24. The computer-implemented method according to any one of claims 11 to 23, wherein, The different therapeutic-specific skill areas include one or more of the following: autonomous requests, listener responses, turn-taking, shared attention, social decency, and play.

25. The computer-implemented method according to any one of claims 11 to 24, further comprising: At the network server, input for selecting a third-party system from multiple third-party systems presented on the user interface is received; as well as After a connection is established between the network server and the selected third-party system, the network server retrieves data related to the patient from the selected third-party system, wherein the data related to the patient includes at least one of the following: the patient's previous clinical data, the patient's previous treatment data, or reference data of other patients.

26. The computer-implemented method according to claim 25, further comprising: At the networked server, a new treatment plan for the patient is generated based on the treatment data and the data related to the patient.

27. A computer-implemented method, comprising: At the network server, a request for an assessment result of the patient based on the patient's session data is received. The session data is collected during a data collection playlist of visual stimuli presented to the patient in a session used to assess the patient's developmental disorder. as well as The network server outputs the patient assessment results. The evaluation results include: The corresponding scores of the developmental disability index associated with the patient's developmental disability, and For each of the developmental disability indices, the correlation between the corresponding score of that developmental disability index and the corresponding reference assessment indicator.

28. The computer-implemented method according to claim 27, wherein, The results of the correlation include at least one of the following: A summary describing the correlation, or The correlation is presented graphically.

29. The computer-implemented method according to claim 27 or 28, wherein, The assessment results also include at least one of the following: Assessment results used to indicate whether the patient has the developmental disorder, or Indicative information for each corresponding score of the developmental disability index.

30. The computer-implemented method according to any one of claims 27 to 29, wherein, The developmental disability index includes at least one of the following: social deficit index, language proficiency index, or non-language learning index.

31. The computer-implemented method according to claim 30, wherein: The corresponding reference evaluation indicators for the social deficit index score include the ADOS-2 index. The corresponding reference assessment indicators for the language proficiency index scores include Mullen's language age equivalent, and The corresponding reference assessment indicators for the scores of the non-language learning index include the Mullen language age equivalent.

32. The computer-implemented method according to any one of claims 27 to 31, wherein, At least one visual scene in the data collection playlist is annotated with at least one skill domain from among multiple skill domains associated with the visual scene in the data collection playlist. The assessment results include: behavioral data of the patient at moments in the session related to the specific skill domain for each of one or more specific skill domains among the plurality of skill domains, with each moment corresponding to a corresponding visual scene in the visual scenes of the data collection playlist.

33. The computer-implemented method according to claim 32, wherein, The behavioral data includes attention percentage, which is defined as the ratio between the number of times the patient pays attention to relevant scene content in the visual stimulus and the total number of times the patient is watching the visual stimulus.

34. The computer-implemented method according to claim 32 or 33, wherein, The evaluation results include: The outline of the distribution map of behavioral data of the reference group, which is based on reference session data collected during a data collection playlist of the visual stimuli presented to each member of the reference group, and For each of the one or more specific skill domains, Representative visual scenes, The representative visual scene highlights one or more areas of interest of the reference group within a predetermined area, or The representative visual scene is highlighted in the patient's area of ​​focus during the session.

35. The computer-implemented method according to any one of claims 32 to 34, wherein, The assessment results include at least one of the following: A first graphical representation of time-by-time measurements of the patient's gaze behavior during the session, or A second graphical representation of the attention funnel of the reference group and the patient's attention during the session.

36. A computer-implemented method, comprising: A session is initiated for the patient by establishing communication with an operator-side computing device and a patient-side portable tablet computing device, the patient-side portable tablet computing device integrating an eye tracker device; Visual scenes of a data collection playlist of visual stimuli are presented sequentially to the patient on the screen of the portable tablet computing device on the patient's side, while the eye-tracking device collects the patient's eye-tracking data. as well as The session data of the session is transmitted to a network server, and the session data includes eye-tracking data of the patient collected during the session. The process of collecting eye tracking data of the patient using the eye tracker device includes: capturing at least one of an image of the patient's eye or the position of the patient's eye, wherein the eye tracking data is determined based on at least one of the captured image of the patient's eye or the position of the eye.

37. The computer-implemented method according to claim 36, wherein, The eye tracker device is configured to: Eye movement data are determined based on at least one of the captured images of the patient's eyes or the location of the eyes, and The patient's eye movement data is converted into eye-tracking data, which includes information associated with at least one of pupil position, gaze vector for each eye, or gaze point.

38. The computer-implemented method according to claim 36 or 37, wherein, Collecting eye-tracking data from the patient using the eye-tracking device also includes: First eye movement data of the patient's eyes are captured by measuring reflected light from the patient's eyes.

39. The computer-implemented method according to claim 38, wherein, The eye-tracking device includes: At least one eye-tracking unit is configured to capture the first eye movement data of the patient's eyes; and At least one image acquisition unit is configured to capture at least one of the patient’s eye image or the location of the patient’s eye.

40. The computer-implemented method according to claim 38 or 39, wherein, The eye tracker device is configured to: Second eye movement data are determined based on at least one of the captured images of the patient's eyes or the location of the eyes, and The eye-tracking data is determined based on the first eye movement data and the second eye movement data.

41. The computer-implemented method according to any one of claims 38 to 40, wherein, The eye tracker device is configured to: The first eye movement data is converted into first eye tracking data. Second eye movement data are determined based on at least one of the captured images of the patient's eyes or the location of the eyes. The second eye movement data is converted into second eye tracking data, and The eye-tracking data is determined based on the first eye-tracking data and the second eye-tracking data.

42. The computer-implemented method according to any one of claims 36 to 41, further comprising: When visual scenes from the data collection playlist of the visual stimuli are presented sequentially, at least one of image data, audio data, or video data collected by one or more recording devices is collected, wherein the one or more recording devices are assembled inside or located outside at least one patient-side computing device. The session data includes at least one of image data, audio data, or video data.

43. An apparatus comprising: At least one processor; as well as One or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a computer-implemented method according to at least one of claims 11 to 42.

44. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform a computer-implemented method according to at least one of claims 11 to 42.

45. A computer-implemented method, comprising: On a networked server, a request is received for an assessment result of the patient based on patient session data collected during a session while a data collection playlist of visual stimuli is presented to the patient; wherein at least one visual scene in the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene in the data collection playlist; and The network server outputs the patient's assessment results, which include: behavioral data of the patient at moments in the session related to each specific skill domain in one or more of the plurality of skill domains, with each moment corresponding to a corresponding visual scene in the visual scenes of the data collection playlist.

46. ​​The computer-implemented method according to claim 45, wherein, The behavioral data includes attention percentage, which is defined as the ratio between the number of times the patient pays attention to relevant scene content in the visual stimulus and the total number of times the patient is watching the visual stimulus.

47. The computer-implemented method according to claim 46, wherein, The session data includes the patient's eye-tracking data, and The computer-implemented method further includes: The total number of moments during which the patient was viewing the visual stimulus was determined based on the patient's eye-tracking data, and The number of times the patient focused on relevant scene content was determined based on the patient's eye-tracking data.

48. The computer-implemented method according to claim 46 or 47, further comprising: At a certain point during the session, it is determined that the patient's area of ​​focus is within a predetermined area; as well as The time is determined to be one of several times when the patient pays attention to the relevant scene content.

49. The computer-implemented method according to claim 48, wherein, The predetermined area corresponds to the outline of the distribution map of the behavioral data of the reference group, which is based on reference session data collected during the data collection playlist of the visual stimuli presented to each member of the reference group.

50. The computer-implemented method according to claim 49, wherein, The values ​​of the contour of the distribution map correspond to the cutoff threshold.

51. The computer-implemented method according to claim 49 or 50, wherein, The evaluation results also include a distribution map of the behavioral data of the reference group.

52. The computer-implemented method according to any one of claims 49 to 51, wherein, The evaluation results also include at least one of the following: for each of the one or more specific skill domains. Representative visual scenes, The representative visual scene highlights one or more areas of interest of the reference group within the predetermined area, or The representative visual scene is highlighted in the patient's area of ​​focus during the session.

53. The computer-implemented method according to any one of claims 45 to 52, wherein, The evaluation results also include at least one of the following: for each of the one or more specific skill domains. Behavioral data from one or more previous sessions of the patient, or A comparison between the behavioral data of the session and the behavioral data of one or more previous sessions of the patient.

54. The computer-implemented method according to claim 53, wherein, The assessment results include graphs showing behavioral data from the session for each of the one or more specific skill domains, compared with behavioral data from one or more previous sessions of the patient.

55. The computer-implemented method according to any one of claims 45 to 54, further comprising: One or more specific skill areas are selected from the plurality of skill areas for the patient's assessment results.

56. The computer-implemented method according to claim 55, wherein, Selecting one or more specific skill areas from the plurality of skill areas includes at least one of the following: Select a specific skill area with reliable data from the plurality of skill areas. Select the most requested skill area from the multiple skill areas. Skill areas with particularly high, low, or representative scores are selected from the plurality of skill areas, where the scores represent the percentage of the patient's attention. Select the skill domain that was previously selected as the target skill domain in the session. Select the skill area chosen to customize the assessment results, or Select the skill area chosen from the patient's previous sessions or previous assessment results.

57. The computer-implemented method according to claim 56, comprising: The session request to initiate the session is received through the portal website on the network server; A list of sessions is displayed on the user interface of the portal website; as well as The user interface receives a selection of the session from the session list.

58. The computer-implemented method according to claim 57, further comprising: In response to receiving a selection of the session, a pop-up window appears for selecting a target skill area from a list of multiple skill areas. Receive user input to select one or more target skill areas in the window, and The session is run based on one or more selected target skill areas. The selected target skill areas include the one or more specific skill areas.

59. The computer-implemented method according to claim 58, further comprising: The data collection playlist of the visual stimuli is adjusted based on one or more selected target skill areas.

60. The computer-implemented method according to claim 59, wherein, The data collection playlist for adjusting the visual stimuli includes at least one of the following: In the data collection playlist, visual scenes related to one or more selected target skill areas are prioritized; The data collection playlist is enriched with additional visual scenes related to one or more selected target skill areas; or In the data collection playlist, visual scenes that are irrelevant to the selected target skill area are reduced or removed.

61. The computer-implemented method according to claim 60, wherein, Prioritizing visual scenes related to one or more selected target skill areas includes at least one of the following: Visual scenes related to one or more selected target skill areas will be arranged at the beginning of the data collection playlist; The visual scenes related to one or more selected target skill areas are arranged in order of their weighted relevance values ​​to the one or more selected target skill areas. or Select only the visual scenes that are relevant to one or more target skill areas in the data collection playlist.

62. The computer-implemented method according to any one of claims 59 to 61, wherein, Receiving user input includes: receiving the user input from an operator-side computing device, wherein the operator-side computing device communicates with the web server through the portal website, and The computer-implemented method further includes: The network server establishes communication between the operator-side computing device and the patient-side computing device, and Information from the adjusted visual stimulus data collection playlist is transmitted to the patient-side computing device, such that the adjusted visual stimulus data collection playlist is presented to the patient on the display screen of the patient-side computing device during the session.

63. The computer-implemented method according to any one of claims 45 to 62, further comprising: Once the session ends, the patient's session data is received from the patient-side computing device, wherein the patient's session data was collected by the patient-side computing device during the session; and The patient's behavioral data is generated by processing the patient's session data based on reference data from a reference group and the one or more specific skill domains.

64. The computer-implemented method according to any one of claims 45 to 63, further comprising: Load reference data for a reference group, the reference data being based on behavioral data of the reference group, the behavioral data of the reference group being based on reference session data collected during the presentation of the data collection playlist of the visual stimuli and the one or more specific skill domains.

65. The computer-implemented method according to claim 64, wherein, The reference data for the reference group includes at least one of the following: for each specific skill domain in the one or more specific skill domains, For a specific visual scene related to that particular skill area, each of the specific visual scenes highlights one or more areas of interest of the reference group; or The distribution map of behavioral data for each specific visual scene for the reference group.

66. The computer-implemented method according to claim 65, wherein, The reference data includes: For each specific skill domain in the one or more specific skill domains and for each specific visual scene in the specific visual scene. The contours in the distribution map, representing thresholds, are used to determine whether the patient is paying attention to relevant scene content of the specific visual scene. The computer-implemented method further includes at least one of the following: If the patient's area of ​​attention is within a predetermined area corresponding to the contour, then it is determined that the patient is paying attention to relevant scene content of the specific visual scene, or If the patient's area of ​​focus is outside the predetermined area, it is determined that the patient is not paying attention to the relevant scene content of the specific visual scene.

67. The computer-implemented method according to claim 66, wherein, The patient's behavioral data includes an attention percentage, defined as the ratio between the number of times the patient pays attention to relevant scene content and the total number of times the patient is viewing the visual stimulus. The computer-implemented method further includes: At some point during the session, the patient's area of ​​focus is determined to be within the predetermined area; and The time is determined to be one of several times when the patient pays attention to the relevant scene content.

68. The computer-implemented method according to claim 66 or 67, wherein, The patient's behavioral data includes a comparison between the patient's percentage of attention and the threshold percentage of attention for the reference group.

69. The computer-implemented method according to claim 68, wherein, The comparison results include at least one of the following: The ratio between the patient's percentage of attention and the threshold percentage of attention for the reference group, or The relationship between the patient's percentage of attention and the threshold percentage of attention for the reference group.

70. The computer-implemented method according to any one of claims 45 to 69, wherein, Receiving the request includes: The system receives user input from the computing device on the user interface of the portal website of the network server, the user input indicating the request, and the user interface is displayed on the screen of the computing device.

71. The computer-implemented method according to claim 70, wherein, The user interface includes at least one of the following: The first user interface element used to view the default assessment report; Secondary user interface elements for customizing assessment reports; or A third user interface element is used to launch an interactive dashboard displaying the evaluation results.

72. The computer-implemented method according to claim 71, further comprising: In response to the selection of the second user interface element, a pop-up window appears on the user interface for selecting the target skill area in the assessment report; Receive second user input in the window for selecting one or more target skill areas; as well as The assessment report is generated based on one or more selected target skill areas, wherein the one or more target skill areas include the one or more specific skill areas in the assessment results.

73. The computer-implemented method according to claim 71, further comprising: In response to the selection of the third user interface element, the interactive dashboard is presented in the user interface. The interactive dashboard includes a sub-window for selecting a skill domain from a list of skill domains for interaction.

74. The computer-implemented method according to claim 73, further comprising: In response to receiving a selection of a specific target skill domain from the list of skill domains, at least one of the following is presented: The change in the patient's attention percentage across a series of consecutive sessions for the specific target skill area. The change in the ratio between the patient's percentage of attention and the threshold percentage of attention for the reference group. The change in the relationship between the patient's percentage of attention and the threshold percentage of attention for the reference group, or For each of a plurality of visual scenes relating to the specific target skill domain, a first scene highlights one or more areas of interest of the reference group in the visual scene, and a second scene highlights the area of ​​interest of the patient in the visual scene.

75. The computer-implemented method according to claim 74, wherein, The multiple visual scenes overlap each other in the user interface, and The interactive dashboard includes a sliding user interface element for selecting each of the plurality of visual scenes.

76. The computer-implemented method according to any one of claims 45 to 75, further comprising: On the network server, annotation data of visual scenes in the data collection playlist of visual stimuli is stored, and the annotation data specifies the corresponding specific skill domain associated with the visual scene; as well as On the network server, reference data for reference groups is stored. This reference data is based on behavioral data, which is based on reference session data collected during the data collection playlist presented with the visual stimuli.

77. The computer-implemented method according to any one of claims 45 to 76, wherein, The session data includes at least one of the following: Eye-tracking data, collected by an eye-tracking device integrated into a patient-side computing device communicating with the networked server, or At least one of image data, audio data, or video data is collected by one or more recording devices, wherein the one or more recording devices are assembled inside or located outside at least one patient-side computing device.

78. A computer-implemented method, comprising: Access to a portal website hosted on a networked server from a computing device; The computing device receives user input on the user interface of the portal website, the user input being for requesting an assessment result for the patient based on patient session data collected during a session while a data collection playlist of visual stimuli is presented to the patient, wherein at least one visual scene in the data collection playlist is annotated with at least one skill domain from a plurality of skill domains associated with the visual scene in the data collection playlist; and The evaluation results are presented on the display screen of the computing device by the computing device, and the evaluation results include: behavioral data of the patient in the session at moments related to that specific skill domain for each of one or more specific skill domains among the plurality of skill domains, each moment corresponding to a corresponding visual scene in the visual scenes of the data collection playlist.

79. The computer-implemented method according to claim 78, further comprising: Establish a wireless connection with a patient-side computing device that integrates an eye tracker; as well as The user interface is presented to communicate with the patient-side computing device to collect the patient's session data.

80. A computer-implemented method, comprising: A session is initiated for the patient by establishing communication with an operator-side computing device and a patient-side portable tablet computing device, the patient-side portable tablet computing device integrating an eye tracker device; Visual scenes of a data collection playlist of visual stimuli are sequentially presented to the patient on the screen of the portable tablet computing device on the patient side, while eye-tracking data of the patient is collected using the eye tracker device, wherein at least one visual scene of the data collection playlist is annotated with at least one skill field from a plurality of skill fields associated with the visual scene of the data collection playlist. as well as The session data of the session is transmitted to a network server, and the session data includes eye-tracking data of the patient collected in the session.

81. The computer-implemented method according to claim 80, wherein, The data collection playlist includes: Visual scenes related to one or more specific skill domains in the plurality of skill domains, the visual scenes being prioritized in the data collection playlist.

82. The computer-implemented method according to claim 80 or 81, further comprising: When visual scenes from the data collection playlist of the visual stimuli are presented sequentially, at least one of image data, audio data, or video data collected by one or more recording devices is collected, wherein at least one of the one or more recording devices is assembled inside or outside the patient-side computing device. The session data includes at least one of the image data, audio data, or video data.

83. An apparatus comprising: At least one processor; as well as One or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a computer-implemented method according to at least one of claims 45 to 82.

84. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform a computer-implemented method according to at least one of claims 45 to 82.

Images