System and method for using a portable computer device with eye-tracking capabilities

Abstract

Embodiments described herein include a portable device having user detection equipment such as an eye-tracking device or other sensors, and a computer system that includes such portable device or data collected from such device (such as eye-tracking data, and / or other multimodal data such as facial expressions, verbal expressions, and / or physical movements). Some examples of the system can provide a useful assessment for developmental disorders and can generate a normative treatment plan having individual time lengths for different treatment-specific skill areas over a given period of time.

Description

Technical Field

[0001] Cross - reference to Related Patent Applications This application claims the benefit of U.S. Provisional Application No. 63 / 506,211, filed Jun. 5, 2023, and U.S. Provisional Application No. 63 / 604,647, filed Nov. 30, 2023, the contents of which are incorporated herein by reference.

[0002] The present disclosure generally relates to portable devices having user detection devices such as eye - tracking sensors and interconnected devices for using eye - tracking data and / or other multimodal data such as facial expressions, verbal expressions, and / or physical movements.

Background Art

[0003] Computer systems have been used to collect eye - tracking data from users such as pediatric patients in clinical settings for the purpose of gathering objective data on user responses to stimuli. In some cases, the objective data can indicate developmental disorders such as autism spectrum disorder (ASD). Various trials by treatment providers (e.g., pediatricians or other medical professionals) to assess the severity of ASD in patients can vary significantly from the perspective of objective assessment tools and the experience of a particular treatment provider. In some environments, the use of conventional "best practice" tools by treatment providers achieves only poor sensitivity and specificity to symptoms, especially for infants or other pediatric patients. Further, treatment providers often lack appropriate tools for objectively measuring the progression of these symptoms over time, especially very early in a patient's life.

Summary of the Invention

Means for Solving the Problems

[0004] This disclosure describes a portable device having a user detection device such as an eye-tracking device or other sensors, and a computer system that includes data collected from such a portable device or such device (such as eye-tracking data, and / or other multimodal data such as facial expressions, verbal expressions, and / or physical movements).

[0005] For example, some of the systems described herein may be implemented with improved portable devices that enable improved objective measurements, and with added convenience for both the therapist and the patient, such as an infant or other young patient. In some embodiments, the system includes at least two separate portable computing devices, e.g., an operator-side portable device and at least one patient-side portable device integrated with an eye-tracking device (or eye-tracker device or eye-tracker). In the specific examples described below, these portable devices, both of which are wirelessly equipped to interact with a network-connected server platform to favorably collect session data in a comfortable and less intrusive manner for the patient, while also adding improved flexibility of control for the therapist. Optionally, session data collected via the patient-side portable device (e.g., using analysis of eye-tracking data generated in response to the display of a predetermined visual stimulus appropriate to the age) may be immediately analyzed for the purpose of outputting a results interface on the operator-side portable device that displays at least one metric based on objective factors. In some versions described herein, the system may be used to make comparative, objective assessments of developmental, cognitive, social, or mental abilities or impairments, including autism spectrum disorder (ASD).

[0006] Some examples described herein, but not limited to, include annotating visual stimuli (e.g., movies or videos) for moment-by-moment skill validity; customizing data collection playlists (e.g., according to target skill areas selected by the user); implementing annotated skill visualization and analysis sections in diagnostic / monitoring reports; customizing monitoring reports with target skill areas that can be automatically selected or selected by the user (when starting a session or viewing diagnostic results); providing users (e.g., therapists or clinicians, or caregivers of patients, such as parents) with interactive dashboards to explore visualizations of any skill areas and behaviors of patients and reference groups; and / or implementing specific skill area (and / or skill) monitoring for developmental assessment, including capturing multimodal data (e.g., audio / video of social interactions showing facial expressions, verbal expressions, and / or physical actions) during data collection sessions, where multimodal data may be used in conjunction with eye-tracking data for developmental assessment.

[0007] One aspect of the present disclosure employs a system using at least one portable computing device having eye-tracking capabilities. The system includes a portable eye-tracking console, which includes a display screen and an eye-tracking device, wherein the eye-tracking device is mounted adjacent to the display screen such that both the display screen and the eye-tracking device are directed toward the patient, and the eye-tracking device is configured to collect patient eye-tracking coordinate data while a predetermined sequence of stimulus video is presented on the display screen during a session; a portable computing device having a touchscreen display interface and being separated from the portable eye-tracking console and portable to various locations relative to the portable eye-tracking console; and a network-connected server, which includes a web portal that wirelessly receives session data from the portable eye-tracking console and exports evaluation results, including graphic correlations of numerical disability index scores correlated to a reference assessment scale. The network-connected server is configured to wirelessly connect to both the portable eye-tracker console and the portable computing device, and as a result, the portable computing device, which wirelessly communicates with the portable eye-tracker console via the network-connected server to control session activation, presents a predetermined sequence of stimulus video on the display screen of the portable eye-tracker console, after which the portable eye-tracker console wirelessly communicates session data to the network-connected server, which includes eye-tracking coordinate data that forms a timestamp relationship with the information of the predetermined sequence of stimulus video displayed by the portable eye-tracker console during the session.

[0008] In some embodiments, the system includes multiple portable eye-tracking consoles that simultaneously communicate wirelessly with a network-attached server.

[0009] In some embodiments, the eye-tracking device includes one or more gaze-tracking sensors mechanically assembled adjacent to the periphery of the display screen.

[0010] In some embodiments, each of one or more eye-tracking sensors includes an illumination source configured to emit detection light and a camera configured to capture eye movement data including at least one of the pupil or the corneal reflection or reflected image of the detection light from the illumination source. The eye-tracking sensor is configured to convert the eye movement data into a data stream containing at least one of the following information: pupil position, gaze vector per eye, or gaze point, and the patient's eye-tracking data includes the patient's corresponding data stream.

[0011] In some embodiments, the detection light includes infrared light, and the camera includes an infrared-sensitive camera. While a predetermined sequence of stimulus video is presented on a display screen directed at the patient during the session, the caregiver holding the patient wears glasses with a filter configured to filter out infrared light so that the camera captures only the patient's eye movement data.

[0012] In some embodiments, the eye-tracking device includes at least one image acquisition device configured to capture an image of at least one of the patient's eyes while a predetermined sequence of a stimulus video is presented on a display screen directed at the patient during a session, and the eye-tracking device is configured to generate corresponding gaze-tracking data of the patient based on the captured image of at least one of the patient's eyes.

[0013] In some embodiments, the system further includes at least one recording device assembled on a portable eye-tracker console and configured to collect at least one of patient-related image data, audio data, or video data while a predetermined sequence of stimulation video is presented on a display screen directed towards the patient during a session, the session data including at least one of image data, audio data, or video data.

[0014] In some embodiments, a portable eye-tracker console includes a housing configured to hold a display screen and an eye-tracker device, and a base connected to the housing through one or more joints. The base is rotatable around one or more joints to adjust the relative position or angle between the display screen and the patient during a session.

[0015] In some embodiments, the network-connected server provides a web portal accessible by a portable computing device, and the network-connected server is configured to output developmental analysis reports, including patient developmental analysis data, to the portable computing device via the user interface of the web portal.

[0016] In some embodiments, a network-connected server is configured to receive patient-related treatment data, wherein the treatment data includes at least one of the following: previous developmental analysis data of the patient, previous treatment plans for the patient, or reference treatment data of another patient; and to generate a prescriptive treatment plan for the patient based on the patient-related treatment data and the patient's developmental analysis data using artificial intelligence.

[0017] Another aspect of the present disclosure involves employing a computer implementation method for obtaining a patient's developmental disorder treatment plan on a network-connected server, wherein the treatment plan has individual time lengths for different treatment-specific skill domains over a certain time period, and the treatment plan has a specific treatment plan format, wherein the network-connected server is configured to process data related to a default treatment plan format; and parsing the treatment plan with the specific treatment plan format on the network-connected server in order to determine treatment data for the patient, wherein the treatment data is consistent with a default treatment plan format.

[0018] In some embodiments, the computer implementation method further includes receiving input to a network-connected server for selecting a treatment plan format from a plurality of treatment plan formats presented on a user interface, wherein the plurality of treatment plan formats are different from one another.

[0019] In some embodiments, multiple treatment plan formats differ from one another in at least one of the following: skill domain name, prompting method, treatment material or training material, reinforcement method, or data acquisition method.

[0020] In some embodiments, parsing a treatment plan with a specific treatment plan format includes parsing a treatment plan with a specific treatment plan format based on a selected treatment plan format and a default treatment plan format.

[0021] In some embodiments, multiple treatment plan formats include two or more of the following: EarliPoint, Early Start Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Joint Attention Symbolic Play Engagement Regulation (JASPER), and Project of Improving Parents As Communication Teachers (Project ImPACT).

[0022] In some embodiments, obtaining a treatment plan for a patient with a developmental disorder includes uploading a treatment plan with a specific treatment plan format from a repository on a network-attached server or storage medium.

[0023] In some embodiments, the treatment data includes at least one of the following: the duration of each different treatment-specific skill area during a time period; the percentage of the duration of each different treatment-specific skill area during a time period; the attention percentage of each different treatment-specific skill area across a series of sessions; the change in the attention percentage of each different treatment-specific skill area between at least two most recent sessions; or the relationship between the percentage of duration and the change in the attention percentage of each different treatment-specific skill area.

[0024] In some embodiments, the computer-implemented method comprises receiving, at a network-connected server, an input for selecting a treatment plan format from a plurality of treatment plan formats presented on a user interface, wherein the plurality of treatment plan formats are different from each other, and further comprising generating, at the network-connected server, a new treatment plan based on treatment data for a patient and the selected treatment plan format.

[0025] In some embodiments, the computer-implemented method further comprises transmitting, by the network-connected server, the new treatment plan with the selected treatment plan format to a computing device or an external server.

[0026] In some embodiments, the computer-implemented method further comprises generating, at the network-connected server, evaluation data for a developmental disorder of a patient based on the patient's eye-tracking session data. Generating the new treatment plan is based on the evaluation data for the developmental disorder of the patient.

[0027] In some embodiments, the computer-implemented method further comprises determining, at the network-connected server, a specific treatment plan format for a new treatment plan for a patient among the plurality of treatment plan formats, and presenting a visual representation on the specific treatment plan format among the plurality of treatment plan formats in the user interface, the visual representation indicating a recommendation of the specific treatment plan format for the new treatment plan for the patient.

[0028] In some embodiments, the computer-implemented method includes receiving, at a network-connected server, a selection of a target session from a list of sessions on a user interface of a web portal on the network-connected server; popping up a window for selecting a target skill area from a plurality of skill areas listed in the window in response to receiving the selection of the target session; automatically selecting one or more target skill areas from the plurality of skill areas based on treatment data, where different treatment-specific skill areas include the one or more target skill areas; and further including executing the target session based on the selected one or more target skill areas.

[0029] In some embodiments, the computer-implemented method further includes presenting, at a network-connected server, an input field for a treatment plan on a user interface of a web portal on the network-connected server; receiving, at the network-connected server, an input for one of the input fields for the treatment plan on the user interface; and updating, at the network-connected server, the treatment plan based on the input for one of the input fields.

[0030] In some embodiments, different treatment-specific skill areas include one or more of manding, listener responding, turn-taking, joint attention, tact, and play.

[0031] In some embodiments, the computer implementation method further includes receiving input in a network-connected server for selecting a third-party system from a plurality of third-party systems presented on a user interface, establishing a connection between the network-connected server and the selected third-party system, and then having the network-connected server retrieve patient-related data from the selected third-party system, wherein the patient-related data includes at least one of the patient's previous clinical data, the patient's previous treatment data, or reference data of other patients.

[0032] In some embodiments, the computer implementation method further includes generating a new treatment plan for the patient on a network-connected server based on treatment data and patient-related data.

[0033] Another aspect of this disclosure involves a computer-aided implementation method in which a network-connected server receives a request for patient assessment results based on patient session data, the session data being collected during the presentation of a visual stimulus data collection playlist to the patient in a session for assessing the patient's developmental disorder, and the network-connected server outputs the patient assessment results. The assessment results include the scores of each developmental disorder index related to the patient's developmental disorder, and for each developmental disorder index, the results of the correlation between each score of the developmental disorder index and the corresponding reference assessment scale.

[0034] In some embodiments, the correlation results include at least one of a summary describing the correlation or a graphical presentation of the correlation.

[0035] In some embodiments, the evaluation results further include at least one of the following: an assessment result indicating whether the patient has a developmental disorder, or information representing each of the scores on the developmental disorder index.

[0036] In some embodiments, the developmental disability index includes at least one of the following: a social disability index, a verbal ability index, or a nonverbal learning index.

[0037] In some embodiments, the corresponding reference assessment scale for each score of the social impairment index includes the ADOS-2 scale, the corresponding reference assessment scale for each score of the verbal ability index includes the Mullen verbal age equivalent, and the corresponding reference assessment scale for each score of the nonverbal learning index includes the Mullen nonverbal age equivalent.

[0038] In some embodiments, at least one visual scene in the data collection playlist is annotated with at least one of a plurality of skill domains associated with the visual scene in the data collection playlist. The evaluation results include, for each of one or more specific skill domains among the plurality of skill domains, patient behavioral data relating to moments in the session associated with a particular skill domain, each of which corresponds to a respective visual scene in the data collection playlist.

[0039] In some embodiments, the behavioral data includes attention percentage, defined as the ratio between the number of moments in which the patient focuses on relevant scene content in a visual stimulus and the total number of moments in which the patient is looking at the visual stimulus.

[0040] In some embodiments, the evaluation results include an outline of a distribution map of behavioral data for a reference group, where the behavioral data is based on reference session data collected during the presentation of a data collection playlist of visual stimuli to each individual in the reference group. The evaluation results may also include, for each of one or more specific skill domains, a representative visual scene, a representative visual scene highlighting one or more areas of attention within a given domain for the reference group, or a representative visual scene highlighting the patient's areas of attention during the session.

[0041] In some embodiments, the evaluation results include at least one of a first graphical presentation of moment-by-moment measurements of the patient's looking behavior during the session, or a second graphical presentation of a reference group attention funnel and the patient's attention during the session.

[0042] Another aspect of the present disclosure involves employing a computer implementation method that initiates a session for a patient by establishing communication between an operator-side computing device and a patient-side portable tablet computing device, wherein the patient-side portable tablet computing device is integrated with an eye-tracking device; sequentially presents the patient with visual scenes from a visual stimulus data collection playlist on the screen of the patient-side portable tablet computing device while collecting patient eye-tracking data using the eye-tracking device; and transmits session data for the session to a network-attached server, wherein the session data includes patient eye-tracking data collected in the session. Collecting patient eye-tracking data using the eye-tracking device includes capturing at least one of the patient's eye image or eye position, wherein the eye-tracking data is determined based on the captured at least one of the patient's eye image or eye position.

[0043] In some embodiments, the eye tracker device is configured to determine eye movement data based on at least one of a captured image of the patient's eyes or eye position, and to convert the patient's eye movement data into gaze tracking data that includes information related to at least one of the pupil position, gaze vector of each eye, or gaze point.

[0044] In some embodiments, collecting patient eye-tracking data using an eye-tracker device further includes capturing first eye movement data of the patient's eyes by measuring reflected light from the patient's eyes.

[0045] In some embodiments, the eye-tracking device includes at least one gaze-tracking unit configured to capture first eye-movement data of the patient's eye, and at least one image acquisition unit configured to capture at least one of the patient's eye image or the patient's eye position.

[0046] In some embodiments, the eye tracker device is configured to determine second eye movement data based on at least one of a captured image of the patient's eyes or eye position, and to determine gaze tracking data based on the first eye movement data and the second eye movement data.

[0047] In some embodiments, the eye tracker device is configured to convert first eye movement data into first eye-tracking data, determine second eye movement data based on at least one of a captured image of the patient's eyes or eye position, convert the second eye movement data into second eye-tracking data, and determine eye-tracking data based on the first and second eye-tracking data.

[0048] In some embodiments, the computer implementation method further includes collecting at least one of image data, audio data, or video data collected by one or more recording devices while visual scenes from a visual stimulus data collection playlist are presented sequentially, wherein one or more recording devices are assembled in or outside of a patient-side computing device. The session data includes at least one of image data, audio data, or video data.

[0049] Another aspect of the present disclosure involves employing an apparatus that includes at least one processor and one or more memories that store instructions, when executed by the at least one processor, causing the at least one processor to execute any one of the computer implementations disclosed herein.

[0050] Another aspect of this disclosure involves employing one or more non-temporary computer-readable media that, when executed by at least one processor, store instructions causing at least one processor to execute any one of the computer implementations disclosed herein.

[0051] Another aspect of the present disclosure involves a computer implementation method comprising receiving a request on a network-connected server for a patient assessment result based on the patient's session data, wherein the session data is collected during the presentation of a data collection playlist of visual stimuli to the patient during the session, wherein at least one visual scene of the data collection playlist is annotated using at least one of a plurality of skill domains associated with the visual scene of the data collection playlist, and outputting the patient assessment result on the network-connected server, wherein the assessment result includes, for each of one or more specific skill domains among the plurality of skill domains, patient behavior data relating to a moment in the session associated with a specific skill domain, with each of those moments corresponding to a respective visual scene of the data collection playlist.

[0052] In some embodiments, the behavioral data includes attention percentage, defined as the ratio between the number of moments in which the patient focuses on relevant scene content in a visual stimulus and the total number of moments in which the patient is looking at the visual stimulus.

[0053] In some embodiments, session data includes patient eye-tracking data. The computer implementation further includes determining the total number of moments in which the patient is looking at a visual stimulus, based on the patient's eye-tracking data, and determining the number of moments in which the patient is focusing on relevant scene content, based on the patient's eye-tracking data.

[0054] In some embodiments, the computer implementation method further includes determining that at a given moment in a session, the patient's area of ​​attention is within a predetermined region, and determining that the moment is one of a certain number of moments in which the patient focuses on relevant scene content.

[0055] In some embodiments, a predetermined region corresponds to the contour of a distribution map of behavioral data of a reference group, and the behavioral data of the reference group is based on reference session data collected during the presentation of a data collection playlist of visual stimuli to each person in the reference group.

[0056] In some embodiments, the contour values ​​of the distribution map correspond to the cutoff threshold.

[0057] In some embodiments, the assessment results further include a distribution map of behavioral data for a reference group.

[0058] In some embodiments, the assessment results further include, for each of one or more specific skill domains, at least one of the following: a representative visual scene, a representative visual scene highlighting one or more areas of attention within a given domain for a reference group, or a representative visual scene highlighting the patient's areas of attention during the session.

[0059] In some embodiments, the assessment results further include, for each of one or more specific skill areas, at least one of the following: behavioral data from one or more previous sessions of the patient, or a comparison between the session's behavioral data and the patient's behavioral data from one or more previous sessions.

[0060] In some embodiments, the assessment results include graphs showing session behavioral data and patient behavioral data from one or more previous sessions for each of one or more specific skill areas.

[0061] In some embodiments, the computer implementation method further includes selecting one or more specific skill areas from a plurality of skill areas based on the patient's assessment results.

[0062] In some embodiments, selecting one or more specific skill areas from a plurality of skill areas includes at least one of the following: selecting a specific skill area from the plurality of skill areas for which reliable data exists; selecting a generally requested skill area from the plurality of skill areas; selecting a skill area from the plurality of skill areas for which a particularly high, low, or representative score exists, wherein the score represents a patient's attention percentage; selecting a skill area previously selected as a target skill area in a session; selecting a skill area selected to customize the assessment results; or selecting a skill area previously selected in a previous session of the patient or in a previous assessment result of the patient.

[0063] In some embodiments, a computer implementation method includes receiving a session request for initiating a session through a web portal on a network-attached server, presenting a list of sessions on the user interface of the web portal, and receiving a selection of a session from the list of sessions on the user interface.

[0064] In some embodiments, the computer implementation method further includes, in response to receiving a session selection, popping up a window for selecting a target skill area from a plurality of skill areas enumerated in the window; receiving user input for selecting one or more target skill areas in the window; and executing a session based on the selected one or more target skill areas, wherein the selected one or more target skill areas include one or more specific skill areas.

[0065] In some embodiments, the computer implementation method includes adjusting a data collection playlist of visual stimuli based on one or more selected target skill areas.

[0066] In some embodiments, adjusting the visual stimulus data collection playlist includes at least one of the following: prioritizing visual scenes related to one or more selected target skill areas within the data collection playlist; increasing the value of additional visual scenes related to one or more selected target skill areas within the data collection playlist; or reducing or removing visual scenes not related to the selected target skill areas within the data collection playlist.

[0067] In some embodiments, prioritizing visual scenes related to one or more selected target skill areas includes at least one of the following: placing visual scenes related to one or more selected target skill areas at the beginning of the data collection playlist; arranging visual scenes related to one or more selected target skill areas in order of their weighted correlation values ​​to one or more selected target skill areas; or selecting only visual scenes related to one or more selected target skill areas within the data collection playlist.

[0068] In some embodiments, receiving user input includes receiving user input from an operator-side computing device communicating with a network-connected server via a web portal. In some embodiments, the computer implementation further includes establishing communication between the operator-side computing device and the patient-side computing device via a network-connected server, and transmitting information of a visually stimulated data collection playlist to the patient-side computing device, so that the visually stimulated data collection playlist is presented to the patient on the display screen of the patient-side computing device during the session.

[0069] In some embodiments, the computer implementation method, upon completion of the session, includes receiving patient session data from a patient-side computing device for the patient, wherein the patient session data is collected by the patient-side computing device during the session, and generating patient behavioral data by processing the patient session data based on reference data of a reference group and one or more specific skill areas.

[0070] In some embodiments, the computer implementation method further includes loading reference data for a reference group, the reference data being based on the behavioral data of the reference group, the behavioral data of the reference group being based on reference session data collected during the presentation of a visual stimulus data collection playlist, and one or more specific skill areas.

[0071] In some embodiments, the reference data of a reference group includes, for each of one or more specific skill domains, at least one of the following: a specific visual scene associated with the specific skill domain, where each specific visual scene highlights one or more attention areas of the reference group; or a distribution map of the behavioral data of the reference group for each specific visual scene.

[0072] In some embodiments, the reference data includes contour lines in a distribution map that represent thresholds for determining whether a patient will focus on the relevant scene content of a particular visual scene, for each of one or more specific skill areas and for each of a particular visual scene. The computer implementation further includes at least one of the following: determining that the patient will focus on the relevant scene content of a particular visual scene if the patient's area of ​​focus lies within a predetermined area corresponding to the contour line; or determining that the patient will not focus on the relevant scene content of a particular visual scene if the patient's area of ​​focus lies outside the predetermined area.

[0073] In some embodiments, patient behavioral data includes an attention percentage, defined as the ratio between the number of moments in which the patient focuses on relevant scene content and the total number of moments in which the patient views the visual stimulus. The computer implementation method may further include determining that at a given moment in a session, the patient's attention area is within a predetermined region, and determining that the moment is one of the moments in which the patient focuses on relevant scene content.

[0074] In some embodiments, patient behavioral data includes the results of a comparison between the patient's attention percentage and the reference group's threshold attention percentage. The results of the comparison may include at least one of the following: a ratio between the patient's attention percentage and the reference group's threshold attention percentage, or a relationship between the patient's attention percentage and the reference group's threshold attention percentage.

[0075] In some embodiments, receiving a request involves receiving user input from a computing device onto the user interface of a web portal of a network-attached server, where the user input indicates the request and the user interface is presented on the display screen of the computing device.

[0076] In some embodiments, the user interface includes at least one of the following: a first user interface element for viewing a default assessment report; a second user interface element for customizing the assessment report; or a third user interface element for launching an interactive dashboard using the assessment results.

[0077] In some embodiments, the computer implementation method further includes, in response to a selection of a second user interface element, popping up a window on the user interface for selecting target skill areas in an assessment report; receiving a second user input for selecting one or more target skill areas in the window; and generating an assessment report based on the selected one or more target skill areas, wherein the one or more target skill areas include one or more specific skill areas in the assessment result.

[0078] In some embodiments, the computer implementation further includes presenting an interactive dashboard within the user interface in response to a selection of a third user interface element, wherein the interactive dashboard includes a subwindow for selecting one of a list of skill areas for interaction.

[0079] In some embodiments, the computer implementation method, in response to receiving a selection of a specific target skill area from a list of skill areas, further includes presenting, over a series of consecutive sessions, a change in the patient's attention percentage for the specific target skill area, a change in the ratio between the patient's attention percentage and the reference group's threshold attention percentage, a change in the relationship between the patient's attention percentage and the reference group's threshold attention percentage, or, for each of a plurality of visual scenes associated with the specific target skill area, at least one of a first scene highlighting one or more attention areas of the reference group in the visual scene, and a second scene highlighting the patient's attention areas in the visual scene.

[0080] In some embodiments, multiple visual scenes are overlaid on each other within the user interface, and the interactive dashboard includes a sliding user interface element for selecting each of the multiple visual scenes.

[0081] In some embodiments, the computer implementation method further includes storing annotation data for visual scenes in a visual stimulus data collection playlist on a network-connected server, wherein the annotation data specifies each particular skill area associated with the visual scene; and storing reference data for reference groups on a network-connected server, wherein the reference data is based on behavioral data based on reference session data collected during the presentation of the visual stimulus data collection playlist.

[0082] In some embodiments, session data includes at least one of the following: eye-tracking data collected by an eye-tracking device assembled in a patient-side computing device communicating with a network-connected server; or image data, audio data, or video data collected by one or more recording devices, wherein one or more recording devices are assembled in or outside the patient-side computing device.

[0083] Another aspect of the present disclosure involves a computer implementation method comprising: receiving user input on the user interface of the web portal on a network-connected server by a computing device, the user input being for requesting a patient assessment result based on the patient's session data, the session data being collected during the presentation of a data collection playlist of visual stimuli to the patient during the session, wherein at least one visual scene of the data collection playlist is annotated using at least one of a plurality of skill domains related to the visual scene of the data collection playlist; and presenting the assessment result on the display screen of the computing device, wherein the assessment result includes, for each of one or more specific skill domains among the plurality of skill domains, patient behavioral data relating to a moment in the session related to a specific skill domain, with each of those moments corresponding to a respective visual scene of the data collection playlist.

[0084] In some embodiments, the computer implementation method further includes establishing a wireless connection with a patient-side computing device integrated with an eye-tracking device, and presenting a user interface for communicating with the patient-side computing device to acquire patient session data.

[0085] Another aspect of the present disclosure involves a computer implementation method that initiates a session for a patient by establishing communication between an operator-side computing device and a patient-side portable tablet computing device, wherein the patient-side portable tablet computing device is integrated with an eye-tracking device, and the patient is continuously presented with visual scenes from a data collection playlist of visual stimuli on the screen of the patient-side portable tablet computing device while the eye-tracking device is used to collect the patient's eye-tracking data, wherein at least one visual scene from the data collection playlist is annotated using at least one of a plurality of skill areas associated with the visual scenes in the data collection playlist, and the session data of the session is transmitted to a network-attached server, wherein the session data includes the patient's eye-tracking data collected in the session.

[0086] In some embodiments, the data collection playlist includes visual scenes related to one or more specific skill areas among a plurality of skill areas that are prioritized within the data collection playlist.

[0087] In some embodiments, the computer implementation method further includes collecting at least one of image data, audio data, or video data collected by one or more recording devices while visual scenes from a visual stimulus data collection playlist are presented sequentially, wherein one or more recording devices are assembled in or outside of a patient-side computing device, and the session data includes at least one of image data, audio data, or video data.

[0088] Another aspect of the present disclosure employs a system for developmental assessment via eye tracking, which includes a patient-side mobile computing device including a screen for presenting visual stimuli to a patient; an eye-tracking device mounted with the patient-side mobile computing device and oriented to a fixed position relative to the screen of the patient-side mobile computing device for collecting eye-tracking data of the patient while visual stimuli are presented to the patient on the screen of the patient-side mobile computing device; and an operator-side mobile computing device configured to present a user interface for controlling the activation of visual stimuli presented to the patient on the screen of the patient-side mobile computing device.

[0089] In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection. In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other wirelessly via a network-attached server. In some embodiments, the network-attached server includes a cloud computing system or cloud server implemented in a cloud environment.

[0090] In some embodiments, the patient's mobile computing device is configured to transmit data to a network-connected server, which includes patient eye-tracking data collected during a session while visual stimuli from a given list of visual stimuli are presented to the patient.

[0091] In some embodiments, the patient-side mobile computing device is configured to automatically transmit data in response to the completion of the presentation of all visual stimuli in a given list of visual stimuli during the session. In some embodiments, the patient-side mobile computing device is configured to transmit data in response to the receipt of a completion indicator from the operator-side mobile computing device or a network-connected server. In some embodiments, the operator-side mobile computing device or network-connected server is configured to generate a completion indicator in response to a decision to end the session or the receipt of input indicating the completion of the session.

[0092] In some embodiments, the data includes information about a given list of visual stimuli and patient eye-tracking data collected during the session. In some embodiments, the patient-side mobile computing device is configured to transmit data in two files, including a first file containing the patient's eye-tracking data and associated timestamp information, and a second file containing information about the list of visual stimuli. In some embodiments, the associated timestamp information for the eye-tracking data includes a timestamp when the eye-tracking data is generated or collected, and the information about the given list of visual stimuli includes a timestamp when each visual stimulus in the list is presented.

[0093] In some embodiments, the operator-side mobile computing device is configured to access a web portal in a network-attached server. In some embodiments, the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices through the web portal in the network-attached server. In some embodiments, the user interface of the operator-side mobile computing device includes at least one of the following: information about one or more patients associated with the operator, information about one or more patient-side computing devices associated with the operator, or information about the operator.

[0094] In some embodiments, the patient-side mobile computing device is configured to display connection information, including an access code, on the screen of the patient-side mobile computing device in response to a connection to a network-attached server. In some embodiments, the operator-side mobile computing device is configured to connect to the patient-side computing device by receiving input of connection information, including an access code, in its user interface. In some embodiments, the user interface of the operator-side mobile computing device presents a request for connection information in response to receiving a selection of a patient-side mobile computing device from one or more patient-side computing devices presented in the user interface.

[0095] In some embodiments, the operator-side mobile computing device is configured to present a user interface for an operator application running on either an operator-side computing device or a network-attached server. In some embodiments, the operator application is configured to present a user interface element for initiating desensitization within the user interface and, in response to the selection of the user interface element, to send a command to the patient-side computing device to play back visual desensitization information.

[0096] In some embodiments, the patient-side mobile computing device is configured to respond to the receipt of a command by playing visual desensitization information for the patient on the screen of the patient-side mobile computing device, and to control the eye-tracking device so as not to collect the patient's eye-tracking data while the visual desensitization information is displayed on the screen.

[0097] In some embodiments, the operator application is configured to present a user interface for the operator to set up a session for a patient by selecting a patient from a list of patients or creating a profile for a patient, while visual desensitization information is displayed on the screen of the patient's mobile computing device.

[0098] In some embodiments, the operator application is configured to display instructions within the user interface for adjusting the position of the eye-tracking device relative to the patient or the patient's position relative to the patient-side mobile computing device, based on the sensing position of the eye-tracking device relative to at least one of the patient's eyes. In some embodiments, the sensing position is determined based on image data of at least one of the patient's eyes, captured by using an image acquisition device included in or adjacent to the eye-tracking device. In some embodiments, the sensing position is determined by the patient-side mobile computing device or the operator application.

[0099] In some embodiments, the operator application is configured to send commands to the patient-side mobile computing device for calibration between the patient and the eye-tracking device in response to either the selection of a user interface element for calibration within the user interface, or the decision that a session for the patient has been set up.

[0100] In some embodiments, the patient-side mobile computing device is configured to, in response to receiving a command, continuously present one or more calibration targets at one or more predetermined locations on the patient-side mobile computing device's screen while capturing patient eye-tracking calibration data using an eye-tracking device.

[0101] In some embodiments, the patient-side mobile computing device is configured to process the patient's captured eye-tracking calibration data for each of one or more calibration targets in order to determine the position of the patient's corresponding visual gaze relative to the calibration target, compare the position of the patient's corresponding visual gaze with a corresponding predetermined location on the screen where the calibration target is presented, and, based on the result of the comparison, determine whether the calibration target is calibrated for the patient.

[0102] In some embodiments, the patient-side mobile computing device is configured to determine that calibration is complete in response to a determination that one or more calibration targets have been calibrated. In some embodiments, the patient-side mobile computing device is configured to recalibrate the calibration targets in response to a determination that the calibration targets have not been calibrated.

[0103] In some embodiments, the patient-side mobile computing device is configured to replay desensitization information between presenting two adjacent calibration targets. In some embodiments, the operator application is configured to initiate calibration verification in response to receiving an indication that calibration is complete. In some embodiments, the patient-side mobile computing device is configured to present at least one additional calibration target on the screen while capturing additional eye-tracking calibration data of the patient using an eye-tracking device, in response to receiving a request to verify calibration, and to process the captured additional eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual gaze to the at least one additional calibration target.

[0104] In some embodiments, the patient-side mobile computing device is configured to compare the position of the patient's corresponding visual gaze to at least one additional calibration target with a corresponding predetermined location on the screen where at least one additional calibration target is presented, and to determine, based on the result of the comparison, whether the calibration has been validated.

[0105] In some embodiments, the operator application is configured to simultaneously present in the user interface at least one additional calibration target at a corresponding predetermined location and at least one depiction of the patient's corresponding visual gaze at a determined position of the patient's corresponding visual gaze toward the at least one additional calibration target, and to present in the user interface a first user interface element for calibration verification and a second user interface element for recalibration.

[0106] In some embodiments, the operator application is configured to send a command to the patient-side computing device for data acquisition in response to either the selection of a user interface element to initiate data acquisition, or a decision that calibration is complete or verified.

[0107] In some embodiments, the patient-side mobile computing device is configured to respond to the receipt of a command by sequentially presenting the patient with a predetermined list of visual stimuli on the screen of the patient-side mobile computing device while capturing the patient's eye-tracking data using an eye-tracking device.

[0108] In some embodiments, the patient-side mobile computing device is configured to present the patient with a centering target on the device's screen before presenting each of a given list of visual stimuli. In some embodiments, the patient-side mobile computing device is configured to perform patient calibration for an eye-tracking device between presenting two adjacent visual stimuli from the given list of visual stimuli. In some embodiments, eye-tracking data collected during calibration is used to recalibrate the patient's eye-tracking data or to determine the accuracy of the calibration.

[0109] In some embodiments, the operator application is configured to display within the user interface at least one of the following: a progress indicator that is continuously updated while a given list of visual stimuli is presented; a user interface element for skipping visual stimuli from the given list of visual stimuli; and information about visual stimuli that have already been presented or are being presented, or information about visual stimuli that should be presented.

[0110] In some embodiments, a network-connected server is configured to provide a patient's diagnostic results based on the patient's eye-tracking data, and the diagnostic results include at least one index value associated with a developmental disorder. In some embodiments, an operator-side mobile computing device is configured to present the diagnostic results within a user interface.

[0111] In some embodiments, the visual stimuli are predetermined based at least on the patient's age or symptoms. In some embodiments, each visual stimulus includes at least one of the following: a static visual stimulus, a dynamic visual stimulus, a pre-recorded visual stimulus, a pre-recorded audiovisual stimulus, a live visual stimulus, a live audiovisual stimulus, a two-dimensional stimulus, or a three-dimensional stimulus. In some embodiments, each visual stimulus is normed to elicit a specific eye movement response with statistical confidence greater than 95%. In some embodiments, each visual stimulus is configured to elicit an eye movement response to a specific spatial-temporal location with statistical confidence greater than 95%.

[0112] In some embodiments, the eye-tracking device is connected to a patient-side mobile computing device via a wired connection. In some embodiments, the eye-tracking device and the screen are housed together in a holder. In some embodiments, the patient-side mobile computing device includes a screen holder that holds the screen and the eye-tracking device in a fixed position relative to the screen.

[0113] In some embodiments, the eye-tracking device includes one or more eye-tracking units positioned at one or more locations adjacent to the periphery of the screen. In some embodiments, at least one of the patient-side mobile computing device or the operator-side mobile computing device is a tablet computing device. In some embodiments, the operator-side computing device is configured to communicate with the patient-side mobile computing device via bidirectional communication.

[0114] Another aspect of the present disclosure involves employing an apparatus that includes a patient-side computing device, which includes a screen for presenting visual stimuli to a patient, and an eye-tracking device integrated with the patient-side computing device and configured to collect eye-tracking data of the patient while visual stimuli are presented to the patient on the screen of the patient-side computing device. The patient-side computing device includes the patient-side computing device described above.

[0115] Another aspect of this disclosure involves employing a device that includes an operator-side computing device as described above.

[0116] Another aspect of the present disclosure involves employing a device that includes at least one processor and at least one non-temporary memory coupled to the at least one processor and storing programming instructions for the at least one processor to execute in order to perform an operation, the operation of which includes establishing a wireless connection with a patient-side computing device integrated with an eye-tracking device and presenting a user interface for communicating with the patient-side computing device for acquiring patient eye-tracking data.

[0117] In some embodiments, the operation further includes accessing a web portal of a network-connected server, where a wireless connection is established through the web portal. In some embodiments, the operation further includes presenting diagnostic results in a user interface based on the patient's eye-tracking data.

[0118] Another aspect of the present disclosure employs a computerized method for conducting a developmental assessment via eye-tracking. The computerized method includes initiating a session for a patient by establishing communication between an operator-side computing device and a patient-side computing device, wherein the patient-side computing device is integrated with an eye-tracking device; continuously presenting the patient with a predetermined list of visual stimuli on the screen of the patient-side computing device while collecting the patient's eye-tracking data using the eye-tracking device; and transmitting session data of the session to a network-attached server, wherein the session data includes the patient's eye-tracking data collected during the session.

[0119] In some embodiments, at least one of the operator-side computing device or the patient-side computing device is a portable device. In some embodiments, establishing communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device.

[0120] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes the operator-side computing device accessing a web portal on a network-attached server and wirelessly connecting the operator-side computing device to the patient-side computing device in response to receiving a selection of the patient-side computing device within the web portal. In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes the patient-side computing device displaying connection information on the screen of the patient-side computing device and establishing a wireless connection between the operator-side computing device and the patient-side computing device in response to receiving input of connection information by the operator-side computing device.

[0121] In some embodiments, the computer implementation method further includes, after establishing communication, displaying visual desensitization information to the patient on the screen of the patient-side computing device. In some embodiments, the computer implementation method further includes controlling the eye-tracking device so as not to collect the patient's eye-tracking data while displaying the visual desensitization information.

[0122] In some embodiments, the computer-assisted method further includes the operator-side computing device accessing a web portal on a network-connected server to set up a session for the patient while displaying visual desensitization information. In some embodiments, setting up a session includes selecting a patient from a list of patients or creating a profile for the patient on the network-connected server. In some embodiments, the computer-assisted method further includes determining the relative position between the eye-tracking device and at least one eye of the patient, and displaying instructions on the user interface of the operator-side computing device for adjusting the position of the eye-tracking device or the patient.

[0123] In some embodiments, the computer implementation method further includes determining that the patient is aligned with the eye-tracking device in response to a determination that the relative location of at least one of the patient's eyes is at a predetermined location within the detection area of ​​the eye-tracking device.

[0124] In some embodiments, the computer implementation method further includes calibrating the patient for an eye-tracking device by displaying one or more calibration targets to the patient on the screen of the patient-side computing device.

[0125] In some embodiments, calibrating a patient for an eye-tracking device includes: sequentially presenting one or more calibration targets at corresponding predetermined locations on the screen of a patient-side computing device while capturing the patient's eye-tracking calibration data using the eye-tracking device; processing the patient's captured eye-tracking calibration data for each of the one or more calibration targets to determine the location of the patient's corresponding visual gaze toward the calibration target; comparing the location of the patient's corresponding visual gaze with the corresponding predetermined location where the calibration target is presented; and determining, based on the result of the comparison, whether the calibration target is calibrated for the eye-tracking device.

[0126] In some embodiments, calibrating a patient for an eye-tracking device further includes determining that a calibration target is calibrated and displaying the next calibration target in response to a determination that the deviation between the patient's corresponding visual gaze position and a corresponding predetermined location is below a predetermined threshold, or determining that a calibration target is not calibrated and redisplaying the calibration target for calibration in response to a determination that the deviation is greater than a predetermined threshold.

[0127] In some embodiments, the computer implementation method further includes calibrating the patient for an eye-tracking device and then verifying the calibration using one or more new calibration targets. In some embodiments, verifying the calibration includes sequentially presenting each of the one or more new calibration targets at a corresponding predetermined location on the screen of the patient-side computing device while capturing the patient's eye-tracking calibration data using the eye-tracking device, and processing the patient's captured eye-tracking calibration data to determine the position of the patient's corresponding visual gaze for each of the one or more new calibration targets.

[0128] In some embodiments, verifying calibration includes simultaneously presenting one or more new calibration targets at one or more corresponding predetermined locations and depictions of one or more corresponding visual gazes of the patient at one or more determined locations on the user interface of the operator-side computing device, and determining that the calibration has been verified in response to receiving an indication to validate the results of the calibration, or initiating recalibration of the patient for the eye-tracking device in response to receiving an indication to invalidate the results of the calibration.

[0129] In some embodiments, verifying calibration includes determining the number of new calibration targets that each pass the calibration for, based on the patient's corresponding visual gaze position and corresponding predetermined position; determining that the calibration is enabled if the number or associated percentage is greater than or equal to a predetermined threshold; or determining that the calibration is disabled and initiating recalibration of the patient for the eye-tracking device if the number or associated percentage is less than the predetermined threshold.

[0130] In some embodiments, sequentially presenting a predetermined list of visual stimuli to the patient on the screen of the patient-side computing device includes presenting a centering target to the patient on the screen of the patient-side computing device before presenting each of the predetermined list of visual stimuli.

[0131] In some embodiments, sequentially presenting a predetermined list of visual stimuli to the patient on the screen of a patient-side computing device includes performing patient calibration for an eye-tracking device between presenting two adjacent visual stimuli from the predetermined list, wherein eye-tracking data collected during the calibration is used for at least one of calibrating the patient's eye-tracking data or determining the accuracy of the calibration.

[0132] In some embodiments, the computer implementation method further includes presenting on the user interface of the operator-side computing device at least one of the following: a progress indicator that is continuously updated while a given list of visual stimuli is presented; information on visual stimuli that have already been presented or are being presented; information on visual stimuli to be presented; or a user interface element for skipping visual stimuli from the given list of visual stimuli.

[0133] In some embodiments, sending session data to a network-connected server includes the patient-side computing device automatically sending session data to the network-connected server in response to either a decision to complete the presentation of a predetermined list of visual stimuli on a screen, or the receipt of a session completion indication from the operator-side computing device.

[0134] In some embodiments, the session data includes information relating to a list of presented visual stimuli. In some embodiments, the information relating to the list of presented visual stimuli includes the name of the presented visual stimuli and the associated timestamp when the visual stimuli are presented.

[0135] In some embodiments, session data includes eye-tracking data and associated timestamps of when the eye-tracking data is generated or collected. In some embodiments, session data is stored in a first file that stores the patient's eye-tracking data and a second file that stores information relating to a given list of presented visual stimuli.

[0136] Another aspect of this disclosure involves employing a computer-assisted method for developmental assessment using eye-tracking data via a network-connected server. The computer-assisted method includes receiving session data for multiple sessions, wherein each session's session data includes eye-tracking data of the corresponding patient in the session; processing the session data for multiple sessions in parallel to generate processed session data for the multiple sessions; and for each session of the multiple sessions, analyzing the processed session data of the session based on corresponding reference data to generate assessment results for the corresponding patient in the session.

[0137] In some embodiments, the computer implementation further includes loading corresponding reference data for multiple sessions in parallel with processing session data for multiple sessions.

[0138] In some embodiments, the network-attached server includes multiple processing cores. Processing session data for multiple sessions in parallel includes using a first set of processing cores to process session data for multiple sessions in parallel, and using a second set of different processing cores to load corresponding reference data for multiple sessions, wherein the number of first processing cores is greater than the number of second processing cores.

[0139] In some embodiments, analyzing processed session data for multiple sessions based on loaded corresponding reference data for multiple sessions involves using multiple processing cores, including a first set of multiple processing cores and a second set of multiple processing cores.

[0140] In some embodiments, analyzing processed session data from multiple sessions based on loaded corresponding reference data for multiple sessions includes at least one of the following: comparing the processed session data of a session with the corresponding reference data; using the corresponding reference data to infer assessment results for the corresponding patient from the processed session data; or using at least one of a statistical model or an artificial intelligence (AI) model. In some embodiments, the corresponding reference data includes historical eye-tracking data or results for patients of substantially the same age or condition as the corresponding patient.

[0141] In some embodiments, the computer implementation method further includes generating assessment results based on previous session data of the corresponding patient. In some embodiments, the computer implementation method includes, for each session of a plurality of sessions, allocating a separate container for the session, processing the session data of the session within each container, and analyzing the processed session data of the session based on corresponding model data to generate assessment results for the corresponding patient in the session.

[0142] In some embodiments, while eye-tracking data is collected during a session, the eye-tracking data is associated with a predetermined list of visual stimuli presented to the patient, where session data includes information related to the predetermined list of visual stimuli in the session.

[0143] In some embodiments, the computer implementation method further includes linking session eye-tracking data to a predetermined list of visual stimuli in the session. In some embodiments, linking session eye-tracking data to a predetermined list of visual stimuli in the session includes breaking down the eye-tracking data into multiple parts within each container based on information related to the predetermined list of visual stimuli, with each part of the eye-tracking data being associated with one of the respective predetermined visual stimuli or corresponding calibrations.

[0144] In some embodiments, processing session data for a session includes processing portions of eye-tracking data associated with each given visual stimulus, based on information about each given visual stimulus.

[0145] In some embodiments, the computer implementation method further includes recalibrating, within each container, a portion of the eye-tracking data associated with each given visual stimulus based on at least one portion of the eye-tracking data associated with the corresponding calibration.

[0146] In some embodiments, the computer implementation further includes determining the calibration accuracy using, within each container, at least one portion of eye-tracking data associated with the corresponding calibration, and a plurality of predetermined locations where a plurality of calibration targets are presented in the corresponding calibration.

[0147] In some embodiments, receiving session data for multiple sessions includes receiving session data for multiple sessions from multiple computing devices associated with the corresponding entities via a web portal. In some embodiments, the computer implementation further includes, in response to receiving session data for a session, adding a file pointer to the session data for the session to a processing queue to be processed. In some embodiments, the computer implementation further includes storing the session data for the session in a database using the file pointer to the session, and retrieving the session data for the session from the database using the file pointer to the session.

[0148] In some embodiments, the computer implementation further includes storing session data from one or more computing devices associated with each entity in its respective repository. In some embodiments, each repository for an entity includes at least one of the following: information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted in the entity, information about one or more patients associated with the entity, or historical information for the respective repository.

[0149] In some embodiments, each repository is contained within a NoSQL database. In some embodiments, each repository is isolated from one or more other repositories and inaccessible to one or more other entities.

[0150] In some embodiments, the computer implementation further includes dynamically adjusting the resources of the network-attached server based on the number of computing devices accessing the network-attached server.

[0151] In some embodiments, the computer implementation further includes replicating data from a first data center to a second data center and automatically diverting traffic to the second data center in response to a determination that the first data center is inaccessible. In some embodiments, each of the first and second data centers includes at least one of the following: a web portal accessible to an operator-side computing device, an operator application, or an application layer for data processing and data analysis.

[0152] In some embodiments, the computer implementation further includes storing the same data in multiple data centers, where the data includes application data for entities and information related to eye-tracking data.

[0153] In some embodiments, the computer implementation method further includes associating the generated assessment results with the corresponding patient in the session and generating an assessment report for the corresponding patient.

[0154] In some embodiments, the computer implementation method further includes outputting assessment results or assessment reports to be presented in the user interface of an operator-side computing device. In some embodiments, the assessment report includes at least one of the following: information about the corresponding patient, information about the entity performing the session for the corresponding patient, information about the accuracy in the session, information about session data collection, or assessment results for the corresponding patient.

[0155] In some embodiments, the assessment results indicate the likelihood that the corresponding patient has a developmental, cognitive, social, or mental impairment or ability. In some embodiments, the assessment results indicate the likelihood that the corresponding patient has or does not have autism spectrum disorder (ASD). In some embodiments, the assessment results include scores for one or more of the following: social impairment index, verbal ability index, nonverbal ability, social adaptability index, and social communication index.

[0156] In some embodiments, the assessment results include at least one of the following: a visualization of eye-tracking data overlaid on a corresponding visual stimulus still image from a socially relevant moment; an animation of the visualization of eye-tracking data overlaid on a corresponding visual stimulus still image from a socially relevant moment; a visualization of aggregated reference data from multiple reference patients aligned with a corresponding patient in one or more patient attributes; or an annotation describing at least one of visual stimulus content or an eye-gaze pattern.

[0157] In some embodiments, the corresponding patient has an age range of 5 months to 7 years, including ages in the range of 5 months to 43 months or 48 months, ages in the range of 16 months to 30 months, ages in the range of 18 months to 36 months, ages in the range of 16 months to 48 months, or ages in the range of 16 months to 7 years.

[0158] Another aspect of the present disclosure involves employing a system that includes at least one processor and one or more memories that store instructions, when executed by the at least one processor, causing the at least one processor to perform a computer implementation as described herein.

[0159] Another aspect of the present disclosure involves employing one or more non-temporary computer-readable media that, when executed by at least one processor, store instructions causing at least one processor to perform a computer implementation as described herein.

[0160] One or more of the embodiments described herein can achieve several technical effects and advantages. In the first example, some embodiments can provide a convenient, miniaturized, and effective computing system for advantageously collecting eye-tracking data and then communicating such data for analysis and diagnostic results. The computing system may include at least two separate portable computing devices, e.g., an operator-side portable device and at least one patient-side portable device integrated with an eye-tracking device. These portable devices can be equipped in various ways (various peripherals or devices, various user interfaces, etc.) and can be wirelessly connected to each other or to a network-connected server platform (providing communication between the operator-side portable device and the patient-side portable device) without using physical connections. For example, the operator may use the operator-side portable device to calibrate the eye-tracking device with the patient and to control the playback of predetermined visual stimuli on the patient-side portable device to acquire the patient's eye-tracking data using the eye-tracking device. Once the session is complete, the patient-side portable device can transmit session data, e.g., detected eye-tracking data and information on the visual stimuli played, to a cloud server for data storage, data processing, and data analysis. Cloud servers can be remotely connected to these portable devices, for example, via a network.

[0161] In another example, the technology implemented herein can provide a much more detailed, interactive reporting output that allows users to delve into behaviors and metrics for specific scenes or groups of scenes related to developmentally important skills. Annotations made by specialist clinicians in light of the behaviors of reference groups enable precise identification of specific skill domains / skills for the diagnosis and / or treatment of a patient, effective tailoring of data collection playlists for the patient in selected skill domains / skills, monitoring of patient improvement or treatment effectiveness in selected skill domains / skills, and / or automated, accurate, consistent, rapid, effortless, and / or cost-effective assessment of developmental disorders for the patient. The technology allows operators / users to manage and / or explore session outcomes at multiple customizable levels with detail. Skill-specific behavioral visualizations and metrics may be configured to give users objective quantification of how well the patient generalizes target skills outside of the treatment context, and to inform which aspects of treatment are aligned with patient progress. Users (for example, treatment providers, clinicians, or patient caregivers) can see whether the patient has made any improvements in one or more target skill areas, whether a treatment for the patient is working or effective, and / or whether a new or modified treatment could be used to replace the current treatment.

[0162] The techniques described herein may be used, for example, to enable earlier identification and assessment of developmental, cognitive, social, verbal, or nonverbal abilities, or mental abilities or the risk of impairment, in patients by measuring visual attention to social information in the environment compared to a normative age-specific benchmark. Patients may have ages ranging from 5 months to 7 years, for example, from 16 months to 7 years, from 12 months to 48 months, from 16 months to 30 months, or from 18 months to 36 months.

[0163] In some embodiments, changes in a patient's visual fixation over time in relation to certain dynamic stimuli provide markers of a patient's possible developmental, cognitive, social, or mental capacity or impairment (such as ASD). Visual fixation is a type of eye movement used to stabilize visual information on the retina and generally occurs simultaneously when a person looks at or "stares" at a point or area on a display plane. In some embodiments, a patient's visual fixation is identified, monitored, and tracked over time through repeated eye-tracking sessions and / or through comparison with model data based on a large number of patients of similar age and / or background. Data relating to visual fixation are then compared to a relative norm to determine a possible increased risk of such a condition in the patient. Changes in visual fixation (specifically, a decrease or increase in visual fixation to the eyes, body, or other areas of interest of a person or object displayed in a visual stimulus), compared to similar visual fixation data of typically developing patients or the patient's own previous visual fixation data, result in an indication of developmental, cognitive, or mental impairment. This technology can be applied to quantitatively measure and monitor the overall symptoms of each ability or impairment, and in some cases, can provide patients, families, and service providers with more accurate, relevant, and normative information. In additional embodiments, this technology may be used to predict outcomes (and thus normative abilities) in individuals with autism, while also providing similar diagnostic and normative scales for overall developmental, cognitive, social, or mental abilities or impairments.

[0164] As detailed below, the techniques described herein for detecting developmental disorders, cognitive disorders, social disorders, or mental disorders may be applicable to the detection of symptoms including, but are not limited to, expressive and receptive language development delays, nonverbal development delays, intellectual disabilities, intellectual disabilities of known or unknown genetic origin, traumatic brain injury, disorders of infancy not otherwise specified (DOI-NOS), social communication disorders, and autism spectrum disorder (ASD), as well as symptoms such as attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), concussions, sports injuries, and dementia.

[0165] It is understood that the methods provided herein may include any combination of the embodiments described herein. That is, the methods provided herein are not limited to any combination of the embodiments described herein, but also include any combination of the embodiments provided.

[0166] Details of one or more embodiments of the present disclosure are described in the accompanying drawings and the following description. Other embodiments and advantages of the present disclosure will become apparent from the description and drawings and from the claims. [Brief explanation of the drawing]

[0167] [Figure 1A] This is a block diagram of an exemplary environment for assessing developmental disorders according to one or more embodiments of the present disclosure. [Figure 1B] This figure shows an example of an exemplary visual stimulus presented on the screen of a patient device according to one or more embodiments of the present disclosure. [Figure 1C] This figure shows an example of a patient device according to one or more embodiments of the present disclosure. [Figure 1D] This figure shows an example of an exemplary user interface presented on an operator device according to one or more embodiments of the present disclosure. [Figure 2A] This is a block diagram of an exemplary system for assessing developmental disorders via eye tracking, according to one or more embodiments of the present disclosure. [Figure 2B] This figure shows an example of managing session data in the system shown in Figure 2A according to one or more embodiments of the present disclosure. [Figure 2C] This figure shows an example of managing multiple session data in parallel within the system shown in Figure 2A, according to one or more embodiments of the present disclosure. [Figure 2D] Figure 2A shows an exemplary database that stores various types of documents as application data in the system according to one or more embodiments of the present disclosure. [Figure 2E] This figure shows an example of a multi-tenant architecture in the system shown in Figure 2A, according to one or more embodiments of the present disclosure. [Figure 2F] This figure shows an example of data backup for the system shown in Figure 2A, according to one or more embodiments of the present disclosure. [Figure 2G] This figure shows an example of data backup for the system shown in Figure 2A, according to one or more embodiments of the present disclosure. [Figure 3] This is a flowchart illustrating an exemplary process for acquiring session data according to one or more embodiments of the present disclosure. [Figure 4A] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4B]This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4C] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4D] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4E] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4F] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4G] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4H] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4I] This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 4J]This figure shows a set of exemplary user interfaces presented on an operator device (Figure a) and a participant device (Figure b) during session data acquisition, according to one or more embodiments of the present disclosure. [Figure 5] This figure shows exemplary session data, including (a) movie playlist data and (b) eye-tracking sensor data, according to one or more embodiments of the present disclosure. [Figure 6] This is a flowchart illustrating an exemplary process for managing session data according to one or more embodiments of the present disclosure. [Figure 7A] This is a flowchart of another exemplary process for managing session data according to one or more embodiments of the present disclosure. [Figure 7B] This is a flowchart of another exemplary process for managing session data according to one or more embodiments of the present disclosure. [Figure 8A] This figure shows an exemplary results interface for displaying at least one index value based on eye-tracking data, according to one or more embodiments of the present disclosure. [Figure 8B] This figure shows another exemplary results interface displaying behavior-based scales of eye-tracking data for developmental assessment in instances of nonverbal communication and gestures (A) and joint attention and mutual gaze (B) in one or more embodiments of the present disclosure. [Figure 8C] Figure 8C shows another exemplary results interface displaying behavior-based scales of developmental assessments based on eye-tracking data in instances of facial emotion (C) and pointing and social monitoring (D) in one or more embodiments of the present disclosure. [Figure 9] This is a flowchart illustrating an exemplary process for acquiring session data according to one or more embodiments of the present disclosure. [Figure 10]This is a flowchart illustrating an exemplary process for managing session data according to one or more embodiments of the present disclosure. [Figure 11] This figure shows an example of a comparison between annotated video scenes, information on typical looking behavior groups, and information on patient looking behaviors for different specific skill areas, according to one or more embodiments of the present disclosure. [Figure 12A] This figure shows an example of an exemplary user interface presented on an operator device for session initiation according to one or more embodiments of the present disclosure. [Figure 12B] This figure shows an example of an exemplary window presented on an operator device for selecting a target skill area for a target monitoring session, according to one or more embodiments of the present disclosure. [Figure 13A] This figure shows an example of an exemplary user interface for reviewing session information about a user device, according to one or more embodiments of the present disclosure. [Figure 13B-1] This figure shows an exemplary portion of an evaluation report illustrating a comparison between annotated video scenes, information on typical viewing behavior groups, and information on patient viewing behaviors for different specific skill areas, according to one or more embodiments of the present disclosure. [Figure 13B-2] This figure shows another exemplary portion of an evaluation report illustrating the monitoring of information about treatment-specific skills and characterized skills according to one or more embodiments of the present disclosure. [Figure 13C] This figure shows an example of an exemplary window presented on a user device for selecting a target skill area for which a custom report should be generated, according to one or more embodiments of the present disclosure. [Figure 13D] This figure shows an exemplary interactive results dashboard presented on a user device according to one or more embodiments of the present disclosure. [Figure 14]This is a flowchart illustrating an exemplary process for managing specific skills for developmental disability assessment, according to one or more embodiments of the present disclosure. [Figure 15A] This figure shows an example of an exemplary user interface presented on a computing device when a data aggregator application is running on a cloud server, according to one or more embodiments of the present disclosure. [Figure 15B] This figure shows an example of an exemplary user interface presented on a computing device when a cloud server runs a data aggregator application for aggregating data from external tools, according to one or more embodiments of the present disclosure. [Figure 15C] This figure shows an example of an exemplary user interface presented on a computing device when a cloud server runs a data aggregator application for an operator to manually input patient information, according to one or more embodiments of the present disclosure. [Figure 15D] This figure shows an example of an exemplary user interface presented on a computing device for session initiation according to one or more embodiments of the present disclosure. [Figure 15E] This is a breakdown graph illustrating exemplary treatment-specific skill area approaches in patient treatment planning according to one or more embodiments of the present disclosure. [Figure 15F] A graph showing a patient's attention to feature skill-related scenes over a session during a time period, according to one or more embodiments of the present disclosure. [Figure 15G] This is a graph illustrating the relationship between approaches to different skill areas and their impact, as described in one or more embodiments of the present disclosure. [Figure 15H] This figure shows an example of an exemplary user interface presented on a computing device when a cloud server outputs a treatment plan, according to one or more embodiments of the present disclosure. [Figure 16A]This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 16B] This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 16C] This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 16D] This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 16E] This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 16F] This figure shows an exemplary results interface for an exemplary evaluation report of an evaluation system according to one or more embodiments of the present disclosure. [Figure 17A] This is a flowchart illustrating an exemplary process for managing a treatment plan for developmental disorder assessment, according to one or more embodiments of the present disclosure. [Figure 17B] This is a flowchart illustrating an exemplary process for managing evaluation reports according to one or more embodiments of the present disclosure. [Figure 18] This figure shows an architecture for a cloud computing system according to one or more embodiments of the present disclosure. [Figure 19] This figure shows an architecture for a computing device according to one or more embodiments of the present disclosure. [Modes for carrying out the invention]

[0168] Similar reference numbers and designations in various drawings refer to the same elements. It should also be understood that the various exemplary embodiments shown in the drawings are illustrative depictions only and are not necessarily drawn to a specific scale.

[0169] This disclosure describes portable devices having user detection equipment such as eye-tracking devices or other sensors, and computer systems and methods that include data (such as eye-tracking data and / or other multimodal data such as facial expressions, verbal expressions, and / or bodily movements) collected from such portable devices or such devices. In some examples, the systems, devices, methods, and techniques described herein may use eye-tracking data and / or multimodal data (such as facial expressions, verbal expressions, and / or bodily movements) generated in response to the display of a particular given visual stimulus (e.g., one or more movies or videos) to provide patients with an improved objective assessment of their developmental, cognitive, social, or mental abilities or impairments, including autism spectrum disorder (ASD). The techniques may provide detailed interactive reporting outputs that enable users to delve into behaviors and metrics for specific scenes or groups of scenes relating to a particular skill domain (and / or skill), such as a developmentally important skill domain / skill (or therapy-specific skill). In this disclosure, the term “skill domain” refers to a group of skills relating to one another. The term “skill domain” may be used interchangeably with the terms “developmental concept” or “skill category.” Exemplary skill domains may include manding, listener response, turn-taking, joint attention, tact, and / or play. Skill domains may be associated with developmental assessment and / or treatment. For illustrative purposes, treatment-specific skill domains (or skills) are used as examples of specific skill domains (or skills).

[0170] In some embodiments, for example, as illustrated with further detail in Figure 11, a given visual stimulus can be pre-annotated moment by moment for skill validity by associating specific skill domains and / or skills with scenes of visual stimuli related to those skill domains and / or skills. These skill domains or skills may be therapeutic targets that are important to, for example, Board Certified Behavior Analysts (BCBA). Exemplary specific skill domains may include manding, listener response, turn-taking, joint attention, tact, and / or play. Annotations may be made by one or more specialist clinicians observing the scenes of visual stimuli and, optionally, the behavior (e.g., viewing behavior, facial expressions, verbal expressions, and / or physical actions) of a reference group (e.g., typical children of similar age) when viewing the same visual stimuli. Scene annotations may be for any developmental skill domain (or concept), therapeutic prompts / scales, severity indicators, or any other skills present in or related to the scene content. Visualizations of behavior in exemplary scenes can be considered representative of a skill domain or skill. Behavioral convergence can be quantified for scenes annotated for a particular skill domain or skill, in light of reference groups that can be used as additional skill-specific metrics.

[0171] Annotations made by specialist clinicians in light of the behavior of reference groups enable the precise identification of specific skill areas / skills for the diagnosis and / or treatment of a patient, the effective tailoring of data collection playlists for the patient in selected skill areas / skills, the monitoring of patient improvement or treatment effectiveness in selected skill areas / skills, and / or the automated, accurate, consistent, rapid, effortless, and / or cost-effective assessment of developmental disorders for the patient. This technique allows operators / users to manage and / or explore session outcomes at multiple customizable levels with detail. Skill-specific behavioral visualizations and metrics can be configured to provide users with objective quantification of how well patients generalize target skills outside of the treatment context, and to inform which aspects of treatment are aligned with patient progress.

[0172] For example, this technique allows for the customization of the visual stimulus data collection playlist. When a patient session is initiated from a web portal on a network-connected server, the operator (e.g., a therapist or clinician) may be prompted on a user interface presented on the screen of the operator-side computing device to select the session type (e.g., a diagnostic session, a monitoring session, or a targeted monitoring session), as illustrated with further detail in Figures 12A-12B. If a targeted monitoring session is selected, a window may be prompted for the operator to select a set of skill areas they wish to target. The default selection may be any skill areas selected in a previous targeted monitoring session. The data collection playlist, which should be presented on the operator-side computing device and / or the patient-side computing device communicating with the network-connected server, can be customized to prioritize data collection in videos that have moments of relevance to selected skill areas, for example, by rearranging the standard playlist, adding new videos that are particularly valuable for selected skill areas, and / or reducing or removing videos that are not relevant to selected skill areas.

[0173] In some embodiments, as illustrated with further detail in Figures 13A–13C, for example, the technique allows a user (e.g., a therapist, clinician, or patient's guardian) to see patient information (e.g., behaviors) within one or more specific skill areas in a patient's diagnostic report. For example, patient information may be presented in comparison to information of reference groups, such as a distribution map (e.g., a spleness map) or frames / moments showing areas of typical looking behaviors. Patient information may include the patient's convergent looking percentage (or attention percentage) for moments associated with a particular skill area. For example, one or more specific skill areas may be automatically selected for a patient based on the skill area with the largest amount of reliable data, the most widely and commonly requested skills, skill areas with particularly high, low, or representative scores, or a combination thereof. One or more specific skill areas may have been previously selected as target skill areas when initiating a targeted monitoring session or when customizing diagnostic results for a patient. If a monitoring session is running and there are one or more previous sessions run with the patient, the monitoring report can show the change in the percentage of the patient converging compared to the previous session. In this way, the user (e.g., a therapist, clinician, or patient caregiver) can see whether the patient has made any improvements in one or more target skill areas, whether the treatment for the patient is working or effective, and / or whether a new or adjusted treatment could be used to replace the current treatment.

[0174] This technique can also allow users to select an interactive results dashboard from a patient session page on a web portal, for example, as illustrated with further detail in Figure 13D. Users can interactively explore results in any skill area, such as a patient's score for a specific skill over a period of time or over several consecutive sessions, as well as / or moment-by-moment (or frame-by-frame) comparisons of patient and reference group behaviors (e.g., viewing behaviors). For example, users can view possible skills grouped by skill area or developmental concept, age, or treatment type. The interactive results dashboard allows users to select a subset of the target skill area in question and view combined metrics for the selected subset. The interactive results dashboard can also allow users to watch videos or view moment-by-moment / frames of patient behavior at each moment contributing to skill-specific metrics, and / or together with still images of reference group behaviors and / or scene content.

[0175] In some embodiments, the patient-side computing device may be assembled with recording devices other than the eye-tracking device (for example, as described with further detail in Figures 1A-1B), or one or more external recording devices may be configured to record the patient's video and / or audio during viewing sessions, unstructured social interactions, and / or therapy sessions. These videos and / or audios may be processed by artificial intelligence (AI) models, such as machine learning (ML) models, single-layer neural network models, multi-layer neural network models, or another trained AI model trained with reference group video / audio during the same session as the expert clinician's guidance / annotations on a list of therapy-specific skills, to generate multimodal data. The multimodal data can replace, augment, validate, or provide additional context to therapy-specific skill monitoring or developmental disorder diagnosis, assessment, and / or severity scales.

[0176] In some embodiments, the eye-tracking device includes one or more eye-tracking units configured to directly capture / track the patient's eye movements (for example, by detecting reflected and / or scattered illumination light such as infrared light). In some embodiments, the eye-tracking device includes one or more image acquisition devices (e.g., cameras) configured to determine the patient's eye movements based on captured images of the patient's eyes or eye position and / or captured images of head movement / facial data while the patient is looking at a visual stimulus. In some embodiments, the eye-tracking device includes one or more eye-tracking units and one or more image acquisition devices, and the eye movement data may include at least one of the patient's direct eye movements, captured images and / or positions of the patient's eyes, or eye movements derived from captured images and / or positions.

[0177] In some embodiments, the eye-tracking device is configured to convert patient eye movement data into eye-tracking data that may include information such as pupil position, gaze vector for each eye, and / or gaze point. In some embodiments, the eye-tracking device is configured to determine first eye-tracking data based on the patient's direct eye movements and second eye-tracking data based on the patient's derived eye movements, based on captured images and / or positions of the patient's eyes. The first and second eye-tracking data can replace, supplement, validate, or provide additional context to each other. The eye-tracking device may be configured to generate final eye-tracking data for the patient based on the first and second eye-tracking data.

[0178] An eye-tracking device can, for example, transmit patient eye-tracking data to an evaluation system (such as EarliPoint) for processing to generate an assessment or evaluation report for the patient. The evaluation system may include cloud servers as described herein, such as cloud server 110 in Figure 1A or the cloud servers relating to Figures 2A-2G. In some cases, the eye-tracking device transmits first eye-tracking data to the evaluation system for processing. In some cases, the eye-tracking device transmits second eye-tracking data to the evaluation system for processing. In some cases, the eye-tracking device transmits the first and second eye-tracking data to the evaluation system for processing, where the processed results based on the first and second eye-tracking data can replace, augment, verify, or provide each other with additional context. In some cases, the eye-tracking device transmits final eye-tracking data to the evaluation system for processing. In some cases, the eye-tracking device transmits the first eye-tracking data, the second eye-tracking data, and the final eye-tracking data to the evaluation system for processing.

[0179] In some embodiments, the assessment system for developmental disorders provides a data aggregator, as illustrated with further detail in Figures 15A-15B, for example. The data aggregator may be configured to connect to one or more third-party tools to ingest patient treatment data, including data from EHR (Electronic Health Record) / EMR (Electronic Medical Record) and ABA (Applied Behavior Analysis) practice management tools, as well as optionally reference patient data (e.g., by parsing), and to ingest patient (and / or other patients') treatment plans, goals, behavioral presses, patient responses over time, and other relevant clinical or treatment data (e.g., by parsing). The data aggregator can be combined with assessment data by the assessment system to build a large and unique data repository of clinical treatment and patient trajectories, such a data repository can enable a comprehensive understanding of the patient. This technique allows a data aggregator to retrieve reference data based on patient information (for example, based on similar groups, age, background, developmental stage, demographics, or domain), and as a result, the evaluation system can generate a specific treatment plan based on all relevant data, including the patient's own data and the reference data.

[0180] In some embodiments, the evaluation system can be configured to provide a practice management tool that gives clinicians, for example, their own direct input practice management tool for clinicians who have never used a third-party system for tracking treatment plans and data (as described with further detail in Figures 15C-15D and 15H). The practice management tool may be configured so that clinicians can manually input treatment plan information or upload existing treatment plan documents via a web browser application of the evaluation system. Clinicians can also add notes to treatment goals or input real-time treatment data during treatment delivery, which may also be used to track treatment billing operations. The input can be automatically parsed and processed by the evaluation system to extract and tabulate relevant data, including time spent on each skill.

[0181] The assessment system may also be configured to run targeted monitoring sessions or customizable eye-tracking sessions using playlists configured to quantify the patient's treatment plan, strengths, weaknesses, and / or progress in skill areas most relevant to their developmental stage (as illustrated with further detail in, for example, Figures 12A–12B or 15D). The assessment system may be configured for automated comparison of the patient's current treatment plan with treatment progress monitoring, e.g., objective skill level progress in eye-tracking sessions, to identify areas of the patient's treatment that correlate with measurable skill improvements. The assessment system may be configured, for example, by an artificial intelligence (AI) model or AI algorithm to generate a normative treatment plan based on the model and aggregated treatment data. The assessment system may recommend the optimal treatment approach (e.g., EarliPoint, ESDM (Early Initiation Denver Model), ESI (Early Social Interaction), DTT (Discrete Trial Training), Jasper (Joint Attention Symbolic Play Involvement Regulation), or Project ImPACT (Improving Parents as Communication Teachers)) and may generate a specific plan based on the patient's unique presentations. Treatment plans can be custom formatted for easy import into third-party tools.

[0182] In some embodiments, the evaluation system can provide the provider with specific tutorials (e.g., video / audio / text) related to the selected treatment plan, thereby enabling the provider to understand the selected treatment plan and learn how to implement it. The specific tutorials are content-based and can be selected from several tutorials based on the selected treatment plan, allowing the provider to understand the selected treatment plan based only on selected tutorials (e.g., fewer than 10 tutorials) without having to browse through a large number of tutorials (e.g., around 100 tutorials). In this way, the evaluation system enables inexperienced or little-experienced providers (e.g., providers in rural areas) to understand, interpret, and / or implement the selected treatment plan. This also enables experienced providers to use the selected tutorials as a basis or reference, or to support them in understanding, interpreting, and / or implementing the selected treatment plan.

[0183] In some embodiments, the assessment system generates a developmental disability assessment report for the patient, as described with further detail in, for example, Figures 8A–8C or Figures 16A–16F. The assessment report may include patient information, session information, and a summary of the assessment results (e.g., ASD or non-ASD). The assessment results may also include one or more index scores, such as a social disability index score, a verbal ability index score, and a nonverbal learning index score, which may be obtained from an artificial intelligence (AI) model, such as a machine learning (ML) model, a single-layer neural network model, a multi-layer neural network model, or another trained AI model, in response to the input of processed session data for a particular session and the corresponding model data (as described above). The assessment report may include correlations (e.g., side-by-side graphic correlations) that present one or more of the individual index scores of the assessment system (such as those obtained from the AI ​​model described above) that correlate to a “reference assessment scale,” such as the ADOS-2 scale or the Mullen scale of the Early Learning Scale, thereby providing healthcare providers reviewing the assessment report with additional understanding (even if those providers are less experienced). As used herein, the term “reference assessment scale” refers to measurements from an assessment scale, tool, or system that has been professionally adopted, administered, and / or peer-reviewed by medically trained individuals in diagnosing one or more developmental disorders (such as ASD). The assessment scale, tool, or system may include at least one of the following: ADOS-2 (Autism Spectrum Disorder Diagnostic Observation Schedule - 2nd Edition), MSEL (Mullen Scale for Early Learning), ADI-R (Autism Spectrum Disorder Diagnostic Interview - Revised Edition), CARS (Childhood Autism Rating Scale), VABS (Vineland Adaptive Behavior Scale), DAS-II (Difference Ability Scale II), WISC (Wechsler Intelligence Scale for Children), WASI (Wechsler Shortened Intelligence Scale), or VB-MAPP (Verbal Behavior Milestones Assessment and Placement Program).The term “reference assessment scale” may also be referred to as “clinical reference assessment scale,” “developmental assessment scale,” “developmental reference scale,” “standard clinical assessment,” “standard developmental reference scale,” or any other suitable term. In some cases, a reference assessment scale is used to assess one or more developmental disorders or one or more developmental skills. In some cases, a reference assessment scale is used for therapeutic purposes. Assessment reports may also include visualized individual test results (including, for example, the patient’s viewing behavior while viewing a visual stimulus) and / or attention funnels (e.g., moment-by-moment viewing behavior across several visual scenes).

[0184] To provide an overall understanding of the systems, devices, and methods described herein, several exemplary embodiments are described. It will be understood that if such data does not indicate a measure of impairment, it may provide a measure of the degree of typicality in normative development, providing an indication of variability in typical development. Furthermore, all the components and other features outlined below may be combined with each other in any preferred manner and adapted and applied to systems outside of medical diagnosis. For example, the interactive visual stimuli of this disclosure may be used as a therapeutic tool. Furthermore, the data collected may provide a measure of several types of visual stimuli that patients preferentially pay attention to. Such measures of preference have both applications in the fields of medical diagnosis and medical therapy, including, for example, the advertising industry or other industries, where data relating to visual stimulus preferences are targeted, and applications that do not involve such fields.

[0185] Exemplary Environment and System Figure 18A is a block diagram of an exemplary environment 100 for assessing developmental disorders via eye-tracking, according to one or more embodiments of the present disclosure. Environment 100 comprises a cloud server 110 and a plurality of computing systems 120-1, ..., 120-n (collectively referred to as computing system 120 or individually as computing system 120) communicating via a network 102. The cloud server 110 can provide developmental disorder assessment or diagnostic services to several users (e.g., therapists). The therapists can use the corresponding computing systems 120 to suitably and reliably collect data on patients in sessions (e.g., procedures related to data collection) of any age from neonates to elderly, with exemplary embodiments described below, particularly suitable for infants or other young patients. The data collected in a session (i.e., session data) may include eye-tracking data generated in response to the display of predetermined specific visual stimuli (e.g., one or more videos) to the patient. The computing system 120 can securely transmit session data to the cloud server 110, which can store, process, and analyze the session data for the diagnosis of ASD or other cognitive, developmental, social, or mental abilities or disorders for the patient, and provide the diagnostic results or reports to the therapist in an extremely secure, robust, rapid, and accurate manner, as will be further described below.

[0186] The treatment provider may be a single healthcare organization, including, but not limited to, an autism center, a healthcare facility, a specialist, a physician, or a clinical research institution. The healthcare organization may provide the patient with developmental assessment and diagnosis, clinical nursing, and / or therapeutic services. As shown in Figure 1A, the patient (e.g., an infant or child) may be brought to the healthcare facility by a caregiver (e.g., a parent). The operator (e.g., a specialist, physician, medical assistant, technician, or other healthcare professional within the healthcare facility) may use the computing system 120 to collect non-invasive eye-tracking data from the patient while the patient views visual stimuli (e.g., dynamic visual stimuli such as movies) that demonstrate general social interaction (e.g., two-way or three-way conversation). The stimuli presented to the patient for data collection purposes may be specific to the patient, for example, based on the patient's age and symptoms. The stimuli may be any suitable visual image (whether static or dynamic), including movies or videos as well as still images or any other visual stimuli. It will be understood that movies or videos are merely references for illustrative purposes, and that any such description may apply to other forms of visual stimuli as well.

[0187] In some embodiments, as shown in Figure 1A, the computing system 120 includes at least two separate computing devices 130 and 140, for example, an operator-side computing device 140 and at least one patient-side computing device 130. Optionally, the two computing devices 130 and 140 may be connected wirelessly, for example, via a wireless connection 131, without using a physical connection. The wireless connection 131 may be via a cellular network, a wireless network, Bluetooth, near-field communication (NFC), or other standard wireless network protocols. In some embodiments, the patient-side computing device 130 is configured to connect to the operator-side computing device 140 via a wired connection, such as a Universal Serial Bus (USB), when the wireless connection 131 is not functioning.

[0188] In some cases, the two computing devices 130 and 140 communicate with each other separately by communicating with the cloud server 110 via the network 102, and the cloud server 110 then provides communication between the operator-side computing device 140 and the patient-side computing device 130. For example, as illustrated with further detail in Figures 4A and 4B, the operator can log in to a web portal running on the cloud server 110 for device management, patient management, and data management, for example, through a web-based operator application. The operator can use the operator-side computing device 140 (e.g., a tablet) to communicate with multiple patient-side portable devices 130 within the same medical facility, for example, to acquire eye-tracking data for multiple patients in multiple sessions, which can greatly simplify the computing system 120, reduce system costs, improve work efficiency, and reduce the operator's workload.

[0189] The computing devices 130 and 140 may include any suitable type of device, such as a tablet computing device, a camera, a handheld computer, a portable device, a mobile device, a personal digital assistant (PDA), a cellular phone, a network appliance, a smart mobile phone, an Extended General-Purpose Packet Radio Services (EGPRS) mobile phone, or any two or more suitable combinations of these data processing devices or other data processing devices. As an example, Figure 19 shows an architecture for a computing device that may be implemented as computing device 130 or 140.

[0190] At least one of the computing devices 130 or 140 may be a portable device, such as a tablet device. In some cases, both computing devices 130 and 140 are portable and wirelessly connected to each other. In this way, the computing system 120 can be moved and repositioned more easily, allowing the operator more flexibility in choosing the operator's position relative to the patient. For example, the operator (carrying the operator-side computing device 140) is not physically tied to the patient-side computing device 130 and can easily position themselves in an optimal location (e.g., away from the patient's direct line of sight) during setup and data acquisition. Furthermore, the patient (e.g., an infant or other child) can be carried by a caregiver (e.g., a parent) in a more suitable location and in a more comfortable manner, which may allow the patient to be more engaged with the visual stimuli being played for effective and accurate eye-tracking data acquisition. The patient-side computing device 130 can be carried by the caregiver or positioned (e.g., adjustable) in front of the patient and caregiver.

[0191] As shown in Figure 1A, the patient-side computing device 130 includes a screen 132 (e.g., a display screen) for displaying or presenting visual stimuli to the patient. The patient-side computing device 130 may also include an eye-tracking device 134, or may be integrated with the eye-tracking device 134 within the same housing 136. In some embodiments, the patient-side computing device 130 integrated with the eye-tracking device 134 may be referred to as an eye-tracking console or eye-tracking system.

[0192] The eye-tracking device 134 may be connected to the patient-side computing device 130 via a wired connection, for example, using a USB cable or electrical wire, or using electrical pins. In some cases, the eye-tracking device 134 may be configured to connect to the patient-side computing device 130 via a wireless connection, for example, Bluetooth or NFC. The eye-tracking device 134 may be positioned in a suitable location with respect to the screen 132 and / or the patient, where the eye-tracking device 134 can capture the patient's eye movements while viewing a visual stimulus, while also minimizing visual disturbance from the patient's field of view.

[0193] As shown in Figure 1A, the eye-tracking device 134 may include one or more eye-tracking units (or sensors) 135 positioned below the bottom of the screen 132. The one or more eye-tracking units 135 may be positioned on one or both sides of the screen 132, on top of the screen 132, and / or around the screen 132. The one or more eye-tracking units 135 may be mechanically mounted on the patient-side computing device 130 in a location adjacent to the periphery of the screen 132. For example, the patient-side computing device 130 may include the screen 132 and a screen holder structure that holds the eye-tracking device 134 in a fixed, predetermined location relative to the screen 132. In some embodiments, the eye-tracking device 134 includes a first eye-tracking unit configured to capture or collect eye movements of the patient's left eye, and a second eye-tracking unit configured to capture or collect eye movements of the patient's right eye. The eye-tracking device 134 may further include a third eye-tracking unit configured to capture the position of the patient's eyes, or an image acquisition unit (e.g., a camera) configured to capture an image of the patient's eyes. In some embodiments, the eye-tracking device 134 is configured to determine eye movements based on the captured position and / or image of the patient's eyes by a third eye-tracking unit. In some embodiments, the patient's eye movement data includes at least one of the following: eye movements collected from the patient's eyes (e.g., by the first and second eye-tracking units), eye movements derived from the captured position and / or image (e.g., by the third eye-tracking unit), the captured position, or the captured image. As described below, the eye movement data may be converted into eye-tracking data to be processed in the cloud server 110.

[0194] An eye-tracking unit includes sensors that can detect the presence of a person and track what that person is looking at in real time, or measure where that person is looking or how their eyes respond to a stimulus. The sensors can convert the person's eye movements into a data stream containing information such as pupil position, gaze vector for each eye, and / or gaze point. In some embodiments, the eye-tracking unit includes a camera (e.g., an infrared-sensitive camera), an illumination source (e.g., infrared (IR) illumination), and algorithms for data acquisition and / or data processing. The eye-tracking unit may be configured to track pupil or corneal reflection or reflex (CR). The algorithms may be configured for pupil center and / or corneal detection and / or artifact suppression. In some embodiments, the eye-tracking unit includes an image acquisition unit (e.g., a camera) configured to capture an image of the patient's eyes while the patient is looking at a visual stimulus. The eye-tracking unit may be configured to process the captured image of the patient's eyes to determine eye movements while the patient is looking at a visual stimulus. The eye-tracking device 134 (for example, an eye-tracking unit) may be configured to convert eye movement data into eye-tracking data that may also include information such as pupil position, gaze vector for each eye, and / or gaze point. In some embodiments, the eye-tracking device 134 may send the eye-tracking data, based on captured images of the eyes, to a cloud server 110 for further processing.

[0195] In some embodiments, the eye-tracking device 134 determines first eye-tracking data based on the reflection or CR of the tracked pupil or cornea, and second eye-tracking data based on a captured image of the eye. The first and second eye-tracking data may be processed by the eye-tracking device 134 to replace, supplement, verify, or provide additional context with respect to each other. The eye-tracking device 134 may be configured to generate final eye-tracking data for the patient based on the first and second eye-tracking data. The eye-tracking device 134 may send the final eye-tracking data based on the captured image of the eye to a cloud server 110 for further processing. In some embodiments, the eye-tracking device 134 sends the first and second eye-tracking data to a cloud server 110 for further processing, where the first processed result based on the first eye-tracking data and the second processed result based on the second eye-tracking data may replace, supplement, verify, or provide additional context with respect to each other. The assessment results may be determined based on the first processed results and the second processed results.

[0196] Because there can be variations in eye size, foveal position, and general physiology that may be relevant to each individual, an eye-tracking unit may be calibrated before being used to collect eye-tracking data from a participant (e.g., a patient). In calibration, the physical position of the eye is algorithmically associated with a point in space that the participant is looking at (e.g., fixating on). The fixation position may be a function of the participant's perception. In some embodiments, calibration requires the participant to look at a fixed, known calibration target (e.g., a point) in the visual field. Calibration can include a single target placed in the center, or two, five, nine, or even thirteen targets. The algorithm can generate a mathematical transformation between the eye position (without CR) and the fixation position for each target, and then, for example, interpolation between each target can be used to create a matrix that covers the entire calibration area. The more targets used, the higher and more uniform the accuracy may be across the entire visual field. The calibration area defines the portion of the eye-tracking unit's range that offers the highest accuracy, and accuracy decreases when the eye moves at an angle greater than the target being used.

[0197] In some embodiments, the eye-tracking unit can perform self-calibration, for example, by creating an eye model and passively measuring each individual's characteristics. Calibration can also be performed without active participation from the participant by effectively "hiding" the calibration target among other visual information and making assumptions about gaze position based on the content. In some embodiments, calibration is not performed on the eye-tracking unit if useful data can be obtained from raw pupil position, for example, using a medical vestibulo-ocular reflex (VOR) system or fatigue monitoring system.

[0198] In some cases, validation may be performed to measure the success of the calibration, for example, by presenting a new target and measuring the accuracy of the calculated gaze. The tolerance for calibration accuracy may depend on the application of the eye-tracking unit. For example, an error between 0.25 and 0.5 degrees of the visual angle may be considered acceptable. Depending on the application, an error exceeding 1 degree may be considered a failed calibration and require another trial. Participants may improve in the second or third attempt. Participants who consistently have large validation errors may have visual or physiological problems that prevent them from participating in the experiment. Validation results can be expressed in terms of the degree of the visual angle and may be represented graphically.

[0199] The patient-side computing device 130 may include an eye-tracking application (or software) configured to retrieve or receive raw eye-tracking data collected by the eye-tracking device 134, as shown in Figure 2A. The patient-side computing device 130 may generate session data based on the raw eye-tracking data and store the raw eye-tracking data together with relevant information (timestamp information) in a data file (for example, in .tsv format, .idf format, or any preferred format), as shown in Figure 5(b). The session data may also include information about the visual stimuli being played or presented in a separate data file (for example, in .tsv format or any preferred format), as shown in Figure 5(a). The information may include timestamp information for each visual stimulus being played.

[0200] In some embodiments, the patient-side computing device 130 stores several predetermined visual stimuli (e.g., movies or video files) that are grouped to correspond to patients in a specific age group and / or symptom group. For example, a first list of predetermined visual stimuli can be configured for ASD assessments for patients in a first age range (e.g., 5–16 months), and a second list of predetermined visual stimuli can be configured for ASD assessments for patients in a second age range different from the first age range (e.g., 16–30 months). In some embodiments, the operator can use the operator-side computing device 140 to control which list of predetermined visual stimuli should be played for a particular patient, based on information about that particular patient. In some embodiments, the operator application sends age information to the eye-tracking application in patient selection, and the eye-tracking application then dynamically selects a pre-configured appropriate playlist based on the age information, without operator intervention or operator selection. In some embodiments, the number of predetermined visual stimuli can also be stored in the operator-side computing device 140.

[0201] The test method relies on the patient being awake and looking at the screen 132 of the patient-side computing device 130. During both the calibration procedure and the data acquisition procedure, predetermined movies and / or other visual stimuli are presented to the patient via the patient-side computing device 130. These movies and / or other visual stimuli may include human or animated actors performing hand / face / body movements. As illustrated with further detail in Figure 5, during the data acquisition period, the computing system 120 may periodically show the patient a calibration target or gaze target (which may be animated). This data may be used later to verify accuracy.

[0202] The visual stimuli (e.g., movies or video scenes) presented to the patient may be age-dependent; that is, the visual stimuli may be age-specific. In some embodiments, processing session data includes measuring the amount of time the patient spends looking at an actor's eyes, mouth, or body, or other predetermined areas of interest, and the amount of time the patient spends looking at background areas in the video. As shown in Figure 1B, the video scenes shown to the patient via the screen 132 of the patient-side computing device 130 may depict scenes of social interaction 133 (e.g., an actor looking directly into the camera, attempting to engage the viewer, or scenes of children in play). In some embodiments, the video scenes may include other preferred stimuli, such as animation and priority viewing tasks, as will be further illustrated with more detail in Figures 4C–4F. Measures of gaze time for specific spatial locations in the video may relate to the patient's level of social and / or cognitive development. For example, children between 12 and 15 months of age may exhibit increased mouth fixation as a result of their developmental stage in language development, alternating between eye fixation and mouth fixation. As another example, a patient's decreasing visual fixation over time, particularly regarding an actor's eyes in a video, may be an indicator of ASD or another developmental disorder in the patient. For the diagnosis and monitoring of developmental, cognitive, social, or mental abilities or impairments, including ASD, an analysis of a patient's viewing patterns (during the movie being shown, and across multiple viewing sessions, or compared to historical data of patients of substantially the same age and / or symptoms) may be performed.

[0203] When both the patient and the caregiver face the eye-tracking device 134 on the patient-side computing device, the detection light (e.g., infrared light) emitted from the eye-tracking device 134 can propagate toward the patient's eyes and the caregiver's eyes. In some embodiments, the patient's caregiver (e.g., parent) is provided with glasses 122 to wear while holding the patient to view a visual stimulus displayed on the patient-side computing device 130, for example, as shown in Figures 1A-1B. The glasses 122 may be configured to filter or block the detection light from the eye-tracking device 134 so that the eye-tracking device 134 can collect only reflected or scattered light from the patient's eyes to track / capture the patient's eye movements while the patient (and caregiver) are viewing the visual stimulus. In this way, the detection accuracy of the eye-tracking device 134 can be improved without interference from the caregiver's eye movement data.

[0204] In some embodiments, the patient-side computing device 130 includes a recording device 138 configured to record images, audio, and / or video of the patient while the patient is viewing visual stimuli presented on the screen 132 of the patient-side computing device 130 during viewing sessions, unstructured social interactions, and / or therapy sessions (e.g., with a therapist). The recording device 138 may be a camera, an audio recorder, or a video recorder. In some embodiments, as shown in Figure 1A, the recording device 138 may be located in a housing 136, for example, at the top of the screen 132, compared to an eye-tracking device 134 located below the bottom of the screen 132.

[0205] Figure 1C shows an example of a patient-side computing device 130 including an eye-tracking device 134 and a recording device 138. The housing 136 may have a recess in the center of the top of the screen 132, and the recording device 138 may be configured to be placed within the recess. As shown in Figure 1C, the patient-side computing device 130 may include a foldable base (or support) 137 through one or more joints 139 between the housing 136 and the base 137. The base 137 may be rotatable so as to be closed to function as a cover to enclose the screen 132, or open to function as a support. The patient-side computing device 130 may be adjustable to accommodate the operation of the recording device 138, for example, based on the patient's height and / or the patient's viewing angle. The patient-side computing device 130 may be carried by the patient's guardian (such as a parent) (for example, as shown in Figure 1A) or placed on a table, for example, to play video during a session.

[0206] In some embodiments, the patient-side computing device 130 includes one or more recording devices 138 located, for example, above the screen 132. In some examples, one recording device is in the center of the top and two other recording devices are on the two sides of the top. In some examples, the two recording devices are distributed across the top. In some embodiments, as an alternative or addition, one or more external recording devices are located within the healthcare facility where the patient is (e.g., on the ceiling and / or corners and / or walls) and are configured to record images, audio, and / or video about the patient's information.

[0207] Compared to the eye-tracking device 134 configured to capture patient eye-tracking data, the recording device 138 and / or one or more external recording devices may be configured to capture other patient information, such as facial information (e.g., facial expressions), verbal information, and / or physical behavior. For example, while the patient is looking at a visual stimulus, the patient may repeat what a character in the visual stimulus has said, speak to another person, smile, raise a hand, point, stand up and down, or remain still, and these may be captured by the recording device 138 and / or one or more external recording devices. The other information, along with the eye-tracking data, may be referred to as multimodal data, for extending the data input to a system for assessing developmental disorders, such as a cloud server 110. Multimodal data, in addition to or in conjunction with eye-tracking data, may replace, supplement, validate, and / or provide additional context for developmental disorder assessment.

[0208] In some embodiments, as described with further detail below, multimodal data, as well as eye-tracking data, may be used to monitor one or more specific (such as therapy-specific) skill areas, e.g., manding, listener response, joint attention, tact, play, turn-taking, and / or any other skill areas. In some cases, clinicians or therapists may review one or more videos of individual patients to assess individual patient developmental impairments in these skill areas, which can be subjective, time-consuming, unreliable, and / or inconsistent. In contrast, the techniques implemented in this disclosure may use one or more artificial intelligence (AI) models (e.g., machine learning (ML) models) to automatically analyze multimodal data for individual patients to identify one or more specific skill areas for assessing patient developmental impairments, which can significantly improve processing speed, consistency, and accuracy, and / or reduce time / cost for clinicians or therapists.

[0209] In some embodiments, for example, during a session of viewing visual stimuli, before that session, and / or after that session, multimodal data (e.g., in the form of image data, audio data, and / or video data) is acquired for a reference group (e.g., typical children of similar age, sex, and / or circumstances). One or more specialist clinicians can analyze the multimodal data for the reference group and annotate the multimodal data using one or more specific skill domains (and skills). The annotated multimodal data for the reference group, along with eye-tracking data taken for the reference group, can be provided to one or more AI or ML models for training. Once the multimodal data for individual patients is input into one or more trained AI or ML models, the AI ​​or ML models can, at their discretion, automatically analyze the individual patient's multimodal data together with eye-tracking data to identify one or more specific skills for assessing developmental disorders in individual patients.

[0210] The operator-side computing device 140 is configured to run an operator application (or software). In some embodiments, the operator application is installed and run within the operator-side computing device 140. In some embodiments, the operator application runs on a cloud server 110, and the operator can log in to a web portal to interact with the operator application through a user interface presented on the screen 142 of the operator-side computing device 140, for example, as shown in Figure 1D. In some embodiments, as described with further detail in Figures 4A-4J, 12A-12B, and 13A-13D, the operator application may be configured to supervise or control steps of the eye-tracking application or software on the patient-side computing device 130, for example, to select and play specific visual stimuli for the patient and collect raw eye-tracking data, and / or to review the results or reports.

[0211] In some examples, as shown in Figure 1D, and further described with more detail in Figures 12A-12B, the operator application may present different sessions (e.g., diagnostic sessions, monitoring sessions, target monitoring sessions) in a user interface 150 that the operator can choose from. For example, when an operator chooses to initiate a target monitoring session within the same healthcare facility with a patient, the operator application may pop up a new window 160 for the operator to select target skill areas (e.g., manding, listener response, joint attention, and play) to monitor the patient's behavior within those target skill areas during the session. As described further with more detail below (e.g., in Figure 11), individual moments or frames in a playlist of visual stimuli may be annotated to specify one or more specific skill areas (and / or skills) by a specialist clinician, for example, in light of the viewing behavior of a reference group. When the operator selects a target skill area in a session, the operator application can adjust the visual stimuli to be presented to the patient on the patient-side computing device 130 based on the selected target skill area, for example, by prioritizing annotated / known videos to monitor the selected target skill area, and / or increasing the value of additional videos related to the selected target skill area, and / or removing frames unrelated to the selected target skill area, and / or optimizing the playlist to maximize the target skill area. When the operator selects a user interface element 162 in a new window 160 to run the session, the adjusted visual stimuli can be presented to the patient on the patient-side computing device 130.

[0212] In some examples, as illustrated with further detail in Figures 13A and 13C, the operator may review diagnostic results / reports using the operator-side computing device 140 (or any other computing device associated with the operator). The operator application may present a user interface on the screen 142 of the operator-side computing device 140. The user interface may include options for the operator to select, such as different patients, or different sessions or histories of patients. Through the user interface, the operator can also view default results (as shown, for example, in Figures 8A-8C and Figures 13B-1 and 13B-2), customized reports (as shown, for example, in Figure 13C), and / or launch an interactive results dashboard (as shown, for example, in Figure 13D). For example, when the operator chooses to view a customized report, the operator application may pop up a new window for the operator to select target skill areas (e.g., manding, listener response, joint attention, and play) to customize the diagnostic or monitoring report. In some examples, when an operator selects a target monitoring session, the operator application can automatically customize the target monitoring session report to select the same target skill area selected for the playlist for the target monitoring session. The new window 160 can be overlaid on user interface 150, be alongside user interface 150, or have an overlap with user interface 150. User interface 150 can be changed to the new window 160.

[0213] In some embodiments, the operator application interfaces with the eye-tracking software via a Software Development Kit (SDK). In some embodiments, communication between the patient-side computing device 130 and the operator-side computing device 140, or between the operator application and the eye-tracking application, may be performed using WebSockets communication. WebSockets communication enables bidirectional communication between the two devices. This bidirectional communication allows the operator to control the patient-side computing device 130 while simultaneously receiving information from it. WebSockets communication may be performed using a secure embodiment of WebSockets called WebSockets Secure (WSS). As described above, communication between the patient-side computing device 130 and the operator-side computing device 140 (for example, communication between the operator application and the eye-tracking application) may be conducted through a cloud server 110. For example, the operator can log in to a web portal running on the cloud server 110 and use the operator-side computing device 140 to establish a wireless connection with the patient-side computing device 130 for acquiring patient eye-tracking data.

[0214] The operator application may also be used to present the operator with an interface that displays other functions, such as information related to stimuli shown to the patient (e.g., movies), such as the patient's name and date of birth. The operator can also use the operator-side computing device 140 to log in to the web portal of the cloud server 110 for device management, patient management, and data management. In some embodiments, the operator application runs on the cloud server 110 and is controlled by the operator using the operator-side computing device through the web portal. The operator can operate the computing system 120 with minimal training.

[0215] As illustrated with further detail in Figures 3, 4A–4J, and 5, the computing system 120 may be configured for session data acquisition. In some embodiments, a session is initialized by establishing a connection between an operator-side computing device 140 and a patient-side computing device 130. After inputting patient information into an operator application (e.g., custom software) running on the operator-side computing device 140, the operator application can control an eye-tracking application running on the patient-side computing device 130 to instruct the operator or the patient's caregiver to select age-specific stimuli and position the patient-side computing device 130 in front of the patient in an appropriate orientation and / or location. The operator may use the operator-side computing device 140 to control the operator application and / or the eye-tracking application or software to (a) calibrate the eye-tracking device 134 for the patient, (b) verify that the calibration is accurate, and (c) collect eye-tracking data from the patient as the patient views dynamic videos or other visual stimuli in a session, for example, from the patient moving their eyes in response to a given movie or other visual stimulus. After the session ends, both the eye-tracking data and the stimulus-related information (e.g., a list of stimuli viewed by the patient) can be stored as session data in two separate data files. The session data can then be automatically transferred, for example, by the patient-side computing device 130 to a secure database in a cloud server 110 via network 102. The database may be remote from computing system 120 and may be configured to accommodate and aggregate data collected from several computing systems 120.

[0216] Network 102 may include a large-scale computer network such as a local area network (LAN), wide area network (WAN), internet, cellular network, or a combination thereof, connecting any number of mobile computing devices, stationary computing devices, and server systems. Each of the computing devices 130, 140 in the computing system 120 can communicate with the cloud server 110 through network 102.

[0217] In some embodiments, communication between on-premises computing devices 130, 140 and a cloud server 110 may be conducted using the Hypertext Transfer Protocol (HTTP). HTTP follows a request-response model in which a client sends a request to a server (for example, through a browser or desktop application) and the server sends a response. Responses sent from the server may include various types of information, such as documents, structured data, or authentication information. HTTP communication may be conducted using a secure embodiment of HTTP called Hypertext Transfer Protocol Secure (HTTPS). Information passed over HTTPS is encrypted to protect both the privacy and integrity of the information.

[0218] The cloud server 110 may be a computing system hosted in a cloud environment. The cloud server 110 may include one or more computing devices and one or more machine-readable repositories or databases. In some embodiments, the cloud server 110 may be a cloud computing system including one or more server computers in a local or distributed network, each having one or more processing cores. The cloud server 110 may be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device having one or more processors. As an example, Figure 18 is an architecture for a cloud computing system that may be implemented as the cloud server 110.

[0219] As shown in Figure 1A, the cloud server 110 includes a cloud platform 112 and a data pipeline system 114. As will be explained with further detail in Figures 2A to 2G, the cloud platform 112 may be configured to provide a web portal, store application data related to the therapist or tenant, and store data, such as raw eye-tracking data, processed data, analytical results and / or diagnostic results. The data pipeline system 114 is configured to perform data processing and data analysis.

[0220] In some embodiments, as described with further detail in Figures 6 and 7A-7B, the cloud server 110 is configured to automatically receive, process, and analyze session data from multiple computing systems. Furthermore, the cloud server can process and analyze session data from several sessions from numerous computing systems in parallel, which significantly improves session processing speed and allows for the provision of diagnostic results within a short timeframe, for example, within a 24-hour window. For example, the receipt of session data by the cloud server 110 (e.g., by the cloud platform 112) can initiate an automated software execution processing and analysis process (e.g., by the data pipeline system 114). In this process, individual patient data can be compared to a model of previously generated eye-tracking data from historical eye-tracking data of patients with substantially the same age, background, and / or symptoms. The results of the comparison may include, but are not limited to, a diagnosis of neurodevelopmental disorder, including normative recommendations for ASD, measures of the patient's developmental / cognitive function, and / or treatment plans. Alternatively or additionally, the collected data may be compared and / or reviewed for a given patient over multiple sessions (and over a predetermined time period) to identify potential changes in visual gaze (e.g., decreased visual gaze). The results may be condensed into a diagnostic report for use by the patient's physician. In some embodiments, once the diagnostic results are ready, the cloud server 110 may transfer the diagnostic results to the operator-side computing device 140, which may then be presented on the user interface of the operator-side computing device 140, as described with further detail in, for example, Figures 8A-8C or Figures 16A-16F.

[0221] In some embodiments, a large amount of model data, including data relating to patients of similar age, similar background, and / or similar circumstances, can be used together with processed session data for patients to generate diagnostic results for patients using comparison or inference via artificial intelligence (AI) models such as statistical models, algorithms, machine learning models, or artificial neural network models, which can significantly increase the accuracy of the diagnostic results.

[0222] Environment 100 involves three main steps corresponding to the three parts of Environment 100 shown in Figure 18 (for example, a computing system 120 for data acquisition, a cloud platform 112, and a data pipeline system 114). As illustrated with further details in Figures 2A to 2G, the three parts can be configured together to reliably collect data from patients and to efficiently process and analyze the collected data for the diagnosis of ASD or other cognitive, developmental, social, or mental abilities or disorders.

[0223] Figure 2A is a block diagram of an exemplary system 200 for assessing developmental disorders via eye tracking, according to one or more embodiments of the present disclosure. System 200 may be implemented within the environment 100 of Figure 1. System 200 may be considered an assessment system for developmental disorders. In some examples, the assessment system is represented as EarliPoint. According to three steps of the data process, system 200 includes three subsystems, namely, a data acquisition subsystem 210, a platform subsystem 220, and a data pipeline subsystem 230. Each subsystem may consist of corresponding hardware and software components. The platform subsystem 220 and the data pipeline subsystem 230 may form a cloud server, for example, the cloud server 110 in Figure 1A.

[0224] The data acquisition subsystem 210 is configured to collect patient eye-tracking data. The data acquisition subsystem 210 may be the computing system 120 in Figure 1. As shown in Figure 2A, the data acquisition subsystem 210 includes an eye-tracking console 212 that runs an eye-tracker application 214, and an operator-side computing device (e.g., 140 in Figure 1) that runs an operator application 216. In some embodiments, the operator application 216 is deployed within the operator-side computing device. In some embodiments, the operator application 216 is deployed within a platform subsystem 220, and the operator can log in to the platform subsystem 220 via a web portal 222 and use the operator-side computing device to run the operator application 216 on the platform subsystem 220. Deploying the operator application 216 within the platform subsystem 220 avoids deploying the operator application 216 within one or more operator-side computing devices, which can reduce the software and hardware requirements for the operator-side computing devices, for example, allowing the operator application 216 to be suitably maintained or updated without maintaining or updating operator information on each of the one or more operator-side computing devices.

[0225] The eye-tracking console 212 may be an integrated device including the patient-side computing device 130 (e.g., a tablet) and the eye-tracking device 134 shown in Figure 1. As described above, the data acquisition subsystem 210 may include several movie files 218 that are stored in the eye-tracking console 212 and optionally in the operator-side computing device. The movie files 218 may be predetermined age-specific visual stimuli for patients at different ages and / or with different symptoms.

[0226] As illustrated in Figure 1A, the platform subsystem 220 and the data pipeline subsystem 230 can be contained within a network-attached server such as a cloud server (e.g., cloud server 110 in Figure 1) and can be implemented in a centralized cloud-hosted environment provided by a cloud provider, such as Microsoft Azure. In some embodiments, the platform subsystem 220 is configured for managing and organizing resources in the cloud-hosted environment. The platform subsystem 220 may be the cloud platform 112 in Figure 1. As shown in Figure 2A, the platform subsystem 220 includes a web portal 222, a database 224 for storing application data, and a database 226.

[0227] The web portal 222 may be a web-based interface. Through the web portal 222, an operator (e.g., a medical professional) can log in to the platform subsystem 220, for example using an operator-side computing device, to manage (view and / or query) application data stored in database 224 and / or data in database 226. For example, the web portal 222 allows the operator to view diagnostic results. Based on the diagnostic results, a prewritten process of actions (e.g., requesting further evaluation) may be provided.

[0228] As an example of database 224, Figure 2D shows database 240, which stores various types of documents. Database 240 could be a NoSQL database such as Azure Cosmos DB. Various types of documents can be stored in database 240 as application data. Unlike relational databases, NoSQL databases do not have strong relationships between documents. The dotted lines in Figure 2D indicate the references and information embedded between documents.

[0229] In some embodiments, the database 240 stores corresponding application data for the therapist (or tenant). The therapist may be a healthcare organization, including, but not limited to, an autism center, a healthcare facility, a specialist, a physician, or a clinical research institution. The organization may vary in structure, patient volume, and lifespan. As shown in Figure 2D, the corresponding application data may include organizational documents 242, user documents 244, device documents 246, patient documents 248, session documents 250, and history documents 252. The user may be an operator associated with the healthcare organization, such as a medical assistant, specialist, physician, or any other healthcare professional.

[0230] Organizational documents 242 include settings and customizations for the organization. User documents 244 include identifier information along with user roles and permissions. User roles indicate whether the user is an administrator or operator associated with different security levels or permissions. Device documents 246 include identifier information for each eye-tracking console associated with the organization, e.g., 212 in Figure 2A. Patient documents 248 include information about patients, e.g., infants or children treated as patients for developmental assessment. Session documents 250 include session-related information, which may consist of a session identifier (session ID), a reference to the patient, a reference to the user running the session, a pointer to eye-tracking data, and the results of data processing and data analysis. History documents 252 may be used to maintain a version history of changes to a document. The document mirrors the structure of its parent document and includes additional audit information. In some embodiments, database 224 allows URL-based queries to query across multiple variables (e.g., to find people with administrative roles). For example, variables may include patient / device / session, adverse events, etc.

[0231] In some embodiments, a cloud server including a platform subsystem 220 and a data pipeline subsystem 230 may be implemented in a centralized cloud environment, which can provide greater flexibility for extending the functionality of the cloud server. For example, a cloud server may utilize a multi-tenant architecture to provide a Software-as-a-Service (SaaS) subscription-based diagnostic service to a healthcare provider. In a multi-tenant architecture, healthcare providers share a single version of the software across various geographical locations. The term “tenant” in a multi-tenant architecture represents a single healthcare provider of the system. The resources of the cloud server may be managed dynamically based on the total number of tenants and the expected average workload, for example, how many tenants are accessing the cloud server at a given point in time. The cloud server may employ horizontal scaling techniques, such as autoscaling, to handle sudden increases in resource workload.

[0232] In a multi-tenant architecture where applications are shared, it is crucial to isolate tenant data and prevent other tenants from accessing it. This is called isolation. There are three different isolation strategies that can be implemented: shared database, tenant-specific database, and tenant-specific application. In the shared database strategy, tenants share a single instance of the application, and all data is stored in a single database. In the tenant-specific database strategy, for example, strategy 260 shown in Figure 2E(a), tenants share a single instance of the application in application layer 262, but have their own database 264 (for example, database 224 in Figure 2A or 240 in Figure 2D). In the tenant-specific application strategy, for example, strategy 270 shown in Figure 2E(b), each tenant has its own instance of the application in its own application layer 272, and its own database 274 (for example, database 224 in Figure 2A or 240 in Figure 2D). The cloud server can deploy a tenant-based database policy 260 or a tenant-based application policy 270 to treatment providers.

[0233] Continuing to refer to Figure 2A, the database 226 is configured to store raw eye-tracking data or session data, processed session data, analysis results, and / or diagnostic results or diagnostic reports. The database 226 may be a storage platform (e.g., Azure Blob) and can be paired with a tool written in any preferred programming language (e.g., Python, Matlab) to enable URL-based interfaces and queries to the database 226. Additionally, the database 226 may be compatible with the programming language (e.g., Python, Matlab) used to transfer data from the data acquisition subsystem 210 to the database 226 and from the database 226 to the data pipeline subsystem 230. For example, if the patient-side computing device (e.g., 130 in Figure 1) is located in a healthcare facility, data collection takes place at that facility and data is transferred between the database 226 and the patient-side computing device. The database 226 may be secure, HIPAA compliant, and protected by redundant backup systems.

[0234] In some embodiments, the platform subsystem 220 is configured to enable one or more operations, including (a) acquiring new patient information, (b) storing raw data files (e.g., including eye-tracking data), (c) automated and secure transfer of files between data acquisition devices (e.g., the eye-tracking console 212 in Figure 2A), a data processing computer, and a database, (d) tabulating and querying data for the purpose of assessing device usage and other data quality metrics, and (e) physician access to the results of processing. One or more of operations (a) to (c) may be performed by the upload function module 221 within the platform subsystem 220.

[0235] Continuing to refer to Figure 2A, the data pipeline subsystem 230 is configured to process and analyze patient eye-tracking data in conjunction with generating diagnostic results. In some embodiments, the data pipeline subsystem 230 includes a data processing module 232, a data analysis module 234, and model data 236. As will be described with further detail in Figures 7A-7B below, the data processing module 232 is configured to process session data, including eye-tracking data, to obtain processed session data, and the data analysis module 234 is configured to analyze the processed session data using the model data 236 to generate diagnostic results.

[0236] In some embodiments, system 200 includes interfaces for devices and subsystems. Interfaces may exist between subsystems. For example, system 200 may also include interfaces between the data acquisition subsystem 210 and the cloud platform subsystem 220, and interfaces from the cloud platform subsystem 220 to the data pipeline subsystem 230. Interfaces may exist within subsystems. For example, system 200 may include interfaces between eye-tracking console hardware (e.g., a tablet and an eye-tracking device) and eye-tracking application software.

[0237] Figure 2B shows an example of processing single session data in the system 200 of Figure 2A according to one or more embodiments of the present disclosure. As described above, after a data collection session is completed, the eye-tracking console 212 can automatically transfer the session data to the platform subsystem 220. The session data may include two files, one containing raw eye-tracking data (e.g., gaze position coordinates, blink data, pupil size data, or a combination thereof) and the other containing stimulus-related information (e.g., a list or playlist of those movies viewed by the patient). Through an upload function module 221 implemented in the platform subsystem 220, the session data can be stored in database 226 and in application data in database 224. The stored session data can then be automatically transferred from the platform subsystem 220 to the data pipeline subsystem 230 for data processing and data analysis without human intervention. For example, a software script written in any suitable programming language (e.g., Python, Matlab) may be used to transfer raw, unprocessed data files from database 226 to data pipeline subsystem 230 for processing. Session data is first processed by data processing module 232 and then analyzed by data analysis module 234, which provides diagnostic information about the patient.

[0238] In some embodiments, three files are generated: a file containing processed eye-tracking data, a file containing a summary of eye-tracking statistics, and a file containing diagnostic information. The file containing diagnostic information may be uploaded to database 224 and associated with the patient in the application data, as shown in Figure 2D. The three files may then be uploaded to database 226 for storage. In some cases, the processed eye-tracking data is tabulated into a session table. A summary of the eye-tracking information (e.g., gaze samples / movies) can be read from the processed summary file and tabulated in database 226 for subsequent queries. Summary values ​​(e.g., percentage gaze / movies) may then be calculated in database 226.

[0239] Figure 2C shows an example of processing multiple session data in parallel within the system 200 of Figure 2A according to one or more embodiments of the present disclosure. As shown in Figure 2C, multiple eye-tracking consoles 212a, 212b can send multiple session data 213a, 213b, 213c (collectively referred to as session data 213 or individually referred to as session data 213) of a session to the platform subsystem 220. Within the data pipeline subsystem 230, the data processing module 232 and the data analysis module 234 can be written in a preferred programming language (e.g., Python), which allows the data processing module 232 and the data analysis module 234 to be deployed in containers 231a, 231b, 231c (collectively referred to as container 231 or individually referred to as container 231). Each session may be processed using its own instance of data processing and data analysis. The use of containers allows data processing and analysis to be performed as session data is uploaded from the data acquisition subsystem 210, which can result in the session being returned within a short time period, for example, within a 24-hour window.

[0240] As illustrated with further detail in Figures 7A and 7B, the cloud server can process and analyze session data from several sessions from numerous computing systems in parallel. Firstly, the cloud server can deploy a separate container (e.g., 231) for each session, each container may include a corresponding data processing module 232 and a corresponding data analysis module 234. In this way, when a session's session data (e.g., 213) is uploaded by the corresponding eye-tracking console 212, the session's session data can be processed and analyzed using its own container (e.g., 231 having its own instance of data processing and data analysis). Secondly, while session data from multiple sessions is being processed in the corresponding container using, for example, the majority of processing units (or cores) in the cloud server, model data for analyzing the processed session data can be pre-loaded in parallel into the corresponding container using, for example, the remaining or a small number of processing units in the cloud server. Thirdly, all of the processed session data and loaded model data can be analyzed in parallel in the corresponding container using, for example, the total number of processing units in the cloud server. The use of parallelization in multiple ways can significantly improve the speed of session data processing and analysis, enabling the provision of rapid diagnostic results within short timeframes, such as a 24-hour window. For example, once diagnostic results become available, the cloud server can send them to the corresponding operator-side computing device (e.g., 140 in Figure 1), and the results can then be displayed in the results interface of the operator application 216. Parallelization can also make the cloud server more efficient in resource utilization, which can further improve system performance.

[0241] Figure 2F shows an exemplary configuration 280 for data backup for the system 200 of Figure 2A, according to one or more embodiments of the present disclosure. Configuration 280 can enable high availability of services to therapies, as a result, therapies can access their services despite any outage in one or more specific areas of the cloud server (e.g., platform subsystem 220 and data pipeline subsystem 230).

[0242] High availability refers to a provider's ability to access their services regardless of whether the cloud service provider is down. Availability can be achieved by replicating resources in different physical locations. The cloud servers implemented herein may be provided by cloud service providers that can provide Platform as a Service (PaaS) resources with either built-in or configurable high availability. Resources hosted in a cloud environment can have high availability using high availability service level agreements or through the use of geo-redundancy.

[0243] Figure 2F illustrates an example of high availability through geo-redundancy. As shown in Figure 2F(a), cloud server resources may be hosted in a first data center 282 having a web portal 222a. The resources are replicated in a second data center 284. When the first data center 282 is functioning correctly, the second data center 282b is a mirror, and therapist traffic is directed to the first data center 282. However, as shown in Figure 2F(b), if the first data center 282 fails, therapist traffic is redirected to the replicated resources in the second data center 284 running the replicated web portal 222b. The switching process can be seamless, and therapists may not even notice the switch to different resources in the replicated data center.

[0244] Figure 2G shows an exemplary data backup for system 200, for example, the platform subsystem 220 and the data pipeline subsystem 230. A database 224 that stores application data, and a database 226 that stores raw and processed eye-tracking data and analyzed or diagnostic results, may be stored in multiple data centers. The web portal 222 in the platform subsystem 220, and the data processing module 232 and data analysis module 234 in the data pipeline subsystem 230, as well as optionally the operator application 216 (which runs on the platform subsystem 220), can be included in the active data center 282 and replicated in the backup data center 284.

[0245] An example process for obtaining session data Figure 3 is a flowchart of an exemplary process 300 for session data acquisition according to one or more embodiments of the present disclosure. Process 300 may be performed by a system, for example, the computing system 120 in Figure 1A or the data acquisition subsystem 210 in Figure 2A. The system includes an operator-side computing device (for example, 140 in Figure 1) and one or more patient-side computing devices (for example, 130 in Figure 1) integrated with an associated eye-tracking device (for example, 134 in Figure 1). Each of the operator-side computing device and one or more patient-side computing devices may communicate with a network-based server or a cloud server (for example, the cloud server 110 in Figure 1A, or a cloud server as described in Figures 2A to 2G) via a network (for example, network 102 in Figure 1). The system may be associated with, for example, a therapist providing developmental disorder assessment and / or treatment services to a patient. The cloud server may be associated with a service provider for providing services, for example, data processing, data analysis, and diagnostic results to the therapist. For illustrative purposes, Figures 4A to 4J show a series of exemplary display screens (or user interfaces) presented on an operator-side computing device (a) and a patient-side computing device (b) during session data acquisition (for example, in process 300 of Figure 3) according to one or more embodiments of the present disclosure.

[0246] In step 302, the session is initiated, for example, by establishing a connection or communication between the operator-side computing device and the patient-side computing device. In some embodiments, the two computing devices 130 and 140 may be connected wirelessly, for example, via a wireless connection, without using a physical connection. The wireless connection may be via a cellular network, a wireless network, Bluetooth, near-field communication (NFC), or other standard wireless network protocols. In some cases, the patient-side computing device may also be configured to connect to the operator-side computing device via a wired connection, such as a Universal Serial Bus (USB), when the wireless connection is not working.

[0247] In some embodiments, the connection between the operator-side computing device and the patient-side computing device is established by the two computing devices communicating with a cloud server that provides communication between the operator-side computing device and the patient-side computing device. For example, as shown in Figure 4A, the operator (e.g., any other person representing a medical assistant, medical professional, or therapist) can log in to a web portal (e.g., 222 in Figure 2A) running on the cloud server for device management, patient management, and data management. The operator may have corresponding user roles and permissions, as described in Figure 2D, for example. Figure 4A(a) shows a user interface (UI) presented on the display screen of the operator-side computing device after the operator has logged in to the web portal using the operator-side computing device. The UI may be the user interface of an operator application (e.g., 216 in Figure 2A) running on the cloud server or on the operator-side computing device.

[0248] As shown in Figure 4A (a), the UI includes a menu with buttons labeled "Home," "Patient," "Device," and "User." By clicking a button, corresponding information (e.g., patient information, device information, or user information) may be presented within the UI. For example, when the "Device" button is clicked, the UI displays a list of names of patient-side computing devices controllable by the operator, e.g., Device 1, Device 2, Device 3, Device 4, Device 5. If patient-side computing devices, e.g., Device 4, Device 5, are connected to the cloud server, a "Connect" indicator, e.g., a string of characters, may be presented adjacent to the patient-side computing device names. The operator can select one of the names, e.g., Device 4, to connect the corresponding patient-side computing device to the operator-side computing device. Once a name is selected, the UI displays a request for an access code to be entered to connect the corresponding patient-side computing device, as shown in Figure 4B (a).

[0249] Figure 4A(b) shows the user interface presented on the screen (e.g., 132 in Figure 1) of a patient-side computing device, for example, device 4. For example, the UI may be presented after the patient-side computing device is turned on and logged in by the operator. The UI may show a button, for example, "Start," which can be clicked by the operator to begin a session. After the "Start" button is clicked, the patient-side computing device connects to a cloud server, for example, a web portal. The cloud server can associate the patient-side computing device with the operator based on the identifier of the patient-side computing device, for example, as shown in Figure 2D. Once the patient-side computing device has successfully connected to the cloud server, the UI presented on the patient-side computing device may show information about an access code generated by the web portal for connection with the operator-side computing device, for example, "5678," as shown in Figure 4B(b). The operator can obtain the access code from the UI presented on the patient-side computing device, enter the access code on the UI presented on the operator-side computing device, and then submit the access code to the web portal. After the web portal verifies that the access code entered in the operator-side computing device matches the access code generator for the patient-side computing device, the web portal can establish a wireless connection between the operator-side computing device and the patient-side computing device.

[0250] Once a connection is established between the operator-side computing device and the patient-side computing device (e.g., device 4), connection information, such as "Connected to device 4," may be displayed on the UI of the operator-side computing device, for example, as shown in Figure 4C(a). The UI may also display buttons to initiate displaying visual information, such as a movie, to the patient on the screen of the patient-side computing device. The patient's human caregiver, such as a parent, can bring (or carry) the patient to view the movie presented on the screen of the patient-side computing device. In some embodiments, the patient's human caregiver may wear glasses configured to filter or block light (e.g., IR light) from the eye-tracking device so that the eye-tracking device can collect only reflected or scattered light from the patient's eyes and not the human caregiver's eyes, in order to track / capture the patient's eye movements while the patient (and the human caregiver) are viewing the visual stimuli on the patient-side computing device.

[0251] In step 304, for example, the operator may click a "Start Movie" button on the UI of the operator-side computing device, which can cause visual desensitization information (e.g., a movie) to be displayed to the patient on the screen of the patient-side computing device, as shown in Figure 4C(b).

[0252] During the display of a desensitizing movie, data is generally not recorded. Instead, the movie is displayed to gain the patient's attention. The movie may reflexively elicit extrinsic cues from the patient without the need for verbal mediation or instructions from the operator. For example, the operator does not need to give instructions to look at the screen of the patient's computing device because the movie itself gains the patient's attention.

[0253] As shown in Figure 4D, Figure (b), while the desensitization movie is displayed on the screen of the patient-side computing device, the operator can select the patient's patient information through the UI of the operator-side computing device, as shown in Figure 4D, Figure (a). The operator can select a patient from a list of existing patients associated with the operator in the cloud server, for example, as shown in Figure 2D, or create a patient profile for a new patient. After the patient is confirmed, the process begins setting up the eye-tracking device (or patient-side computing device) for the patient by displaying setup information on the UI of the operator-side computing device, as shown in Figure 4E, Figure (a). The operator can also select "Pause movie" or "Skip movie" on the UI of the operator-side computing device.

[0254] During setup, the desensitization movie may continue to play on the screen of the patient-side computing device, as shown in Figure 4E(b) and Figure 4F(b). As shown in Figure 4F(a), the relative position between the eye-tracking device and the patient's eye is indicated on the UI of the operator-side computing device, for example, by text or diagram. This relative position may be determined by capturing image data of the patient's eye using an image acquisition device (e.g., a camera) that is included in or adjacent to the eye-tracking device. In some embodiments, after the operator clicks the "Start Setup" button in the UI of the operator-side computing device, an operator application running on a cloud server may send a command to the patient-side computing device to capture an image of the patient's eye using an image acquisition device, as shown in Figure 4E(a). The patient-side computing device can then send the captured image to the cloud server, and the operator application can process the image to determine the relative position between the eye-tracking device and the patient's eye. The relative position may include the distance between the eye-tracking device and the patient's eye, and the horizontal and / or vertical deviation between the center of the eye and the center of the eye-tracking device's field of view (or detection area). Based on the relative position, the operator application may display instructions on the operator-side computing device's UI for adjusting the position of the eye-tracking device, for example, “Move console down,” as shown in Figure 4F(a). Once the relative position between the patient's eye and the eye-tracking device is acceptable, the operator can confirm the setup, for example, by clicking a button for “Confirm setup” in the UI. In some embodiments, in response to a determination that the relative location between the patient's eye and the eye-tracking device is less than a predetermined threshold (e.g., horizontal / vertical deviation less than 0.1 cm), the operator application may determine that the setup is complete and display an indication to the operator.

[0255] In step 306, the patient is calibrated using the eye tracking device. After the setup is completed, the operator application can present a button for "starting calibration" on the UI of the operator-side computing device, as shown in FIG. 4G(a). In some embodiments, calibration requires the patient to look at one or more fixed known calibration targets (e.g., dots or icons) in the field of view. The calibration target or fixation target reflexively captures the patient's attention, resulting in intermittent movements towards the known target location and fixation on such location. The target ensures fixation on a radially symmetric target that spreads less than 0.5 degrees of visual angle, for example, at a finite location. Other examples include concentric patterns, shapes, or shrinking stimuli that ensure fixation on a fixed target location, even if initially larger in size.

[0256] For example, when the operator clicks the button to start calibration, a plurality of calibration targets can be presented sequentially at a predetermined location (i.e., the target location) (e.g., the center, the upper left corner, or the lower right corner) on the screen of the patient-side computing device, as shown in FIG. 4G(b). While presenting a plurality of calibration targets on the screen of the patient-side computing device, the eye tracking device can be activated to capture the patient's eye tracking calibration data in response to receiving a command from the operator application. An eye tracking application (e.g., 214 in FIG. 2A) can be executed on the patient-side computing device to collect the patient's eye tracking calibration data.

[0257] In some embodiments, a patient-side computing device (e.g., an eye-tracking application) is configured to determine the position of the corresponding visual fixation on a calibration target and then compare the determined position of the patient's corresponding visual fixation with a predetermined location where the calibration target was presented. Based on the result of the comparison, the eye-tracking application can determine whether the calibration target is calibrated. If the distance between the position of the patient's corresponding visual fixation and the predetermined location for the calibration target is within a predetermined threshold, the eye-tracking application can determine that the patient's corresponding visual fixation aligns with the predetermined location for the calibration target, i.e., the calibration target is calibrated. If the distance is greater than or equal to the predetermined threshold, the eye-tracking application can determine that the patient's corresponding visual fixation does not align with the predetermined location, i.e., the calibration target has failed to calibrate.

[0258] In some embodiments, the patient-side computing device transmits the captured eye-tracking calibration data of the patient and / or information about the predetermined location to an operator-side computing device or a cloud server, and the operator application can determine the position of the patient's corresponding visual fixation, compare the determined position with a plurality of predetermined locations, and / or determine whether the calibration target is calibrated based on the result of the comparison.

[0259] In some embodiments, a first calibration target may be initially presented at the center of the screen, and calibration may continue with four additional calibration targets presented at each corner of the screen along the rotational direction. The operator application can alert the operator to the active status of the calibration, for example, calibrating point 1, calibrating point 2, calibrating point 3, or that calibration has completed 4 / 5 of the targets. Between each calibration target, a desensitization movie is played for a set time period before a new calibration target is presented. Each calibration target may loop a set number of times before determining that the calibration target is not calibrated and moving on to the next calibration target. If a calibration target fails to calibrate, it may be retried after all remaining calibration targets have been presented and gaze acquisition has been attempted.

[0260] In step 308, the calibration is validated. Validation may be performed to measure the success of the calibration, for example, by showing new targets and measuring the calculated accuracy of gaze. Validation may show calibration targets for calibration step 306, for example, fewer than five, such as three calibration targets. A desensitization movie may be played between showing two adjacent calibration targets.

[0261] In some embodiments, initial validation with varying levels of success (e.g., the number of calibration targets calibrated or validated) based on a comparison between the determined location of the patient's corresponding visual gaze and a predetermined location where the calibration targets are presented can automatically instruct the operator to (1) recalibrate the eye-tracking device using the patient, (2) revalidate any calibration targets that could not be validated, or (3) accept the calibration and proceed to data collection in step 310.

[0262] In some embodiments, the operator may have discretion in deciding whether to accept the calibration. As shown in Figure 4H, calibration targets are simultaneously presented at multiple predetermined locations on the display screen of the operator-side computing device, along with a depiction (e.g., a dot) of the patient's corresponding visual gaze at a determined position of the patient's corresponding visual gaze. The UI may also show a first button for “accept verification” and a second button for “recalibrate.” The operator can see the alignment between the multiple calibration targets and the depiction of the patient's corresponding visual gaze and decide whether to accept verification (by clicking the first button) or to recalibrate the patient for the eye-tracking device (by clicking the second button).

[0263] In step 310, patient eye-tracking data is collected, for example, after calibration is validated or the operator accepts validation, by presenting the patient with a playlist of predetermined visual stimuli (e.g., stimulus movies) on the screen of the patient-side computing device. As shown in Figure 5(a), the predetermined list of visual stimuli may include several social stimulus videos specific to the patient (e.g., 0075PEER, 0076PEER, 0079PEER), based on the patient's age and / or symptoms. Between each social stimulus video, or before each social stimulus video is presented, a center-stim video (e.g., a centerstim video) may be shown to temporarily center the patient's gaze. In some embodiments, as shown in Figure 5(a), a calibration check (e.g., similar to that in step 306) is performed in the data collection step, for example, between showing the center-stim video. For example, a calibration check could include indicating five calibration targets: CCTL for the top-left calibration check, CCTR for the top-right calibration check, CCBL for the bottom-left calibration check, CCCC for the center-center calibration check, and CCBR for the bottom-right calibration check. The data related to the calibration check can be used in post-processing, for example, to recalibrate eye-tracking data and / or to determine the accuracy of the calibration.

[0264] In a specific example, the data collection sequence in step 310 may be as follows: 1. Center alignment stim, 2. Stimulating movies, 3. Center alignment stim, 4. Stimulation movie or calibration check (for example, displaying 5 calibration targets that are randomly played between 2-4 stimulation movies), 5. Repeat steps 1-4 until the designated playlist of stimulating videos is complete.

[0265] In some embodiments, as shown in Figure 4I, the UI on the operator-side computing device displays a button for “Start Collection.” After the operator clicks the “Start Collection” button, a predetermined playlist of visual stimuli may be presented sequentially on the patient-side computing device screen according to a predetermined sequence. On the operator-side computing device screen, as shown in Figure 4I, the UI may indicate the status of running the playlist in text format (e.g., Movie in Play: Focusing Stim) or showing the status of showing the same content as the content presented on the patient-side computing device screen (e.g., Showing Focusing Stim Video).

[0266] In some embodiments, as shown in Figure 4J, the UI may display a running playlist of videos that have been played or are currently playing, for example, centering stim, PEER1234, centering stim, PEER5678. The UI may also display the video being presented on the screen of the patient-side computing device. The UI may also display a progress bar showing the percentage of a given stimulus movie that has been played out of a playlist of stimulus movies. The UI may also display buttons for the operator to skip movies.

[0267] In some embodiments, the accuracy of the eye-tracking data collected (e.g., 812 in Figure 8A) can be assessed by presenting visual stimuli that reflexively acquire attention and result in intermittent movement toward a known target location and fixation on such location. The target is designed to reliably elicit fixation on a finite location, e.g., a radially symmetrical target that extends less than 0.5 degrees of the visual angle. Other examples include concentric patterns, shapes, or contracting stimuli that reliably elicit fixation on a fixed target location, even if initially larger in size. Such stimuli may be tested under data collection with a headrest to ensure they reliably elicit fixation under ideal test conditions, and their use may then be extended to include data collection without head restraint.

[0268] In some embodiments, the numerical assessment of the accuracy of the collected eye-tracking data may include the following steps: (1) presenting gaze targets that reliably elicit gaze to a small area of ​​the visual display unit; (2) recording eye-tracking data throughout the target presentation; (3) identifying gazes within the collected eye-tracking data; (4) calculating the difference between gaze location coordinates and target location coordinates; and (5) storing the calculated difference between gaze location coordinates and target location coordinates as vector data (direction and magnitude) for as few as one target, or as many as possible (e.g., five or nine, but more). In some embodiments, the recalibration or post-processing step may be performed, for example, by applying spatial transformations to align gaze location coordinates with actual target location coordinates, by techniques including, but not limited to, (a) trilinear interpolation, (b) linear interpolation in centroid coordinates, (c) affine transformation, and (d) piecewise polynomial transformation.

[0269] The session ends when the predetermined playlist of visual stimuli has been fully played. The patient-side computing device can generate session data based on the raw eye-tracking data collected by the eye-tracking device, storing the raw eye-tracking data in a data file (for example, in .tsv format, .idf format, or any preferred format) along with relevant information (timestamp information), as shown in Figure 5(b). The raw eye-tracking data may include values ​​for several eye-tracking parameters at different timestamps. The eye-tracking parameters may include gaze coordinate information for the left eye, right eye, left pupil, and / or right pupil.

[0270] Session data may also include information about the visual stimuli being played or presented in a separate data file (for example, in .tsv format or any preferred format), as shown in Figure 5(a). This information may include timestamp information and a name for each visual stimulus being played. The timestamp information of the visual stimuli can be associated with the timestamp information of the eye-tracking data, so that the eye-tracking data for each visual stimulus (and / or calibration check) can be determined individually based on the timestamp information in these two data files.

[0271] In step 312, session data is sent to the cloud server. Once session data is generated by the patient-side computing device, the patient-side computing device can send the session data to the cloud server. As will be explained with further detail in Figures 2A to 2G, and in Figures 6, 7A to 7B, and 8, the cloud server can first store the session data in a centralized database, for example, database 226 in Figures 2A to 2B, and then process the session data, analyze the processed data, and generate patient diagnostic results, which may be accessible or viewable by an operator or medical professional.

[0272] Exemplary data processing and data analysis Figure 6 is a flowchart of an exemplary process 600 for managing session data by a cloud server (for example, cloud server 110 in Figure 1A or a cloud server as described in Figures 2A to 2G) according to one or more embodiments of the present disclosure, for example, for data processing and data analysis. Figures 7A to 7B show in more detail than Figure 6 a flowchart of an exemplary process 700 for managing session data by a cloud server according to one or more embodiments of the present disclosure.

[0273] In step 702, if the session is complete, the corresponding patient-side computing device (e.g., 130 in Figure 1) or eye-tracking console (e.g., 212 in Figures 2A-2G) sends the session data to the cloud server's cloud platform, for example, via a web portal. The cloud platform may be platform 112 in Figure 1A or platform subsystem 220 in Figures 2A-2G. In response to receiving the session data, the cloud server's cloud platform stores the session data in a database within the cloud platform (e.g., database 226 in Figures 2A-2G). The cloud platform then automatically transfers the session data to a data pipeline system (e.g., 114 in Figure 1A or 230 in Figures 2A-2G) for data processing and data analysis.

[0274] In step 704, a file pointer to the session data of the session is added to the processing queue (step 704). The session data of all completed sessions awaits processing according to the processing queue. As soon as the session data of a session is uploaded and stored in the cloud server, the corresponding file pointer can be assigned to the session data of the session and added to the processing queue. The file pointer can be an identifier for the session data of each session. The session data of each session can be retrieved from a database in the cloud platform based on the file pointer.

[0275] In step 706, for example, based on an auto-scaling technology that can implement session parallelization, a respective container is created for the session data of each session. For example, in response to the addition of a file pointer for a new session to a processing queue, a new container can be created for the new session. Each container (e.g., 231 in FIG. 2C) can have its own instance of, for example, a data processing module and a data analysis module as shown in FIG. 2C.

[0276] Within each container, steps 708 - 714 are executed on the session data of the corresponding session by, for example, the data processing module 232 in FIGS. 2A - 2G. Note that steps 708 - 714 can be executed in parallel for the session data of multiple sessions in multiple containers.

[0277] (Corresponding to step 602 in FIG. 6) In step 708, session data is retrieved from a database in a cloud platform using the corresponding file pointer. As described above, the session data can include two files, namely, a gaze tracking data file (e.g., as shown in FIG. 5(b)) and a playlist file (e.g., as shown in FIG. 5(a)).

[0278] Referring to FIG. 6, step 602 can correspond to step 708. In step 604, the session data is prepared for processing. Step 604 can include one or more steps as described in steps 710 - 714 of FIG. 7.

[0279] Step 604 may include linking eye-tracking data in an eye-tracking data file to movies played in a playlist file. In some embodiments, as shown in Figure 7A, in step 710, the eye-tracking data is broken down into separate steps based, for example, timestamp information in these two files. Each step may correspond to playing a corresponding movie (e.g., a centering target, a given visual stimulus, or one or more calibration targets). For example, eye-tracking data corresponding to timestamps within a range defined by the timestamps of two adjacent movies may be included in a step. In step 712, the eye-tracking data in each step is linked to a corresponding movie from the playlist based on timestamp information in these two files. In some embodiments, the eye-tracking data is not broken down into separate steps, but instead is processed as a continuous stream with data samples linked to corresponding movies in the playlist.

[0280] In step 714, the eye-tracking data is recalibrated to account for drift or deviation. In some embodiments, the eye-tracking data collected in the calibration step while presenting the playlist may be used to calibrate or align eye-tracking data collected while playing individual movies in different sequences, for example, as shown in Figure 5(a). Any discrepancies in gaze position are corrected using data from adjacent time points when an additional calibration target is presented. Some larger discrepancies may cause some data to be excluded from subsequent analysis.

[0281] In step 606, the prepared session data is processed. In some embodiments, the data processing module extracts relevant information from the prepared session data, such as the patient's visual gaze and / or visual gaze to a target object or area in the movie. In some embodiments, the data is resampled to account for any temporal variability between samples. The data may be resampled using any preferred interpolation and / or smoothing techniques. The data may be transformed from the specified original resolution and / or coordinate system of the collected eye-tracking data to a resolution and / or coordinate system suitable for analysis. For example, the raw data may be collected at a higher resolution (e.g., 1024 x 768 pixels) than the resolution of the presented stimulus (e.g., rescaled to 640 x 480 pixels). In some embodiments, the data processing module can automatically identify basic eye-movement events (such as unsolicited gaze, intermittent movement, blinking, off-screen data, or lost data), and can automatically identify when the subject was gazing (in an undesirable way), moving intermittently, blinking, or not looking at the screen. The data processing module can be adjusted for aberrations in gaze position estimation as output from the eye-tracking device.

[0282] In some embodiments, as shown in step 716 of Figure 7B, session data for multiple sessions of a patient is processed within multiple session containers in parallel with the preloading of corresponding model data for the patient into multiple session containers. In some examples, while the corresponding model data is preloaded in parallel into multiple session containers using a small number of processing units (e.g., M processing cores) in the cloud server, the session data for multiple sessions is being processed within multiple session containers using the majority of processing units (e.g., N processing cores) in the cloud server. The processing units or cores may be central processing units (CPUs). Parallelization can avoid additional time spent waiting to upload model data.

[0283] The cloud server can pre-store model data in a database, for example, in Figures 2A-2G, 226. The model data may include data on numerous instances of significant differences in gaze position for patients (e.g., infants, toddlers, or children) across fluctuating levels of social, cognitive, or developmental functioning. Corresponding model data for patients may include data relating to patients of similar age, similar background, and / or similar symptoms, and such data may be used together with processed session data for patients to generate diagnostic results for the patients. Corresponding model data for patients can be identified and retrieved from the database, for example, based on the patient's age, patient background, and / or patient's symptoms. Step 608 of process 600, where the processed data is prepared for analysis, may include retrieving the processed data in multiple session containers and pre-loading the corresponding model data into the multiple session containers.

[0284] In step 610, the processed data is analyzed to generate the analyzed results. In some embodiments, for a session, the processed data is compared with the corresponding model data in the corresponding session container to obtain comparison results. In some embodiments, the data analysis module generates results using the processed data and the corresponding model data, for example, by using comparison or inference via an artificial intelligence (AI) model such as a statistical model, algorithm, machine learning model, or artificial neural network model. In some embodiments, as shown in step 718 of Figure 7B, the processed session data and preloaded model data are analyzed in parallel using a total number of processing units, for example, N+M cores, within multiple session containers.

[0285] In some embodiments, processed session data is compared to a corresponding data model to determine the level of developmental, cognitive, social, or psychiatric symptoms. The generated scores are then compared to predetermined cutoffs or other values ​​to determine the level of diagnosis and symptom severity for patients with ASD. In some other embodiments, patient gaze point data (e.g., visual gaze data) is analyzed over a predetermined time period (e.g., across multiple sessions spanning several months) to identify decreases, increases, or other significant changes in visual gaze (e.g., gaze point data initially corresponding to typically developing children changing to more unstable gaze point data corresponding to children exhibiting ASD, or changing to gaze point data that becomes more similar to typically developing children in response to targeted therapy).

[0286] In step 720 (corresponding to step 612), a summary of the results is calculated. As described above, the analyzed results can be used to determine a score for at least one index, such as the social impairment index, verbal ability index, nonverbal ability index, social adaptability index, and / or social communication index. Based on a comparison of the score with at least one predetermined cutoff value, the diagnosis of ASD and the level of symptom severity can be calculated for the patient. For example, as shown in Figure 8A, based on the analyzed results and / or any other suitable information (e.g., from other relevant analyses of the patient), a social impairment index score of 6.12 is shown within the range of -50 (social impairment) to 50 (social ability), indicating no problem with social impairment; a verbal ability index score of 85.89 is shown within the range of 0 to 100, indicating above-average verbal ability; and a nonverbal ability index score of 85.89 is shown within the range of 0 to 100, indicating above-average nonverbal ability. Furthermore, based on the analyzed data, a diagnosis of not having ASD can also be calculated.

[0287] In some embodiments, the results summary includes a visualization of individual eye-tracking data (e.g., gaze point data) overlaid on movie stills from socially relevant moments, enabling clinicians and parents to better understand how patients visually focus on social information. For example, in step 610, movie stills for which the patient has available data can be cross-referenced against a list of movie stills predetermined to evoke eye-gaze behavior using information about diagnostic status, including symptom severity. The visualizations may also include visualizations of aggregated reference data from typically developing children, matched by patient attributes such as age and sex. These visualizations may be side-by-side so that clinicians and / or parents can compare individual patient data with reference data and see how gaze patterns align or deviate. These visualizations may include annotations describing movie content, eye-gaze patterns, and various other things.

[0288] In some embodiments, the results summary includes an animation that visualizes the patient's eye-tracking data, overlaid on movie stills from socially relevant moments. For example, a web portal may include a dashboard that allows clinicians to view stimulus movies to be shown to the patient, with the patient's eye-gaze data overlaid. The dashboard may be configurable to allow the user to select which movies should be visualized and whether frames should be visualized to obtain information about social impairment indices, verbal ability indices, nonverbal indices, and any other indices calculated in the report.

[0289] Continuing to refer to Figure 7B, in step 722, the data pipeline subsystem returns the results output to the web portal, for example, as shown in Figure 2B. In some embodiments, the results output includes three files: a file containing the processed eye-tracking data, a file containing a summary of eye-tracking statistics, and a file containing diagnostic information (e.g., a summary of results). The three files may then be uploaded to a database (e.g., 226 in Figures 2A-2G) for storage. In some cases, the processed eye-tracking data is tabled into a session table. A summary of the eye-tracking information (e.g., gaze samples / movies) can be read from the processed summary file and tabled in the database for subsequent queries. Summary values ​​(e.g., percentage gaze / movies) may then be calculated in the database.

[0290] In step 724, the result output is recombined with patient information to generate a diagnostic report or diagnosis result for the patient. For example, a file containing diagnostic information may be uploaded to an application data database (e.g., 224 in Figures 2A-2G) so that it is associated with the patient in the application data, as shown in Figure 2D. The diagnostic report or diagnosis result may be presented in any preferred manner to a user associated with the patient (an operator, or a medical professional such as a physician) or a caregiver associated with the patient in the application data database.

[0291] In some embodiments, once a patient's diagnostic report or result is generated, the user may be notified (e.g., by email or message) to log in to view the diagnostic report or result via a web portal. The diagnostic report or result may be presented on a user interface, for example, as shown in Figures 8A or 8B-8C or 16A-16F. In some embodiments, once a patient's diagnostic report or result is generated, the diagnostic report or result may be sent to an operator-side computing device for presentation to the user. The diagnostic report or result may also be sent to the operator via secure email or message. The diagnostic report or result may be stored in an application data database (e.g., 224 in Figures 2A-2G) and / or a database (e.g., 226 in Figures 2A-2G).

[0292] Figure 8A shows an exemplary results interface 800 that displays an evaluation report (or diagnostic report or diagnostic result) including at least one index value based on eye-tracking data, according to one or more embodiments of the present disclosure. The results interface 800 displays patient information 802, requesting physician / organization information 804, device ID 806 of the patient-side computing device, processing date 807 (indicating the time to retrieve session data for processing), and report issuance date 808.

[0293] The results interface 800 also displays the collected information 810, including the accuracy 812, the oculomotor function 814, and the data collection summary 816. The accuracy 812 and the oculomotor function 814 may be presented graphically. The data collection summary 816 may include the number of videos viewed, the number of videos excluded, the duration of the collected data, the time spent watching videos, the time spent not watching videos, and at least one of the accuracy, the oculomotor scale, or the quality control scale.

[0294] The results interface 800 also displays neurodevelopmental test results 820, which may include diagnostic results 822 (e.g., ASD or non-ASD), social disability index information 824, verbal ability index information 826, and nonverbal ability index information 828. The results interface 800 can graphically represent these index information 824, 826, and 828 along with corresponding explanations.

[0295] Figures 8B–8C show another exemplary outcome interface 850 that displays behavior-based scales of developmental assessment in instances of nonverbal communication and gestures (A) and joint attention and mutual gaze (B) in Figure 8B, and facial affect (C) and pointing and social surveillance (D) in Figure 8C, according to one or more embodiments of the present disclosure.

[0296] The results interface 850 shows behavior-based measures of children's individual vulnerabilities and opportunities for skill development. Neurodevelopmental assessment via eye tracking measures how children engage with social and asocial cues that occur continuously within a naturalistic environmental context (shown as still frames from the test video, left column 852). With respect to those contexts, a normative reference metric provides an objective quantification of not having ASD and a visual engagement of age-expected (both illustrated as density distributions in a pseudo-color format in the middle column 854, and as a fade from color to grayscale overlaid on the corresponding still frames in the middle column 856). The age-expected reference metric can be used to measure and visualize patient comparisons that reveal individual strengths, vulnerabilities, and opportunities for skill improvement (right column 858, exemplary patient data shown as overlaid circular openings encompassing portions of the video in the fovea of ​​each patient, e.g., each opening extends approximately 5.2 degrees in the center of the patient's field of vision). Individual patients with ASD appear to be less likely to gaze at instances of (A) verbal and nonverbal dialogue and gestures (860), (B) joint attention and mutual gaze cueing (870), (C) dynamic facial affect (880), and (D) joint attention and social surveillance (890). As shown in Figures 8B–8C, children with ASD appear to be more likely to engage with object toys (1, 3, 5, 7), color and contrast cues (2, 6, 8), objects (10, 11, 12), background elements not directly related to the social context (4, 9, 13), and recursive visual features (14, 15, 16, 17, 18). The elapsed time in the lower right of the static frames highlights the rapidly changing nature of social dialogue, and within approximately 12 minutes of viewing time, hundreds of verbal and nonverbal communication cues are presented, each eliciting patterns of age expectations of engagement and providing a corresponding opportunity for objective quantitative comparison of patient behavior.

[0297] Exemplary process Figure 9 is a flowchart of an exemplary process 900 for session data acquisition according to one or more embodiments of the present disclosure. Process 900 may be performed by a system, for example, the computing system 120 in Figure 1A or the data acquisition subsystem 210 in Figures 2A to 2G. Process 900 may be similar to process 300 in Figure 3 and may be described with reference to Figures 4A to 4J.

[0298] The system includes an operator-side computing device (e.g., 140 in Figure 1) and one or more patient-side computing devices (e.g., 130 in Figure 1) integrated with an associated eye-tracking device (e.g., 134 in Figure 1). At least one of the operator-side computing device or the patient-side computing device may be a portable device. Each of the operator-side computing device and one or more patient-side computing devices can communicate with a network-based server or a cloud server (e.g., cloud server 110 in Figure 1A or a cloud server as described in Figures 2A to 2G) via a network (e.g., network 102 in Figure 1). The system may be associated with, for example, a therapist providing developmental disorder assessment and / or treatment services to a patient. The cloud server may be associated with a service provider for providing services, such as data processing, data analysis, and diagnostic results to the therapist. Process 900 may include several steps, some of which are performed by the operator-side computing device, some by the patient-side computing device and / or eye-tracking device, and some by a combination of the operator-side computing device and the patient-side computing device.

[0299] In step 902, a session for the patient is initiated by establishing communication between the operator-side computing device and the patient-side computing device. In some embodiments, establishing communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device, for example, the wireless connection 131 in Figure 1.

[0300] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes the operator-side computing device accessing a web portal on a network-attached server (e.g., 222 in Figures 2A to 2G) and wirelessly connecting the operator-side computing device to the patient-side computing device in response to receiving a selection of the patient-side computing device within the web portal.

[0301] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes, for example, displaying connection information on the screen of the patient-side computing device by the patient-side computing device, as shown in Figure 4B, and establishing a wireless connection between the operator-side computing device and the patient-side computing device in response to receiving connection information input by the operator-side computing device.

[0302] In some embodiments, the process 900 further includes, for example, displaying visual desensitization information to the patient on the screen of a patient-side computing device after establishing communication, as shown in Figure 4C. The eye-tracking device may be configured not to collect the patient's eye-tracking data while displaying the visual desensitization information.

[0303] In some embodiments, process 900 further includes, for example, accessing a web portal on a network-connected server by an operator-side computing device to set up a session for the patient while displaying visual desensitization information, as shown in Figures 4D and 4E. In some cases, setting up a session includes selecting a patient from a list of patients or creating a profile for the patient on the network-connected server.

[0304] In some embodiments, process 900 further includes determining the relative position between the eye-tracking device and at least one eye of the patient, as shown in Figure 4F, for example, and displaying instructions on the user interface of the operator-side computing device for adjusting the position of the eye-tracking device or the patient's position. In some cases, process 900 further includes determining that the patient is aligned with the eye-tracking device in response to the determination that the relative location of at least one eye of the patient is at a predetermined location within the detection area of ​​the eye-tracking device.

[0305] In step 904, the patient is calibrated for the eye-tracking device by displaying one or more calibration targets to the patient on the screen of the patient-side computing device, for example, as shown in Figure 4G. While the eye-tracking device is used to capture the patient's eye-tracking calibration data, each of the one or more calibration targets may be presented sequentially at corresponding predetermined locations on the screen of the patient-side computing device. Process 900 may include, for each of the one or more calibration targets, processing the patient's captured eye-tracking calibration data to determine the location of the patient's corresponding visual gaze to the calibration target; comparing the location of the patient's corresponding visual gaze to the corresponding predetermined location where the calibration target is presented; and, based on the result of the comparison, determining whether the calibration target is calibrated for the eye-tracking device.

[0306] In some embodiments, calibrating a patient for an eye-tracking device further includes determining that a calibration target is calibrated and displaying the next calibration target in response to a determination that the deviation between the patient's corresponding visual gaze position and a corresponding predetermined location is below a predetermined threshold, or determining that a calibration target is not calibrated and redisplaying the calibration target for calibration in response to a determination that the deviation is greater than a predetermined threshold.

[0307] In some embodiments, process 900 further includes validating the calibration using one or more new calibration targets after calibrating the patient for the eye-tracking device. Similar to the calibration described in step 904, validating the calibration includes sequentially presenting each of the one or more new calibration targets at a corresponding predetermined location on the screen of the patient-side computing device while capturing the patient's eye-tracking calibration data using the eye-tracking device, and processing the patient's captured eye-tracking calibration data to determine the location of the patient's corresponding visual gaze for each of the one or more new calibration targets.

[0308] In some embodiments, for example as shown in Figure 4H, verifying calibration includes simultaneously presenting one or more new calibration targets at one or more corresponding predetermined locations and depictions of one or more corresponding visual gazes of the patient at one or more determined locations on the user interface of the operator-side computing device, and determining that the calibration has been verified in response to receiving an indication to validate the results of the calibration, or initiating recalibration of the patient for the eye-tracking device in response to receiving an indication to invalidate the results of the calibration.

[0309] In some embodiments, verifying calibration includes determining the number of new calibration targets that each pass the calibration for, based on the patient's corresponding visual gaze position and corresponding predetermined position; determining that the calibration is enabled if the number or associated percentage is greater than or equal to a predetermined threshold; or determining that the calibration is disabled and initiating recalibration of the patient for the eye-tracking device if the number or associated percentage is less than the predetermined threshold.

[0310] In step 906, following the decision to enable calibration, a predetermined list of visual stimuli is presented to the patient sequentially on the screen of the patient-side computing device while eye-tracking data from the patient is collected using the eye-tracking device.

[0311] In some embodiments, as shown, for example, in Figure 4I, Figure 4J, or Figure 5, a centering target may be presented to the patient on the screen of the patient-side computing device to center the patient's gaze before each of a given list of visual stimuli is presented.

[0312] In some embodiments, for example as shown in Figure 5, patient calibration for an eye-tracking device is performed between presenting two adjacent visual stimuli from a given playlist of visual stimuli. Eye-tracking data collected during calibration can be used for at least one of the following: calibrating the patient's eye-tracking data, or determining the accuracy by a network-connected server.

[0313] In some embodiments, for example as shown in Figure 4J, the process 900 further includes presenting on the user interface of the operator-side computing device at least one of the following: a progress indicator that is continuously updated while presenting a predetermined playlist of visual stimuli; information on visual stimuli that have already been presented or are currently being presented; information on visual stimuli to be presented; or a user interface element for skipping visual stimuli from the predetermined playlist of visual stimuli.

[0314] In step 908, the session data is transmitted by the patient-side computing device to the network-connected server, and the session data includes the patient's eye-tracking data collected during the session. The patient-side computing device may automatically transmit the session data to the network-connected server in response to either the decision to complete the presentation of a predetermined playlist of visual stimuli on the screen, or the receipt of a session completion indication from the operator-side computing device, for example, through a web portal on the network-connected server.

[0315] In some embodiments, session data includes information relating to a playlist in which a given visual stimulus is presented, which may include, for example, the name of a given visual stimulus to be presented and an associated timestamp when the given visual stimulus is presented, as shown in Figure 5(a). Session data may also include eye-tracking data and an associated timestamp when the eye-tracking data is generated or collected, as shown in Figure 5(b). In some embodiments, transmitting session data includes transmitting a first file storing the patient's eye-tracking data and a second file storing information relating to a list in which a given visual stimulus is presented.

[0316] Figure 10 is a flowchart of an exemplary process 1000 for data processing and data analysis according to one or more embodiments of the present disclosure. Process 1000 may be executed by a network-attached server, which may be a cloud server in a cloud environment, for example, cloud server 110 in Figure 1A or a cloud server as described in Figures 2A-2G. For example, the network-attached server may include a platform, for example, 112 in Figure 1A or 220 in Figures 2A-2G, and a data pipeline system, for example, 114 in Figure 1A or 230 in Figures 2A-2G. The platform may include a web portal (for example, 222 in Figures 2A-2G), an application data database (for example, 224 in Figures 2A-2G), and a database (for example, 226 in Figures 2A-2G). The data pipeline system may include one or more data processing modules (for example, 232 in Figures 2A-2G) and one or more data analysis modules (for example, 234 in Figures 2A-2G). Process 1000 may be similar to process 600 in Figure 6 or process 700 in Figures 7A to 7B.

[0317] In step 1002, for example, as shown in Figure 2B, session data for multiple sessions is received, and the session data for each session includes eye-tracking data of the corresponding patient in the session. In step 1004, the session data for multiple sessions is processed in parallel to generate processed session data for the multiple sessions. In step 1006, for each session in the multiple sessions, the processed session data for the session is analyzed based on the corresponding reference data to generate assessment results for the corresponding patient in the session.

[0318] In some embodiments, process 1000 further includes loading corresponding reference data for multiple sessions in parallel with processing session data for multiple sessions.

[0319] In some embodiments, the network-attached server includes multiple processing cores. Processing session data for multiple sessions in parallel may include, for example, using a first set of processing cores to process session data for multiple sessions in parallel, as shown in step 716 of Figure 7B, and using a second set of different processing cores to load corresponding reference data for multiple sessions. The number of first processing cores may be greater than the number of second processing cores. In some embodiments, analyzing processed session data for multiple sessions based on loaded corresponding reference data for multiple sessions may include using multiple processing cores, including a first set of processing cores and a second set of processing cores, as shown in step 718 of Figure 7B.

[0320] In some embodiments, analyzing processed session data from multiple sessions based on loaded corresponding reference data for multiple sessions includes at least one of the following: comparing the processed session data of a session with the corresponding reference data; using the corresponding reference data to infer assessment results for the corresponding patient from the processed session data; or using at least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model.

[0321] In some embodiments, the corresponding reference data includes historical eye-tracking data or results for patients of substantially the same age or condition as the corresponding patient. In some embodiments, process 1000 includes generating assessment results based on the corresponding patient's previous session data.

[0322] In some embodiments, for example, as shown in Figure 2C or Figure 7A, each session of a group of sessions is assigned its own container. Process 1000 may include processing session data for each session within each container and analyzing the processed session data of the session based on corresponding model data to generate assessment results for the corresponding patient in the session.

[0323] In some embodiments, while eye-tracking data is collected during a session, the eye-tracking data is associated with a predetermined list of visual stimuli presented to the patient, and the session data includes information related to the predetermined list of visual stimuli in the session.

[0324] In some embodiments, the process 1000 further includes, for example, as shown in Figure 7A, breaking down the eye-tracking data into multiple parts within each container based on information related to a given list of visual stimuli, with each part of the eye-tracking data associated with either the respective given visual stimuli or the corresponding calibration.

[0325] In some embodiments, processing session data for a session includes processing portions of eye-tracking data associated with each given visual stimulus based on information about each given visual stimulus. In some embodiments, process 1000 further includes recalibrating portions of eye-tracking data associated with each given visual stimulus within each container based on at least one portion of eye-tracking data associated with the corresponding calibration.

[0326] In some embodiments, the process 1000 further includes determining the calibration accuracy within each container using at least one portion of eye-tracking data associated with the corresponding calibration, and a plurality of predetermined locations where a plurality of calibration targets are presented in the corresponding calibration.

[0327] In some embodiments, receiving session data for multiple sessions includes, for example, receiving session data for multiple sessions from multiple computing devices associated with the corresponding entity via a web portal, as shown in Figure 2C.

[0328] In some embodiments, process 1000 further includes, for example, adding a file pointer to the session's session data into a processing queue to be processed in response to the receipt of the session's session data, as shown in Figure 7A. Process 1000 may further include storing the session's session data in a database using the file pointer to the session, and retrieving the session's session data from the database using the file pointer to the session.

[0329] In some embodiments, process 1000 further includes, for each entity, storing session data from one or more computing devices associated with the entity in a separate repository within the application data database, as shown, for example, in Figure 2E. Each repository can be isolated from one or more other repositories and may be inaccessible to one or more other entities. The application data database may be a NoSQL database.

[0330] In some examples, each repository for an entity may contain at least one of the following, as shown in Figure 2D: information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted with the entity, information about one or more patients associated with the entity, or historical information for each repository.

[0331] In some embodiments, process 1000 further includes dynamically adjusting the resources of the network-attached server based on the number of computing devices accessing the network-attached server, for example, as shown in Figure 2F. Process 1000 may further include replicating data from a first data center to a second data center and automatically diverting traffic to the second data center in response to a determination that the first data center is inaccessible.

[0332] In some embodiments, each of the first and second data centers includes at least one instance of a web portal, operator application, or application layer for data processing and data analysis accessible to an operator-side computing device, as shown, for example, in Figure 2G. The process may further include storing the same data in multiple data centers. The data may include application data for entities and information related to eye-tracking data.

[0333] In some embodiments, process 1000 further includes associating the generated assessment results with the corresponding patient in the session and generating an assessment report for the corresponding patient, as shown in step 724 of Figure 7B, for example.

[0334] In some embodiments, process 1000 further includes outputting assessment results or assessment reports to be presented in the user interface of an operator-side computing device, for example, via a web portal.

[0335] In some embodiments, as shown for example in Figure 8A, the assessment report includes at least one of the following: information about the corresponding patient, information about the entity performing the session for the corresponding patient, information about the accuracy in the session, information about session data collection, or assessment results for the corresponding patient. In some embodiments, the assessment results indicate the possibility that the corresponding patient has a developmental, cognitive, social, or mental disability or ability. For example, the assessment results indicate the possibility that the corresponding patient has or does not have autism spectrum disorder (ASD). In some embodiments, the assessment results include, for example, scores for one or more of each of the following: social disability index, verbal ability index, and nonverbal ability.

[0336] In some embodiments, the corresponding patient has an age range of 5 months to 7 years, including ages in the range of 5 months to 43 months or 48 months, ages in the range of 16 months to 30 months, ages in the range of 18 months to 36 months, ages in the range of 16 months to 48 months, or ages in the range of 16 months to 7 years.

[0337] Exemplary specific skill monitoring For example, with respect to Figure 8A, as described above, the diagnostic outcome can provide an overall diagnostic result (e.g., ASD or non-ASD), as well as scores and information on three severity indicators (e.g., social impairment, verbal ability, and nonverbal learning). Embodiments of this disclosure can provide much more detailed interactive reporting outputs that allow users to delve into behaviors and metrics for specific scenes or groups of scenes related to developmentally important skills, such as treatment-specific skill domains / skills, as illustrated with further detail in Figures 11-14, for example.

[0338] Figure 11 shows an example 1100 of a comparison between annotated video scenes 1120, information on typical viewing behavior groups 1130, and information on patient viewing behaviors 1140 for different specific skill areas 1110 according to one or more embodiments of the present disclosure. Information on typical viewing behavior groups 1130 may include a distribution map 1132 and exemplary highlighted video scenes 1134. The distribution map 1132 may be a splendor map. Information on patient viewing behaviors 1140 may include representative video scenes 1142 (which may be highlighted video scenes) and specific skill metrics (e.g., convergent viewing percentage or attention percentage) 1144.

[0339] A patient's developmental assessment may relate to one or more specific skill domains (or developmental concepts or skill categories). A skill domain may include one or more skills that can relate to one or more skills. A skill may be associated with one or more skill domains. Specific skill domains may include manding, listener response, turn-taking, joint attention, tact, or play. A specific skill domain may correspond to one or more therapies, and a therapy may be associated with one or more specific skill domains.

[0340] For example, the skill domain "Joint Attention" may include multiple skills, such as pointing at something, following where someone else is pointing, and / or watching someone else point. Another example is the skill domain "Manding," which may include, for example, pointing at something (while posing) and / or verbally requesting something. Another example is that pointing at something may be associated with the skill domains "Joint Attention" and "Manding."

[0341] For a session (e.g., a diagnostic session, a monitoring session, or a targeted monitoring session), the visual stimulus data collection playlist may include multiple videos (or movies), as described with respect to Figures 4A to 4J. A video may include multiple video scenes (moments or frames), such as the exemplary video scene 1120 shown in Figure 11. A video scene may relate to one or more skill domains or skills.

[0342] As an example, in exemplary video scene 1120a, boy A (on the right) extends his hand to boy B (on the left) and asks for a toy. Boy B, holding the toy, says no. While watching video scene 1120a, a typical (normal) child might look at boy A's hand and / or boy B's toy, while in contrast, a child with a developmental disorder might look at the surroundings, for example, a picture on the wall, or the feet of boy A or boy B. As another example, in exemplary video scene 1120b, girl C turns her head towards someone who is speaking and listens to it. While watching video scene 1120b, a typical (normal) child might look at girl C's head or eyes, while in contrast, a child with a developmental disorder might look somewhere else, for example, a nearby table.

[0343] Information on a typical viewing behavior group 1130 may include a distribution map 1132 and a highlight video scene 1134. The highlight video scene 1134 may be a representative video scene from several video scenes for the reference group. The distribution map 1132 (for example, map 1132a for video scene 1120a or map 1132b for video scene 1120b) shows the distribution of viewing areas (or viewing behaviors) of some individuals (e.g., children) in the reference group, obtained from analysis of eye-tracking data, for example, as described above. The distribution map 1132 may be a sampling map. The distribution map 1132 may be shown in color or grayscale. The higher the map value, the higher the normal level of the child. The reference group may include children with normal development and children with abnormal development (or developmental disorders). Children in the reference group may have similar ages, sexes, and / or other similar circumstances. Compared to the original video scene 1120a, the enhanced video scene 1134 (for example, enhanced video scene 1134a compared to video scene 1120a or enhanced video scene 1134b compared to video scene 1120b) enhances two regions that correspond to two areas seen by the majority of children in the reference group.

[0344] One or more specialist clinicians can examine each video scene in the video (for example, frame by frame or moment by moment) and annotate which skill domain and / or specific skill the video scene relates to. For example, a specialist clinician might annotate video scene 1120a as relating to the skill domain manding and / or the skill of requesting something by pointing, and video scene 1120b as relating to the skill domain listener response 1120b and / or the skill of responding to someone's utterance. In some cases, a specialist clinician might also annotate video scenes with respect to information about typical viewing behavior groups 1130, for example, distribution maps 1132 and modified video scenes 1134. In some examples, a specialist clinician might annotate videos relating to a specific skill domain, where a first set of video scenes relates to a first skill related to a particular skill domain, and a second set of video scenes relates to a second skill related to a particular skill domain. In some examples, a specialist clinician may annotate a first set of video scenes in a video as relating to a first specific skill domain, and a second set of video scenes in a video as relating to a second specific skill domain. The specialist clinician may further annotate individual video scenes within the first set of video scenes for each skill relating to the first specific skill domain, and individual video scenes within the second set of video scenes for each skill relating to the second specific skill domain. Each of the specialist clinician's annotations can be associated with a video scene and stored together with information on video scene 1120 and the typical viewing behavior 1130 of the reference group, and can be stored in a network-connected server, for example, the network-connected server 110 in Figure 1A, such as in the platform subsystem 112 in Figure 1A, or 220 in Figure 2A, or in a library or cloud storage.

[0345] If the area the patient views is closer to (or converges to) a highlighted area in map 1132 or a highlighted video scene 1134 for a particular skill area, this may indicate that the patient is more normal in that particular skill area. If the area the patient views is further away from (or diverges from) a highlighted area in map 1132 or a highlighted video scene 1134 for a particular skill, this may indicate that the patient may be abnormal in that particular skill. In some embodiments, a cutoff threshold, for example, a contour line around one or more highlighted areas in map 1132, for example, contour line 1133a in map 1132a or contour line 1133b in map 1132b, is determined to evaluate the patient's viewing behavior towards video scenes annotated with a particular skill area. If the area the patient views is within the contour line, for example, at a map value higher than the cutoff threshold, it may be determined that the patient is normal with respect to that particular skill area at that moment. If a patient's viewing area is outside the contour line, for example, at a map value lower than the cutoff threshold, it may be determined that the patient is abnormal at that moment, or, for example, abnormal over a certain number of moments, that they have a developmental disorder in a particular skill area. Note that whether a patient's viewing area is inside or outside the contour line corresponding to the cutoff threshold at a single moment does not indicate overall normality / abnormality for the patient, but rather only the normality / abnormality of their behavior at that moment (or frame). Overall abnormality / impairment may be indicated from an entire session, for example, if abnormality occurs over a threshold number of moments. The cutoff threshold and / or contour line may be determined, for example, by an AI or ML model analyzing the viewing behavior of a typical group based on the viewing behavior of known normal children, or by a specialist clinician while referring to the model's analysis.

[0346] In some embodiments, a skill-specific metric for assessing a patient's developmental impairment in a particular skill domain is defined as the percentage of moments determined to be normal by the patient, compared to the total number of moments related to that particular skill domain in the data collection playlist while the patient is watching. The skill-specific metric is sometimes called the convergent viewing percentage or attention percentage. As an example, in a session for patient Ben, there are 200 video scenes in the data collection playlist annotated using skill domain manding. Compared to the 200 video scenes, there are, for example, 100 moments (corresponding to 100 video scenes) where the patient is looking at the screen of the patient-side computing device, based on the patient's captured eye data during the session. That is, there may be another 100 moments where the patient is not looking at the screen of the patient-side computing device. Compared to the 100 moments, there are 29 moments where the area the patient is looking at is within the contour or has a map value higher than the cutoff threshold, i.e., the patient is normal in 29 moments. There may be another 71 moments where the patient is looking at a video scene but is not within the contour or has a map value lower than the cutoff threshold. Therefore, the skill-specific metric for the patient's manding is 29% for that session. For example, as shown in Figure 11, information on Ben's viewing behavior 1140 includes a representative video scene 1142a and a manding-specific metric 1144a that refers to Ben paying attention to 29% of the manding-related moments in the session. Similarly, information on Ben's viewing behavior 1140 includes a representative video scene 1142b and a skill-specific metric 1144b that refers to Ben paying attention to 37% of the listener response-related moments, indicating that Ben is determined to be normal in the skill domain listener response for 37% of the moments while he is viewing the video scene related to listener response. In some embodiments, as an alternative or addition, information on Ben's viewing behavior 1140 is compared with reference data from a reference group.For example, the attention percentage relative to a reference group can be used as a baseline to determine what attention percentage is normal, expected, or typical. The patient's attention percentage can be compared to the reference group's attention percentage, and information on the patient's behavior can include the results of the comparison as a metric to indicate expected / target / typical attention percentages and / or normative percentiles. For example, the attention percentage relative to the reference group in listener responses is 40%, and the patient's (e.g., Ben) attention percentage is 37%. Information on the patient's behavior can refer to "37% being within the 92.5 percentile for the listener response skill domain" or "37% being within the expected 90-94% attention percentage for typical age-matched peers for the listener response skill domain."

[0347] As will be explained with further detail below, video scenes with annotations created by specialist clinicians in light of the viewing behavior of the reference group enable the precise identification of specific skill areas / skills for the diagnosis and / or treatment of patients, the effective tailoring of data collection playlists for patients in selected skill areas / skills, and the monitoring of patient improvement or treatment effectiveness in selected skill areas / skills.

[0348] Figure 12A shows an example of an exemplary user interface 1200 presented on an operator device for session initiation according to one or more embodiments of the present disclosure. The operator device may be an operator-side computing device such as the computing device 140 in Figures 1A-1B or the operator-side computing device described with respect to Figures 4A-4J.

[0349] For example, an operator (e.g., any other person representing a medical assistant, medical professional, or healthcare provider) can log in to a web portal (e.g., web portal 222 in Figure 2A) running on a network-connected server (e.g., cloud server 110 in Figure 1A or platform subsystem 220 in Figure 2A) for device management, patient management, and data management. The operator may have corresponding user roles and permissions, as described in Figure 2D, for example. After the operator logs in to the web portal using the operator-side computing device, a user interface (UI) 1200 may be presented on the display screen of the operator-side computing device. The UI 1200 may be the user interface of an operator application (e.g., operator application 216 in Figure 2A) running on the network-connected server or the operator-side computing device.

[0350] As shown in Figure 12A, UI 1200 includes a menu 1210 with buttons "Home," "Patient," and "Appointment." By clicking a button, corresponding information (e.g., patient information, device information, or appointment information) may be presented within UI 1200. As described in Figures 4A-4B, the operator-side computing device can establish communication with the patient-side computing device (e.g., the patient-side computing device 130 in Figure 1A or the patient-side computing device described in Figures 4A-4J) for example, through a network-attached server. After communication is established, the operator can select a patient (or create a new patient) to start a session for the patient. If the operator chooses to start a session from the web portal, UI 1200 may be presented to the operator showing a session startup 1220 for session setup.

[0351] As shown in Figure 12A, below session initiation 1220, the operator can select session types 1230 that may include a diagnostic session 1232, a monitoring session 1234, and a targeted monitoring session 1236. The diagnostic session 1232 is configured to run the session, for example as shown in Figures 4C-4J, and generate a diagnostic report, for example as shown in Figure 8A. The diagnostic report may include a diagnostic outcome, for example, whether the patient is ASD or not, and / or scores for three indicators (social impairment indicator, verbal ability indicator, and nonverbal ability indicator). The monitoring session 1234 is configured to monitor behavioral (or performance) changes across a series of sessions for an existing patient by running the session, for example as described in Figures 4C-4J, and generate a monitoring report, for example as described with further detail in Figures 13A-13D. In some examples, as shown in Figure 12A, for instance, the UI 1200 may indicate the time (e.g., date) when the last run was performed for monitoring session 1234 and / or diagnostic session 1232.

[0352] In some embodiments, the diagnostic session 1232 and the monitoring session 1234 have the same data collection playlist of visual stimuli. In some embodiments, the monitoring session 1234 may have a different data collection playlist of visual stimuli from the diagnostic session 1232. The monitoring session 1234 may run a default playlist, which may be the same as the playlist run in the previous session for the patient. The playlist run in the previous session can be customized for one or more specific skill areas, for example, the previous session may be a targeted monitoring session.

[0353] If target monitoring session 1236 is selected, when the operator clicks the “Next” button 1238 in UI 1200, a window 1250 may be prompted on UI 1200 for the operator to select a set of skill areas that the operator wishes to target, as shown in Figure 12B, for example. Window 1250 may be overlaid on user interface 1200, be alongside user interface 1200, or have an overlap with user interface 1200. User interface 1200 may be changed to the new window 1250.

[0354] The set of skill areas may include, but is not limited to, manding, listener response, turn-taking, joint attention, tact, and play. The default selection may be any skill areas selected in a previous target monitoring session. The network-connected server may recommend consistent skill area selections across multiple consecutive monitoring sessions. In some cases, the network-connected server may remember previously selected target skill areas and / or personalized playlists, so that the operator can repeat the same target monitoring session in subsequent sessions without re-selecting target skill areas, if desired. Window 1250 may also present a note, “Automatically select from previous target monitoring session,” which the operator can click to automatically select target skill areas that should be the same as those in a previous target monitoring session. The operator can choose to retain the same selected target skill areas or change one or more target skill areas. In some embodiments, there is a maximum number of target skill areas to be set for selection, for example, up to four. This maximum number may be determined, for example, by the length of the videos in the data collection playlist. After the target skill area is selected or confirmed, the operator can click button 1260 in window 1250 to run a session for the target skill area.

[0355] Based on selected target skill areas, personalized video playlists can be constructed and valued for those target skill areas by, for example, including and prioritizing videos that are determined or known to best monitor those target skill areas. A network-attached server can optimize the playlist to maximize video content relevant to the target skill areas. In some embodiments, the video playlist is rearranged to place videos (or video scenes) annotated in relation to the target skill areas at the beginning of a session where the patient is likely to have more attention and should watch the video. In some embodiments, new videos that are particularly valued for one or more selected skill areas are added to the playlist. In some embodiments, videos unrelated to the selected skill areas are reduced or removed from the playlist to maintain a reasonable video length for the session. In some embodiments, visual scenes related to one or more selected target skill areas form a weighted correlation order to one or more selected target skill areas. In some embodiments, only visual scenes related to one or more selected target skill areas are selected in the data collection playlist.

[0356] The video playlist may be personalized, for example, by a network-attached server or an operator-side computing device or a patient-side computing device, when a target skill area is selected in window 1250 or when button 1260 is clicked to run a session for a target skill area. In some embodiments, the network-attached server receives input from the operator via a web portal, personalizes the video playlist, and sends the personalized playlist information (e.g., the sequence of videos in the playlist) to a patient-side computing device which may be configured to adjust the playlist according to the personalized playlist information. In some cases, the patient-side computing device may download new videos from the network-attached server if no new videos have been previously installed on the patient-side computing device. In some embodiments, commands may be sent from the network-attached server to a patient-side computing device which may be configured to personalize the playlist on the patient-side computing device based on the command.

[0357] Figures 13A to 13D show a user interface for a user to review session information about a user device through a web portal (e.g., web portal 222 in Figure 2A) of a network-connected server (e.g., cloud server 110 in Figure 1A or platform subsystem 220 in Figure 2A). The user may be, for example, a healthcare provider, a clinician, or a patient's guardian, or any other suitable person with permission or authority to access patient information. The user device may be any suitable computing device, which may be the same as or different from the operator-side computing device 140 in Figures 1A to 1B or the operator-side computing device described in Figures 4A to 4J.

[0358] Figure 13A shows an example of an exemplary user interface 1300 for viewing session information about a user device, according to one or more embodiments of the present disclosure. The user interface 1300 includes a menu 1302 (for example, menu 1210 in Figure 12A) with buttons “Home,” “Patient,” and “Appointment.” By clicking one of the buttons, the corresponding information (for example, patient information, device information, or appointment information) may be presented within the user interface 1300. The user can select a patient to view patient information. The user interface 1300 also includes another menu 1304 with buttons “Information,” “Session,” and “History.” By clicking one of the buttons, the corresponding information (for example, patient or appointment information, session information, or history information) may be presented within the user interface 1300.

[0359] For example, by selecting "Session" from menu 1304, session information 1306 associated with the patient is presented in the user interface 1300. The session information 1306 includes session background information 1308, which may include session date, patient name, session age (in months) indicating how long the session lasted, session status (uploaded or not uploaded), whether the quality check was passed (yes or no), device (which patient-side computing device is used to capture session data), and / or operator (who is using the operator-side computing device to run the session).

[0360] In addition to the session background information 1308, the session information 1306 further includes a button 1310 for reviewing session results (for example, as described in detail in Figures 8A, 8B-8C, 13B-1, and / or 13B-2), a button 1312 for customizing diagnostic / monitoring reports (for example, as described in detail in Figure 13C), and a button 1314 for launching an interactive results dashboard (for example, as described in detail in Figure 13D).

[0361] Figure 13B-1 illustrates an exemplary portion 1320 of an assessment report, showing a comparison between annotated video scenes, information on typical viewing behavior groups, and information on patient viewing behaviors for different specific skill areas, according to one or more embodiments of the present disclosure. Figure 13B-2 illustrates another exemplary portion 1330 of an assessment report, showing monitoring of treatment-specific skills in section 1332 and information on characteristic skills in section 1334, according to one or more embodiments of the present disclosure.

[0362] In some embodiments, the evaluation report is a diagnostic report that may include information presented in exemplary section 1320 and / or in Figures 8A, 8B, and / or 8C. As described above, the diagnostic report may be generated by performing a diagnostic session as described in Figure 12A. In some embodiments, the evaluation report is a monitoring report that may include information presented in exemplary section 1320, exemplary section 1330, and / or in Figures 8A, 8B, and / or 8C. As described above, the monitoring report may be generated by performing a monitoring session or a targeted monitoring session as described in Figures 12A-12B.

[0363] In some embodiments, an exemplary portion of the evaluation report 1330, as shown in Figure 13A, for example, may include an introduction section 1322 and a chart section 1324. The introduction section 1322 describes the information in the chart section 1324, including, for example, "the individual's viewing behavior metrics quantify how much of their viewing behaviors, aligned with age, were expected viewing behaviors during moments identified by the specialist clinician as being related to a specific skill area..."

[0364] Chart section 1324 may be similar to Example 1100 in Figure 11, including relevant skill areas, exemplary video scenes, information on typical group viewing behaviors (distribution map and highlighted video scenes), and information on patient viewing behaviors (highlighted video scenes and statistical scores or convergent viewing percentages). Relevant skill areas may be automatically selected by a network-connected server, or selected for skill areas with the largest amount of reliable data or the most widely and commonly requested skills / skill areas, or selected for skill areas with particularly high, low, or representative scores, or selected in previous assessment reports for the patient, or selected when initiating a targeted monitoring session, or a combination thereof. For example, as shown in Figure 13B-1, chart section 1324 includes information on the skill areas “Munding,” “Listener Response,” and “Joint Attention,” which may be annotated for the specialist clinician’s video scenes. It shows that patient Ben focused on 29% of the moments related to manding, 37% of the moments related to listener response, and 32% of the moments related to joint attention.

[0365] If a patient has had a previous monitoring session, an exemplary portion of the assessment report 1330 may show the change from the previous session. As shown in Figure 13B-2, the exemplary portion 1330 includes a monitoring section 1332 showing changes in relevant skill areas, including a chart showing the convergent viewing percentages that change over a series of sessions for each relevant skill area (e.g., manding, listener response, and joint attention). The monitoring section 1332 may refer to a comparison of the convergent viewing percentages between the current section and the previous section, for example, Ben is paying attention to 29% of manding-related moments, an increase from the previous session by 21%, Ben is paying attention to 37% of listener response-related moments, no change from the previous session, and Ben is paying attention to 14% of joint attention-related moments, an increase from the previous session by 13%. In this way, the user can easily determine whether the patient has had any improvement in each of the relevant skill areas and / or whether the current treatment is effective or beneficial. Exemplary section 1330 may also include information 1334 about characterized skills / skill areas, which may include definitions of the relevant skill areas (e.g., manding, listener response, and joint attention).

[0366] If the user selects button 1310 to review results within the user interface 1300, a default report with automatically selected skill areas, such as the assessment report described in Figures 8A-8C, 13B-1, and / or 13B-2, may be provided to the user. In contrast, if the user selects button 1312 to view a customized report rather than generating a default report, the network-connected server may present a new window 1340, such as shown in Figure 13C, for the user to select target skill areas (e.g., manding, listener response, joint attention, and play) to customize the assessment report. There may be a maximum number of target skill areas (e.g., up to four) to be selected for customization. After the user has selected the target skill areas, the user can click the "View Custom Report" button 1342 to generate a customized report. The customized report may be similar to the assessment report shown in Figures 13B-1 and / or 13B-2. The relevant skill areas in the customized report are the target skill areas selected through window 1340. In some examples, when a user selects a target monitoring session, the network-attached server (for example, an operator application running on the network-attached server) can automatically customize the monitoring report for the target monitoring session to select the same target skill areas that are selected for the playlist for the target monitoring session. The new window 1340 can be overlaid on user interface 1300, be alongside user interface 1300, or have an overlap with user interface 1300. User interface 1300 can be changed to the new window 1340.

[0367] Referring again to Figure 13A, if the user selects button 1314 to launch the interactive results dashboard, an exemplary interactive results dashboard 1360, such as shown in Figure 13D, may be presented within the user interface 1350 on the user device's display screen. The dashboard 1360 may include a section 1362 for the user to select a target skill area (e.g., manding) to interact with, a section 1364 for the user to select a specific session from a series of consecutive sessions, and a section 1366 showing, for example, moment-by-moment (or frame-by-frame) the patient's viewing behavior compared to the viewing behavior of a reference group. For example, section 1366 could show a comparison between a highlighted video scene 1370 of the reference group (e.g., highlighted video scene 1134 in Figure 11) and a highlighted video scene 1372 of the patient (e.g., highlighted video scene 1142 in Figure 11). Section 1366 may also include a playback sliding bar 1368 that allows the user to watch the video or pause or select a specific moment / frame (still image) to compare or view. Section 1366 may also include percentage information showing the convergence of the current session and previous sessions.

[0368] Figure 14 is a flowchart of an exemplary process 1400 for managing specific skills for developmental disability assessment, according to one or more embodiments of the present disclosure. Process 1400 may be executed by a network-attached server, which may be a cloud server in a cloud environment, for example, a cloud server 110 in Figure 1A or a cloud server as described in Figures 2A–2G. For example, the network-attached server may include a platform, for example, 112 in Figure 1A or 220 in Figures 2A–2G, and a data pipeline system, for example, 114 in Figure 1A or 230 in Figures 2A–2G. The platform may include a web portal (for example, 222 in Figures 2A–2G), an application data database (for example, 224 in Figures 2A–2G), and a database (for example, 226 in Figures 2A–2G). A data pipeline system may include one or more data processing modules (e.g., 232 in Figures 2A to 2G) and one or more data analysis modules (e.g., 234 in Figures 2A to 2G).

[0369] In step 1402, a request is received by a network-connected server. The request seeks the patient's assessment results based on the patient's session data. Session data is collected during the presentation of a data collection playlist of visual stimuli to the patient during the session. At least one visual scene in the data collection playlist is annotated using at least one of several skill domains associated with the visual scene in the data collection playlist.

[0370] In step 1404, the patient's assessment results are output by a network-connected server. The assessment results include, for each of one or more specific skill areas among multiple skill areas, patient behavioral data related to a specific skill area in the session, with each of those moments corresponding to a visual scene in the data collection playlist.

[0371] In some embodiments, for example as illustrated in Figure 11, the behavioral data includes attention percentage, defined as the ratio between the number of moments in which the patient pays attention to the relevant scene content and the total number of moments in which the patient is looking at the visual stimulus.

[0372] In some embodiments, session data includes patient eye-tracking data. Based on the patient's eye-tracking data, a network-connected server can determine the total number of moments the patient is looking at a visual stimulus, and based on the patient's eye-tracking data, it can determine the number of moments the patient is focusing on relevant scene content.

[0373] In some embodiments, process 1400 further includes determining that at a given moment in a session, the patient's area of ​​attention is within a predetermined region, and that this moment is one of the many moments in which the patient focuses on relevant scene content. The predetermined region corresponds to a contour line (e.g., contour line 1133a or 1133b in Figure 11) of a distribution map of reference group behavioral data (e.g., map 1132 in Figure 11). The reference group behavioral data may be based on reference session data collected during the presentation of a data collection playlist of visual stimuli to each person in the reference group. The contour line values ​​of the distribution map may correspond to a cutoff threshold.

[0374] In some embodiments, the assessment results further include a distribution map of behavioral data for a reference group, as shown in Figure 13B-1, for example. The assessment results may further include, for each of one or more specific skill domains, at least one of the following: a representative visual scene, a representative visual scene highlighting one or more areas of attention within a given domain for the reference group, or a representative visual scene highlighting the patient's areas of attention during the session.

[0375] In some embodiments, for example as shown in Figure 13B-2, the assessment results further include, for each of one or more specific skill areas, at least one of the following: behavioral data from one or more previous sessions of the patient, or a comparison between the session behavioral data and the patient's behavioral data from one or more previous sessions. The assessment results may include graphs showing the session behavioral data and the patient's behavioral data from one or more previous sessions for each of one or more specific skill areas.

[0376] In some embodiments, process 1400 further includes selecting one or more specific skill areas from a plurality of skill areas for the patient's assessment results. Selecting one or more specific skill areas from a plurality of skill areas includes at least one of the following: selecting a specific skill area from the plurality of skill areas for which reliable data exists; selecting a widely requested skill area from the plurality of skill areas; selecting a skill area from the plurality of skill areas for which a particularly high, low, or representative score exists, where the score represents the patient's attention percentage; selecting a skill area previously selected as a target skill area in a session; selecting a skill area selected to customize the assessment results; or selecting a skill area previously selected in a previous session of the patient or in a previous assessment result of the patient.

[0377] In some embodiments, for example as shown in Figure 12A, process 1400 includes receiving a session request for initiating a session through a web portal on a network-attached server, presenting a list of sessions on the user interface of the web portal (for example, user interface 1200 in Figure 12A), and receiving a session selection from the list of sessions on the user interface.

[0378] In some embodiments, for example as shown in Figure 12B, process 1400 further includes, in response to receiving a session selection, popping up a window (e.g., window 1250 in Figure 12B) for selecting a target skill area from a plurality of skill areas enumerated in the window, receiving user input for selecting one or more target skill areas in the window, and executing the session based on the selected one or more target skill areas. The selected one or more target skill areas may include one or more specific skill areas in the assessment results.

[0379] In some embodiments, the network-connected server adjusts the data collection playlist of visual stimuli based on one or more selected target skill areas by, for example, prioritizing visual scenes annotated to monitor one or more selected target skill areas in the data collection playlist, increasing the value of additional visual scenes related to one or more selected target skill areas in the data collection playlist, or reducing or removing visual scenes not related to the selected target skill areas in the data collection playlist.

[0380] In some embodiments, prioritizing visual scenes annotated to monitor one or more selected target skill areas includes at least one of the following: placing visual scenes annotated to monitor one or more selected target skill areas at the beginning of the data collection playlist; arranging visual scenes related to one or more selected target skill areas in order of their weighted correlation values ​​to the selected target skill areas; or selecting only visual scenes related to one or more selected target skill areas within the data collection playlist. User input may be received from an operator-side computing device communicating with a network-attached server via a web portal. The network-attached server can establish communication between the operator-side computing device and the patient-side computing device via the network-attached server and can transmit information about the visually stimulated data collection playlist to the patient-side computing device, so that the visually stimulated data collection playlist is presented to the patient on the display screen of the patient-side computing device during the session. The operator-side computing device may be the computing device 140 in Figures 1A to 1B or the operator-side computing device relating to Figures 4A to 4J and / or Figures 12A to 12B and Figures 13A to 13D. The patient-side computing device may be the computing device 130 in Figure 1A or the patient-side computing device relating to Figures 4A to 4J.

[0381] In some embodiments, process 1400 further includes receiving patient session data from a patient-side computing device for the patient once the session is completed. The patient session data is collected by the patient-side computing device during the session and generates patient behavioral data by processing the patient session data based on reference data of a reference group and one or more specific skill areas.

[0382] In some embodiments, process 1400 further includes loading reference data for reference groups from, for example, a library or cloud storage. The reference data may be based on reference session data collected during the presentation of a visual stimulus data collection playlist, and behavioral data for reference groups based on one or more specific skill areas.

[0383] In some embodiments, for example, as shown in Figures 11 and 13B-1, the reference data includes, for each of one or more specific skill domains, at least one of the following: a specific visual scene associated with the specific skill domain, where each specific visual scene highlights one or more attention areas of the reference group; or a distribution map of behavioral data of the reference group for each of the specific visual scenes.

[0384] In some embodiments, the reference data includes contour lines in a distribution map that represent thresholds for determining whether the patient will focus on the relevant scene content of a particular visual scene, for each of one or more specific skill areas and for each of a particular visual scene. Process 1400 may further include at least one of the following: determining that the patient will focus on the relevant scene content of a particular visual scene if the patient's area of ​​focus lies within a predetermined region corresponding to the contour line; or determining that the patient will not focus on a particular visual scene if the patient's area of ​​focus lies outside the predetermined region.

[0385] In some embodiments, patient behavior data includes attention percentage, defined as the ratio between the number of moments in which the patient focuses on relevant scene content and the total number of moments in which the patient views the visual stimulus. Process 1400 may further include determining that at a given moment in a session, the patient's attention area is within a predetermined region, and determining that the moment is one of the moments in which the patient focuses on relevant scene content. In some embodiments, patient behavior data includes the result of a comparison between the patient's attention percentage and a reference group's threshold attention percentage. For example, the result of the comparison may include at least one of the following: the ratio between the patient's attention percentage and the reference group's threshold attention percentage, or the relationship between the patient's attention percentage and the reference group's threshold attention percentage.

[0386] In some embodiments, receiving a request involves receiving user input from a user device on a user interface of a web portal of a network-attached server (for example, user interface 1300 in Figure 13A), where the user input indicates the request, and the user interface is presented on the display screen of the user device.

[0387] In some embodiments, for example as shown in Figure 13A, the user interface includes at least one of the following: a first user interface element for viewing a default assessment report (e.g., button 1310 in Figure 13A), a second user interface element for customizing the assessment report (e.g., button 1312 in Figure 13A), or a third user interface element for launching an interactive dashboard (e.g., dashboard 360 in Figure 13D) using the assessment results (e.g., button 1314 in Figure 13A).

[0388] In some embodiments, for example as shown in Figure 13C, the process 1400 further includes, in response to a selection of a second user interface element, popping up a window on the user interface for selecting target skill areas in an assessment report; receiving a second user input for selecting one or more target skill areas in the window; and generating an assessment report based on the selected one or more target skill areas, wherein the one or more target skill areas include one or more specific skill areas in the assessment result.

[0389] In some embodiments, for example as shown in Figure 13D, process 1400 further includes presenting an interactive dashboard within the user interface in response to a selection for a third user interface element. The interactive dashboard may include a subwindow for selecting one of a list of skill areas for interaction. In response to receiving a selection of a specific target skill area from the list of skill areas, process 1400 may present, over a series of consecutive sessions, a change in the patient's attention percentage for a specific target skill area, a change in the ratio between the patient's attention percentage and the reference group's threshold attention percentage, a change in the relationship between the patient's attention percentage and the reference group's threshold attention percentage, or, for each of a set of visual scenes associated with a specific target skill area, at least one of a first scene highlighting one or more attention areas of reference groups in the visual scene, and a second scene highlighting the patient's attention areas in the visual scene.

[0390] In some embodiments, multiple visual scenes are overlaid on each other within the user interface, and the interactive dashboard includes a sliding user interface element (for example, a sliding bar 1368 in Figure 13D) for selecting each of the multiple visual scenes.

[0391] In some embodiments, the network-connected server is configured to store annotation data for visual scenes in a visual stimulus data collection playlist, wherein the annotation data specifies each particular skill area associated with the visual scene, and to store reference data for reference groups, wherein the reference data is based on behavioral data based on reference session data collected during the presentation of the visual stimulus data collection playlist. The annotation data and reference data may be stored in a library or cloud storage.

[0392] In some embodiments, session data includes at least one of the following: eye-tracking data collected by an eye-tracking device assembled within a patient-side computing device communicating with a network-connected server (e.g., eye-tracking device 134 in Figure 1A), or image data, audio data, or video data collected by one or more recording devices (e.g., recording device 138 in Figure 1A). One or more recording devices may be assembled within or outside the patient-side computing device (e.g., as shown in Figure 1A).

[0393] In some embodiments, the operator-side computing device is configured to access a web portal on a network-connected server and receive user input on the web portal's user interface, which is used to request patient assessment results based on patient session data, and the session data is collected during the presentation of a data collection playlist of visual stimuli to the patient during the session. Each visual scene in the data collection playlist is annotated with at least one of several skill domains associated with the visual scene in the data collection playlist. The operator-side computing device may be configured to present the assessment results on the operator-side computing device's display screen. The assessment results may include patient behavioral data for each of one or more specific skill domains from the several skill domains, relating to moments in the session associated with a particular skill domain, each of which corresponds to a visual scene in the data collection playlist.

[0394] In some embodiments, the patient-side computing device is configured to initiate a session for the patient by establishing communication with the operator-side computing device and the patient-side portable tablet computing device, wherein the patient-side portable tablet computing device is integrated with an eye-tracking device (e.g., eye-tracking device 134 in Figure 1A), and to present the patient with a series of visual scenes from a data collection playlist of visual stimuli on the screen of the patient-side portable tablet computing device while collecting patient eye-tracking data using the eye-tracking device, wherein each visual scene in the data collection playlist is annotated with at least one of a plurality of skill areas associated with the visual scene in the data collection playlist, and to transmit the session data of the session to a network-attached server, wherein the session data includes the patient's eye-tracking data collected in the session. The data collection playlist may include visual scenes associated with one or more specific skill areas from a plurality of preferred skill areas in the data collection playlist.

[0395] In some embodiments, the patient-side computing device is configured to collect at least one of image data, audio data, or video data collected by one or more recording devices while visual scenes from a visual stimulus data collection playlist are presented sequentially. One or more recording devices may be assembled inside or outside the patient-side computing device. Session data may include at least one of image data, audio data, or video data.

[0396] Exemplary management of treatment plans The assessment system disclosed herein may be configured for the assessment of developmental disorders, such as autism spectrum disorder (ASD). The assessment system may be system 200 in Figures 2A–2G and may be implemented in environment 100 in Figure 1A. The assessment system may include a cloud server (or network-connected server), such as cloud server 110 in Figures 1A–1D or the cloud server relating to Figures 2A–2G. In some examples, the assessment system disclosed herein is represented as EarliPoint.

[0397] As described in detail above (for example, in Figures 1A to 14), the assessment system can perform one or more assessment sessions (e.g., a diagnostic session or a monitoring session) to assess a patient's developmental disorder and generate a corresponding assessment report (for example, as shown in Figures 8A to 8C, 13A to 13D, or 16A to 16F). As described in further detail in Figures 15A to 15H, the assessment system can also generate a normative specific treatment plan for the patient based on at least one of treatment plan data, patient data, or reference data.

[0398] In some embodiments, the cloud server includes a cloud platform (e.g., cloud platform 112 in Figure 1A or platform subsystem 220 in Figures 2A-2G) and a data pipeline system (e.g., data pipeline system 114 in Figure 1A or data pipeline subsystem 230 in Figures 2A-2G). The cloud platform may be configured to provide a web portal, store application data related to the therapist, and store data, such as raw eye-tracking data, processed data, analysis results and / or diagnostic results, and / or treatment plan data. The data pipeline system may be configured to perform data processing and data analysis. As illustrated with detail in Figures 6 and 7A-7B, the cloud server can automatically receive, process, and analyze session data from multiple computing systems and can process and analyze session data from several sessions from numerous computing systems in parallel.

[0399] In some embodiments, during processing and analysis processes (e.g., by a data pipeline system), individual patient data may be compared (e.g., using an artificial intelligence algorithm or model) with reference data previously generated from historical eye-tracking data of patients belonging to the same or similar group and / or having similar age, background, and / or symptoms. The results of the comparison may include, but are not limited to, a diagnosis of a neurodevelopmental disorder, including normative recommendations for ASD, measures of the patient's developmental / cognitive function, and / or treatment plans. Alternatively or additionally, the collected data may be compared and / or reviewed for a given patient over multiple sessions (and over a predetermined time period) to identify potential changes in visual fixation (e.g., decreased visual fixation). The results may be condensed into a diagnostic report for use by the patient's physician. Once the diagnostic results are ready, the evaluation system can transfer them to a computing device for review (e.g., the operator-side computing device 140 in Figure 1), and the diagnostic results may be presented on the user interface of the operator-side computing device 140, as described, for example, in Figures 8A-8C and 13B-1, 13B-2, 13C, and 13D, and Figures 16A-16F. In some embodiments, a large amount of model data, including data relating to patients of similar age, similar background, and / or similar circumstances, may be used along with processed session data for patients to generate diagnostic results and / or normative treatment plans for patients, using comparison or inference via artificial intelligence (AI) models such as statistical models, algorithms, machine learning models, or artificial neural network models, which can significantly increase the accuracy of the diagnostic results.

[0400] For patients with developmental disorders (e.g., ASD), different treatment plans may be tailored to the patient depending on the patient's age, background, circumstances, assessment reports, developmental stage, demographics, geography, and / or available resources / practitioner. Furthermore, the same patient may experience different treatment plans. Below, exemplary treatment plans are described, which include, but are not limited to, at least one of the following: EarliPoint, Early Initiation Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Joint Attention Symbolic Play Involvement Regulation (JASPER), or Project ImPACT. It should be noted that the above treatment plans are examples only, and any other suitable treatment plans may be implemented in this disclosure.

[0401] The EarliPoint assessment system evaluates a patient's developmental disability using three developmental disability indices, including social disability indices, verbal ability indices, and nonverbal learning indices (for example, as shown in Figures 8A and 16A-16E), as well as specific skill domains, including manding, listener response, turn-taking, responding to joint attention or joint attention (RJA), tact, and / or play (for example, as shown in Figures 11-14 and 16F). As mentioned above, skill domains can include several specific skills (for example, 10, 20, 50, or 100 or more). The EarliPoint assessment system can generate a treatment plan based on the scores of the developmental disability indices and the skill domains / skills.

[0402] The Early Initiated Denver Model (ESDM) is a behavioral therapy for children with autism between the ages of 12 and 48 months. It is based on the methods of applied behavior analysis (ABA). Parents and therapists use play to build positive and enjoyable relationships. Through play and collaborative activities, children are encouraged to improve their language, social, and cognitive skills. ESDM therapy is based on a normal understanding of early childhood learning and development, with instruction given during natural play and daily activities, and play used to encourage dialogue and communication, with a focus on building positive relationships. ESDM can help children advance in their social, language, and cognitive skills.

[0403] Early Social Interaction (ESI) provides early intervention for young children with autism spectrum disorder (ASD) and their families. ESI educates parents on how to support their child's social communication, emotional regulation, and play in everyday activities and settings.

[0404] Discrete Trial Training (DTT) is an intensive therapeutic program used to help children with developmental disorders such as autism. DTT involves training children with autism in various skills they would not otherwise acquire. This method focuses on teaching skills through a step-by-step process rather than teaching all the desired skills at once. DTT uses a basic process for teaching a new skill or behavior and repeats it until the child learns. This process involves giving commands such as "Pick up the cup." If necessary, the child carries out the command using physical or verbal prompts, such as pointing to the cup.

[0405] JASPER is a therapeutic approach based on a combination of developmental and behavioral principles. This intervention model targets the foundations of social communication (joint attention, imitation, and play), employs naturalistic strategies to increase the rate and complexity of social communication, and includes parents and teachers as implementers of the intervention to promote generalization across settings and activities and to ensure maintenance over time.

[0406] Project ImPACT is a parent-mediated intervention for young children with autism spectrum disorder (ASD) and associated social communication delays. Parent-mediated means that Project ImPACT coaches instruct parents on techniques to be used with their children. Project ImPACT coaches use systematic commands to enhance parental responsiveness to their child's behavior and instruct parents on how to use prompts and reinforcements to teach their children to use new communication, imitation, and play skills within guided conversations. It is a naturalistic developmental behavioral intervention (NDBI), a newer class of medical intervention known in the fields of developmental science, communication science, and applied behavior analysis (ABA). Project ImPACT can be implemented in individual or group coaching models and can be adapted for telemedicine.

[0407] Different treatment plans may have different names for different skill domains, different prompting techniques, different treatment / training materials, different reinforcement techniques, and / or different data collection methods. For example, discrete trial training (DTT) is more rigorous in its structure, materials, locations, and delivery methods. Naturalistic developmental behavioral interventions (NDBI), such as ESDM and JASPER, maintain the core principles of ABA but are less rigorous, and their locations and materials are more naturalistic and flexible. Within both of these ABA categories, there are also different subgroups that use their own specific vocabulary, prompting structures, data collection sheets, etc. Some of the main differences between therapeutic approaches may include a) the precise names of skills (e.g., "demanding" vs. "manding," "listener response" vs. "direction following"), b) prompting techniques (e.g., most to least, least to most, when and how prompts should be adjusted, and the level of verbal or nonverbal communication in prompts), c) materials (e.g., intentional or naturalistic, pre-ordered for use in the therapeutic setting, or based on what is in the child's environment), and d) reinforcement techniques, which may include physical rewards, intrinsic rewards, screen time, food, etc.

[0408] For illustrative purposes, Table 1 shows two exemplary treatment curriculum plans (ESDM and DTT) for the same skill area. Note that clinicians may have flexibility in creating patient-specific curriculum plans.

[0409] [Table 1A] [Table 1B]

[0410] As illustrated with further detail in Figures 15A to 15H, embodiments of the present disclosure provide techniques for integrating treatment data, patient data, and reference data (e.g., different treatment plan formats) to generate a specific treatment plan for a patient.

[0411] Figures 15A to 15H show examples of exemplary user interfaces presented on a computing device according to one or more embodiments of the present disclosure. The computing device may be an operator device or operator-side computing device, such as the computing device 140 in Figures 1A to 1B or the operator-side computing device described with respect to Figures 4A to 4J. For example, an operator (e.g., any other person representing a medical assistant, medical professional, or healthcare provider) may log in to a web portal running on a cloud server (e.g., web portal 222 in Figure 2A) for device management, patient management, and data management. The operator may have corresponding user roles and permissions, as described, for example, in Figure 2D. After the operator logs in to the web portal using the operator-side computing device, a user interface (UI) may be presented on the display screen of the operator-side computing device. The UI may be the user interface of an operator application running on the cloud server or operator-side computing device (e.g., operator application 216 in Figure 2A).

[0412] Figure 15A shows an example of an exemplary user interface 1500 presented on a computing device when the data aggregator application is running on a cloud server. The data aggregator application may be configured to connect to one or more third-party tools to ingest patient treatment data, including data from EHR (Electronic Health Record) / EMR (Electronic Medical Record) and ABA (Applied Behavior Analysis) practice management tools, as well as optionally reference patient data (e.g., by parsing), and to ingest patient (and / or other patients') treatment plans, goals, behavioral press, patient responses over time, and other relevant clinical or treatment data (e.g., by parsing). The data aggregator application may be configured to be combined with assessment data by an evaluation system to build a large and unique data repository of clinical treatment and patient trajectories.

[0413] As shown in Figure 15A, the data aggregator application allows the operator to upload the patient's most recent treatment plan. The treatment plan may be in a format different from the default treatment plan format associated with the evaluation system (e.g., EarliPoint). As shown in Figure 15A, the user interface 1500 includes a selection element 1502 for the operator to select the plan format of the treatment plan to be uploaded, a dropdown list 1504 showing a list of treatment plan formats (e.g., EarliPoint, ESDM, ESI, DTT, JASPER, and Project ImPACT), and an upload document element 1506 for uploading the treatment plan document. The data aggregator application can upload the treatment plan document from a repository that stores patient data in a cloud server (e.g., in platform 220 in Figure 2A), from a computing device, or from a storage medium (e.g., USB, disk, or storage disk) coupled to a computing device.

[0414] After the treatment plan document is uploaded, the data aggregator application can automatically parse the document to extract relevant information based on the treatment plan's format. As mentioned above, different treatment plans may have different names for the same skill domain. The data aggregator application can convert different skill domain names to the same skill domain name used in the evaluation system (e.g., the default skill domain name). For example, the skill domain name "Requesting" in ESDM may be converted to "Manding" in EarliPoint. The data aggregator application can also summarize the treatment plan in relation to the skill domains. For example, how many hours per week are each treatment-specific skill domains trained, and what is the impact of the treatment plan on the treatment-specific skill domains. In some examples, the impact of the treatment plan may be determined based on the percentage of patients that converge.

[0415] Figure 15E shows a breakdown graph 1540 illustrating exemplary treatment-specific skill area engagement in a patient's treatment plan. Exemplary treatment-specific skill areas include manding, play, tacting, and RJA. The treatment plan may include (e.g., per week) time spent on these exemplary treatment-specific skill areas after the final assessment report has been generated for the patient. The engagement may be determined based on the percentage of time allocated to the corresponding treatment-specific skill area relative to the total time in the treatment plan. For example, graph 1540 shows breakdown percentages for different treatment-specific skill areas in a treatment plan, including 41% for play, 25% for tacting, 17% for manding, and 17% for RJA.

[0416] As ...

Claims

1. A system using at least one portable computing device having eye-tracking capabilities, A portable eye-tracking console comprising a display screen and an eye-tracking device, wherein the eye-tracking device is mounted adjacent to the display screen such that both the display screen and the eye-tracking device are directed toward the patient, and the eye-tracking device is configured to collect the patient's gaze tracking coordinate data while a predetermined sequence of a stimulus video is presented on the display screen during a session, A portable computing device having a touchscreen display interface, which is separated from the portable eye-tracker console and can be carried to various locations relative to the portable eye-tracker console, A network-connected server includes a web portal that wirelessly receives session data from the portable eye-tracker console and exports evaluation results, including a graphical correlation of numerical impairment index scores correlated with a reference assessment scale. Equipped with, The network-connected server is configured to wirelessly connect to both the portable eye-tracker console and the portable computing device, and as a result, the portable computing device, which wirelessly communicates with the portable eye-tracker console via the network-connected server to control the activation of the session, presents the predetermined sequence of the stimulus video on the display screen of the portable eye-tracker console, and then the portable eye-tracker console wirelessly communicates the session data, including the eye-tracking coordinate data which has a timestamp relationship with the information of the predetermined sequence of the stimulus video displayed by the portable eye-tracker console during the session, to the network-connected server. system.

2. The system according to claim 1, further comprising a plurality of portable eye-tracker consoles that communicate wirelessly simultaneously with the aforementioned network-connected server.

3. The system according to claim 1 or 2, wherein the eye-tracking device comprises one or more gaze-tracking sensors mechanically assembled adjacent to the periphery of the display screen.

4. Each of the one or more eye-tracking sensors is A light source configured to emit detection light, A camera configured to capture eye movement data including at least one of the pupil or the reflection or reflected image of the cornea of ​​the detected light from the illumination source, and Equipped with, The eye-tracking sensor is configured to convert the eye movement data into a data stream that includes at least one of the following information: pupil position, gaze vector for each eye, or gaze point, and the eye-tracking data of the patient includes the corresponding data stream of the patient. The system according to claim 3.

5. The detected light includes infrared light, and the camera includes an infrared-sensitive camera. During the session, while the predetermined sequence of the stimulation video is presented on the display screen directed at the patient, the caregiver holding the patient wears glasses having a filter configured to filter the infrared light so that the camera captures only the patient's eye movement data. The system according to claim 4.

6. The eye-tracker device comprises at least one image acquisition device configured to capture an image of at least one of the patient's eyes while the predetermined sequence of the stimulation video is presented on the display screen directed towards the patient during the session. The eye tracker device is configured to generate corresponding eye-tracking data of the patient based on a captured image of at least one of the patient's eyes. The system according to any one of claims 1 to 5.

7. The portable eye-tracker console further comprises at least one recording device configured to collect at least one of the patient-related image data, audio data, or video data while the predetermined sequence of the stimulation video is presented on the display screen directed towards the patient during the session, The session data includes at least one of the following: image data, audio data, or video data. The system according to any one of claims 1 to 6.

8. The aforementioned portable eye-tracking console, A housing configured to hold the display screen and the eye tracker device, A base connected to the housing through a 1 or 1 joint and Equipped with, The base is rotatable around one or more joints to adjust the relative position or angle between the display screen and the patient during the session. The system according to any one of claims 1 to 7.

9. The network-connected server provides the web portal accessible by the portable computing device, The network-connected server is configured to output a developmental analysis report, including the patient's developmental analysis data, to the portable computing device via the user interface of the web portal. The system according to any one of claims 1 to 8.

10. The aforementioned network-connected server Receiving treatment data related to the said patient, wherein the treatment data includes at least one of the said patient's previous developmental analysis data, the said patient's previous treatment plan, or reference treatment data of another patient. Using artificial intelligence, generate a normative treatment plan for the patient based on the treatment data and developmental analysis data related to the patient. A system according to any one of claims 1 to 9, configured to perform the following:

11. A computer implementation method, A step of obtaining a treatment plan for a patient with a developmental disorder on a network-connected server, wherein the treatment plan has individual time lengths for different treatment-specific skill areas over a certain time period, the treatment plan has a specific treatment plan format, and the network-connected server is configured to process data related to a default treatment plan format. The network-connected server comprises the step of parsing the treatment plan with the specific treatment plan format in order to determine the treatment data for the patient, wherein the treatment data is consistent with the default treatment plan format. A computer implementation method, including

12. The computer implementation method according to claim 11, further comprising the step of receiving input for selecting a treatment plan format from a plurality of treatment plan formats presented on a user interface in the network-connected server, wherein the plurality of treatment plan formats are different from each other.

13. The computer implementation method according to claim 12, wherein at least one of the following is different from the others: skill domain name, prompt method, treatment material or training material, reinforcement method, or data acquisition method.

14. The step of parsing the treatment plan with the aforementioned specific treatment plan format is: A computer-aided method according to claim 12 or 13, comprising the step of parsing the treatment plan with the specific treatment plan format based on the selected treatment plan format and the default treatment plan format.

15. The computer-assisted method according to any one of claims 11 to 14, wherein the plurality of treatment plan formats include two or more of the following: EarliPoint, Early Initiation Denver Model (ESDM), Early Social Interaction (ESI), Discrete Trial Training (DTT), Joint Attention Symbolic Play Involvement Regulation (JASPER), and Project ImPACT (Project ImPACT).

16. The step of obtaining a treatment plan for developmental disorders for patients is, A computer-aided method according to any one of claims 11 to 15, comprising the step of uploading the treatment plan, which has the specified treatment plan format, from a repository on the network-connected server or storage medium.

17. The aforementioned treatment data, The respective time durations of the different treatment-specific skill areas during the aforementioned time period, The respective percentages of the duration of each of the different treatment-specific skill areas during the aforementioned time period, The percentage of attention given to each of the aforementioned different treatment-specific skill areas across a series of sessions, The change in the percentage of attention for each of the aforementioned different treatment-specific skill areas between at least two recent sessions, or The relationship between the respective percentages of the duration of each of the different treatment-specific skill areas and the respective percentage changes in the attention of each of the different treatment-specific skill areas. A computer implementation method according to any one of claims 11 to 16, comprising at least one of the following:

18. The step of receiving input for selecting a treatment plan format from a plurality of treatment plan formats presented on the user interface of the network-connected server, wherein the plurality of treatment plan formats are different from each other. The network-connected server comprises the steps of generating a new treatment plan based on the patient's treatment data and the selected treatment plan format. A computer implementation method according to any one of claims 11 to 17, further comprising:

19. The computer implementation method according to claim 18, further comprising the step of transmitting the new treatment plan, with the selected treatment plan format, to a computing device or an external server via the network-connected server.

20. The network-connected server further includes the step of generating developmental disorder assessment data for the patient based on the patient's eye-tracking session data, The step of generating a new treatment plan is based on the evaluation data of the patient's developmental disorder, The computer implementation method according to claim 18 or 19.

21. The network-connected server includes the step of determining a specific treatment plan format from among the plurality of treatment plan formats for the new treatment plan for the patient, A step of presenting a visual display on a specific treatment plan format among the plurality of treatment plan formats in the user interface, wherein the visual display indicates a recommendation of the specific treatment plan format for the new treatment plan for the patient. A computer implementation method according to any one of claims 18 to 20, further comprising:

22. The network-connected server includes the step of receiving a selection of a target session from a list of sessions on the user interface of a web portal on the network-connected server, In response to receiving the selection of the target session, the step of popping up the window for selecting a target skill area from a plurality of skill areas listed in the window, A step of automatically selecting one or more target skill areas from the plurality of skill areas based on the treatment data, wherein the different treatment-specific skill areas include the one or more target skill areas. The steps include: executing the target session based on the selected one or more target skill areas; A computer implementation method according to any one of claims 11 to 21, further comprising:

23. The steps include: presenting the input fields for the treatment plan on the user interface of the web portal on the network-connected server; The network-connected server includes the step of receiving input to one of the input fields of the treatment plan on the user interface, The network-connected server includes the step of updating the treatment plan based on the input to one of the input fields. A computer implementation method according to any one of claims 11 to 22, further comprising:

24. The computer-assisted method according to any one of claims 11 to 23, wherein the different therapy-specific skill domains include one or more of manding, listener response, turn-taking, joint attention, tact, and play.

25. The network-connected server includes the step of receiving input for selecting a third-party system from a plurality of third-party systems presented on the user interface, The steps include: after establishing a connection between the network-connected server and a selected third-party system, retrieving patient-related data from the selected third-party system, wherein the patient-related data includes at least one of the patient's previous clinical data, the patient's previous treatment data, or reference data of another patient; A computer implementation method according to any one of claims 11 to 24, further comprising:

26. The computer implementation method according to claim 25, further comprising the step of generating a new treatment plan for the patient based on the treatment data and the patient-related data in the network-connected server.

27. A computer implementation method, A network-connected server receives a request for an evaluation result of the patient based on the patient's session data, wherein the session data is collected during the presentation of a playlist of visual stimulus data collection to the patient in a session for evaluating the patient's developmental disorder. The network-connected server outputs the evaluation results for the patient. Includes, The aforementioned evaluation results, The respective scores of the developmental disorder indicators related to the aforementioned developmental disorder for the aforementioned patient, For each of the aforementioned developmental disability indicators, the results of the correlation between the respective scores of the aforementioned developmental disability indicators and the corresponding reference assessment scales and including, Computerized implementation method.

28. The results of the aforementioned correlation are A summary describing the aforementioned correlation, or Graphical presentation of the aforementioned correlation The computer implementation method according to claim 27, comprising at least one of the following.

29. The aforementioned evaluation results, Assessment results indicating whether the patient has the aforementioned developmental disorder, or Display information for each of the scores of the aforementioned developmental disability index The computer implementation method according to claim 27 or 28, further comprising at least one of the following.

30. The computer implementation method according to any one of claims 27 to 29, wherein the developmental disability index includes at least one of a social disability index, a language ability index, or a non-verbal learning index.

31. The corresponding reference assessment scale for each of the aforementioned scores of the social disability index includes the ADOS-2 scale, The corresponding reference assessment scale for each of the scores of the language proficiency index includes the Mullen language age equivalent value, The corresponding reference assessment scale for each of the scores of the aforementioned nonverbal learning indicators includes the Mullen nonverbal age equivalent value, The computer implementation method according to claim 30.

32. At least one visual scene in the data collection playlist is annotated using at least one of a plurality of skill areas related to the visual scene in the data collection playlist. The evaluation results include, for each of the one or more specific skill areas among the plurality of skill areas, patient behavioral data relating to moments in the session that are associated with the specific skill area, and each of the moments corresponds to each of the visual scenes in the data collection playlist. The computer implementation method according to any one of claims 27 to 31.

33. The computer implementation method according to claim 32, wherein the behavioral data includes an attention percentage defined as the ratio of the number of moments in which the patient pays attention to the relevant scene content in the visual stimulus to the total number of moments in which the patient is looking at the visual stimulus.

34. The aforementioned evaluation results, An outline of the distribution map of behavioral data of a reference group, wherein the behavioral data of the reference group is based on reference session data collected during the presentation of the data collection playlist of visual stimuli to each person in the reference group, and For each of the one or more specific skill areas, Representative visual scenes, The representative visual scene highlighting one or more areas of interest within a predetermined region for the reference group, or The representative visual scene in the session that highlights the patient's area of ​​focus. A computer implementation method according to claim 32 or 33, including the method described in claim 32 or 33.

35. The aforementioned evaluation results, A first graphical presentation of moment-by-moment measurements of the patient's observed behavior during the session, or A reference group attention funnel and a second graphical presentation of the patient's attention during the session. A computer implementation method according to any one of claims 32 to 34, comprising at least one of the following:

36. A computer implementation method, A step of initiating a patient session by establishing communication between an operator-side computing device and a patient-side portable tablet computing device, wherein the patient-side portable tablet computing device is integrated with an eye-tracking device. The steps include: collecting eye-tracking data from the patient using the eye-tracking device, while continuously presenting the patient with visual scenes from a visual stimulus data collection playlist on the screen of the patient's portable tablet computing device; A step of transmitting session data of the session to a network-connected server, wherein the session data includes the eye-tracking data of the patient collected in the session. Includes, Using the aforementioned eye-tracking device to collect eye-tracking data from the patient, Capturing at least one of the patient's eye image or the patient's eye position, wherein the eye-tracking data is determined based on the captured at least one of the patient's eye image or eye position. Computerized implementation method.

37. The aforementioned eye-tracking device, Based on the captured image of the patient's eye or the position of the eye, eye movement data is determined. Convert the eye movement data of the patient into eye-tracking data that includes information related to at least one of the following: pupil position, gaze vector of each eye, or gaze point. The computer implementation method according to claim 36, configured as described above.

38. Using the aforementioned eye-tracking device to collect eye-tracking data from the patient, The computer-assisted method according to claim 36 or 37, further comprising capturing first eye movement data of the patient's eye by measuring reflected light from the patient's eye.

39. The aforementioned eye-tracking device, At least one eye-tracking unit configured to capture first eye movement data of the patient's eye, At least one image acquisition unit configured to capture at least one of the images of the patient's eye or the position of the patient's eye, A computer implementation method according to claim 38, comprising:

40. The aforementioned eye-tracking device, A second eye movement data is determined based on the captured image of the patient's eye or the position of the eye, The eye-tracking data is determined based on the first eye movement data and the second eye movement data. A computer implementation method according to claim 38 or 39, configured as follows.

41. The aforementioned eye-tracking device, The first eye movement data is converted into the first gaze tracking data. A second eye movement data is determined based on the captured image of the patient's eye or the position of the eye, The second eye movement data is converted into second gaze tracking data. The eye-tracking data is determined based on the first eye-tracking data and the second eye-tracking data. A computer implementation method according to any one of claims 38 to 40, configured as follows.

42. A step of collecting at least one of image data, audio data, or video data collected by one or more recording devices while the visual scenes of the data collection playlist of visual stimuli are presented sequentially, further comprising the step of assembling the one or more recording devices inside or outside the patient-side computing device, The session data includes at least one of the following: image data, audio data, or video data. A computer implementation method according to any one of claims 36 to 41.

43. At least one processor, One or more memory locations that store instructions and Equipped with, When the instruction is executed by the at least one processor, it causes the at least one processor to execute the computer implementation method according to any one of claims 11 to 42. Device.

44. One or more non-temporary computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to execute the computer implementation method according to any one of claims 11 to 42.

45. A computer implementation method, A network-connected server receives a request for an assessment result of a patient based on the patient's session data, wherein the session data is collected during the presentation of a data collection playlist of visual stimuli to the patient during the session, and at least one visual scene of the data collection playlist is annotated using at least one of a plurality of skill areas related to the visual scene of the data collection playlist. A step of outputting the assessment results of the patient using the network-connected server, wherein the assessment results include, for each of one or more specific skill areas among the plurality of skill areas, the patient's behavioral data relating to moments in the session that are associated with the specific skill area, and each of the moments corresponds to each of the visual scenes in the data collection playlist. A computer implementation method, including

46. The computer implementation method according to claim 45, wherein the behavioral data includes an attention percentage defined as the ratio of the number of moments in which the patient pays attention to the relevant scene content in the visual stimulus to the total number of moments in which the patient is looking at the visual stimulus.

47. The session data includes the patient's eye-tracking data, The aforementioned computer implementation method A step of determining the total number of moments when the patient is looking at the visual stimulus, based on the eye-tracking data of the patient. A step of determining the number of times the patient focuses on the relevant scene content based on the eye-tracking data of the patient. Further including, The computer implementation method according to claim 46.

48. The steps include determining that, at a certain moment during the session, the patient's area of ​​focus is within a predetermined region, The step of determining that the aforementioned moment is one of the aforementioned number of moments in which the patient pays attention to the relevant scene content. A computer implementation method according to claim 46 or 47, further comprising:

49. The computer implementation method according to claim 48, wherein the predetermined region corresponds to the contour line of a distribution map of behavioral data of a reference group, and the behavioral data of the reference group is based on reference session data collected during the presentation of the data collection playlist of visual stimuli to each person of the reference group.

50. The computer implementation method according to claim 49, wherein the value of the contour line of the distribution map corresponds to a cutoff threshold.

51. The computer implementation method according to claim 49 or 50, wherein the assessment results further include the distribution map of the behavioral data of the reference group.

52. The assessment results, for each of the one or more specific skill areas, Representative visual scenes, The representative visual scene highlighting one or more areas of interest within the predetermined region for the reference group, or The representative visual scene in the session that highlights the patient's area of ​​focus. A computer implementation method according to any one of claims 49 to 51, further comprising at least one of the following.

53. The assessment results, for each of the one or more specific skill areas, Behavioral data from one or more preceding sessions of the aforementioned patient, or Comparison between the behavioral data of the session and the behavioral data of the patient from one or more preceding sessions. A computer implementation method according to any one of claims 45 to 52, further comprising at least one of the following.

54. The computer implementation method according to claim 53, wherein the assessment results include a graph showing the behavioral data of the session and the behavioral data of the patient from one or more preceding sessions for each of the one or more specific skill areas.

55. The computer implementation method according to any one of claims 45 to 54, further comprising the step of selecting one or more specific skill areas from the plurality of skill areas with respect to the assessment results of the patient.

56. The step of selecting one or more specific skill areas from the aforementioned plurality of skill areas is: A step of selecting a specific skill area from among the aforementioned multiple skill areas for which reliable data exists. A step of selecting a skill area from among the aforementioned multiple skill areas that is generally in demand, A step of selecting a skill area from among the aforementioned multiple skill areas that has a particularly high, low, or representative score, wherein the score represents the patient's attention percentage. A step of selecting a skill area previously selected as the target skill area in the aforementioned session, A step of selecting skill areas to customize the assessment results, or The step of selecting a skill area previously selected in the patient's previous sessions or previous assessment results. The computer implementation method according to claim 55, comprising at least one of the following.

57. The steps include receiving a session request to start the aforementioned session through a web portal on the network-connected server, The steps include: presenting a list of sessions on the user interface of the aforementioned web portal; The steps include receiving the selection of a session from the list of sessions on the user interface, and The computer implementation method according to claim 56, further comprising:

58. In response to receiving the selection for the session, the step of popping up the window for selecting a target skill area from the multiple skill areas listed in the window, The steps include receiving user input to select one or more target skill areas within the aforementioned window, The steps include: executing the session based on one or more selected target skill areas; It further includes, The selected one or more target skill areas include the one or more specific skill areas. The computer implementation method according to claim 57.

59. The computer implementation method according to claim 58, further comprising the step of adjusting the data collection playlist of visual stimuli based on one or more selected target skill areas.

60. The step of adjusting the data collection playlist for visual stimuli is, The step of prioritizing visual scenes related to one or more selected target skill areas within the aforementioned data collection playlist, A step of enhancing the value of additional visual scenes related to one or more selected target skill areas within the data collection playlist, or The step of reducing or removing visual scenes from the data collection playlist that are not related to the selected target skill area. The computer implementation method according to claim 59, comprising at least one of the following.

61. The step of prioritizing visual scenes related to one or more selected target skill areas is: The steps include placing the visual scenes related to the selected one or more target skill areas at the beginning of the data collection playlist, The steps of arranging the visual scenes related to the selected one or more target skill areas in order of their weighted correlation values ​​to the selected one or more target skill areas, or The step of selecting only the visual scenes related to one or more selected target skill areas from the data collection playlist. The computer implementation method according to claim 60, comprising at least one of the following.

62. The step of receiving user input is The step includes receiving user input from an operator-side computing device that is communicating with the network-connected server via the web portal, The aforementioned computer implementation method The steps include establishing communication between the operator-side computing device and the patient-side computing device through the network-connected server, A step of transmitting information of a adjusted data collection playlist of visual stimuli to the patient-side computing device, wherein as a result, the adjusted data collection playlist of visual stimuli is presented to the patient on the display screen of the patient-side computing device during the session. Further including, A computer implementation method according to any one of claims 59 to 61.

63. Upon completion of the session, the patient receives the session data from a patient-side computing device for the patient, wherein the patient's session data is collected by the patient-side computing device during the session. A step of generating the patient's behavioral data by processing the patient's session data based on reference data of a reference group and one or more specific skill areas. A computer implementation method according to any one of claims 45 to 62, further comprising:

64. A computer implementation method according to any one of claims 45 to 63, further comprising the step of loading reference data for a reference group, wherein the reference data is based on behavioral data for the reference group, the behavioral data for the reference group is based on reference session data collected during the presentation of the data collection playlist of visual stimuli, and one or more specific skill areas.

65. The reference data of the reference group applies to each of the one or more specific skill areas. A specific visual scene related to the aforementioned specific skill domain, wherein each of the aforementioned specific visual scenes highlights one or more areas of attention in the aforementioned reference group, or Distribution map of the behavioral data of the reference group for each of the specified visual scenes The computer implementation method according to claim 64, comprising at least one of the following.

66. The aforementioned reference data is for each of the one or more specific skill areas and for each of the specific visual scenes, The distribution map includes contour lines representing thresholds for determining whether the patient pays attention to the relevant scene content of the particular visual scene, The aforementioned computer implementation method If the patient's area of ​​focus is within a predetermined region corresponding to the contour line, the step of determining that the patient will focus on the relevant scene content of the particular visual scene, or If the patient's area of ​​attention is outside the predetermined area, the step of determining that the patient does not focus on the relevant scene content of the particular visual scene. Further including at least one of the following: The computer implementation method according to claim 65.

67. The patient's behavioral data includes an attention percentage defined as the ratio of the number of moments when the patient focuses on the relevant scene content to the total number of moments when the patient is looking at the visual stimulus. The aforementioned computer implementation method The steps include determining that, at a certain moment during the session, the patient's area of ​​focus is within the predetermined region, The step of determining that the aforementioned moment is one of the aforementioned number of moments in which the patient pays attention to the relevant scene content. Further including, The computer implementation method according to claim 66.

68. The computer implementation method according to claim 66 or 67, wherein the patient's behavioral data includes the result of a comparison between the patient's attention percentage and the reference group's threshold attention percentage.

69. The above results of the comparison are, The ratio of the patient's attention percentage to the reference group's threshold attention percentage, or The relationship between the patient's attention percentage and the reference group's threshold attention percentage. The computer implementation method according to claim 68, comprising at least one of the following.

70. The step of receiving a request is A computer implementation according to any one of claims 45 to 69, comprising the step of receiving user input from a computing device on the user interface of a web portal of the network-connected server, wherein the user input indicates the request and the user interface is presented on the display screen of the computing device.

71. The aforementioned user interface The first user interface element for viewing the default evaluation report, A second user interface element for customizing the evaluation report, or A third user interface element for launching an interactive dashboard using the aforementioned assessment results. The computer implementation method according to claim 70, comprising at least one of the following.

72. In response to a selection for the second user interface element, a window for selecting a target skill area in the evaluation report is popped up on the user interface; The steps include receiving a second user input for selecting one or more target skill areas within the aforementioned window, A step of generating the assessment report based on one or more selected target skill areas, wherein the one or more target skill areas include the one or more specific skill areas in the assessment result. The computer implementation method according to claim 71, further comprising:

73. The process further includes the step of presenting the interactive dashboard within the user interface in response to a selection of the third user interface element, The aforementioned interactive dashboard includes a subwindow for selecting one of a list of skill areas for interaction. The computer implementation method according to claim 71.

74. In response to receiving the selection of a specific target skill area from the aforementioned list of skill areas, Changes in the patient's attention percentage to the specific target skill area over a series of consecutive sessions, The change in the ratio between the patient's attention percentage and the reference group's threshold attention percentage, Changes in the relationship between the patient's attention percentage and the reference group's threshold attention percentage, or For each of the multiple visual scenes associated with the specific target skill domain, a first scene highlights one or more areas of attention within the reference group in the visual scene, and a second scene highlights the patient's area of ​​attention within the visual scene. The computer implementation method according to claim 73, further comprising the step of presenting at least one of the following.

75. The aforementioned multiple visual scenes are overlaid on each other within the user interface. The interactive dashboard includes a sliding user interface element for selecting each of the multiple visual scenes. The computer implementation method according to claim 74.

76. The network-connected server stores annotation data of visual scenes in the data collection playlist of visual stimuli, wherein the annotation data specifies each particular skill area associated with the visual scene. The steps include storing reference data of a reference group in the network-connected server, wherein the reference data is based on behavioral data based on reference session data collected during the presentation of the data collection playlist of visual stimuli, and A computer implementation method according to any one of claims 45 to 75, further comprising:

77. The aforementioned session data, Eye-tracking data collected by an eye-tracking device assembled in a patient-side computing device communicating with the aforementioned network-connected server, or At least one of the following: image data, audio data, or video data collected by one or more recording devices. It includes at least one of the following: The one or more recording devices are assembled inside the patient-side computing device or outside the patient-side computing device, A computer implementation method according to any one of claims 45 to 76.

78. A computer implementation method, The steps involve using a computing device to access a web portal on a network-attached server, The computing device receives user input on the user interface of the web portal, wherein the user input is for requesting an assessment result of the patient based on the patient's session data, the session data is collected during the presentation of a data collection playlist of visual stimuli to the patient during the session, and at least one visual scene of the data collection playlist is annotated using at least one of a plurality of skill areas related to the visual scene of the data collection playlist. The steps include: presenting the assessment results on the display screen of the computing device, wherein the assessment results include, for each of the one or more specific skill areas among the plurality of skill areas, patient behavioral data relating to moments in the session that are associated with the specific skill area, and each of the moments corresponds to each of the visual scenes in the data collection playlist; A computer implementation method, including

79. Steps include establishing a wireless connection between the eye-tracking device and the patient-side computing device that will be integrated with it, The steps include presenting the user interface for communicating with the patient's computing device in order to acquire the patient's session data, and The computer implementation method according to claim 78, further comprising:

80. A computer implementation method, A step of initiating a patient session by establishing communication between an operator-side computing device and a patient-side portable tablet computing device, wherein the patient-side portable tablet computing device is integrated with an eye-tracking device. The steps include: collecting eye-tracking data from the patient using the eye-tracker device, while sequentially presenting the patient with visual scenes from a data collection playlist of visual stimuli on the screen of the patient's portable tablet computing device, wherein at least one visual scene from the data collection playlist is annotated using at least one of a plurality of skill areas associated with the visual scene from the data collection playlist; A step of transmitting session data of the session to a network-connected server, wherein the session data includes the eye-tracking data of the patient collected in the session. A computer implementation method, including

81. The computer implementation method according to claim 80, wherein the data collection playlist includes visual scenes related to one or more specific skill areas among the plurality of skill areas that are prioritized within the data collection playlist.

82. A step of collecting at least one of image data, audio data, or video data collected by one or more recording devices while the visual scenes of the data collection playlist of visual stimuli are presented sequentially, further comprising the step of assembling the one or more recording devices inside or outside the patient-side computing device, The session data includes at least one of the following: image data, audio data, or video data. Computerized implementation method according to claim 80 or 81.

83. At least one processor, One or more memory locations that store instructions and Equipped with, When the instruction is executed by the at least one processor, it causes the at least one processor to execute the computer implementation method described in any one of claims 45 to 82. Device.

84. One or more non-temporary computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to execute the computer implementation method according to any one of claims 45 to 82.

Images