Assessment of Developmental Disorders via Gaze Tracking

Abstract

Systems, devices, apparatuses, methods, and computer-readable storage media for developmental assessment using eye tracking are provided. In one aspect, a system for developmental assessment via eye tracking includes a patient-side computing device having a screen for presenting visual stimuli to a patient, an eye tracking device integrated with the patient-side computing device and configured to collect eye tracking data of the patient while the visual stimuli are presented to the patient on the screen of the patient-side computing device, and an operator-side computing device configured to present a user interface for the operator to communicate with the patient-side computing device.

Description

Technical Field

[0001] Cross - reference to Related Patent Applications This application is a continuation of U.S. Patent Application No. 17 / 881,911, filed on August 5, 2022, which claims the benefit of U.S. Provisional Application No. 63 / 350,741, filed on June 9, 2022, the contents of which are incorporated herein by reference; a continuation of U.S. Patent Application No. 17 / 901,369, filed on September 1, 2022, which claims the benefit of priority; a continuation of U.S. Patent Application No. 17 / 902,415, filed on September 2, 2022, which claims the benefit of priority; and claims the priority of U.S. Patent Application No. 17 / 903,318, filed on September 6, 2022, the contents of which are incorporated herein by reference.

[0002] The present disclosure generally relates to developmental assessment, such as using an eye - tracking device to assess developmental disorders in patients.

Background Art

[0003] Developmental disorders, such as autism spectrum disorder (ASD), affect approximately 14% of children in the United States. Diagnostic methods for symptoms such as ASD vary significantly in terms of objective assessment tools and the experience of treatment providers. In some settings, the use of "best practice" tools gives symptoms particularly poor sensitivity and specificity, especially for infants or other young patients. Delayed diagnosis of developmental disorders reduces the effectiveness of treatment and often results in poor outcomes. Additionally, treatment providers (e.g., pediatricians or other medical specialists) often lack appropriate tools to objectively measure the progression of these symptoms, especially very early in a patient's life.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure describes methods, apparatuses, and systems for assessing developmental disorders via eye tracking. For example, some of the systems described herein can be used in diagnostic methods for developmental, cognitive, social, or mental abilities or disorders, including autism spectrum disorder (ASD). The system may optionally be implemented using improved objective measurements, as well as an improved portable device that provides added convenience to treatment providers and patients such as infants or other young patients. In some embodiments, the system includes at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye tracking device. In the specific examples described below, these portable devices can be variously equipped to wirelessly interact with a network-connected server platform to advantageously collect session data in a manner that is comfortable and less intrusive for the patient, while also adding improved flexibility of control for the treatment provider. Optionally, session data collected via the patient-side portable device (e.g., using analysis of eye tracking data generated in response to the display of a predetermined visual stimulus appropriate for the age) can be immediately analyzed for the purpose of outputting at the operator-side portable device a result interface that displays at least one indicator based on objective factors. **Means for Solving the Problems**

[0005] One aspect of the present disclosure enables a system for developmental assessment via eye tracking, including a patient-side mobile computing device including a screen for presenting visual stimuli to a patient, an eye tracking device mounted with the patient-side mobile computing device and directed at a fixed position relative to the screen of the patient-side mobile computing device for collecting eye tracking data of the patient while the visual stimuli are presented to the patient on the screen of the patient-side mobile computing device, and an operator-side mobile computing device configured to present a user interface for controlling the activation of the visual stimuli presented to the patient on the screen of the patient-side mobile computing device.

[0006] In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection. In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to wirelessly communicate with each other via a network-connected server. In some embodiments, the network-connected server includes a cloud computing system or cloud server implemented within a cloud environment.

[0007] In some embodiments, the patient-side mobile computing device is configured to send data to a network-connected server, the data including the patient's eye tracking data collected during a session while visual stimuli from a predetermined list of visual stimuli are presented to the patient.

[0008] In some embodiments, the patient-side mobile computing device is configured to automatically transmit data in response to the completion of the presentation of all visual stimuli in a predetermined list of visual stimuli during a session. In some embodiments, the patient-side mobile computing device is configured to transmit data in response to receiving an indication of completion from an operator-side mobile computing device or a network-connected server. In some embodiments, the operator-side mobile computing device or the network-connected server is configured to generate an indication of completion in response to determining that the session has ended or in response to receiving an input indicating the completion of the session.

[0009] In some embodiments, the data includes information on a predetermined list of visual stimuli and the patient's eye-tracking data collected during the session. In some embodiments, the patient-side mobile computing device is configured to transmit data in two files, including a first file containing the patient's eye-tracking data and associated timestamp information, and a second file containing information on the list of visual stimuli. In some embodiments, the associated timestamp information for the eye-tracking data includes the timestamp when the eye-tracking data was generated or collected, and the information on the list of predetermined visual stimuli includes the timestamp when the individual visual stimuli in the list were presented.

[0010] In some embodiments, the operator-side mobile computing device is configured to access a web portal in a network-connected server. In some embodiments, the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices through the web portal in the network-connected server. In some embodiments, the user interface of the operator-side mobile computing device includes at least one of information about one or more patients associated with the operator, information about one or more patient-side computing devices associated with the operator, or information about the operator.

[0011] In some embodiments, the patient-side mobile computing device is configured to display connection information including an access code on the screen of the patient-side mobile computing device in response to a connection to the network-connected server. In some embodiments, the operator-side mobile computing device is configured to connect to the patient-side computing device by receiving an input of the connection information including the access code in the user interface. In some embodiments, the user interface of the operator-side mobile computing device presents a request for connection information in response to receiving a selection of the patient-side mobile computing device among one or more patient-side computing devices presented in the user interface.

[0012] In some embodiments, the operator-side mobile computing device is configured to present a user interface of an operator application that runs on one of the operator-side computing devices or a network-connected server. In some embodiments, the operator application presents a user interface element for initiating desensitization within the user interface and, in response to the selection of the user interface element, is configured to send a command to the patient-side computing device to play visual desensitization information.

[0013] In some embodiments, in response to receiving the command, the patient-side mobile computing device plays visual desensitization information for the patient on the screen of the patient-side mobile computing device and controls the eye-tracking device so as not to collect the patient's eye-tracking data while the visual desensitization information is being displayed on the screen.

[0014] In some embodiments, the operator application is configured to present a user interface for the operator to set up a patient session by selecting a patient from a list of patients or creating a profile for the patient while the visual desensitization information is being displayed on the screen of the patient-side mobile computing device.

[0015] In some embodiments, the operator application is configured to display, within the user interface, instructions for adjusting the position of the eye-tracking device relative to the patient or the position of the patient relative to the patient-side mobile computing device based on the sensed position of the eye-tracking device with respect to at least one of the patient's eyes. In some embodiments, the sensed position is determined based on image data of at least one of the patient's eyes captured by using an image acquisition device included in or adjacent to the eye-tracking device. In some embodiments, the sensed position is determined by the patient-side mobile computing device or the operator application.

[0016] In some embodiments, the operator application is configured to send commands to the patient-side mobile computing device for calibration between the patient and the eye-tracking device in response to one of a selection of a user interface element for calibration within the user interface or a determination that a session for the patient is set up.

[0017] In some embodiments, in response to receiving the commands, the patient-side mobile computing device is configured to continuously present one or more calibration targets at one or more predetermined locations on the screen of the patient-side mobile computing device while using the eye-tracking device to capture the patient's eye-tracking calibration data.

[0018] In some embodiments, for each of the one or more calibration targets, the patient-side mobile computing device processes the captured eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual fixation with respect to the calibration target, compares the position of the patient's corresponding visual fixation with the corresponding predetermined location on the screen where the calibration target is presented, and determines whether the calibration target is calibrated for the patient based on the result of the comparison.

[0019] In some embodiments, the patient-side mobile computing device is configured to determine that the calibration is complete in response to a determination that one or more calibration targets have been calibrated. In some embodiments, the patient-side mobile computing device is configured to recalibrate the calibration target in response to a determination that the calibration target has not been calibrated.

[0020] In some embodiments, the patient-side mobile computing device is configured to reproduce desensitization information while presenting two adjacent calibration targets. In some embodiments, the operator application is configured to begin verifying the calibration in response to receiving an indication that the calibration is complete. In some embodiments, in response to receiving a request to verify the calibration, the patient-side mobile computing device presents at least one additional calibration target on the screen while using the eye-tracking device to capture additional eye-tracking calibration data of the patient, and processes the captured additional eye-tracking calibration data of the patient to determine the position of the patient's corresponding visual fixation on the at least one additional calibration target.

[0021] In some embodiments, the patient-side mobile computing device compares the position of the patient's corresponding visual fixation on at least one additional calibration target with a corresponding predetermined location on the screen where the at least one additional calibration target is presented, and is configured to determine whether the calibration is verified based on the result of the comparison.

[0022] In some embodiments, the operator application simultaneously presents in the user interface at least one additional calibration target at a corresponding predetermined location and at least one depiction of the corresponding visual fixation of the patient at the determined location of the corresponding visual fixation of the patient on the at least one additional calibration target, and presents in the user interface a first user interface element for calibration verification and a second user interface element for recalibration.

[0023] In some embodiments, the operator application is configured to send a command for data collection to the patient-side computing device in response to either the selection of a user interface element to initiate data collection or the determination that calibration is complete or verified.

[0024] In some embodiments, the patient-side mobile computing device is configured to continuously present a list of predetermined visual stimuli to the patient on the screen of the patient-side mobile computing device while using an eye-tracking device to capture the patient's eye-tracking data in response to receiving the command.

[0025] In some embodiments, the patient-side mobile computing device is configured to present a centering target to the patient on the screen of the patient-side mobile computing device before presenting each of the list of predetermined visual stimuli. In some embodiments, the patient-side mobile computing device is configured to perform calibration of the patient for the eye-tracking device during the presentation of two adjacent visual stimuli of the list of predetermined visual stimuli. In some embodiments, the eye-tracking data collected when performing the calibration is used for at least one of recalibrating the patient's eye-tracking data or determining calibration accuracy.

[0026] In some embodiments, the operator application is configured to present, within a user interface, at least one of a progress indicator that continuously updates throughout the presentation of a list of predetermined visual stimuli, a user interface element for skipping a visual stimulus among the list of predetermined visual stimuli, information about a visual stimulus that has already been presented or is being presented, or information about a visual stimulus to be presented.

[0027] In some embodiments, the network-connected server is configured to provide a patient's diagnosis result based on the patient's eye-tracking data, and the diagnosis result includes at least one metric value associated with a developmental disorder. In some embodiments, the operator-side mobile computing device is configured to present the diagnosis result within a user interface.

[0028] In some embodiments, the visual stimuli are pre-determined based at least on the patient's age or the patient's symptoms. In some embodiments, each of the visual stimuli includes at least one of a static visual stimulus, a dynamic visual stimulus, a pre-recorded visual stimulus, a pre-recorded audiovisual stimulus, a live visual stimulus, a live audiovisual stimulus, a two-dimensional stimulus, or a three-dimensional stimulus. In some embodiments, each of the visual stimuli is normed to elicit a specific eye movement response with a statistical reliability greater than 95%. In some embodiments, each of the visual stimuli is configured to elicit an eye movement response to an individual spatio-temporal location with a statistical reliability greater than 95%.

[0029] In some embodiments, the eye-tracking device is connected to the patient-side mobile computing device through a wired connection. In some embodiments, the eye-tracking device and the screen are housed together in a holder. In some embodiments, the patient-side mobile computing device includes a screen holder that holds the eye-tracking device at a fixed position relative to the screen.

[0030] In some embodiments, the eye-tracking device includes one or more eye-tracking units disposed at one or more locations adjacent to the periphery of the screen. In some embodiments, at least one of the patient-side mobile computing device or the operator-side mobile computing device is a tablet computing device. In some embodiments, the operator-side computing device is configured to communicate with the patient-side mobile computing device via two-way communication.

[0031] Another aspect of the present disclosure enables an apparatus including a patient-side computing device including a screen for presenting visual stimuli to a patient, and an eye-tracking device integrated with the patient-side computing device and configured to collect eye-tracking data of the patient while the visual stimuli are presented to the patient on the screen of the patient-side computing device. The patient-side computing device includes a patient-side computing device as described above.

[0032] Another aspect of the present disclosure enables an apparatus including an operator-side computing device as described above.

[0033] Another aspect of the present disclosure enables an apparatus including at least one processor and at least one non-transitory memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform operations, the operations including establishing a wireless connection with a patient-side computing device integrated with an eye-tracking device and presenting a user interface for communicating with the patient-side computing device to obtain eye-tracking data of the patient.

[0034] In some embodiments, the operation further includes accessing a web portal of a network-connected server, where the wireless connection is established through the web portal. In some embodiments, the operation further includes presenting a diagnostic result within a user interface based on the patient's eye-tracking data.

[0035] Another aspect of the present disclosure employs a computer-implemented method for developmental assessment via eye tracking. The computer-implemented method includes starting a session for a patient by establishing communication with an operator-side computing device and a patient-side computing device, where the patient-side computing device is integrated with an eye-tracking device; continuously presenting a list of predetermined visual stimuli to the patient on a screen of the patient-side computing device while collecting the patient's eye-tracking data using the eye-tracking device; and transmitting session data of the session to a network-connected server, where the session data includes the patient's eye-tracking data collected during the session.

[0036] In some embodiments, at least one of the operator-side computing device or the patient-side computing device is a portable device. In some embodiments, establishing communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device.

[0037] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes accessing, by the operator-side computing device, a web portal in a network-connected server, and wirelessly connecting the operator-side computing device to the patient-side computing device in response to receiving a selection of the patient-side computing device within the web portal. In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes displaying, by the patient-side computing device, connection information on a screen of the patient-side computing device, and establishing a wireless connection between the operator-side computing device and the patient-side computing device in response to receiving an input of the connection information by the operator-side computing device.

[0038] In some embodiments, the computer-implemented method further includes, after establishing communication, displaying, to the patient, visual desensitization information on a screen of the patient-side computing device. In some embodiments, the computer-implemented method further includes controlling a gaze tracking device so as not to collect gaze tracking data of the patient while displaying the visual desensitization information.

[0039] In some embodiments, the computer-implemented method further includes accessing, by an operator-side computing device, a web portal in a network-connected server to set up a session for a patient while displaying visual desensitization information. In some embodiments, setting up the session includes one of selecting a patient from a list of patients or creating a patient profile in the network-connected server. In some embodiments, the computer-implemented method further includes determining a relative position between a gaze tracking device and at least one eye of the patient and displaying, on a user interface of the operator-side computing device, an instruction for adjusting the position of the gaze tracking device or the position of the patient.

[0040] In some embodiments, the computer-implemented method further includes determining that the patient is aligned with the gaze tracking device in response to determining that a relative location of at least one eye of the patient is at a predetermined location within a detection area of the gaze tracking device.

[0041] In some embodiments, the computer-implemented method further includes calibrating the patient for the gaze tracking device by displaying one or more calibration targets to the patient on a screen of a patient-side computing device.

[0042] In some embodiments, calibrating a patient for a gaze tracking device includes continuously presenting each of one or more calibration targets at corresponding predetermined locations on a screen of a patient-side computing device while capturing the patient's gaze tracking calibration data using the gaze tracking device; processing the captured gaze tracking calibration data of the patient to determine the location of the patient's corresponding visual fixation for each of the one or more calibration targets; comparing the location of the patient's corresponding visual fixation with the corresponding predetermined location at which the calibration target is presented; and determining whether the calibration target is calibrated for the gaze tracking device based on the result of the comparison.

[0043] In some embodiments, calibrating a patient for a gaze tracking device further includes determining that the calibration target is calibrated and displaying the next calibration target in response to determining that the deviation between the location of the patient's corresponding visual fixation and the corresponding predetermined location is below a predetermined threshold, or determining that the calibration target is not calibrated and redisplaying the calibration target for calibration in response to determining that the deviation is greater than the predetermined threshold.

[0044] In some embodiments, the computer-implemented method further includes verifying the calibration using one or more new calibration targets after calibrating the patient for the gaze tracking device. In some embodiments, verifying the calibration includes continuously presenting each of one or more new calibration targets at corresponding predetermined locations on a screen of a patient-side computing device while capturing the patient's gaze tracking calibration data using the gaze tracking device; and processing the captured gaze tracking calibration data of the patient to determine the location of the patient's corresponding visual fixation for each of the one or more new calibration targets.

[0045] In some embodiments, validating a calibration involves presenting one or more new calibration targets at one or more corresponding predetermined locations and depicting one or more corresponding visual fixations of the patient at one or more determined locations simultaneously on a user interface of an operator-side computing device, and determining that the calibration is validated in response to receiving an indication to validate the result of the calibration, or initiating re-calibrating the patient for the eye tracking device in response to receiving an indication to invalidate the result of the calibration.

[0046] In some embodiments, validating a calibration involves determining the number of new calibration targets, each passing the calibration, based on the corresponding visual fixation positions of the patient and the corresponding predetermined positions, and determining that the calibration is valid if the number or associated percentage is above a predetermined threshold, or determining that the calibration is invalid and initiating re-calibrating the patient for the eye tracking device if the number or associated percentage is less than the predetermined threshold.

[0047] In some embodiments, continuously presenting a list of predetermined visual stimuli to the patient on a screen of a patient-side computing device includes presenting a centering target to the patient on the screen of the patient-side computing device before presenting each of the list of predetermined visual stimuli.

[0048] In some embodiments, continuously presenting a list of predetermined visual stimuli to the patient on a screen of a patient-side computing device includes performing calibration of the patient for the eye tracking device during the presentation of two adjacent visual stimuli of the list of predetermined visual stimuli, where the eye tracking data collected during performing the calibration is used for at least one of calibrating the patient's eye tracking data or determining calibration accuracy.

[0049] In some embodiments, the computer-implemented method further includes presenting, on a user interface of an operator-side computing device, at least one of a progress indicator that continuously updates throughout the presentation of a list of predetermined visual stimuli, information about visual stimuli that have already been presented or are being presented, information about visual stimuli to be presented, or user interface elements for skipping visual stimuli from a list of predetermined visual stimuli.

[0050] In some embodiments, transmitting session data of a session to a network-connected server includes automatically transmitting, by a patient-side computing device, the session data of the session to the network-connected server in response to one of a determination of completion of presenting a list of predetermined visual stimuli on a screen or receipt of a session completion indication from the operator-side computing device.

[0051] In some embodiments, the session data includes information related to a presented list of predetermined visual stimuli. In some embodiments, the information related to a presented list of predetermined visual stimuli includes the name of the predetermined visual stimulus presented and the associated timestamp when the predetermined visual stimulus is presented.

[0052] In some embodiments, the session data includes gaze-tracking data and the associated timestamp when the gaze-tracking data is generated or collected. In some embodiments, the session data is stored in a first file that stores the patient's gaze-tracking data and a second file that stores information related to a presented list of predetermined visual stimuli.

[0053] Another aspect of the present disclosure employs a computer-implemented method for developmental assessment using gaze tracking data by a network-connected server. The computer-implemented method includes receiving session data for a plurality of sessions, wherein the session data for each session includes gaze tracking data of a corresponding patient in the session, and processing the session data for the plurality of sessions in parallel to generate processed session data for the plurality of sessions, and for each session of the plurality of sessions, analyzing the processed session data of the session based on corresponding reference data to generate an assessment result for the corresponding patient in the session.

[0054] In some embodiments, the computer-implemented method further includes loading corresponding reference data for the plurality of sessions in parallel with processing the session data for the plurality of sessions.

[0055] In some embodiments, the network-connected server includes a plurality of processing cores. Processing the session data for the plurality of sessions in parallel includes using a first plurality of processing cores to process the session data for the plurality of sessions in parallel and using a second different plurality of processing cores to load the corresponding reference data for the plurality of sessions, wherein the number of the first plurality of processing cores is greater than the number of the second plurality of processing cores.

[0056] In some embodiments, analyzing the processed session data of the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions includes using a plurality of processing cores including the first plurality of processing cores and the second plurality of processing cores.

[0057] In some embodiments, analyzing processed session data for multiple sessions based on the corresponding reference data loaded for the multiple sessions includes at least one of comparing the processed session data of a session with the corresponding reference data, using the corresponding reference data to infer assessment results for the corresponding patient from the processed session data, or using at least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model. In some embodiments, the corresponding reference data includes historical eye-tracking data or results for patients having substantially the same age or symptoms as the corresponding patient.

[0058] In some embodiments, the computer-implemented method further includes generating assessment results based on previous session data of the corresponding patient. In some embodiments, the computer-implemented method includes, for each of a plurality of sessions, allocating a respective container for the session, processing the session data of the session within each respective container, and analyzing the processed session data of the session based on the corresponding model data to generate an assessment result for the corresponding patient in the session.

[0059] In some embodiments, while eye-tracking data is collected during a session, the eye-tracking data is associated with a list of predetermined visual stimuli presented to the patient, where the session data includes information related to the list of predetermined visual stimuli in the session.

[0060] In some embodiments, the computer-implemented method further includes associating the gaze-tracking data of a session with a list of predetermined visual stimuli in the session. In some embodiments, associating the gaze-tracking data of a session with a list of predetermined visual stimuli in the session includes, within each container, decomposing the gaze-tracking data into a plurality of portions based on information related to the list of predetermined visual stimuli, and each portion of the gaze-tracking data is associated with one of the respective predetermined visual stimuli or corresponding calibration.

[0061] In some embodiments, processing the session data of a session includes processing portions of the gaze-tracking data associated with each respective predetermined visual stimulus based on the information of each respective predetermined visual stimulus.

[0062] In some embodiments, the computer-implemented method further includes, within each container, recalibrating portions of the gaze-tracking data associated with each respective predetermined visual stimulus based on at least one portion of the gaze-tracking data associated with the corresponding calibration.

[0063] In some embodiments, the computer-implemented method further includes, within each container, determining calibration accuracy using at least one portion of the gaze-tracking data associated with the corresponding calibration and a plurality of predetermined locations at which a plurality of calibration targets are presented in the corresponding calibration.

[0064] In some embodiments, receiving session data for a plurality of sessions includes receiving session data for the plurality of sessions from a plurality of computing devices associated with a corresponding entity through a web portal. In some embodiments, the computer-implemented method further includes adding a file pointer for the session data of a session into a processing queue to be processed in response to receiving the session data of the session. In some embodiments, the computer-implemented method further includes storing the session data of a session into a database using the file pointer for the session and retrieving the session data of the session from the database using the file pointer for the session.

[0065] In some embodiments, the computer-implemented method further includes storing, for each entity, session data from one or more computing devices associated with the entity into respective repositories. In some embodiments, each repository for an entity includes at least one of entity information, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted at the entity, information about one or more patients associated with the entity, or history information of each repository.

[0066] In some embodiments, each repository is included in a NoSQL database. In some embodiments, each repository is separated from one or more other repositories and is inaccessible by one or more other entities.

[0067] In some embodiments, the computer-implemented method further includes dynamically adjusting the resources of the network-connected server based on the number of computing devices accessing the network-connected server.

[0068] In some embodiments, the computer-implemented method further includes replicating the data of the first data center to the second data center and automatically directing traffic to the second data center in response to a determination that the first data center is inaccessible. In some embodiments, each of the first data center and the second data center includes at least one of a web portal accessible to the operator-side computing device, an operator application, or an application layer for data processing and data analysis.

[0069] In some embodiments, the computer-implemented method further includes storing the same data in multiple data centers, where the data includes application data for the entity and information related to the gaze tracking data.

[0070] In some embodiments, the computer-implemented method further includes associating the generated assessment result with the corresponding patient in the session and generating an assessment report for the corresponding patient.

[0071] In some embodiments, the computer-implemented method further includes outputting the assessment result or the assessment report to be presented in the user interface of the operator-side computing device. In some embodiments, the assessment report includes at least one of information about the corresponding patient, information about the entity executing the session for the corresponding patient, information about the calibration accuracy in the session, information about the session data collection, or the assessment result for the corresponding patient.

[0072] In some embodiments, the assessment results indicate the likelihood that the corresponding patient has a developmental, cognitive, social, or mental disorder or ability. In some embodiments, the assessment results indicate the likelihood that the corresponding patient has or does not have an autism spectrum disorder (ASD). In some embodiments, the assessment results include respective scores for each of one or more of a social impairment indicator, a language ability indicator, a non-verbal ability, a social adaptability indicator, and a social communication indicator.

[0073] In some embodiments, the assessment results include at least one of a visualization of eye-tracking data overlaid on a corresponding visual stimulus still photograph from a socially relevant moment, an animation visualizing eye-tracking data overlaid on a corresponding visual stimulus still photograph from a socially relevant moment, a visualization of aggregated reference data from a plurality of reference patients matched to the corresponding patient in one or more patient attributes, or an annotation describing at least one of visual stimulus content or an eye-gaze pattern.

[0074] In some embodiments, the corresponding patient has an age in the range of 5 months to 7 years, including an age in the range of 5 months to 43 months or 48 months, an age in the range of 16 months to 30 months, an age in the range of 18 months to 36 months, an age in the range of 16 months to 48 months, or an age in the range of 16 months to 7 years.

[0075] Another aspect of the disclosure employs a system that includes at least one processor and one or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to execute a computer-implemented method as described herein.

[0076] Another aspect of the present disclosure employs one or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to execute a computer-implemented method as described herein.

[0077] One or more of the embodiments described herein can achieve several technical effects and advantages. In a first example, some embodiments can provide a convenient, miniaturized, and effective computing system for advantageously collecting eye-tracking data and then communicating such data for analysis and diagnostic results. The computing system may include at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye-tracking device. These portable devices can be variously equipped (with various peripheral devices or instruments, various user interfaces, etc.) and can be wirelessly connected to each other or to a network-connected server platform (providing communication between the operator-side portable device and the patient-side portable device) without using a physical connection. For example, an operator may use the operator-side portable device to control the playback of a predetermined visual stimulus on the patient-side portable device to calibrate the eye-tracking device using the patient and to acquire the patient's eye-tracking data using the eye-tracking device. When the session is completed, the patient-side portable device can transmit session data, such as the detected eye-tracking data and information on the visual stimulus played back, to a cloud server for data storage, data processing, and data analysis. The cloud server can be remotely connected to these portable devices, for example, via a network.

[0078] In a second example, some implementations of the computing systems described below can have a smaller form factor (e.g., compared to a larger seat-mounted computer device), can be more easily moved and repositioned, thereby improving convenience for the patient, the treatment provider, or both. The computing system can also allow for more flexibility for both the operator and the patient, and the operator and patient can more easily position themselves in a suitable location during setup and data collection. Moreover, in some embodiments described in detail below that use two separate portable computing devices, e.g., tablets, the system can advantageously impose fewer requirements on the computing devices themselves and provide more flexibility in implementing various functions within the two separate computing devices. Further, in some of the methods described below, for improved comfort in the patient-side portable device and accurate eye-tracking data acquisition, during the display of visual stimuli, the patient (e.g., an infant or other young patient) can be held by a caregiver (e.g., a parent) / seated with a caregiver (e.g., a parent), and the treatment provider (who carries the operator-side portable device to control the playback of a given visual stimulus) can simultaneously select a position spaced apart from the patient and the caregiver, thereby reducing jitter for the patient. Additionally, the present technology enables the treatment provider to use the operator-side portable device to log in to a web portal of a cloud server for device management, patient management, and data management. For example, through the web portal of the cloud server, the treatment provider can use the operator-side portable device to communicate with a plurality of patient-side portable devices for eye-tracking data acquisition of a plurality of patients in a plurality of sessions, which can significantly improve work efficiency and reduce the workload of the treatment provider.In addition, the web-based operator application can be run on a cloud server, select a patient-side portable device for connection, and provide a user interface to the treatment provider to perform calibration, verification, and session data acquisition. The web-based operator operation can be maintained or updated on the cloud server, which can be less costly, more convenient, and more efficient than installing the operator application in individual operator-side portable devices. Further, the operator-side portable device can be paired with multiple patient-side portable devices, which can simplify the overall system, reduce system costs, and improve system performance.

[0079] In a third example, several embodiments detailed below can advantageously support services to a number of treatment providers across various geographical locations. In some embodiments, the cloud server is implemented in a centralized cloud environment and can communicate with multiple computing devices or systems as described above. The cloud server can utilize a software as a service (SaaS) model by providing a subscription-based diagnostic service to treatment providers. The centralized cloud environment can provide more flexibility for expanding the capabilities of the cloud server. For example, the cloud server can implement a multi-tenant architecture. A tenant can be, but is not limited to, a single healthcare organization including an autism center, a healthcare facility, an expert, a physician, or a clinical study. The resources of the cloud server can be dynamically managed based on the total number of tenants and the expected average workload, e.g., how many tenants are accessing the cloud server at a given point in time. The cloud server can employ horizontal scaling techniques such as auto-scaling to handle a sudden increase in resource workload.

[0080] Additionally, in a fourth example, some versions of the techniques detailed below can provide highly efficient and rapid diagnostic results to a treatment provider. As described above, a computing system, such as a patient-side portable device, can automatically upload session data to a cloud server when a session is completed. The cloud server can also be configured to automatically receive, process, and analyze session data from multiple computing systems without human intervention. Moreover, the cloud server can process and analyze session data from multiple sessions from a number of computing systems in parallel, which can significantly improve the session processing speed and provide diagnostic results within a short time period, e.g., within a 24-hour window. In some embodiments, the cloud server can deploy a respective container for each session, and each container can include a corresponding data processing module and data analysis module. In this way, when the session data of a session is uploaded, the session data of the session can be processed and analyzed using its own container (e.g., its own instance of data processing and data analysis). In some embodiments, for example, while session data of multiple sessions is being processed in a corresponding container using a majority of the processing units (or cores) in the cloud server, model data for analyzing the processed session data can be pre-loaded in parallel into the corresponding container using, for example, the remaining or minority of the processing units in the cloud server. Then, all of the processed session data and the loaded model data can be analyzed in parallel in the corresponding container using, for example, the total number of processing units in the cloud server. The present technology enables automation and parallelization in multiple ways, which can significantly improve the speed of session data processing and analysis and provide rapid diagnostic results within a short time period, e.g., within a 24-hour window.Parallelization can also make the cloud server more efficient in resource utilization, which can further improve system performance.

[0081] In a fifth example, some of the embodiments described herein can also provide improved accuracy of diagnostic results for patients based on efficient data analysis. For example, using comparisons or inferences via artificial intelligence (AI) models such as statistical models, algorithms, machine learning models, or artificial neural network models, a large amount of model data including data related to patients in similar age, similar background, and / or similar circumstances can be used together with the processed session data for the patient to generate diagnostic results for the patient, which can significantly improve the accuracy of the diagnostic results.

[0082] In a sixth example, some versions of the techniques described herein can enable the privacy, confidentiality, and security of treatment provider data. In some embodiments, the cloud server stores different types of data related to, for example, organizations, users, sessions, patients, and devices, as application data or documents in a database (e.g., a NoSQL database). Different tenants can have their own databases, which can separate data between tenants and prevent others from accessing their data, thereby improving data integrity and data confidentiality. Tenants can share a single instance of an application in the cloud server, or each tenant can obtain their own instance of the application.

[0083] In a seventh example, some embodiments described herein can guarantee high availability of services to treatment providers such that the treatment providers can access their services regardless of any outages in the cloud server. The cloud server can be hosted in a cloud environment having built-in high availability or configurable high availability, for example, using a high-availability service level agreement or through the use of geo-redundancy. As an example, the technology can replicate data centers in different physical locations. When a data center fails, traffic can be automatically redirected to the replicated data center. The process can be seamless and the treatment provider may not notice the switch to the replicated data center. In some embodiments, the data center includes a web portal accessible to the treatment provider / user as well as data processing and data analysis modules. Additionally, the technology can replicate databases in multiple locations. The databases can include application data and raw eye-tracking data, processed data, data analysis, diagnostic results, and / or assessment reports.

[0084] In an eighth example, some versions of the technology described herein can perform unified data input (such as patient identifiers and metadata for augmenting eye-tracking data) in, for example, a cloud server, which can reduce the workload of treatment providers, reduce human errors, and guarantee data accuracy. In some embodiments, the technology provides two-way communication between different devices or systems. Compared to one-way communication that requires data to be entered multiple times into separate devices or systems, two-way communication can reduce data input for treatment providers and enable autonomous data transmission.

[0085] In a ninth example, some of the embodiments described herein can also provide secure communication between different devices and systems. For example, communication between an on-premises computing system (including an operator-side portable device and a patient-side portable device) and a cloud server can use Hypertext Transfer Protocol Secure (HTTPS) communication through which information is encrypted to protect both the privacy and integrity of the information. Communication between the patient-side portable device and the operator-side portable device can be performed using WebSocket communication, for example, using a secure implementation of WebSocket called WebSocket Secure (WSS). WebSocket communication can also provide two-way communication between devices that enables the operator-side portable device to control the patient-side portable device while receiving information from the patient-side portable device.

[0086] In some implementations, the technology can be used to perform a diagnosis (based on objective data) on pediatric patients at a young age, where such diagnosis and medical treatment can achieve the most effects. Additionally, the technology described herein can be used to monitor risk severity, which can provide insights into severity at a much younger age and confirm severity and impairment status over time. The technology can provide a quick and reliable baseline of a child's status and progress to support individualized therapy for the child. The technology can provide an objective baseline and confirmatory diagnosis for informing an individualized treatment plan, minimizing inefficiencies and costs associated with patient-scale and redundant assessments. The technology can reduce nursing variability by providing clear standards and measures of progress, providing clear and quantifiable standards that can be managed and understood regardless of provider / staff skill. The technology can create access among populations that are underserved and under resourced, and can be used in various care settings at an appropriate price point that is both affordable and less time consuming than current assessment alternatives.

[0087] The technology implemented herein can be used, for example, to perform earlier identification and assessment of developmental, cognitive, social, language, or non-verbal abilities, or risk of mental abilities or disorders in a patient, by measuring visual attention to social information in the environment compared to normative age-specific benchmarks. The patient can have an age in the range of, for example, from 5 months to 7 years, from 16 months to 7 years, from 12 months to 48 months, from 16 months to 30 months, or from 18 months to 36 months.

[0088] According to some aspects, changes in a patient's visual fixation over time with respect to some dynamic stimuli provide markers of the patient's putative developmental, cognitive, social, or mental abilities or disorders (such as ASD). Visual fixation is a type of eye movement used to stabilize visual information on the retina and generally occurs simultaneously with a person looking at or "staring at" a point or region on a display plane. In some embodiments, a patient's visual fixation is identified, monitored, and tracked over time through repeated eye-tracking sessions and / or through comparison with model data based on a number of patients in a similar age and / or background. Data related to visual fixation is then compared to relative norms to determine the putative increased risk of such symptoms in the patient. Changes in visual fixation (specifically, decreases or increases in visual fixation to images of the eyes, body, or other areas of interest of a person or object displayed in a visual stimulus), compared to similar visual fixation data of typically developing patients or the patient's own previous visual fixation data, result in an indication of a developmental disorder, cognitive disorder, or mental disorder. The technology can be applied to quantitatively measure and monitor the overall symptoms of each ability or disorder and, in some cases, provide more accurate and relevant, conventional information to patients, families, and service providers. According to additional aspects, the technology can be used to predict outcomes (and thus provide conventional capabilities) in patients with autism while also providing similar diagnostic and conventional measures for overall developmental, cognitive, social, or mental abilities or disorders.

[0089] As detailed below, the techniques described herein for the detection of developmental, cognitive, social, or mental disorders are applicable to the detection of symptoms including, but not limited to, expressive and receptive language delays, non-verbal developmental delays, intellectual disabilities, intellectual disabilities of known or unknown genetic origin, traumatic brain injuries, disorders of infancy not otherwise specified (DOI-NOS), social communication disorders, and autism spectrum disorders (ASD), as well as attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), concussion, sports injuries, and symptoms such as dementia.

[0090] It is understood that the methods according to the present disclosure may include any combination of the embodiments described herein. That is, the methods according to the present disclosure are not particularly limited to the combinations of the embodiments described herein, but also include any combination of the provided embodiments.

[0091] Details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the following description. Other embodiments and advantages of the present disclosure will become apparent from the description and drawings, and from the claims.

Brief Description of the Drawings

[0092]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 4F

Figure 4G

Figure 4H

Figure 4I

Figure 4J

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 8A

Figure 8B

Figure 8C

Figure 9

Figure 10

Figure 11

Figure 12

DETAILED DESCRIPTION OF THE INVENTION

[0093] Like reference numerals and designations in the various drawings indicate like elements. It is to be understood that the various illustrative implementations shown in the drawings are only illustrative depictions and are not necessarily drawn to scale.

[0094] This disclosure describes systems, devices, and methods for detecting and assessing developmental, cognitive, social, or mental abilities or disorders, including autism spectrum disorder (ASD), in a patient using eye-tracking data generated in response to the display of a predetermined specific visual stimulus (e.g., one or more videos) to the patient.

[0095] To provide an overall understanding of the systems, devices, and methods described herein, several illustrative embodiments are described. It will be understood that such data may provide a measure of the degree of stereotypy in normative development, providing an indication of variability in typical development if such data do not indicate a measure of the disorder. Further, all of the components and other features outlined below may be combined with each other in any suitable manner and may be adapted and applied to systems outside of medical diagnosis. For example, the interactive visual stimuli of the present disclosure may be used as a therapeutic tool. Additionally, the data collected may provide a measure of several types of visual stimuli to which the patient preferentially attends. Such measures of preference have applications in both the fields of medical diagnosis and medical therapy, including, for example, the advertising industry or other industries, where data related to visual stimulus preference are of interest, and applications without such fields.

[0096] Exemplary Environments and Systems FIG. 1 is a block diagram of an exemplary environment 100 for assessing developmental disorders via eye tracking, according to one or more embodiments of the present disclosure. Environment 100 involves a cloud server 110 and a plurality of computing systems 120 - 1, ..., 120 - n (collectively referred to as computing system 120 or individually as computing system 120) that communicate via network 102. The cloud server 110 can provide developmental disorder assessment or diagnostic services to several users (e.g., treatment providers). Treatment providers can use the corresponding computing system 120 to suitably and reliably collect data on patients in sessions (e.g., procedures related to collecting data) of any age from newborns to the elderly, with the exemplary embodiments described below, particularly suitable for infants or other young patients. The data collected during the session (i.e., session data) can include eye - tracking data generated in response to the display of a pre - determined specific visual stimulus (e.g., one or more videos) to the patient. The computing system 120 can securely transmit the session data to the cloud server 110, and the cloud server 110 can store, process, and analyze the session data for diagnosing ASD or other cognitive, developmental, social, or mental abilities or disorders in the patient, and can provide a diagnosis result or diagnostic report to the treatment provider in a highly secure, robust, fast, and accurate manner, as will be described in further detail below.

[0097] The treatment provider can be a single healthcare organization, including but not limited to, an autism center, a healthcare facility, an expert, a physician, or a clinical study. The healthcare organization can provide the patient with developmental assessment and diagnosis, clinical care, and / or therapy services. As shown in FIG. 1, the patient (e.g., an infant or a child) may be brought to the healthcare facility by a caregiver (e.g., a parent). An operator (e.g., an expert, a physician, a medical assistant, a technician, or another medical professional within the healthcare facility) can use the computing system 120 to collect non-invasive eye-tracking data from the patient while the patient views a visual stimulus (e.g., a dynamic visual stimulus such as a movie) that exhibits general social interaction (e.g., a two-person or three-person interaction). The stimulus presented to the patient for the purpose of data collection can be unique to the patient, for example, based on the patient's age and symptoms. The stimulus can be any suitable visual image (whether static or dynamic), including movies or videos as well as still images or any other visual stimulus. It will be understood that movies or videos are merely referenced as examples and that any such description applies equally to other forms of visual stimulus.

[0098] In some embodiments, as shown in FIG. 1, computing system 120 includes at least two separate computing devices 130 and 140, such as operator-side computing device 140 and at least one patient-side computing device 130. Optionally, the two computing devices 130 and 140 can be wirelessly connected via, for example, wireless connection 131 without using a physical connection. Wireless connection 131 can be through a cellular network, a wireless network, Bluetooth, near field communication (NFC), or other standard wireless network protocols. In some embodiments, the patient-side computing device 130 is configured to connect to the operator-side computing device 140 through a wired connection, such as a Universal Serial Bus (USB), for example, when the wireless connection 131 does not function.

[0099] In some cases, the two computing devices 130 and 140 communicate with each other by separately communicating with cloud server 110 via network 102, and cloud server 110 in turn provides communication between operator-side computing device 140 and patient-side computing device 130. For example, as will be described in further detail in FIGS. 4A-4B, an operator can log in to a web portal running on cloud server 110 for device management, patient management, and data management, for example, through a web-based operator application. The operator can use an operator-side computing device 140 (e.g., a tablet) to communicate with multiple patient-side portable devices 130, such as within the same medical facility, for obtaining gaze tracking data of multiple patients in multiple sessions, which can greatly simplify computing system 120, reduce system costs, improve work efficiency, and reduce the operator's workload.

[0100] Computing devices 130, 140 can include any suitable type of device, such as tablet computing devices, cameras, handheld computers, portable devices, mobile devices, personal digital assistants (PDAs), cellular telephones, network appliances, smart mobile phones, enhanced general packet radio service (EGPRS) mobile phones, or any suitable combination of any two or more of these data processing devices or other data processing devices. As an example, FIG. 12 shows an architecture for a computing device that can be implemented as computing device 130 or 140.

[0101] At least one of computing device 130 or computing device 140 can be a portable device, such as a tablet device. In some cases, both computing devices 130, 140 are portable and wirelessly connected to each other. In this way, computing system 120 can be more easily moved and repositioned, allowing for more flexibility for an operator to select the operator's position relative to the patient. For example, an operator (carrying operator-side computing device 140) is not physically tied to patient-side computing device 130 and can easily position himself at an optimal location (e.g., away from the direct line of sight of the patient) during setup and data collection. Further, a patient (e.g., an infant or other child) can be carried by a caregiver (e.g., a parent) in a more suitable location and in a more comfortable manner, which may allow the patient to be more engaged with the visual stimuli being presented for effective and accurate gaze-tracking data acquisition. Patient-side computing device 130 can be carried by a caregiver or positioned (e.g., adjustably) in front of the patient and the caregiver.

[0102] As shown in FIG. 1, the patient-side computing device 130 includes a screen 132 (e.g., a display screen) for displaying or presenting visual stimuli to the patient. The patient-side computing device 130 can also include an eye-tracking device 134 or can be integrated with the eye-tracking device 134 within the same housing 136. In some embodiments, the patient-side computing device 130 integrated with the eye-tracking device 134 may be referred to as an eye-tracking console or an eye-tracking system.

[0103] The eye-tracking device 134 can be connected to the patient-side computing device 130 via a wired connection, for example, using a USB cable or an electrical wire, or using electrical pins. In some cases, the eye-tracking device 134 is configured to be connected to the patient-side computing device 130 via a wireless connection, such as Bluetooth or NFC. The eye-tracking device 134 can be positioned at a suitable location with respect to the screen 132 and / or the patient, where the eye-tracking device 134 can capture the patient's eye movements while viewing the visual stimuli while also minimizing visual jitter from the patient's field of view.

[0104] As shown in FIG. 1, the gaze tracking device 134 can include one or more gaze tracking units (or sensors) 135 disposed below the lower part of the screen 132. The one or more gaze tracking units 135 can be disposed on one or both sides of the screen 132, above the screen 132, and / or around the screen 132. The one or more gaze tracking units 135 can be mechanically mounted on the patient-side computing device 130 at a location adjacent to the peripheral portion of the screen 132. For example, the patient-side computing device 130 can include the screen 132 and a screen holder structure that holds the gaze tracking device 134 at a fixed predetermined location with respect to the screen 132. In some embodiments, the gaze tracking device 134 includes a first gaze tracking unit configured to capture or collect the eye movement of the patient's left eye, and a second gaze tracking unit configured to capture or collect the eye movement of the patient's right eye. The gaze tracking device 134 can further include a third gaze tracking unit configured to capture the position of the patient's eyes, or an image acquisition unit (e.g., a camera) configured to capture an image of the patient's eyes.

[0105] The gaze tracking unit includes sensors that can detect the presence of a person and can either track in real time what the person is looking at or measure where the person is looking or how the eyes respond to stimuli. The sensors can convert the eye movement of a person into a data stream that includes information such as pupil position, per-eye fixation vector, and / or fixation point. In some embodiments, the gaze tracking unit includes a camera (e.g., an infrared-sensitive camera), an illumination source (e.g., infrared light (IR) illumination), and algorithms for data collection and / or data processing. The gaze tracking unit can be configured to track the reflection or reflected image (CR: corneal reflection or reflex) of the pupil or the cornea. The algorithms can be configured for pupil center and / or cornea detection and / or artifact suppression.

[0106] Since eye size, foveal position, and variability in general physiology can vary for each individual, the eye tracking unit can be initially calibrated before using it to collect eye tracking data for a participant (e.g., a patient). In calibration, the physical position of the eye is algorithmically associated with points in the space that the participant is looking at (e.g., fixating on). The fixation position can be a function of the participant's perception. In some embodiments, calibration requires the participant to look at a fixed known calibration target (e.g., a point) in the visual field. Calibration can include a single target placed in the center, or can include two, five, nine, or even thirteen targets. The algorithm can generate a mathematical transformation between the (CR-free) eye position and the fixation position for each target, and then can create a matrix covering the entire calibration area, for example, using interpolation in the middle of each target. The more targets are used, the higher and more uniform the accuracy can be across the entire visual field. The calibration area defines the part of the eye tracking unit's range with the highest accuracy, and the accuracy drops when the eye moves at an angle larger than the targets used.

[0107] In some embodiments, the eye tracking unit can perform self-calibration, for example, by creating a model of the eye and passively measuring the characteristics of each individual. Calibration can also be performed without the active cooperation of the participant by effectively "hiding" the calibration target among other visual information and making assumptions about the fixation position based on the content. In some embodiments, calibration is not performed on the eye tracking unit if useful data can be taken from the raw pupil position, for example, using a medical vestibulo-ocular reflex (VOR) system or a fatigue monitoring system.

[0108] In some cases, for example, verification can be performed to measure the success of calibration by indicating a new target and measuring the calculated fixation accuracy. The tolerance for calibration accuracy can depend on the application example of the eye tracking unit. For example, an error between 0.25 degrees and 0.5 degrees of the viewing angle may be considered acceptable. In some application examples, exceeding 1 degree is considered a failed calibration and requires another attempt. Participants can improve on the second or third attempt. Participants who consistently have large verification errors may have visual or physiological problems that prevent their participation in the experiment. The verification results can be expressed in terms of the degree of the viewing angle and can be displayed graphically.

[0109] The patient-side computing device 130 can include an eye tracking application (or software) configured to extract or receive the raw eye tracking data collected by the eye tracking device 134, as shown in FIG. 2A. The patient-side computing device 130 can generate session data based on the raw eye tracking data and store the raw eye tracking data, along with related information (timestamp information), in a data file (e.g., in a.tsv format,.idf format, or any suitable format), as shown in FIG. 5(b) for example. The session data can also include information about the visual stimuli to be reproduced or presented in another data file (e.g., in a.tsv format or any suitable format), as shown in FIG. 5(a). The information can include timestamp information for each visual stimulus to be reproduced.

[0110] In some embodiments, the patient-side computing device 130 stores a number of predetermined visual stimuli (e.g., movies or video files) that are grouped to correspond to patients in a particular age group and / or symptom group. For example, for an ASD assessment for patients in a first age range (e.g., 5 - 16 months), a first list of predetermined visual stimuli can be configured, and for an ASD assessment for patients in a second age range different from the first age range (e.g., 16 - 30 months), a second list of predetermined visual stimuli can be configured. In some embodiments, an operator can use the operator-side computing device 140 to control which list of predetermined visual stimuli to play for a particular patient based on information about that particular patient. In some embodiments, the operator application sends age information to the eye-tracking application in patient selection, and the eye-tracking application then dynamically selects a pre-set appropriate playlist based on the age information without operator intervention or operator selection. In some embodiments, the number of predetermined visual stimuli can also be stored in the operator-side computing device 140.

[0111] The test method depends on the patient being awake and looking at the screen 132 of the patient-side computing device 130. During both the calibration procedure and the data collection procedure, predetermined movies and / or other visual stimuli are presented to the patient via the patient-side computing device 130. These movies and / or other visual stimuli may include humans or animation actors performing hand / face / body movements. As will be described in further detail in FIG. 5, during the data collection period, the computing system 120 can periodically show the patient a calibration target or a fixation target (which may be animated). This data can be used later to confirm accuracy.

[0112] The visual stimuli presented to the patient (e.g., a movie or video scene) may depend on the patient's age. That is, the visual stimuli can be age-specific. In some embodiments, processing the session data includes measuring the amount of fixation time the patient spends looking at the eyes, mouth, or body of an actor, or other predetermined area of interest, and the amount of time the patient spends looking at background areas in the video. The video scene shown to the patient via the screen 132 of the patient-side computing device 130 may depict a scene of social interaction (e.g., an actor looking directly into the camera, attempting to engage the viewing patient, e.g., or a scene of children at play). In some embodiments, the video scene can include other suitable stimuli, such as, for example, animation and prioritized viewing tasks. The measure of fixation time with respect to a particular spatial location in the video may be related to the level of social and / or cognitive development of the patient. For example, children between 12 and 15 months of age show increasing mouth fixation as a result of their stage of language development, going back and forth between eye fixation and mouth fixation. As another example, a decrease in visual fixation over time by the patient with respect to the eyes of an actor in the video may be an indicator of ASD or another developmental condition in the patient. Analysis of the patient's viewing patterns (during the movie presented, and over multiple viewing sessions, or compared to historical data of patients having substantially the same age and / or symptoms) may be performed for the diagnosis and monitoring of developmental, cognitive, social, or mental abilities or disorders, including ASD.

[0113] The operator-side computing device 140 is configured to execute an operator application (or software). In some embodiments, the operator application is installed and executed within the operator-side computing device 140. In some embodiments, the operator application runs on the cloud server 110, and the operator can log in to the web portal to interact with the operator application through a user interface presented on the operator-side computing device 140, as described in further detail with reference to FIGS. 4A - 4J. The operator application can be configured to oversee or control steps of a gaze tracking application or software in the patient-side computing device 130, for example, to select and play specific visual stimuli for the patient and to collect raw gaze tracking data. In some embodiments, the operator application interfaces with the gaze tracking software via a software development kit (SDK). In some embodiments, the communication between the patient-side computing device 130 and the operator-side computing device 140, or between the operator application and the gaze tracking application, can be performed using web socket communication. Web socket communication enables two-way communication between two devices. This two-way communication allows the operator to receive information from the patient-side computing device 130 while simultaneously controlling the patient-side computing device 130. Web socket communication can be performed using a secure implementation of web socket called web socket secure (WSS). As described above, the communication between the patient-side computing device 130 and the operator-side computing device 140 (e.g., the communication between the operator application and the gaze tracking application) can be through the cloud server 110.For example, an operator can use the operator-side computing device 140 to log in to a web portal running on the cloud server 110 and establish a wireless connection with the patient-side computing device 130 for obtaining the patient's eye-tracking data.

[0114] The operator application can additionally be used to perform an interface that presents the operator with other functions, such as information related to the patient, such as the patient's name, date of birth, and information related to the stimuli (e.g., movies) shown to the patient. The operator can also use the operator-side computing device 140 to log in to the web portal of the cloud server 110 for device management, patient management, and data management. In some embodiments, the operator application runs on the cloud server 110 and is controlled by the operator using the operator-side computing device through the web portal. The operator can operate the computing system 120 with minimal training.

[0115] As will be described in further detail with reference to FIGS. 3, 4A-4J, and 5, computing system 120 may be configured for session data acquisition. In some embodiments, a session is initialized by establishing a connection between operator computing device 140 and patient computing device 130. After entering patient information into an operator application (e.g., custom software) running on operator computing device 140, the operator application can control a gaze tracking application running on patient computing device 130 to select age-appropriate stimuli and instruct the operator or the patient's caregiver to position patient computing device 130 in front of the patient in the appropriate orientation and / or location. The operator can use operator computing device 140 to control the operator application and / or the gaze tracking application or software to (a) calibrate the gaze tracking device 134 for the patient, (b) verify that the calibration is accurate, and (c) collect gaze tracking data from the patient as the patient views dynamic video or other visual stimuli in the session, such as by moving their eyes in response to a predetermined movie or other visual stimulus. After the session ends, both the gaze tracking data and information related to the stimuli (e.g., a list of stimuli viewed by the patient) can be stored as session data in two separate data files. The session data can then be automatically transferred, for example, by patient computing device 130, to a secure database in cloud server 110 via network 102. The database can be remote from computing system 120 and can be configured to store and aggregate data collected from several computing systems 120.

[0116] Network 102 can include a large-scale computer network such as a local area network (LAN), wide area network (WAN), Internet, cellular network, or a combination thereof that connects any number of mobile computing devices, fixed computing devices, and server systems. Each of the computing devices 130, 140 in the computing system 120 can communicate with the cloud server 110 through the network 102.

[0117] In some embodiments, the communication between the on-premises computing devices 130, 140 and the cloud server 110 can be performed using the Hypertext Transfer Protocol (HTTP). HTTP follows a request and response model where a client sends a request to the server (e.g., through a browser or a desktop application) and the server sends a response. The response sent from the server can include various types of information such as documents, structured data, or authentication information. HTTP communication can be performed using a secure implementation of HTTP called Hypertext Transfer Protocol Secure (HTTPS). Information passed through HTTPS is encrypted to protect both the privacy and integrity of the information.

[0118] The cloud server 110 can be a computing system hosted within a cloud environment. The cloud server 110 can include one or more computing devices and one or more machine-readable repositories or databases. In some embodiments, the cloud server 110 can be a cloud computing system that includes one or more server computers in a local network or a distributed network, each having one or more processing cores. The cloud server 110 can be implemented in a parallel processing or peer-to-peer infrastructure or on a single device having one or more processors. As an example, FIG. 11 is an architecture for a cloud computing system that can be implemented as the cloud server 110.

[0119] As shown in FIG. 1, the cloud server 110 includes a cloud platform 112 and a data pipeline system 114. As will be described in further detail in FIGS. 2A-2G, the cloud platform 112 can provide a web portal and store application data related to a treatment provider or tenant, and can be configured to store data such as raw eye-tracking data, processed data, analysis results, and / or diagnostic results. The data pipeline system 114 is configured to perform data processing and data analysis.

[0120] In some embodiments, as will be described in further detail with reference to FIGS. 6 and 7A - 7B, the cloud server 110 is configured to automatically receive, process, and analyze session data from multiple computing systems. Moreover, the cloud server can process and analyze session data for multiple sessions from a number of computing systems in parallel, which can significantly improve the session processing speed and provide diagnostic results within a short time period, e.g., within a 24 - hour window. For example, the reception of session data by the cloud server 110 (e.g., by the cloud platform 112) can initiate an automated software implementation process and analysis process (e.g., by the data pipeline system 114). In the process, an individual patient's data can be compared to a model of previously generated gaze - tracking data from the gaze - tracking data of patients with substantially the same age, background, and / or symptoms. The results of the comparison can be a diagnosis of a neurodevelopmental disorder, including, but not limited to, conventional recommendations for ASD, measures of the patient's developmental / cognitive function, and / or treatment plans. Alternatively or additionally, the collected data can be compared and / or examined for a given patient over multiple sessions (and over a given time period) to identify potential changes in visual fixation (e.g., a decrease in visual fixation). Those results can be condensed into a diagnostic report for use by the patient's physician. In some embodiments, when the diagnostic results are ready, the cloud server 110 can transfer the diagnostic results to the operator - side computing device 140, and the diagnostic results can be presented on the user interface of the operator - side computing device 140, as will be described in further detail with reference to FIGS. 8A - 8C, for example.

[0121] In some embodiments, for example, to generate a diagnostic result for a patient, comparison or inference through an artificial intelligence (AI) model such as a statistical model, an algorithm, a machine learning model, or an artificial neural network model is used, and a large amount of model data including data related to patients in similar ages, similar backgrounds, and / or similar circumstances can be used together with the processed session data for the patient, which can significantly improve the accuracy of the diagnostic result.

[0122] Environment 100 involves three main steps corresponding to the three parts of environment 100 shown in FIG. 1 (for example, computing system 120 for data acquisition, cloud platform 112, and data pipeline system 114). As will be described in further detail with reference to FIGS. 2A - 2G, the three parts can be configured together to reliably collect data for a patient and efficiently process and analyze the collected data for the diagnosis of ASD or other cognitive, developmental, social, or mental abilities or disorders.

[0123] FIG. 2A is a block diagram of an exemplary system 200 for assessing developmental disorders through eye tracking according to one or more embodiments of the present disclosure. System 200 can be implemented within environment 100 of FIG. 1. According to the three steps of the data process, system 200 includes three subsystems, namely, data acquisition subsystem 210, platform subsystem 220, and data pipeline subsystem 230. Each subsystem can be composed of corresponding hardware and software components.

[0124] The data acquisition subsystem 210 is configured to collect the patient's eye-tracking data. The data acquisition subsystem 210 can be the computing system 120 of FIG. 1. As shown in FIG. 2A, the data acquisition subsystem 210 includes an eye-tracking console 212 that executes an eye-tracker application 214, and an operator-side computing device (e.g., 140 of FIG. 1) that executes an operator application 216. In some embodiments, the operator application 216 is deployed within the operator-side computing device. In some embodiments, the operator application 216 is deployed within the platform subsystem 220, and the operator can use the operator-side computing device to log in to the platform subsystem 220 through the web portal 222 and execute the operator application 216 on the platform subsystem 220. Deploying the operator application 216 within the platform subsystem 220 can avoid deploying the operator application 216 within one or more operator-side computing devices, which can reduce the software and hardware requirements for the operator-side computing devices. For example, it can enable the operator application 216 to be suitably maintained or updated without maintaining or updating the operator information 216 in each of one or more operator-side computing devices.

[0125] The eye-tracking console 212 can be an integrated device that includes the patient-side computing device 130 (e.g., a tablet) and the eye-tracking device 134 of FIG. 1. As described above, the data acquisition subsystem 210 can include some movie files 218 that are stored within the eye-tracking console 212 and optionally within the operator-side computing device. The movie files 218 can be predetermined age-specific visual stimuli for patients of different ages and / or with different symptoms.

[0126] As described with reference to FIG. 1, the platform subsystem 220 and the data pipeline subsystem 230 can be included within a network-connected server, such as a cloud server (e.g., cloud server 110 of FIG. 1), and can be implemented within a centralized cloud-hosted environment provided by a cloud provider, such as Microsoft Azure. In some embodiments, the platform subsystem 220 is configured for managing and orchestrating resources of the cloud-hosted environment. The platform subsystem 220 can be the cloud platform 112 of FIG. 1. As shown in FIG. 2A, the platform subsystem 220 includes a web portal 222, a database 224 for storing application data, and a database 226.

[0127] The web portal 222 can be a web-based interface. Through the web portal 222, an operator (e.g., a medical professional) can log in to the platform subsystem 220, for example, using an operator-side computing device, to manage (view and / or query) the application data 224 stored in the database 224 and / or the data in the database 226. For example, the web portal 222 enables the operator to view diagnostic results. Based on the diagnostic results, a pre-written process for actions (e.g., requesting further evaluation) may be provided.

[0128] As an example of the database 224, FIG. 2D shows a database 240 for storing various types of documents. The database 240 can be a NoSQL database, such as Azure Cosmos DB. Various types of documents can be stored as application data in the database 240. Unlike relational databases, NoSQL databases do not have strong relationships between documents. The dotted lines in FIG. 2D indicate references and information embedded between the documents.

[0129] In some embodiments, the database 240 stores corresponding application data for a treatment provider (or tenant). The treatment provider can be a healthcare organization including, but not limited to, an autism center, a healthcare facility, an expert, a physician, or a clinical study. The organization may vary in structure, patient volume, and duration. As shown in FIG. 2D, the corresponding application data can include an organization document 242, a user document 244, a device document 246, a patient document 248, a session document 250, and a history document 252. The user can be an operator associated with the healthcare organization, such as a healthcare assistant, an expert, a physician, or any other medical professional.

[0130] The organization document 242 includes settings and customizations for the organization. The user document 244 includes identifier information along with user roles and permissions. The user role indicates whether the user is either an administrator or an operator associated with different security levels or permissions. The device document 246 includes identifier information for each eye-tracking console associated with the organization, such as 212 in FIG. 2A. The patient document 248 includes information about a patient, such as an infant or child treated as a patient for developmental assessment. The session document 250 includes information related to a session that can be composed of a session identifier (session ID), a reference to the patient, a reference to the user executing the session, a pointer to the eye-tracking data, and the results of data processing and data analysis. The history document 252 can be used to maintain a version history of changes to the document. The document mirrors the structure of its parent document and includes additional audit information. In some embodiments, the database 224 enables URL-based queries (e.g., to find people with administrative roles) across multiple variables. For example, the variables can include patient / device / session, adverse events, and the like.

[0131] In some embodiments, a cloud server including the platform subsystem 220 and the data pipeline subsystem 230 can be implemented in a centralized cloud environment that provides more flexibility for extending the capabilities of the cloud server. For example, the cloud server can utilize a multi-tenant architecture to provide a software as a service (SaaS)-based diagnostic service to treatment providers. In a multi-tenant architecture, treatment providers share a single version of the software across various geographical locations. The term "tenant" in a multi-tenant architecture represents a single treatment provider of the system. The resources of the cloud server can be dynamically managed based on the total number of tenants and the expected average workload, e.g., how many tenants are accessing the cloud server at a given point in time. The cloud server can employ horizontal scaling techniques such as auto-scaling to handle sudden increases in resource workload.

[0132] In a multi-tenant architecture where applications are shared, it is important to separate their tenant data and prevent other tenants from accessing that tenant data. This is called isolation. There are three different isolation strategies that can be implemented, namely, a shared database, a tenant-specific database, and a tenant-specific application. In the shared database strategy, tenants share a single instance of the application and all data is stored in a single database. In the tenant-specific database strategy, such as strategy 260 shown in FIG. 2E, panel (a), tenants share a single instance of the application in the application layer 262, but have their own databases 264 (e.g., database 224 of FIG. 2A or 240 of FIG. 2D). In the tenant-specific application strategy, such as strategy 270 shown in FIG. 2E, panel (b), each tenant obtains its own instance of the application in its respective application layer 272, and its own database 274 (e.g., database 224 of FIG. 2A or 240 of FIG. 2D). The cloud server can deploy the tenant-specific database strategy 260 or the tenant-specific application strategy 270 to the treatment provider.

[0133] Continuing to refer to FIG. 2A, the database 226 is configured to store raw gaze tracking data or session data, processed session data, analysis results, and / or diagnostic results or diagnostic reports. The database 226 can be a storage platform (e.g., Azure Blob) and can be paired with tools written in any suitable programming language (e.g., Python, Matlab), enabling URL-based interfaces and queries to the database 226. Additionally, the database 226 can be compatible with the programming language (e.g., Python, Matlab) used to transfer data from the data acquisition subsystem 210 to the database 226 and from the database 226 to the data pipeline subsystem 230. For example, if the patient-side computing device (e.g., 130 in FIG. 1) is located at a medical facility, data collection occurs at that facility, and data is transferred between the database 226 and the patient-side computing device. The database 226 can be secure, HIPAA compliant, and protected by a redundant backup system.

[0134] In some embodiments, the platform subsystem 220 is configured to enable one or more operations including (a) intake of new patient information, (b) storage of raw data files (e.g., including gaze tracking data), (c) automated and secure transfer of files between a data collection device (e.g., the gaze tracking console 212 in FIG. 2A), a data processing computer, and the database, (d) tabulation and querying of data for the purpose of assessing device utilization and other data quality metrics, and (e) access by a physician to the results of the processing. One or more of operations (a)-(c) can be performed by an upload function module 221 within the platform subsystem 220.

[0135] Continuing to refer to FIG. 2A, the data pipeline subsystem 230 is configured to process and analyze patient gaze tracking data along with generating diagnostic results. In some embodiments, the data pipeline subsystem 230 includes a data processing module 232, a data analysis module 234, and model data 236. As will be described in further detail in FIGS. 7A-7B below, the data processing module 232 is configured to process session data including gaze tracking data to obtain processed session data, and the data analysis module 234 is configured to analyze the processed session data using the model data 236 to generate diagnostic results.

[0136] In some embodiments, system 200 includes interfaces for devices and subsystems. There can be interfaces between subsystems. For example, system 200 can also include an interface between the data acquisition subsystem 210 and the cloud platform subsystem 220, and an interface from the cloud platform subsystem 220 to the data pipeline subsystem 230. There can be interfaces within a subsystem. For example, system 200 can include an interface between the gaze tracking console hardware (e.g., a tablet and a gaze tracking device) and the gaze tracking application software.

[0137] FIG. 2B shows an example of processing single session data within the system 200 of FIG. 2A according to one or more embodiments of the present disclosure. As described above, after the data collection session is completed, the eye tracking console 212 can automatically transfer the session data of the session to the platform subsystem 220. The session data can include two files, namely, one including raw eye tracking data (e.g., fixation position coordinates, blink data, pupil size data, or a combination thereof) and the other including information related to the stimuli (e.g., a list or playlist of those movies viewed by the patient). Through the upload function module 221 implemented within the platform subsystem 220, the session data can be stored in the database 226 and can be stored in the application data in the database 224. The stored session data can then be automatically transferred from the platform subsystem 220 to the data pipeline subsystem 230 for data processing and data analysis without human intervention. For example, a software script written in any suitable programming language (e.g., Python, Matlab) can be used to transfer the raw unprocessed data file from the database 226 to the data pipeline subsystem 230 for processing. The session data is first processed by the data processing module 232 and then analyzed by the data analysis module 234, which provides diagnostic information about the patient.

[0138] In some embodiments, three files are generated: a file containing processed gaze-tracking data, a file containing an overview of gaze-tracking statistics, and a file containing diagnostic information. The file containing diagnostic information can be uploaded to the database 224 and associated with a patient within the application data, as shown in FIG. 2D. The three files can then be uploaded to the database 226 for storage. Optionally, the processed gaze-tracking data is tabulated to form a session table. An overview of the gaze-tracking information (e.g., fixation samples / movies, etc.) can be read from the processed overview file and tabulated within the database 226 for subsequent queries. Summary values (e.g., percent fixation / movie, etc.) can then be calculated within the database 226.

[0139] FIG. 2C shows an example of processing multiple session data in parallel within the system 200 of FIG. 2A according to one or more embodiments of the present disclosure. As shown in FIG. 2C, multiple gaze-tracking consoles 212a, 212b can send multiple session data 213a, 213b, 213c (collectively or individually referred to as session data 213) of a session to the platform subsystem 220. Within the data pipeline subsystem 230, the data processing module 232 and the data analysis module 234 can be written in a suitable programming language (e.g., Python), which enables deploying the data processing module 232 and the data analysis module 234 within containers 231a, 231b, 231c (collectively or individually referred to as container 231). Each session can be processed using its own instance of data processing and data analysis. The use of containers enables data processing and data analysis to be performed as session data is uploaded from the data acquisition subsystem 210, which can result in sessions being returned within a short time period, e.g., within a 24-hour window.

[0140] As will be described in further detail with reference to FIGS. 7A-7B, the cloud server can process and analyze session data of several sessions from a number of computing systems in parallel. First, the cloud server can deploy a respective container (e.g., 231) for each session, and each container can include a corresponding data processing module 232 and a corresponding data analysis module 234. In this way, when session data (e.g., 213) of a session is uploaded by the corresponding line-of-sight tracking console 212, the session data of the session can be processed and analyzed using its own container (e.g., 231 having its own instance of data processing and data analysis). Second, for example, while session data of multiple sessions is being processed in a corresponding container using a majority of the processing units (or cores) in the cloud server, model data for analyzing the processed session data can be pre-loaded in parallel into the corresponding container using, for example, the remaining or minority of the processing units in the cloud server. Third, all of the processed session data and the loaded model data can be analyzed in parallel in the corresponding container using, for example, the total number of processing units in the cloud server. The use of parallelization in multiple ways can significantly improve the speed of session data processing and analysis and can provide rapid diagnostic results within a short time period, e.g., within a 24-hour window. For example, when the diagnostic results become available, the cloud server can send the diagnostic results to the corresponding operator-side computing device (e.g., 140 in FIG. 1), and the diagnostic results can then be displayed in the result interface of the operator application 216. Parallelization can also make the cloud server more efficient in resource utilization, which can further improve system performance.

[0141] FIG. 2F shows an exemplary configuration 280 for data backup for the system 200 of FIG. 2A according to one or more embodiments of the present disclosure. The configuration 280 can enable high availability of services to treatment providers, such that treatment providers can access their services regardless of any outage in one or more specific areas of the cloud server (e.g., the platform subsystem 220 and the data pipeline subsystem 230).

[0142] High availability refers to the ability of treatment providers to access their services regardless of whether the cloud service provider experiences an outage. Availability can be achieved by replicating resources in different physical locations. The cloud servers implemented herein can be provided by a cloud service provider that can provide platform as a service (PaaS) resources as a service, with either built-in high availability or configurable high availability. Resources hosted within the cloud environment can have high availability using a high availability service level agreement or through the use of geo-redundancy.

[0143] FIG. 2F shows an example of high availability through geo-redundancy. As shown in FIG. 2F(a), the resources of the cloud server can be hosted within a first data center 282 having a web portal 222a. The resources are replicated within a second data center 284. When the first data center 282 is functioning correctly, the second data center 282b is a mirror and treatment provider traffic is directed to the first data center 282. However, as shown in FIG. 2F(b), when the first data center 282 fails, treatment provider traffic is redirected to the replicated resources within the second data center 284 that execute the replicated web portal 222b. The switching process can be seamless and treatment providers may not notice the switch to different resources within the replicated data center.

[0144] Figure 2G shows an exemplary data backup for system 200, e.g., for platform subsystem 220 and data pipeline subsystem 230. A database 224 that stores application data, as well as a database 226 that stores raw eye-tracking data and processed eye-tracking data and analyzed results or diagnostic results, can be stored among a plurality of data centers. A web portal 222 within the platform subsystem 220, as well as a data processing module 232 and a data analysis module 234 within the data pipeline subsystem 230, and optionally an operator application 216 (running on the platform subsystem 220), can be included within an active data center 282 and replicated within a backup data center 284.

[0145] Exemplary Process for Session Data Acquisition FIG. 3 is a flowchart of an exemplary process 300 for session data acquisition according to one or more embodiments of the present disclosure. Process 300 may be executed by a system, such as computing system 120 of FIG. 1 or data acquisition subsystem 210 of FIG. 2A. The system includes an operator-side computing device (e.g., 140 of FIG. 1), and one or more patient-side computing devices (e.g., 130 of FIG. 1) integrated with a related eye tracking device (e.g., 134 of FIG. 1). Each of the operator-side computing device and the one or more patient-side computing devices can communicate via a network (e.g., network 102 of FIG. 1) with a network-based server or a cloud server (e.g., cloud server 110 of FIG. 1, or a cloud server as described in FIGS. 2A-2G). The system can be related to, for example, a treatment provider that provides developmental disorder assessment and / or treatment services to a patient. The cloud server can be related to a service provider that provides services, such as data processing, data analysis, and services for providing diagnostic results to the treatment provider. For illustration purposes, FIGS. 4A-4J show a series of exemplary display screens presented during session data acquisition on an operator-side computing device (a) and on a patient-side computing device (b) (e.g., in process 300 of FIG. 3) according to one or more embodiments of the present disclosure.

[0146] In step 302, for example, a session is started by establishing a connection or communication between the operator-side computing device and the patient-side computing device. In some embodiments, the two computing devices 130 and 140 can be wirelessly connected via, for example, a wireless connection without using a physical connection. The wireless connection can be through a cellular network, a wireless network, Bluetooth, near-field communication (NFC), or other standard wireless network protocols. In some cases, the patient-side computing device can also be configured to connect to the operator-side computing device through a wired connection such as a universal serial bus (USB) when, for example, the wireless connection fails.

[0147] In some embodiments, the connection between the operator-side computing device and the patient-side computing device is established by the two computing devices communicating with a cloud server that provides communication between the operator-side computing device and the patient-side computing device. For example, as shown in FIG. 4A, an operator (e.g., a medical assistant, a medical professional, or any other person representing a treatment provider) can log in to a web portal (e.g., 222 in FIG. 2A) running on the cloud server for device management, patient management, and data management. The operator can have corresponding user roles and permissions as described, for example, in FIG. 2D. FIG. 4A(a) shows a user interface (UI) presented on the display screen of the operator-side computing device after the operator logs in to the web portal using the operator-side computing device. The UI can be the user interface of an operator application (e.g., 216 in FIG. 2A) running on the cloud server or on the operator-side computing device.

[0148] As shown in FIG. (a) of FIG. 4A, the UI includes a menu showing buttons such as "Home", "Patient", "Device", and "User". By clicking on a button, the corresponding information (e.g., patient information, device information, or user information) can be presented within the UI. For example, when the "Device" button is clicked, the UI shows a list of names of patient-side computing devices that can be controlled by an operator, such as Device 1, Device 2, Device 3, Device 4, Device 5. When patient-side computing devices, such as Device 4, Device 5 are connected to a cloud server, a display indicating "Connected", such as a character string, can be presented adjacent to the name of the patient-side computing device. The operator can select one of the names, such as Device 4, to connect the corresponding patient-side computing device to the operator-side computing device. When a name is selected, the UI shows a request asking for the access code to be entered to connect to the corresponding patient-side computing device, as shown in FIG. (a) of FIG. 4B.

[0149] FIG. (b) of FIG. 4A shows a user interface presented on a patient-side computing device, e.g., device 4, on a screen (e.g., 132 of FIG. 1). For example, the UI may be presented after the patient-side computing device is turned on and logged in by an operator. The UI can show a "Start" button that can be clicked by the operator, for example, to start a session. After the "Start" button is clicked, the patient-side computing device connects to a cloud server, e.g., a web portal. The cloud server can associate the patient-side computing device with the operator based on the identifier of the patient-side computing device, as shown, for example, in FIG. 2D. When the patient-side computing device successfully connects to the cloud server, the UI presented on the patient-side computing device can show information about an access code generated by the web portal for connection with the operator-side computing device, e.g., "5678", as shown in FIG. (b) of FIG. 4B. The operator can obtain the access code from the UI presented on the patient-side computing device, input the access code on the UI presented on the operator-side computing device, and then submit the access code to the web portal. After the web portal confirms that the access code entered in the operator-side computing device matches the access code generator for the patient-side computing device, the web portal can establish a wireless connection between the operator-side computing device and the patient-side computing device.

[0150] When a connection is established between the operator-side computing device and the patient-side computing device (e.g., Device 4), connection information such as "Connected to Device 4" can be displayed on the UI of the operator-side computing device, as shown in, for example, FIG. 4C(a). On the other hand, the UI can show a button for starting to display visual information, such as a movie, to the patient on the screen of the patient-side computing device. A caregiver of the patient, such as a parent, can bring (or carry) the patient to watch the movie presented on the screen of the patient-side computing device.

[0151] In step 304, for example, when the operator clicks a button saying "Start movie" on the UI of the operator-side computing device, desensitization begins, which can cause visual desensitization information (e.g., a movie) to be displayed to the patient on the screen of the patient-side computing device, as shown in FIG. 4C(b).

[0152] During the display of the desensitization movie, data is generally not recorded. Instead, the movie is displayed to gain the patient's attention. The movie may reflexively cause exogenous cuing by the patient without the need for mediation or instructions in words by the operator. For example, since the movie itself gains the patient's attention, the operator does not need to give an instruction to look at the screen of the patient-side computing device.

[0153] As shown in FIG. (b) of FIG. 4D, while the desensitization movie is being displayed on the screen of the patient-side computing device, the operator can select the patient's patient information through the UI of the operator-side computing device as shown in FIG. (a) of FIG. 4D. The operator can, for example, select a patient from a list of existing patients associated with the operator in the cloud server as shown in FIG. 2D, or can create a patient profile for a new patient. After the patient is confirmed, the process begins to set up the eye-tracking device (or the patient-side computing device) with respect to the patient by showing setup information on the UI of the operator-side computing device as shown in FIG. (a) of FIG. 4E. The operator can also select "pause movie" or "skip movie" on the UI of the operator-side computing device.

[0154] During setup, the desensitization movie can continue to play on the screen of the patient-side computing device, as shown in FIGS. 4E(b) and 4F(b). As shown in FIG. 4F(a), on the UI of the operator-side computing device, the relative position between the eye-tracking device and the patient's eyes is indicated, for example, by text or diagrammatically. The relative position can be determined by capturing image data of the patient's eyes using an image acquisition device (e.g., a camera) included in or adjacent to the eye-tracking device. In some embodiments, after the operator clicks a "start setup" button in the UI of the operator-side computing device, the operator application running on the cloud server can send a command to the patient-side computing device to capture an image of the patient's eyes, as shown in FIG. 4E(a). The patient-side computing device can then send the captured image to the cloud server, and the operator application can process the image to determine the relative position between the eye-tracking device and the patient's eyes. The relative position can include the distance between the eye-tracking device and the patient's eyes, the horizontal deviation and / or vertical deviation between the center of the eyes and the center of the field of view (or detection area) of the eye-tracking device. Based on the relative position, the operator application can indicate, on the UI of the operator-side computing device, as shown in FIG. 4F(a), an instruction for adjusting the position of the eye-tracking device, for example, "move the console down". When the relative position between the patient's eyes and the eye-tracking device is acceptable, the operator can confirm the setup, for example, by clicking a "confirm setup" button in the UI. In some embodiments, in response to a determination that the relative location between the patient's eyes and the eye-tracking device is less than a predetermined threshold (e.g., the horizontal / vertical deviation is less than 0.1 cm), the operator application can determine that the setup is complete and can indicate a display to the operator.

[0155] In step 306, the patient is calibrated using the eye tracking device. After the setup is complete, the operator application can present a button for "starting calibration" on the UI of the operator-side computing device, as shown in FIG. 4G(a). In some embodiments, calibration requires the patient to look at one or more fixed known calibration targets (e.g., dots or icons) in the field of view. The calibration target or fixation target reflexively captures the patient's attention, resulting in intermittent movements towards the known target location and fixation on such location. The target ensures fixation on a radially symmetric target that spreads less than 0.5 degrees of visual angle, for example, at a finite location. Other examples include concentric patterns, shapes, or shrinking stimuli that ensure fixation on a fixed target location, even if initially larger in size.

[0156] For example, when the operator clicks the button to start calibration, a plurality of calibration targets can be presented sequentially at a predetermined location (i.e., the target location) (e.g., the center, the upper left corner, or the lower right corner) on the screen of the patient-side computing device, as shown in FIG. 4G(b). While presenting a plurality of calibration targets on the screen of the patient-side computing device, the eye tracking device can be activated to capture the patient's eye tracking calibration data in response to receiving a command from the operator application. To collect the patient's eye tracking calibration data, an eye tracking application (e.g., 214 in FIG. 2A) can be executed on the patient-side computing device.

[0157] In some embodiments, a patient-side computing device (e.g., an eye-tracking application) is configured to determine the position of the corresponding visual fixation on a calibration target and then compare the determined position of the patient's corresponding visual fixation with a predetermined location where the calibration target was presented. Based on the result of the comparison, the eye-tracking application can determine whether the calibration target is calibrated. If the distance between the position of the patient's corresponding visual fixation and the predetermined location for the calibration target is within a predetermined threshold, the eye-tracking application can determine that the patient's corresponding visual fixation aligns with the predetermined location for the calibration target, i.e., the calibration target is calibrated. If the distance is greater than or equal to the predetermined threshold, the eye-tracking application can determine that the patient's corresponding visual fixation does not align with the predetermined location, i.e., the calibration target has failed calibration.

[0158] In some embodiments, the patient-side computing device transmits the captured eye-tracking calibration data of the patient and / or information about the predetermined location to an operator-side computing device or a cloud server, and the operator application can determine the position of the patient's corresponding visual fixation, compare the determined position with a plurality of predetermined locations, and / or determine whether the calibration target is calibrated based on the result of the comparison.

[0159] In some embodiments, the first calibration target can first be presented at the center of the screen, and four additional calibration targets can be presented at each corner of the screen along the rotation direction for calibration to continue. The operator application can alert the operator of the active status of calibration, such as calibrating point 1, calibrating point 2, calibrating point 3, or that the calibration has completed 4 / 5 of the targets. Between each calibration target, a desensitization movie plays for a set time period before a new calibration target is shown. Each calibration target can loop a set number of times before determining that the calibration target has not been calibrated and moving on to the next calibration target. The calibration target can be retried after all remaining calibration targets have been shown and fixation collection has been attempted if the calibration fails.

[0160] In step 308, the calibration is verified. For example, verification can be performed to measure the success of the calibration by showing a new target and measuring the calculated fixation accuracy. The verification can show calibration targets for calibration step 306, for example, fewer than five, for example, three calibration targets. A desensitization movie can play between showing two adjacent calibration targets.

[0161] In some embodiments, based on the result of the comparison between the determined position of the patient's corresponding visual fixation and the predetermined location where the calibration target was presented, initial verification with various levels of success (e.g., the number of calibrated or verified calibration targets) can automatically instruct the operator to (1) recalibrate the eye-tracking device using the patient, (2) re-verify calibration targets that could not be verified, or (3) accept the calibration and proceed to data collection in step 310.

[0162] In some embodiments, the operator may have discretion to decide whether to accept the calibration. As shown in FIG. 4H, on the display screen of the operator-side computing device, calibration targets are simultaneously presented at a plurality of predetermined locations, along with a depiction (e.g., a dot) of the patient's corresponding visual fixation at the determined position of the patient's corresponding visual fixation. The UI can also show a first button for "accept verification" and a second button for "recalibrate". The operator can view the alignment between the plurality of calibration targets and the depiction of the patient's corresponding visual fixation and decide whether to accept the verification (by clicking the first button) or to recalibrate the patient for the eye tracking device (by clicking the second button).

[0163] In step 310, by presenting a playlist of a predetermined visual stimulus (e.g., a stimulus movie) on the screen of the patient-side computing device to the patient, for example, after calibration is verified or the operator accepts the verification, the patient's eye-tracking data is collected. As shown in FIG. 5(a), the list of predetermined visual stimuli can include some social stimulus videos (e.g., 0075PEER, 0076PEER, 0079PEER) specific to the patient, for example, based on the patient's age and / or symptoms. Between each social stimulus video, or before presenting each social stimulus video, a centering video (e.g., a centerstim video) can be shown to temporarily center the patient's gaze. In some embodiments, as shown in FIG. 5(a), for example, a calibration check (similar to that in step 306) is performed in the data collection step while showing the centering video. For example, the calibration check can include indicating five calibration targets, namely, CCTL for the upper left calibration check, CCTR for the upper right calibration check, CCBL for the lower left calibration check, CCCC for the center and center calibration check, and CCBR for the lower right calibration check. Data related to the calibration check can be used in post-processing, for example, to re-calibrate the eye-tracking data and / or to determine the calibration accuracy.

[0164] In a specific example, the sequence of data collection in step 310 can be as follows. 1. Centering stim 2. Stimulus movie 3. Centering stim 4. Stimulus movie or calibration check (e.g., displaying five calibration targets randomly played between 2 to 4 stimulus movies) 5. Repeat steps 1 to 4 until the playlist of the predetermined stimulus movie is completed.

[0165] In some embodiments, as shown in FIG. 4I, the UI on the operator-side computing device shows a button for "Start Collection". After the operator clicks the button for "Start Collection", a playlist of predetermined visual stimuli can be continuously presented on the screen of the patient-side computing device according to a predetermined sequence. On the screen of the operator-side computing device, as shown in FIG. 4I, the UI can show the status of executing the playlist in text form (e.g., Movie in playback: Centering stim), or can show the status of showing the same content as the content presented on the screen of the patient-side computing device (e.g., showing the centering stim video).

[0166] In some embodiments, as shown in FIG. 4J, the UI can show the playlist during execution of the video being played or in playback, e.g., centering stim, PEER1234, centering stim, PEER5678. The UI can also show the video being presented on the screen of the patient-side computing device. The UI can also show a progress bar indicating the percentage of the played stimulus movies in a playlist of predetermined stimulus movies. The UI can also show a button for the operator to skip the movie.

[0167] In some embodiments, for example, the accuracy of gaze-tracking data (e.g., 812 in FIG. 8A) collected via the presentation of visual stimuli that reflexively acquire attention and result in intermittent movement towards and fixation on known target locations can be assessed. The target ensures fixation on a finite location, for example, a radially symmetric target that spreads less than 0.5 degrees of visual angle. Other examples include concentric patterns, shapes, or shrinking stimuli that ensure fixation on a fixed target location, even if initially larger in size. Such stimuli may be tested under data collection using a headrest to ensure that they reliably elicit fixation in an ideal test environment, and then their use can be extended to include data collection without head restraint.

[0168] In some embodiments, the numerical assessment of the accuracy of the collected gaze-tracking data may include the following steps: (1) presenting a fixation target that reliably elicits fixation on a small area of the visual display unit; (2) recording gaze-tracking data over the entire target presentation; (3) identifying fixations in the collected gaze-tracking data; (4) calculating the difference between the fixation location coordinates and the target location coordinates; and (5) storing the calculated differences between the fixation location coordinates and the target location coordinates as vector data (direction and magnitude) for one target, or for as many targets as possible (e.g., 5 or 9, but possibly more). In some embodiments, the recalibration or post-processing step may be performed, for example, by applying a spatial transformation to align the fixation location coordinates with the actual target location coordinates by techniques including, but not limited to: (a) cubic interpolation; (b) linear interpolation in barycentric coordinates; (c) affine transformation; and (d) piecewise polynomial transformation.

[0169] The session ends when the playlist of the specified visual stimuli is completely played. The patient-side computing device can generate session data based on the raw eye-tracking data collected by an eye-tracking device that stores the raw eye-tracking data in a data file (e.g., in.tsv format,.idf format, or any suitable format) along with relevant information (timestamp information), as shown in FIG. 5(b). The raw eye-tracking data can include values for several eye-tracking parameters at different timestamps. The eye-tracking parameters can include fixation coordinate information for the left eye, right eye, left pupil, and / or right pupil.

[0170] As shown in FIG. 5(a), the session data can also include information on the visual stimuli played or presented (e.g., in.tsv format or any suitable format) in another data file. That information can include the timestamp information and name of each visual stimulus played. The timestamp information of the visual stimuli can be associated with the timestamp information of the eye-tracking data, so that the eye-tracking data for each visual stimulus (and / or calibration check) can be determined individually based on the timestamp information in these two data files.

[0171] In step 312, session data is sent to the cloud server. When session data is generated by the patient-side computing device, the patient-side computing device can send the session data to the cloud server. As will be described in further detail with reference to FIGS. 2A-2G, FIGS. 6, 7A-7B, and 8, the cloud server can first store the session data in a centralized database, such as database 226 of FIGS. 2A-2B, and then process the session data, analyze the processed data, and generate a patient diagnosis result, which may be accessible or viewable by an operator or medical professional.

[0172] Exemplary Data Processing and Data Analysis FIG. 6 is a flowchart of an exemplary process 600 for managing session data by a cloud server (e.g., cloud server 110 of FIG. 1 or a cloud server as described in FIGS. 2A-2G), for example, for data processing and data analysis, according to one or more embodiments of the present disclosure. FIGS. 7A-7B show a flowchart of an exemplary process 700 for managing session data by a cloud server in more detail than FIG. 6, according to one or more embodiments of the present disclosure.

[0173] In step 702, if the session is completed, the corresponding patient-side computing device (e.g., 130 in FIG. 1) or the eye-tracking console (e.g., 212 in FIGS. 2A - 2G) sends the session data of the session to the cloud platform of the cloud server through, for example, a web portal. The cloud platform may be the platform 112 in FIG. 1 or the platform subsystem 220 in FIGS. 2A - 2G. In response to receiving the session data, the cloud platform of the cloud server stores the session data in a database (e.g., database 226 in FIGS. 2A - 2G) within the cloud platform. Then, the cloud platform automatically transfers the session data to a data pipeline system (e.g., 114 in FIG. 1 or 230 in FIGS. 2A - 2G) for data processing and data analysis.

[0174] In step 704, a file pointer for the session data of the session is added to the processing queue (step 704). The session data of all completed sessions waits for processing according to the processing queue. As soon as the session data of a session is uploaded and stored in the cloud server, a corresponding file pointer can be assigned to the session data of the session and added to the processing queue. The file pointer may be an identifier for the session data of each session. The session data of each session can be retrieved from the database within the cloud platform based on the file pointer.

[0175] In step 706, for example, based on an auto-scaling technique that can implement session parallelization, a respective container is created for the session data of each session. For example, in response to the addition of a file pointer for a new session to a processing queue, a new container can be created for the new session. Each container (e.g., 231 in FIG. 2C) can have its own instance of, for example, a data processing module and a data analysis module as shown in FIG. 2C.

[0176] Within each container, steps 708 - 714 are executed on the session data of the corresponding session by, for example, the data processing module 232 in FIGS. 2A - 2G. Note that steps 708 - 714 can be executed in parallel for the session data of multiple sessions in multiple containers.

[0177] (Corresponding to step 602 in FIG. 6) In step 708, session data is retrieved from a database in a cloud platform using the corresponding file pointer. As described above, the session data can include two files, namely, a gaze tracking data file (e.g., as shown in FIG. 5(b)) and a playlist file (e.g., as shown in FIG. 5(a)).

[0178] Referring to FIG. 6, step 602 can correspond to step 708. In step 604, session data is prepared for processing. Step 604 can include one or more steps as described in steps 710 - 714 in FIG. 7.

[0179] Step 604 can include linking the eye-tracking data in the eye-tracking data file to the movie being played in the playlist file. In some embodiments, as shown in FIG. 7A, at step 710, the eye-tracking data is broken down into separate runs, for example, based on the timestamp information in these two files. Each run can correspond to playing a corresponding movie (e.g., a centering target, a given visual stimulus, or one or more calibration targets). For example, the eye-tracking data corresponding to timestamps within the range defined by the timestamps of two adjacent movies is included in the run. At step 712, the eye-tracking data within each run is linked to the corresponding movie from the playlist based on the timestamp information in these two files. In some embodiments, the eye-tracking data is not broken down into separate runs, but instead is processed as a continuous stream with data samples linked to the corresponding movies in the playlist.

[0180] At step 714, the eye-tracking data is recalibrated to account for drift or deviation. In some embodiments, the eye-tracking data collected during the calibration step while presenting the playlist can be used, for example, as shown in FIG. 5(a), to calibrate or align the eye-tracking data collected while playing the individual movies in different runs. For example, any discrepancies at the fixation position are corrected using data from adjacent times when an additional calibration target is shown. Some larger discrepancies may involve excluding some data from subsequent analysis.

[0181] In step 606, the prepared session data is processed. In some embodiments, the data processing module extracts relevant information from the prepared session data, such as the patient's visual fixation and / or visual fixation on an object or region of interest in the movie. In some embodiments, the data is resampled to account for any temporal variability between samples. The data can be resampled using any suitable interpolation technique and / or smoothing technique. The data can be converted from the specified original resolution and / or coordinate system of the collected eye-tracking data to an appropriate resolution and / or coordinate system for analysis. For example, the raw data may be collected at a higher resolution (e.g., 1024×768 pixels) than the resolution of the presented stimulus (e.g., resampled to 640×480 pixels). In some embodiments, the data processing module can automatically identify basic oculomotor events (such as unwanted fixations, saccades, blinks, off-screen data, or lost data), and can automatically identify whether the subject was fixating (in an unwanted way), saccading, blinking, or the time the subject was not looking at the screen. The data processing module can adjust for aberrations in the fixation position estimate as an output by the eye-tracking device.

[0182] In some embodiments, as shown in step 716 of FIG. 7B, session data for a plurality of sessions of a patient is processed in a plurality of session containers in parallel with preloading corresponding model data for the patient into the plurality of session containers. In some examples, while the corresponding model data is preloaded in parallel into the plurality of session containers using a small number of processing units (e.g., M processing cores) in a cloud server, the session data for the plurality of sessions is being processed in the plurality of session containers using a majority of the processing units (e.g., N processing cores) in the cloud server. The processing unit or core can be a central processing unit (CPU). Parallelization can avoid additional time waiting to upload the model data.

[0183] The cloud server can store the model data in advance in a database, such as 226 of FIGS. 2A-2G. The model data can include data of a number of instances of significant differences in the gaze position for patients (e.g., infants, toddlers, or children) across varying levels of social function, cognitive function, or developmental function. The corresponding model data for a patient can include data related to patients with similar age, similar background, and / or similar symptoms, and such data can be used together with the processed session data for the patient to generate a diagnosis result for the patient. The corresponding model data for a patient can be identified and retrieved from the database, for example, based on the patient's age, the patient's background, and / or the patient's symptoms. Step 608 of process 600, where processed data is prepared for analysis, can include obtaining the processed data in the plurality of session containers and preloading the corresponding model data into the plurality of session containers.

[0184] In step 610, the processed data is analyzed to generate an analyzed result. In some embodiments, for a session, the processed data is compared with corresponding model data in a corresponding session container to obtain a comparison result. In some embodiments, the data analysis module uses, for example, comparison or inference via an artificial intelligence (AI) model such as a statistical model, an algorithm, a machine learning model, or an artificial neural network model to generate a result using the processed data and the corresponding model data. In some embodiments, as shown in step 718 of FIG. 7B, among a plurality of session containers, the processed session data and the pre-loaded model data are analyzed in parallel using a total number of processing units, for example, N+M cores.

[0185] In some embodiments, the processed session data is compared with a corresponding data model to determine the level of developmental, cognitive, social, or mental symptoms. The generated score is then compared with a predetermined cut-off or other value to determine the diagnosis of a patient with ASD and the level of symptom severity. In some other embodiments, the patient's gaze point data (e.g., visual gaze data) is analyzed over a predetermined time period (e.g., over a plurality of sessions spanning several months) to identify a decrease, increase, or other significant change in visual gaze (e.g., the gaze point data initially corresponding to that of a typically developing child changes to more unstable gaze point data corresponding to that of children presenting ASD, or to gaze point data more similar to that of typically developing children in response to the targeted therapy).

[0186] In step 720 (corresponding to step 612), an overview of the results is calculated. As described above, the analyzed results can be used to determine scores for at least one indicator, such as a social impairment indicator, a language ability indicator, a non - language ability indicator, a social adaptability indicator, and / or a social communication indicator. Based on a comparison of the scores with at least one predetermined cut - off value, a diagnosis of a patient with ASD as well as a level of symptom severity can be calculated. For example, as shown in FIG. 8A, based on the analyzed results and / or any other suitable information (e.g., from other relevant analyses in the patient), a social impairment indicator score of 6.12 is shown within a range from - 50 (social impairment) to 50 (social ability), indicating no problem with social impairment, a language ability indicator score of 85.89 is shown within a range from 0 to 100, indicating above - average language ability, and a non - language ability indicator score of 85.89 is shown within a range from 0 to 100, indicating above - average non - language ability. Moreover, based on the analyzed data, a diagnosis of not having ASD can also be calculated.

[0187] In some embodiments, the summary of results includes a visualization of the patient's eye-tracking data (e.g., fixation point data) overlaid on a movie still photo from a socially relevant moment, enabling clinicians and parents to better understand how the patient visually attends to social information. For example, at step 610, movie still photos that the patient has available data for can be cross-referenced against a list of movie still photos that have been pre-determined to elicit eye gaze behavior using information about the diagnostic status including symptom severity. The visualization can also include, for example, a visualization of aggregated reference data from typically developing children that is matched in patient attributes such as age, gender, etc. These visualizations can be side-by-side so that clinicians and / or parents can compare the individual patient data to the reference data and see how the fixation patterns align or deviate. These visualizations may include annotations explaining the movie content, eye gaze patterns, and various other things.

[0188] In some embodiments, the summary of results includes an animation that visualizes the patient's eye-tracking data overlaid on a movie still photo from a socially relevant moment. For example, the web portal may include a dashboard that enables a clinician to view a stimulus movie with the patient's eye gaze data overlaid, shown to the patient. The dashboard may be configurable to enable a user to select which movie to visualize and whether to visualize a frame for information about which social impairment metrics, language ability metrics, non-verbal metrics, or any other metrics were calculated in the report.

[0189] Continuing to refer to FIG. 7B, at step 722, the data pipeline subsystem causes the result output to be returned to the web portal, as shown, for example, in FIG. 2B. In some embodiments, the result output includes three files, namely, a file containing processed gaze-tracking data, a file containing an overview of gaze-tracking statistics, and a file containing diagnostic information (e.g., an overview of the results). The three files can then be uploaded to a database (e.g., 226 in FIGS. 2A-2G) for storage. Optionally, the processed gaze-tracking data is tabulated to form a session table. An overview of the gaze-tracking information (e.g., fixation samples / movies, etc.) can be read from the processed overview file and tabulated in the database for subsequent queries. Summary values (e.g., percent fixation / movie, etc.) can then be calculated within the database.

[0190] At step 724, the result output is recombined with the patient information to generate a diagnostic report or diagnostic result for the patient. For example, the file containing the diagnostic information can be uploaded to the application data database (e.g., 224 in FIGS. 2A-2G) to be associated with the patient within the application data, as shown, for example, in FIG. 2D. The diagnostic report or diagnostic result can be presented to the user associated with the patient (e.g., an operator or a medical professional such as a doctor) or a caregiver associated with the patient within the application data database in any suitable manner.

[0191] In some embodiments, when a diagnostic report or diagnostic result for a patient is generated, the user may be notified (e.g., by email or message) to log in through a web portal to view the diagnostic report or diagnostic result. The diagnostic report or diagnostic result may be presented on a user interface, for example, as shown in FIG. 8A or FIGS. 8B - 8C. In some embodiments, when a diagnostic report or diagnostic result for a patient is generated, the diagnostic report or diagnostic result may be sent to an operator-side computing device for presentation to the user. The diagnostic report or diagnostic result may also be sent to the operator in a secure email or message. The diagnostic report or diagnostic result may be stored in an application data database (e.g., 224 of FIGS. 2A - 2G) and / or a database (e.g., 226 of FIGS. 2A - 2G).

[0192] FIG. 8A shows an exemplary result interface 800 that displays a diagnostic report or diagnostic result including at least one metric value based on eye-tracking data, according to one or more embodiments of the present disclosure. The result interface 800 shows patient information 802, requested doctor / group information 804, device ID 806 of the patient-side computing device, processing date 807 (indicating the time to obtain session data for processing), and report issuance date 808.

[0193] The result interface 800 also shows collected information 810, including calibration accuracy 812, oculomotor function 814, and data collection summary 816. The calibration accuracy 812 and oculomotor function 814 may be presented graphically. The data collection summary 816 can include, for example, the number of videos viewed, the number of videos excluded, the duration of the data collected, the time spent viewing videos, the time spent not viewing videos, at least one of calibration accuracy, oculomotor scale, or quality control scale.

[0194] The result interface 800 also shows a neurodevelopment test result 820 that can include a diagnosis result 822 (e.g., ASD or non-ASD), social impairment index information 824, language ability index information 826, and non-verbal ability index information 828. The result interface 800 can graphically display these index information 824, 826, 828 along with corresponding explanations.

[0195] Figures 8B - 8C show another exemplary result interface 850 that displays behavior-based measures of developmental assessment in instances of non-verbal communication and gestures (A) and joint attention and mutual gaze (B) in Figure 8B, and facial emotions (C) and pointing and social monitoring (D) in Figure 8C, according to one or more embodiments of the present disclosure.

[0196] The Results Interface 850 indicates the individual vulnerability behavioral-based metrics of the children and the opportunities for skill development. Neurodevelopmental assessment via eye tracking measures how children engage with social and non-social cues that occur continuously within a naturalistic environmental context (shown as still frames from the test video, left column 852). With respect to those contexts, normative reference metrics provide an objective quantification of not having ASD and age-expected visual engagement (central column 854 illustrated as a density distribution in a pseudo-color format and central column 856 illustrated as a fade from color to grayscale overlaid on the corresponding still frame). The age-expected reference metrics can be used to measure and visualize patient comparisons that reveal individual strengths, vulnerabilities, and opportunities for skill improvement (right column 858, exemplary patient data shown as an overlaid circular aperture that encompasses the foveal portion of the video for each patient; for example, each aperture extends approximately 5.2 degrees around the center of the patient's visual field). Individual patients with ASD appear not to fixate on instances of (A) verbal and non-verbal conversations and gestures (860), (B) joint attention and mutual gaze cuing (870), (C) dynamic facial affect (880), and (D) joint attention and social monitoring (890). As shown in FIGS. 8B-8C, children with ASD appear to engage with target toys (1, 3, 5, 7), color and contrast cues (2, 6, 8), objects (10, 11, 12), background elements not directly related to the social context (4, 9, 13), and recursive visual features (14, 15, 16, 17, 18). The elapsed time in the lower right of the still frame emphasizes the rapidly changing nature of the social conversation, and within approximately 12 minutes of viewing time, hundreds of verbal and non-verbal communication cues are presented, each of which elicits an age-expected pattern of engagement and provides a corresponding opportunity for objective quantitative comparison of patient behavior.

[0197] Exemplary Process FIG. 9 is a flowchart of an exemplary process 900 for session data acquisition according to one or more embodiments of the present disclosure. Process 900 may be executed by a system, such as computing system 120 of FIG. 1 or data acquisition subsystem 210 of FIGS. 2A-2G. Process 900 may be similar to process 300 of FIG. 3 and may be described with reference to FIGS. 4A-4J.

[0198] The system includes one or more patient-side computing devices (e.g., 130 of FIG. 1) integrated with an operator-side computing device (e.g., 140 of FIG. 1) and an associated eye-tracking device (e.g., 134 of FIG. 1). At least one of the operator-side computing device or the patient-side computing device may be a portable device. Each of the operator-side computing device and the one or more patient-side computing devices can communicate via a network (e.g., network 102 of FIG. 1) with a network-based server or a cloud server (e.g., cloud server 110 of FIG. 1 or a cloud server as described in FIGS. 2A-2G). The system can be associated with, for example, a treatment provider that provides developmental disorder assessment and / or treatment services to patients. The cloud server can be associated with a service provider that provides services, such as data processing, data analysis, and services for providing diagnostic results to the treatment provider. Process 900 can include several steps, some of which are executed by the operator-side computing device, some of which are executed by the patient-side computing device and / or the eye-tracking device, and some of which are executed by a combination of the operator-side computing device and the patient-side computing device.

[0199] In step 902, a session for the patient is started by establishing communication between the operator-side computing device and the patient-side computing device. In some embodiments, establishing the communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device, such as the wireless connection 131 in FIG. 1.

[0200] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes accessing, by the operator-side computing device, a web portal (such as 222 in FIGS. 2A-2G) in a network-connected server, and in response to receiving a selection of the patient-side computing device in the web portal, wirelessly connecting the operator-side computing device to the patient-side computing device.

[0201] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes, for example, as shown in FIG. 4B, displaying, by the patient-side computing device, connection information on the screen of the patient-side computing device, and in response to receiving an input of the connection information by the operator-side computing device, establishing a wireless connection between the operator-side computing device and the patient-side computing device.

[0202] In some embodiments, process 900 further includes, for example, as shown in FIG. 4C, after establishing the communication, displaying visual desensitization information to the patient on the screen of the patient-side computing device. The eye-tracking device may be configured not to collect the patient's eye-tracking data while the visual desensitization information is being displayed.

[0203] In some embodiments, process 900 further includes, for example, as shown in FIGS. 4D and 4E, accessing, by an operator-side computing device, a web portal in a network-connected server to set up a patient session while displaying visual desensitization information. Optionally, setting up the session includes one of selecting a patient from a list of patients or creating a patient profile in the network-connected server.

[0204] In some embodiments, process 900 further includes, for example, as shown in FIG. 4F, determining a relative position between a gaze tracking device and at least one eye of a patient and displaying, on a user interface of an operator-side computing device, instructions for adjusting the position of the gaze tracking device or the position of the patient. Optionally, process 900 further includes determining that the patient is aligned with the gaze tracking device in response to a determination that a relative location of at least one eye of the patient is at a predetermined location within a detection area of the gaze tracking device.

[0205] In step 904, for example, as shown in FIG. 4G, the patient is calibrated for the gaze tracking device by displaying one or more calibration targets to the patient on a screen of a patient-side computing device. While capturing the patient's gaze tracking calibration data using the gaze tracking device, each of the one or more calibration targets may be continuously presented at a corresponding predetermined location on the screen of the patient-side computing device. Process 900 can include, for each of the one or more calibration targets, processing the captured gaze tracking calibration data of the patient to determine a corresponding visual fixation position of the patient with respect to the calibration target, comparing the corresponding visual fixation position of the patient to the corresponding predetermined location at which the calibration target is presented, and determining, based on the result of the comparison, whether the calibration target is calibrated for the gaze tracking device.

[0206] In some embodiments, calibrating a patient for a gaze tracking device further includes determining that a calibration target is calibrated and displaying a next calibration target in response to a determination that a deviation between a location of a corresponding visual fixation of the patient and a corresponding predetermined location is below a predetermined threshold, or determining that a calibration target is not calibrated and redisplaying the calibration target for calibration in response to a determination that the deviation is greater than the predetermined threshold.

[0207] In some embodiments, process 900 further includes verifying the calibration using one or more new calibration targets after calibrating the patient for the gaze tracking device. Similar to the calibration described in step 904, verifying the calibration includes continuously presenting each of the one or more new calibration targets at corresponding predetermined locations on the screen of the patient-side computing device while capturing the patient's gaze tracking calibration data using the gaze tracking device, and processing the captured gaze tracking calibration data of the patient to determine the location of the corresponding visual fixation of the patient for each of the one or more new calibration targets.

[0208] In some embodiments, for example, as shown in FIG. 4H, verifying the calibration includes simultaneously presenting one or more new calibration targets at one or more corresponding predetermined locations and depictions of one or more corresponding visual fixations of the patient at the determined one or more locations on the user interface of the operator-side computing device, and determining that the calibration is verified in response to receiving an indication to validate the result of the calibration, or initiating re-calibrating the patient for the gaze tracking device in response to receiving an indication to invalidate the result of the calibration.

[0209] In some embodiments, verifying calibration involves determining the number of new calibration targets that each pass the calibration, based on the corresponding visual fixation position of the patient and the corresponding predetermined position, and determining that the calibration is valid if that number or associated percentage is above a predetermined threshold, or determining that the calibration is invalid and initiating re-calibration of the patient for the eye tracking device if that number or associated percentage is less than the predetermined threshold.

[0210] In step 906, following the determination that the calibration is valid, while collecting the patient's eye tracking data using the eye tracking device, a list of predetermined visual stimuli is continuously presented to the patient on the screen of the patient-side computing device.

[0211] In some embodiments, for example, as shown in FIGS. 4I, 4J, or 5, a centering target may be presented to the patient on the screen of the patient-side computing device to center the patient's fixation before presenting each of the list of predetermined visual stimuli.

[0212] [[ID=I1]] In some embodiments, for example, as shown in FIG. 5, calibration of the patient for the eye tracking device is performed while presenting two adjacent visual stimuli out of a predetermined playlist of visual stimuli. The eye tracking data collected when performing the calibration can be used for at least one of calibrating the patient's eye tracking data or determining the calibration accuracy by a network-connected server.

[0213] In some embodiments, for example, as shown in FIG. 4J, process 900 further includes presenting, on a user interface of the operator-side computing device, at least one of a progress indicator that continuously updates throughout the presentation of a predetermined playlist of visual stimuli, information about visual stimuli that have already been presented or are being presented, information about visual stimuli to be presented, or user interface elements for skipping visual stimuli in a predetermined playlist of visual stimuli.

[0214] In step 908, the session data of the session is transmitted by the patient-side computing device to a network-connected server, and the session data includes the patient's eye-tracking data collected during the session. The patient-side computing device can automatically transmit the session data of the session to the network-connected server in response to one of a determination of the completion of presenting a predetermined playlist of visual stimuli on the screen or, for example, receiving a session completion indication from the operator-side computing device through a web portal on the network-connected server.

[0215] In some embodiments, the session data can include information related to the presented playlist of predetermined visual stimuli, for example, as shown in FIG. 5(a), the name of the predetermined visual stimulus presented and the associated timestamp when the predetermined visual stimulus is presented. The session data can include, for example, as shown in FIG. 5(b), eye-tracking data and the associated timestamp when the eye-tracking data is generated or collected. In some embodiments, transmitting the session data includes transmitting a first file storing the patient's eye-tracking data and a second file storing information related to the presented list of predetermined visual stimuli.

[0216] FIG. 10 is a flowchart of an exemplary process 1000 for data processing and data analysis according to one or more embodiments of the present disclosure. Process 1000 can be executed by a network-connected server, which can be a cloud server in a cloud environment, such as cloud server 110 of FIG. 1 or a cloud server as described in FIGS. 2A-2G. For example, the network server can include a platform, such as 112 of FIG. 1 or 220 of FIGS. 2A-2G, and a data pipeline system, such as 114 of FIG. 1 or 230 of FIGS. 2A-2G. The platform can include a web portal (e.g., 222 of FIGS. 2A-2G), an application data database (e.g., 224 of FIGS. 2A-2G), and a database (e.g., 226 of FIGS. 2A-2G). The data pipeline system can include one or more data processing modules (e.g., 232 of FIGS. 2A-2G) and one or more data analysis modules (e.g., 234 of FIGS. 2A-2G). Process 1000 can be similar to process 600 of FIG. 6 or process 700 of FIGS. 7A-7B.

[0217] In step 1002, for example, as shown in FIG. 2B, session data of multiple sessions is received, and the session data of each session includes the gaze tracking data of the corresponding patient in the session. In step 1004, the session data of multiple sessions is processed in parallel to generate processed session data for the multiple sessions. In step 1006, for each session of the multiple sessions, the processed session data of the session is analyzed based on the corresponding reference data to generate an assessment result for the corresponding patient in the session. [[ID=,5]]

[0218] In some embodiments, process 1000 further includes loading the corresponding reference data for the multiple sessions in parallel with processing the session data of the multiple sessions.

[0219] In some embodiments, the network-connected server includes a plurality of processing cores. Processing session data of multiple sessions in parallel can include, for example, using a first plurality of processing cores to process session data of multiple sessions in parallel, as shown in step 716 of FIG. 7B, and using a second different plurality of processing cores to load corresponding reference data for the multiple sessions. The number of the first plurality of processing cores may be greater than the number of the second plurality of processing cores. In some embodiments, analyzing the processed session data of multiple sessions based on the loaded corresponding reference data for the multiple sessions can include, for example, using a plurality of processing cores including the first plurality of processing cores and the second plurality of processing cores, as shown in step 718 of FIG. 7B.

[0220] In some embodiments, analyzing the processed session data of multiple sessions based on the loaded corresponding reference data for the multiple sessions includes at least one of comparing the processed session data of the sessions with the corresponding reference data, using the corresponding reference data to infer an assessment result for the corresponding patient from the processed session data, or using at least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model.

[0221] In some embodiments, the corresponding reference data includes historical gaze-tracking data or results for patients having substantially the same age or symptoms as the corresponding patient. In some embodiments, process 1000 includes generating an assessment result based on previous session data of the corresponding patient.

[0222] In some embodiments, for example, as shown in FIG. 2C or FIG. 7A, for each of a plurality of sessions, a respective container is assigned for the session. Process 1000 can include, within each respective container, processing the session data of the session and analyzing the processed session data of the session based on the corresponding model data to generate an assessment result for the corresponding patient in the session.

[0223] In some embodiments, during collection of eye-tracking data within a session, the eye-tracking data is associated with a list of predetermined visual stimuli presented to the patient, and the session data includes information related to the list of predetermined visual stimuli in the session.

[0224] In some embodiments, process 1000 further includes, within each respective container, decomposing the eye-tracking data into a plurality of portions based on information related to the list of predetermined visual stimuli, as shown in FIG. 7A for example, with each portion of the eye-tracking data being associated with one of the respective predetermined visual stimuli or corresponding calibrations.

[0225] In some embodiments, processing the session data of the session includes processing the portion of the eye-tracking data associated with each respective predetermined visual stimulus based on information for each respective predetermined visual stimulus. In some embodiments, process 1000 further includes, within each respective container, recalibrating the portion of the eye-tracking data associated with each respective predetermined visual stimulus based on at least one portion of the eye-tracking data associated with the corresponding calibration.

[0226] In some embodiments, process 1000 further includes, within each respective container, determining calibration accuracy using at least one portion of the eye-tracking data associated with the corresponding calibration and a plurality of predetermined locations at which a plurality of calibration targets are presented in the corresponding calibration.

[0227] In some embodiments, receiving session data for multiple sessions includes, for example, as shown in FIG. 2C, receiving session data for multiple sessions from multiple computing devices associated with a corresponding entity through a web portal.

[0228] In some embodiments, process 1000 further includes, for example, as shown in FIG. 7A, adding a file pointer for the session data of a session to a processing queue to be processed in response to receiving the session data of the session. Process 1000 can further include storing the session data of the session into a database using the file pointer for the session, and retrieving the session data of the session from the database using the file pointer for the session.

[0229] In some embodiments, process 1000 further includes, for example, as shown in FIG. 2E, storing, for each entity, session data from one or more computing devices associated with the entity into respective repositories in an application data database. Each repository can be separated from one or more other repositories and may be inaccessible by one or more other entities. The application data database can be a NoSQL database.

[0230] In some examples, each repository for an entity includes at least one of, for example, as shown in FIG. 2D, information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted in the entity, information about one or more patients associated with the entity, or historical information of each repository.

[0231] In some embodiments, process 1000 further includes dynamically adjusting the resources of the network-connected server based on, for example, the number of computing devices accessing the network-connected server as shown in FIG. 2F. Process 1000 can further include replicating the data of the first data center to the second data center and automatically directing traffic to the second data center in response to a determination that the first data center is inaccessible.

[0232] In some embodiments, each of the first data center and the second data center includes at least one instance of a web portal, an operator application, or an application layer for data processing and data analysis that is accessible to an operator-side computing device, as shown in FIG. 2G, for example. The process can further include storing the same data in multiple data centers. The data can include application data for an entity and information related to gaze tracking data.

[0233] In some embodiments, process 1000 further includes associating the generated assessment result with the corresponding patient in the session and generating an assessment report for the corresponding patient, as shown in step 724 of FIG. 7B, for example.

[0234] In some embodiments, process 1000 further includes outputting an assessment result or an assessment report to be presented in the user interface of the operator-side computing device, for example, through a web portal.

[0235] In some embodiments, for example, as shown in FIG. 8A, the assessment report includes at least one of information about the corresponding patient, information about the entity that performs the session for the corresponding patient, information about the calibration accuracy in the session, information about session data collection, or the assessment result for the corresponding patient. In some embodiments, the assessment result indicates that the corresponding patient may have a developmental, cognitive, social, or mental disorder or ability. For example, the assessment result indicates whether the corresponding patient has or does not have an autism spectrum disorder (ASD). In some embodiments, the assessment result includes, for example, as shown in FIG. 8A, respective scores for each of one or more of social disorder indicators, language ability indicators, and non-verbal ability.

[0236] In some embodiments, the corresponding patient has an age in the range of 5 months to 7 years, with the age in the range of 5 months to 43 or 48 months, the age in the range of 16 months to 30 months, the age in the range of 18 months to 36 months, the age in the range of 16 months to 48 months, or the age in the range of 16 months to 7 years.

[0237] Exemplary Cloud Computing System Architecture FIG. 11 is an exemplary architecture 1100 for a cloud computing system (e.g., the cloud server 110 described with reference to FIG. 1 or the cloud server described with reference to FIGS. 2A-2G) according to one or more embodiments of the present disclosure. Other architectures are possible, including architectures having more or fewer components. In some implementations, architecture 1100 includes one or more processors 1102 (e.g., dual-core Intel® Xeon® processors), one or more network interfaces 1106, one or more storage devices 1104 (e.g., hard disks, optical disks, flash memory), and one or more computer-readable media 1108 (e.g., hard disks, optical disks, flash memory, etc.). These components can communicate and exchange data via one or more communication channels 1110 (e.g., buses) that utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0238] The term "computer-readable media" refers to any media that participates in providing instructions to processor 1102 for execution, including, but not limited to, non-volatile media (e.g., optical disks or magnetic disks), volatile media (e.g., memory), and transmission media. Transmission media includes, but is not limited to, coaxial cables, copper wire, and fiber optics.

[0239] Computer-readable media 1108 can further include instructions 1112 for an operating system (e.g., Mac OS® server, Windows® NT server, Linux® server), instructions 1114 for a network communication module, data processing instructions 1116, and interface instructions 1118.

[0240] The operating system can be multi-user, multi-processing, multi-tasking, multi-threading, real-time, etc. The operating system performs basic tasks including, but not limited to, recognizing inputs from and providing outputs to devices 1102, 1104, 1106, and 1108, tracking and managing files and directories on computer-readable medium 1108 (e.g., memory or storage device), controlling peripheral devices, and managing traffic on one or more communication channels 1110. The network communication module includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols such as TCP / IP, HTTP, etc.), as well as various components for creating a distributed streaming platform using, for example, Apache Kafka (trademark). The data processing instructions 1116 include server-side or backend software for performing server-side operations as described with reference to FIG. 1. The interface instructions 1118 include software for implementing a web server and / or portal for sending and receiving data between user-side computing devices and service-side computing devices.

[0241] Architecture 1100 can be implemented by a cloud computing system and can be included in any computer device including one or more server computers in a local network or distributed network, each having one or more processing cores. Architecture 1100 can be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device having one or more processors. The software can include multiple software components or can be a single piece of code.

[0242] Exemplary Computing Device Figure 12 shows an architecture for a computing device according to one or more embodiments of the present disclosure. Referring now to Figure 12, a schematic diagram of device 1200 is illustrated. Device 1200 includes a processor 1204, a memory 1206, a storage component 1208, an input interface 1210, an output interface 1212, a communication interface 1214, and a bus 1202. In some embodiments, device 1200 corresponds to at least one of the patient-side computing device 130 or the operator-side computing device 140 of Figure 1.

[0243] Bus 1202 includes components that enable communication between the components of device 1200. In some embodiments, processor 1204 is implemented in hardware, software, or a combination of hardware and software. In some examples, processor 1204 includes a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.). Memory 1206 includes random access memory (RAM), read only memory (ROM), and / or another type of dynamic and / or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores data and / or instructions for use by processor 1204.

[0244] The memory component 1208 stores data and / or software related to the operation and use of the device 1200. In some examples, the memory component 1208 includes a hard disk (such as a magnetic disk, optical disk, magneto-optical disk, solid state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, a CD-ROM, a RAM, a PROM, an EPROM, a FLASH-EPROM, an NV-RAM, and / or another type of computer-readable medium together with a corresponding drive.

[0245] The input interface 1210 includes components that enable the device 1200 to receive information, such as via a user input section (such as a touch screen display, keyboard, keypad, mouse, button, switch, microphone, camera, etc.). Additionally or alternatively, in some embodiments, the input interface 1210 includes sensors that sense information (such as a global positioning system (GPS) receiver, accelerometer, gyroscope, actuator, etc.). The output interface 1212 includes components that provide output information from the device 1200 (such as a display, speaker, one or more light emitting diodes (LEDs), etc.).

[0246] In some embodiments, communication interface 1214 includes components such as a transceiver (e.g., a transceiver, separate receiver, and transmitter, etc.) that enable device 1200 to communicate with other devices via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. In some examples, communication interface 1214 enables device 1200 to receive information from and / or provide information to another device. In some examples, communication interface 1214 includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, etc.

[0247] In some embodiments, device 1200 executes one or more of the processes described herein. Device 1200 executes these processes based on software instructions stored by a computer-readable medium such as memory 1206 and / or storage component 1208 being executed by processor 1204. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes a memory space located inside a single physical memory device or a memory space spread across multiple physical memory devices.

[0248] In some embodiments, software instructions are read into memory 1206 and / or storage component 1208 from another computer-readable medium or from another device via communication interface 1214. When executed, the software instructions stored in memory 1206 and / or storage component 1208 cause processor 1204 to execute one or more processes described herein. Additionally or alternatively, instead of or in combination with software instructions for executing one or more processes described herein, a wired-connected circuit configuration is used. Accordingly, the embodiments described herein are not limited to any particular combination of hardware circuit configuration and software, unless otherwise specified.

[0249] Memory 1206 and / or storage component 1208 includes a data storage area or at least one data structure (such as, for example, a database). Device 1200 is capable of receiving information from, storing information in, communicating information to, or searching for information stored in the data storage area or at least one data structure in memory 1206 or storage component 1208. In some examples, the information includes network data, input data, output data, or any combination thereof.

[0250] In some embodiments, device 1200 is configured to execute software instructions stored either in memory 1206 and / or in the memory of another device (e.g., another device that is the same as or similar to device 1200). As used herein, the term "module" refers to at least one instruction stored in memory 1206 and / or in the memory of another device that, when executed by processor 1204 and / or by a processor of another device (e.g., another device that is the same as or similar to device 1200), causes device 1200 (e.g., at least one component of device 1200) to perform one or more processes described herein. In some embodiments, a module is implemented in software, firmware, hardware, etc.

[0251] The number and arrangement of the components shown in FIG. 12 are provided as an example. In some embodiments, device 1200 can include additional components, fewer components, different components, or components arranged differently than those shown in FIG. 12. Additionally or alternatively, a set of components of device 1200 (e.g., one or more components) can perform one or more functions described as being performed by another component of device 1200 or another set of components.

[0252] The disclosed examples and other examples can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them. The term “data processing apparatus” encompasses, by way of example, all apparatus, devices, and machines for processing data, including programmable processors, computers, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0253] The system can encompass, by way of example, all apparatus, devices, and machines for processing data, including programmable processors, computers, or multiple processors or computers. The system can include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0254] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including a compiled or interpreted language, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed for execution on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0255] The processes and logical flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform the functions described herein. The processes and logical flows can also be performed by, and the apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0256] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from a read only memory or a random access memory or both. Essential elements of a computer can include a processor for executing instructions, as well as one or more memory devices for storing instructions and data. Generally, a computer can also include, or be operatively coupled to receive from, or transfer data to, one or more mass storage devices for storing data, such as, magnetic disks, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data can include, by way of example, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices, such as, EPROM, EEPROM, and flash memory devices, and magnetic disks. The processor and the memory can be supplemented by, or incorporated in, dedicated logic circuitry.

[0257] Although this specification may describe many details, these should be construed as descriptions of features specific to particular embodiments rather than as limitations within the scope of the claimed invention or of what may be claimed. Some of the features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable subcombination in a plurality of embodiments. Moreover, features may be described above as acting in some combinations and even initially claimed as such, but one or more features from a claimed combination may in some cases be capable of being excised from that combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination. Similarly, although operations may be shown in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the illustrated operations be performed, to achieve desirable results.

[0258] The foregoing is merely an illustration of the principles of the present disclosure, and the systems, devices, and methods may be practiced by things different from the illustrative embodiments presented for purposes of illustration and not limitation. The embodiments and features herein are described in detail with respect to their use for collecting and analyzing a patient's eye-tracking data for the assessment, screening, monitoring, or diagnosis of autism spectrum disorder (ASD), but it will be understood that the systems, devices, and methods may also be applied to other developmental, cognitive, social, or mental abilities or disorders, and other conditions including, but not limited to, language disorders, intellectual disabilities, developmental disorders, with or without the presence of known genetic disorders, as well as attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), head injuries, concussions, sports injuries, and dementia. It will be understood that such data may provide a measure of the degree of stereotypy in normative development, providing an indication of variability in typical development when not indicative of a measure for a disorder. Further, all of the components and other features outlined herein may be combined with each other in any suitable manner and may be adapted and applied to systems outside of medical diagnosis. For example, the interactive visual stimuli of the present disclosure may be used as a therapeutic tool. Further, the data collected may provide a measure of several types of visual stimuli to which the patient preferentially attends. Such measures of preference have applications in both the fields of medical diagnosis and medical therapy, including, for example, the advertising industry or other industries, where data related to visual stimulus preference is of interest, as well as applications without such fields.

[0259] Several implementations have been described. However, it will be understood that various changes may be made without departing from the spirit and scope of the techniques and devices described herein. For example, the phase perturbation or variation method described above may be implemented in a diffraction structure to remove high-frequency artifacts or intermediate-frequency artifacts in the inference pattern. The features shown in each of the implementations may be used independently or in combination with each other. Additional features and variations may also be included within the implementations. Accordingly, other implementations fall within the scope of the appended claims.

Explanation of Signs

[0260] 100 Environment 102 Network 110 Cloud Server 112 Cloud Platform 114 Data Pipeline System 120 Computing System 130 Patient-Side Computing Device 131 Wireless Connection 132 Screen 134 Eye Tracking Device 135 Eye Tracking Unit 136 Housing 140 Operator-Side Computing Device 200 System 210 Data Acquisition Subsystem 212 Eye Tracking Console 213 Session Data 214 Eye Tracker Application 216 Operator Application[[ID='44']] 218 Movie File 220 Platform Subsystem 221 Upload Function Module 222 Web Portal 224 Database, Application Data 226 Database 230 Data Pipeline Subsystem 231 Container 232 Data Processing Module 234 Data Analysis Module 236 Model Data 240 Database 242 Organizational Document 244 User Document 246 Device Document 248 Patient Document 250 Session Document 252 History Document 260 Tenant-Level Database Policy 262 Application Layer 264 Database 270 Tenant-Level Application Policy 272 Application Layer 274 Database 280 Configuration 282 First Data Center 284 Second Data Center 800 Result Interface 802 Patient Information 804 Requested Physician / Group Information 806 Device ID 807 Processing Date 808 Report Issuance Date 810 Collected Information 812 Calibration Accuracy 814 Oculomotor Function 816 Data Collection Summary 820 Neurodevelopmental Test Results 822 Diagnostic Results 824 Social Impairment Indicator Information 826 Language Ability Indicator Information 828 Non-Verbal Ability Indicator Information 850 Result Interface 860 Verbal and Non-Verbal Conversation and Gestures 870 Joint Attention and Mutual Gaze Cueing 880 Dynamic Facial Emotions 890 Co - attention and Social Monitoring 1100 Architecture 1102 Processor 1104 Memory Device 1106 Network Interface 1108 Computer - readable Medium 1110 Communication Channel 1112 Instructions for Operating System 1114 Instructions for Network Communication Module 1116 Data Processing Instructions 1118 Interface Instructions 1200 Device 1202 Bus 1204 Processor 1206 Memory 1208 Memory Component 1210 Input Interface 1212 Output Interface 1214 Communication Interface

Claims

1. A system for developmental assessment via eye tracking, A patient-side mobile computing device equipped with a screen for presenting visual stimuli to the patient, An eye-tracking device, mounted with the patient-side mobile computing device and directed to a fixed position relative to the screen of the patient-side mobile computing device, for collecting eye-tracking data of the patient while the visual stimulus is presented to the patient on the screen of the patient-side mobile computing device, An operator-side mobile computing device is configured to display a user interface on the screen of the patient-side mobile computing device that controls the activation of the visual stimuli presented to the patient. A system equipped with these features.

2. The system according to claim 1, wherein the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection.

3. The system according to claim 2, wherein the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate wirelessly with each other via a network-connected server.

4. The system according to claim 3, wherein the network-connected server comprises a cloud computing system or cloud server implemented in a cloud environment.

5. The system according to claim 3 or 4, wherein the patient-side mobile computing device is configured to transmit data to the network-connected server, and the data comprises the patient's eye-tracking data collected during a session while the visual stimuli from a predetermined list of visual stimuli are presented to the patient.

6. The system according to claim 5, wherein the patient-side mobile computing device is configured to automatically transmit the data in response to the completion of all visual stimuli in the list of predetermined visual stimuli being presented during the session.

7. The system according to claim 5, wherein the patient-side mobile computing device is configured to transmit the data in response to receiving a completion indication from the operator-side mobile computing device or the network-connected server.

8. The system according to claim 7, wherein the operator-side mobile computing device or the network-connected server is configured to generate the completion indicator in response to a decision that the session is ending or the receipt of an input indicating the completion of the session.

9. The system according to any one of claims 5 to 8, wherein the data comprises information of the list of predetermined visual stimuli and the eye-tracking data of the patient collected during the session.

10. The system according to claim 9, wherein the patient-side mobile computing device is configured to transmit the data in two files, the first file comprising the patient's eye-tracking data and associated timestamp information, and the second file comprising the information of the list of visual stimuli.

11. The system according to claim 10, wherein the associated timestamp information for the eye-tracking data comprises a timestamp when the eye-tracking data is generated or collected, and the information for the list of predetermined visual stimuli comprises a timestamp when each of the visual stimuli in the list is presented.

12. The system according to any one of claims 3 to 11, wherein the operator-side mobile computing device is configured to access a web portal in the network-connected server.

13. The system according to claim 12, wherein the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices through the web portal in the network-connected server.

14. The system according to claim 13, wherein the user interface of the operator-side mobile computing device comprises at least one of the following: information of one or more patients associated with the operator, information of one or more patient-side computing devices associated with the operator, or information of the operator.

15. The system according to any one of claims 3 to 14, wherein the patient-side mobile computing device is configured to display connection information, including an access code, on the screen of the patient-side mobile computing device in response to a connection to the network-attached server.

16. The system according to claim 15, wherein the operator-side mobile computing device is configured to connect to the patient-side computing device by receiving the connection information, which includes the access code, as input at the user interface.

17. The system according to claim 15 or 16, wherein the user interface of the operator-side mobile computing device presents a request for connection information in response to receiving a selection of the patient-side mobile computing device from among the one or more patient-side computing devices presented in the user interface.

18. The operator-side mobile computing device, The operator-side computing device, or The aforementioned network-connected server It is configured to present the user interface of the operator application to be executed in one of the following: The system according to any one of claims 3 to 17.

19. The aforementioned operator application Within the aforementioned user interface, a user interface element for initiating desensitization is presented, In response to the selection of the user interface element, it is configured to send a command to the patient-side computing device to play back visual desensitization information. The system according to claim 18.

20. The aforementioned patient-side mobile computing device In response to receiving the command, the visual desensitization information is played back to the patient on the screen of the patient's mobile computing device. The eye-tracking device is configured to control itself so as not to collect the patient's eye-tracking data while the visual desensitization information is displayed on the screen. The system according to claim 19.

21. The system according to claim 19 or 20, wherein the operator application is configured to present the user interface for the operator to set up a session for the patient by selecting the patient from a list of patients or creating a profile for the patient, while the visual desensitization information is displayed on the screen of the patient-side mobile computing device.

22. The system according to any one of claims 19 to 21, wherein the operator application is configured to display instructions in the user interface for adjusting the position of the eye-tracking device relative to the patient or the position of the patient relative to the patient-side mobile computing device, based on the sensing position of the eye-tracking device relative to at least one eye of the patient.

23. The system according to claim 22, wherein the sensing position is determined based on image data of the patient's at least one eye, captured by using an image acquisition device included in or adjacent to the eye-tracking device.

24. The system according to claim 23, wherein the sensing position is determined by the patient-side mobile computing device or the operator application.

25. The aforementioned operator application Selection of user interface elements for calibration within the aforementioned user interface, or In response to one of the decisions that a session has been set up for the aforementioned patient, It is configured to send commands to the patient's mobile computing device for calibration between the patient and the eye-tracking device. The system according to any one of claims 18 to 24.

26. The aforementioned patient-side mobile computing device In response to receiving the command, the device is configured to continuously present one or more calibration targets at one or more predetermined locations on the screen of the patient-side mobile computing device while capturing the patient's eye-tracking calibration data using the eye-tracking device. The system according to claim 25.

27. The aforementioned patient-side mobile computing device For each of the one or more calibration targets, the captured eye-tracking calibration data of the patient is processed to determine the position of the patient's corresponding visual gaze relative to the calibration target. The position of the corresponding visual gaze of the patient is compared with a corresponding predetermined location on the screen where the calibration target is presented. Based on the results of the comparison, it is configured to determine whether the calibration target is calibrated for the patient. The system according to claim 26.

28. The system according to claim 27, wherein the patient-side mobile computing device is configured to determine that the calibration is complete in response to a determination that one or more calibration targets have been calibrated.

29. The aforementioned patient-side mobile computing device It is configured to recalibrate the calibration target in response to a determination that the calibration target is not calibrated. The system according to claim 27 or 28.

30. The system according to any one of claims 26 to 29, wherein the patient-side mobile computing device is configured to replay desensitization information between presenting two adjacent calibration targets.

31. The aforementioned operator application It is configured to initiate verification of the calibration in response to receiving an indication that the calibration is complete. The system according to any one of claims 25 to 30.

32. The aforementioned patient-side mobile computing device In response to receiving a request to verify the calibration, the eye-tracking device captures additional eye-tracking calibration data of the patient while presenting at least one additional calibration target on the screen. Configured to process the captured additional gaze-tracking calibration data of the patient in order to determine the position of the patient's corresponding visual gaze relative to the at least one additional calibration target. The system according to claim 31.

33. The aforementioned patient-side mobile computing device The position of the patient's corresponding visual gaze toward the at least one additional calibration target is compared with a corresponding predetermined location on the screen where the at least one additional calibration target is presented. Based on the results of the comparison, it is configured to determine whether the calibration has been verified. The system according to claim 32.

34. The aforementioned operator application The user interface simultaneously presents the at least one additional calibration target at the corresponding predetermined location and at least one depiction of the patient's corresponding gaze at the determined position of the patient's corresponding gaze toward the at least one additional calibration target. The user interface is configured to present a first user interface element for verifying the calibration and a second user interface element for recalibration. The system according to claim 32.

35. The aforementioned operator application Selecting a user interface element to initiate data collection, or In response to one of the decisions that the calibration is complete or has been verified, The system is configured to send commands to the patient-side computing device for the purpose of collecting the aforementioned data. The system according to any one of claims 18 to 34.

36. The aforementioned patient-side mobile computing device In response to receiving the command, the eye-tracking device is configured to capture the patient's eye-tracking data while continuously presenting the patient with a predetermined list of visual stimuli on the screen of the patient's mobile computing device. The system according to claim 35.

37. The aforementioned patient-side mobile computing device Before presenting each of the predetermined list of visual stimuli, the device is configured to present a centering target to the patient on the screen of the patient-side mobile computing device, The system according to claim 36.

38. The aforementioned patient-side mobile computing device The device is configured to perform patient calibration for the eye-tracking device between the presentation of two adjacent visual stimuli from the list of predetermined visual stimuli. The system according to claim 36 or 37.

39. The system according to claim 38, wherein the eye-tracking data collected when performing the calibration is used for at least one of recalibrating the patient's eye-tracking data or determining the accuracy of the calibration.

40. The aforementioned operator application A progress indicator that continues to update throughout the presentation of the aforementioned list of predetermined visual stimuli. A user interface element for skipping a visual stimulus from the aforementioned list of predetermined visual stimuli. Information from visual stimuli that have already been presented or are currently being presented, or At least one of the visual stimuli to be presented, The user interface is configured to present the following: The system according to any one of claims 36 to 39.

41. The system according to any one of claims 3 to 40, wherein the network-connected server is configured to provide a diagnosis of the patient based on the patient's eye-tracking data, and the diagnosis of the patient comprises at least one index value associated with a developmental disorder.

42. The system according to claim 41, wherein the operator-side mobile computing device is configured to present the diagnostic results within the user interface.

43. The system according to any one of claims 1 to 42, wherein the visual stimulus is predetermined based at least on the patient's age or the patient's symptoms.

44. The system according to any one of claims 1 to 43, wherein each of the visual stimuli comprises at least one of a static visual stimulus, a dynamic visual stimulus, a pre-recorded visual stimulus, a pre-recorded audiovisual stimulus, a live visual stimulus, a live audiovisual stimulus, a two-dimensional stimulus, or a three-dimensional stimulus.

Images