Assessment of developmental disorders by eye tracking

Abstract

Systems, devices, devices, methods, and computer-readable storage media for development assessment using eye tracking are provided. In one aspect, a system for developmental assessment via eye tracking includes a patient-side computing device having a screen for presenting visual stimuli to a patient; an eye tracking device integrated with the patient-side computing device and configured to collect eye tracking data of the patient while presenting the visual stimuli to the patient on a screen of the patient-side computing device; and an operator-side computing device configured to present a user interface to an operator to communicate with the patient-side computing device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 17 / 903,318, filed on September 6, 2022, which is a continuation of and claims the benefit of priority to U.S. Patent Application No. 17 / 902,415, filed on September 2, 2022, which is a continuation of and claims the benefit of priority to U.S. Patent Application No. 17 / 901,369, filed on September 1, 2022, which is a continuation of and claims the benefit of priority to U.S. Patent Application No. 17 / 881,911, filed on August 5, 2022, and claims the benefit of priority to U.S. Provisional Application No. 63 / 350,741, filed on June 9, 2022, the contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to developmental assessment, such as using an eye tracking device to assess a patient for a developmental disorder. Background Art

[0004] Developmental disorders such as autism spectrum disorder (ASD) affect nearly 14% of children in the United States. Diagnostic methods for conditions such as ASD vary widely in terms of objective assessment tools and the experience of treatment providers. In some cases, the use of "best practice" tools provides fairly poor sensitivity and specificity for the condition, especially for toddlers or other young patients. Late diagnosis of developmental disabilities reduces the effectiveness of treatment and often leads to poor outcomes. In addition, treatment providers (e.g., pediatricians or other medical professionals) often lack appropriate tools to objectively measure the progression of these conditions, especially very early in a patient's life. Summary of the invention

[0005] The present disclosure describes methods, devices and systems for evaluating developmental disorders by eye tracking. For example, some systems described herein can be used in methods for diagnosing developmental, cognitive, social or intellectual abilities or disabilities (including autism spectrum disorders (ASD)). The system can optionally be implemented with an improved portable device that implements improved objective measurements and increases convenience for treatment providers and patients (e.g., toddlers or other young patients). In some embodiments, the system includes at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye tracking device. In the specific examples described below, these portable devices can be equipped differently, but both are wirelessly interacted with a network-connected server platform to advantageously collect session data in a comfortable and less intrusive manner for the patient, while also increasing improved control flexibility for the treatment provider. Optionally, the session data collected via the patient-side portable device (e.g., analysis of eye tracking data generated using a display in response to a predetermined, age-appropriate visual stimulus) can be quickly analyzed to output a result interface of at least one index based on objective factors at the operator-side portable device.

[0006] One aspect of the disclosure features a system for developmental assessment via eye tracking, the system comprising: a patient-side mobile computing device including a screen for presenting visual stimuli to a patient; an eye tracking device mounted with the patient-side mobile computing device and oriented at a fixed position relative to the screen of the patient-side mobile computing device to collect eye tracking data of the patient while the visual stimuli are presented to the patient on the screen of the patient-side mobile computing device; and an operator-side mobile computing device configured to present a user interface that controls activation of the visual stimuli presented to the patient on the screen of the patient-side mobile computing device.

[0007] In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection. In some embodiments, the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other wirelessly via a network connection server. In some embodiments, the network connection server includes a cloud computing system or a cloud server implemented in a cloud environment.

[0008] In some embodiments, the patient-side mobile computing device is configured to send data to the network-connected server, the data comprising eye tracking data of the patient collected in a session while presenting visual stimuli of a predetermined list of visual stimuli to the patient.

[0009] In some embodiments, the patient-side mobile computing device is configured to automatically send the data in response to completing all visual stimuli in the predetermined list of visual stimuli presented in the session. In some embodiments, the patient-side mobile computing device is configured to send the data in response to receiving a completion indication from the operator-side mobile computing device or the network connection server. In some embodiments, the operator-side mobile computing device or the network connection server is configured to generate the completion indication in response to determining that the session is over or receiving an input indicating that the session is complete.

[0010] In some embodiments, the data includes a predetermined visual stimulus list and information about the patient's eye tracking data collected in the session. In some embodiments, the patient-side mobile computing device is configured to send the data in two files, the two files including a first file and a second file, the first file including the patient's eye tracking data and associated timestamp information, the second file including the information about the visual stimulus list. In some embodiments, the associated timestamp information about the eye tracking data includes a timestamp when the eye tracking data is generated or collected, and the information about the predetermined visual stimulus list includes a timestamp when each visual stimulus in the list is presented.

[0011] In some embodiments, the operator-side mobile computing device is configured to access a web portal in the network connection server. In some embodiments, the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices through the web portal in the network connection server. In some embodiments, the user interface of the operator-side mobile computing device includes at least one of information of one or more patients associated with the operator, information of one or more patient-side computing devices associated with the operator, or information of the operator.

[0012] In some embodiments, the patient-side mobile computing device is configured to display connection information including an access code on a screen of the patient-side mobile computing device in response to connecting to the network connection server. In some embodiments, the operator-side mobile computing device is configured to connect to the patient-side computing device by receiving input of the connection information including the access code at the user interface. In some embodiments, the user interface of the operator-side mobile computing device presents a request for the connection information in response to receiving a selection of the patient-side mobile computing device from among the one or more patient-side computing devices presented in the user interface.

[0013] In some embodiments, the operator-side mobile computing device is configured to present a user interface of an operator application running on one of the operator-side computing device or the network-connected server. In some embodiments, the operator application is configured to: present a user interface element for starting desensitization in the user interface; and in response to selection of the user interface element, send a command to the patient-side computing device to play the visual desensitization information.

[0014] In some embodiments, the patient-side mobile computing device is configured to: in response to receiving the command, play the visual desensitization information to the patient on the screen of the patient-side mobile computing device, and when the visual desensitization information is displayed on the screen, control the eye tracking device not to collect eye tracking data of the patient.

[0015] In some embodiments, the operator application is configured to present the user interface so that the operator can establish a session for the patient by selecting the patient in a patient list or creating a profile for the patient, while the visual desensitization information is displayed on the screen of the patient-side mobile computing device.

[0016] In some embodiments, the operator application is configured to display instructions in the user interface to adjust the position of the eye tracking device relative to the patient or the position of the patient relative to the patient-side mobile computing device based on the sensed position of the eye tracking device relative to at least one eye of the patient. In some embodiments, the sensed position is determined based on image data of at least one eye of the patient, the image data being obtained using an image acquisition device included in or adjacent to the eye tracking device. In some embodiments, the sensed position is determined by the patient-side mobile computing device or the operator application.

[0017] In some embodiments, the operator application is configured to send a command to the patient-side mobile computing device for calibration between the patient and the eye tracking device in response to one of: selecting a user interface element for calibration in a user interface, or determining to establish a session for the patient.

[0018] In some embodiments, the patient-side mobile computing device is configured to, in response to receiving the command, sequentially present one or more calibration targets at one or more predetermined positions of the screen of the patient-side mobile computing device, while obtaining eye tracking calibration data of the patient using the eye tracking device.

[0019] In some embodiments, the patient-side mobile computing device is configured to: process the eye tracking calibration data obtained for the patient for each of the one or more calibration targets to determine the position of the patient's corresponding visual gaze for the calibration target; and compare the position of the patient's corresponding visual gaze with the corresponding predetermined position of the calibration target presented on the screen; and determine whether the calibration target is calibrated for the patient based on the result of the comparison.

[0020] In some embodiments, the patient-side mobile computing device is configured to determine that the calibration is complete in response to determining that the one or more calibration targets have been calibrated. In some embodiments, the patient-side mobile computing device is configured to recalibrate the calibration target in response to determining that the calibration target is not calibrated.

[0021] In some embodiments, the patient-side mobile computing device is configured to play desensitized information between presenting two adjacent calibration targets. In some embodiments, the operator application is configured to: in response to receiving an indication that the calibration is complete, start verifying the calibration. In some embodiments, the patient-side mobile computing device is configured to: in response to receiving a request to verify the calibration, present at least one additional calibration target on the screen while obtaining additional eye tracking calibration data for the patient using the eye tracking device; and process the obtained additional eye tracking calibration data for the patient to determine the position of the patient's corresponding visual gaze for the at least one additional calibration target.

[0022] In some embodiments, the patient-side mobile computing device is configured to: compare the position of the patient's corresponding visual gaze for the at least one additional calibration target with the corresponding predetermined position of the at least one additional calibration target presented on the screen; and determine whether the calibration has been verified based on the result of the comparison.

[0023] In some embodiments, the operator application is configured to: simultaneously present in the user interface the at least one additional calibration target at the corresponding predetermined position and present at least one representation of the patient's corresponding visual gaze at the determined position of the patient's corresponding visual gaze for the at least one additional calibration target; and present in the user interface a first user interface element for verifying the calibration and a second user interface element for recalibrating.

[0024] In some embodiments, the operator application is configured to send a command for data collection to the patient-side computing device in response to one of: selecting a user interface element for initiating data collection, or determining that calibration is complete or verified.

[0025] In some embodiments, the patient-side mobile computing device is configured to, in response to receiving the command, sequentially present a predetermined list of visual stimuli to the patient on the screen of the patient-side mobile computing device while obtaining eye tracking data of the patient using the eye tracking device.

[0026] In some embodiments, the patient-side mobile computing device is configured to present a centering target to the patient on a screen of the patient-side mobile computing device before presenting each of the predetermined list of visual stimuli. In some embodiments, the patient-side mobile computing device is configured to perform calibration of the patient for the eye tracking device between presenting two adjacent visual stimuli in the predetermined list of visual stimuli. In some embodiments, the eye tracking data collected when performing the calibration is used to at least one of recalibrate the eye tracking data of the patient or to determine calibration accuracy.

[0027] In some embodiments, the operator application is configured to present at least one of the following in a user interface: a progress indicator that is updated during presentation of the predetermined list of visual stimuli; a user interface element for skipping visual stimuli in the predetermined list of visual stimuli; information about visual stimuli that have been presented or are being presented; or information about visual stimuli to be presented.

[0028] In some embodiments, the network connection server is configured to provide a diagnosis result of the patient based on the eye tracking data of the patient, the diagnosis result including at least one index value associated with a developmental disorder. In some embodiments, the operator-side mobile computing device is configured to present the diagnosis result in a user interface.

[0029] In some embodiments, the visual stimulus is predetermined based on at least the age of the patient or the condition of the patient. In some embodiments, each of the visual stimuli comprises at least one of a static visual stimulus, a dynamic visual stimulus, a pre-recorded visual stimulus, a pre-recorded audiovisual stimulus, a live visual stimulus, a live audiovisual stimulus, a two-dimensional stimulus, or a three-dimensional stimulus. In some embodiments, each of the visual stimuli is standardized to induce a specific eye movement response with a statistical confidence level greater than 95%. In some embodiments, each of the visual stimuli is configured to induce an eye movement response to a discrete space-time location with a statistical confidence level greater than 95%.

[0030] In some embodiments, the eye tracking device is connected to the patient-side mobile computing device via a wired connection. In some embodiments, the eye tracking device and the screen are housed together in a holder. In some embodiments, the patient-side mobile computing device includes a screen holder that holds the screen and the eye tracking device in the fixed position relative to the screen.

[0031] In some embodiments, the eye tracking device includes one or more eye tracking units arranged in one or more locations adjacent to a periphery of the screen. In some embodiments, at least one of the patient-side mobile computing device or the operator-side mobile computing device is a tablet computing device. In some embodiments, the operator-side computing device is configured to communicate with the patient-side mobile computing device via two-way communication.

[0032] Another aspect of the disclosure features an apparatus comprising: a patient-side computing device comprising a screen for presenting visual stimuli to a patient; and an eye tracking device integrated with the patient-side computing device and configured to collect eye tracking data of the patient when the visual stimuli are presented to the patient on the screen of the patient-side computing device, wherein the patient-side computing device comprises the patient-side computing device described above.

[0033] Another aspect of the disclosure features an apparatus that includes an operator-side computing device as described above.

[0034] Another aspect of the present disclosure is characterized by a device comprising: at least one processor; and at least one non-transitory memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform operations including: establishing a wireless connection with a patient-side computing device integrated with an eye tracking device; and presenting a user interface to communicate with the patient-side computing device to obtain eye tracking data of the patient.

[0035] In some embodiments, the operations further include: accessing a web portal of the network connection server, wherein the wireless connection is established through the web portal. In some embodiments, the operations further include: presenting a diagnosis result in a user interface based on the patient's eye tracking data.

[0036] Another aspect of the present disclosure features a computer-implemented method of developmental assessment via eye tracking. The computer-implemented method includes: initiating a session for a patient by establishing communications with an operator-side computing device and a patient-side computing device, the patient-side computing device being integrated with an eye tracking device; sequentially presenting a list of predetermined visual stimuli to the patient on a screen of the patient-side computing device while collecting eye tracking data of the patient using the eye tracking device; and sending session data of the session to a network-connected server, the session data including the eye tracking data of the patient collected in the session.

[0037] In some embodiments, at least one of the operator-side computing device or the patient-side computing device is a portable device. In some embodiments, establishing the communication includes establishing a wireless connection between the operator-side computing device and the patient-side computing device.

[0038] In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes: accessing a web portal at the web connection server by the operator-side computing device; and in response to receiving a selection of the patient-side computing device in the web portal, wirelessly connecting the operator-side computing device to the patient-side computing device. In some embodiments, establishing a wireless connection between an operator-side computing device and a patient-side computing device includes: displaying connection information on a screen of the patient-side computing device by the patient-side computing device; and in response to receiving an input of the connection information by the operator-side computing device, establishing the wireless connection between the operator-side computing device and the patient-side computing device.

[0039] In some embodiments, the computer-implemented method further comprises: after establishing communication, displaying visual desensitization information to the patient on a screen of the patient-side computing device. In some embodiments, the computer-implemented method further comprises: controlling the eye tracking device not to collect eye tracking data of the patient while displaying the visual desensitization information.

[0040] In some embodiments, the computer-implemented method further comprises: while displaying the visual desensitization information, accessing, by the operator-side computing device, a web portal at the network connection server to establish the session for the patient. In some embodiments, establishing the session comprises one of selecting a patient in a patient list or creating a profile for the patient on the network connection server. In some embodiments, the computer-implemented method further comprises: determining a relative position between the eye tracking device and at least one eye of the patient; and displaying instructions on a user interface of the operator-side computing device to adjust a position of the eye tracking device or a position of the patient.

[0041] In some embodiments, the computer-implemented method further comprises determining that the patient is aligned with the eye tracking device in response to determining that the relative position of at least one eye of the patient is at a predetermined position in a detection area of ​​the eye tracking device.

[0042] In some embodiments, the computer-implemented method further includes calibrating the patient relative to the eye tracking device by displaying one or more calibration targets to the patient on a screen of a patient-side computing device.

[0043] In some embodiments, calibrating the patient relative to the eye tracking device includes: sequentially presenting each of the one or more calibration targets at a corresponding predetermined position of the screen of the patient-side computing device, while obtaining eye tracking calibration data of the patient using the eye tracking device; for each of the one or more calibration targets, processing the obtained eye tracking calibration data of the patient to determine a corresponding visual gaze position of the patient relative to the calibration target; comparing the corresponding visual gaze position of the patient with a corresponding predetermined position of the presented calibration target; and determining whether the calibration target is calibrated relative to the eye tracking device based on a result of the comparison.

[0044] In some embodiments, calibrating the patient relative to the eye tracking device also includes: in response to determining that a deviation between the patient's corresponding visual gaze position and the corresponding predetermined position is less than or equal to a predetermined threshold, determining that the calibration target is calibrated and displaying the next calibration target, or in response to determining that the deviation is greater than the predetermined threshold, determining that the calibration target has failed to be calibrated, and redisplaying the calibration target for calibration.

[0045] In some embodiments, the computer-implemented method further comprises: after calibrating the patient relative to the eye tracking device, verifying the calibration using one or more new calibration targets. In some embodiments, verifying the calibration comprises: sequentially presenting each of the one or more new calibration targets at a corresponding predetermined position of a screen of the patient-side computing device while obtaining eye tracking calibration data of the patient using the eye tracking device; and processing the obtained eye tracking calibration data of the patient to determine a corresponding visual gaze position of the patient for each of the one or more new calibration targets.

[0046] In some embodiments, verifying the calibration includes: simultaneously presenting on a user interface of the operator-side computing device the one or more new calibration targets at one or more corresponding predetermined positions and one or more corresponding visual gaze representations of the patient at the determined one or more positions; and in response to receiving an indication of verifying the result of the calibration, determining that the calibration has been verified, or in response to receiving an indication that the calibration result is invalid, starting to recalibrate the patient relative to the eye tracking device.

[0047] In some embodiments, verifying the calibration includes: determining the number of new calibration targets that have each passed the calibration based on the patient's corresponding visual gaze position and the corresponding predetermined position; and if the number or the relevant percentage is greater than or equal to a predetermined threshold, determining that the calibration has been verified, or if the number or the relevant percentage is less than a predetermined threshold, determining that the calibration is invalid and starting to recalibrate the patient relative to the eye tracking device.

[0048] In some embodiments, sequentially presenting to the patient a list of predetermined visual stimuli on the screen of the patient-side computing device comprises presenting to the patient a centering target on the screen of the patient-side computing device prior to presenting each of the list of predetermined visual stimuli.

[0049] In some embodiments, sequentially presenting a predetermined list of visual stimuli to the patient on the screen of the patient-side computing device includes: between presenting two adjacent visual stimuli in the predetermined list of visual stimuli, performing calibration of the patient relative to the eye tracking device, wherein the eye tracking data collected when performing the calibration is used to calibrate the eye tracking data of the patient or to determine at least one of the calibration accuracy.

[0050] In some embodiments, the computer-implemented method also includes: presenting at least one of the following on the user interface of the operator-side computing device: a progress indicator that remains updated during the presentation of the predetermined visual stimulus list; information about visual stimuli that have been presented or are being presented; information about visual stimuli to be presented; or a user interface element for skipping visual stimuli in the predetermined visual stimulus list.

[0051] In some embodiments, sending the session data of the session to the network-connected server includes automatically sending the session data of the session to the network-connected server by the patient-side computing device in response to one of: determining completion of presenting the predetermined list of visual stimuli on the screen, or receiving a completion indication of the session from the operator-side computing device.

[0052] In some embodiments, the session data includes information related to the list of predetermined visual stimuli presented. In some embodiments, the information related to the list of predetermined visual stimuli presented includes the name of the predetermined visual stimulus presented and an associated timestamp when the predetermined visual stimulus was presented.

[0053] In some embodiments, the session data includes the eye tracking data and an associated timestamp when the eye tracking data was generated or collected. In some embodiments, the session data is stored in a first file storing the patient's eye tracking data and a second file storing information associated with the list of predetermined visual stimuli presented.

[0054] Another aspect of the disclosure features a computer-implemented method for performing developmental assessment using eye tracking data by a network-connected server. The computer-implemented method includes: receiving session data for a plurality of sessions, the session data for each session including eye tracking data for a corresponding patient in the session; processing the session data for the plurality of sessions in parallel to generate processed session data for the plurality of sessions; and for each session in the plurality of sessions, analyzing the processed session data for the session based on corresponding reference data to generate an assessment result for the corresponding patient in the session.

[0055] In some embodiments, the computer-implemented method further comprises loading corresponding reference data for the plurality of sessions in parallel with processing session data for the plurality of sessions.

[0056] In some embodiments, the network connection server includes a plurality of processing cores. Processing the session data of the plurality of sessions in parallel includes: using a first plurality of processing cores to process the session data of the plurality of sessions in parallel, and using a second plurality of different processing cores to load corresponding reference data of the plurality of sessions, wherein the number of the first plurality of processing cores is greater than the number of the second plurality of processing cores.

[0057] In some embodiments, analyzing the processed session data for the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions includes using a plurality of processing cores including a first plurality of processing cores and a second plurality of processing cores.

[0058] In some embodiments, analyzing the processed session data of the plurality of sessions based on the loaded corresponding reference data of the plurality of sessions comprises at least one of comparing the processed session data of the sessions with corresponding reference data, using the corresponding reference data to infer an assessment result of a corresponding patient based on the processed session data, or using at least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model. In some embodiments, the corresponding reference data comprises historical eye tracking data or results for a patient having substantially the same age or condition as the corresponding patient.

[0059] In some embodiments, the computer-implemented method further comprises: generating the assessment result based on previous session data of the corresponding patient. In some embodiments, the computer-implemented method comprises: for each session in a plurality of sessions, assigning a corresponding container to the session, and in the corresponding container, processing session data of the session, and analyzing the processed session data of the session based on the corresponding model data to generate an assessment result for the corresponding patient in the session.

[0060] In some embodiments, when the eye tracking data is collected in the session, the eye tracking data is associated with a predetermined list of visual stimuli presented to the patient, and wherein the session data includes information associated with the predetermined list of visual stimuli in the session.

[0061] In some embodiments, the computer-implemented method further comprises: linking the eye tracking data of the session with a list of predetermined visual stimuli in the session. In some embodiments, linking the eye tracking data of the session with the list of predetermined visual stimuli in the session comprises: decomposing the eye tracking data into a plurality of portions in a respective container based on information associated with the list of predetermined visual stimuli, each portion of the eye tracking data being associated with one of a respective predetermined visual stimulus or a corresponding calibration.

[0062] In some embodiments, processing session data for the session includes processing a portion of the eye tracking data associated with a corresponding predetermined visual stimulus based on information of the corresponding predetermined visual stimulus.

[0063] In some embodiments, the computer-implemented method further comprises, in the respective container, recalibrating a portion of the eye tracking data associated with the respective predetermined visual stimulus based on at least a portion of the eye tracking data associated with the corresponding calibration.

[0064] In some embodiments, the computer-implemented method further comprises determining, in the respective container, a calibration accuracy using at least a portion of the eye tracking data associated with the corresponding calibration and a plurality of predetermined locations at which the plurality of calibration targets are presented in the corresponding calibration.

[0065] In some embodiments, receiving the session data for the plurality of sessions comprises: receiving the session data for the plurality of sessions from a plurality of computing devices associated with the corresponding entity through a network portal. In some embodiments, the computer-implemented method further comprises: in response to receiving the session data for the session, adding a file pointer for the session data for the session to a processing queue to be processed. In some embodiments, the computer-implemented method further comprises: storing the session data for the session in a database using the file pointer for the session; and retrieving the session data for the session from the database using the file pointer for the session.

[0066] In some embodiments, the computer-implemented method further comprises: for each entity, storing session data from one or more computing devices associated with the entity in a corresponding repository. In some embodiments, the corresponding repository for the entity includes at least one of: information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted in the entity, information about one or more patients associated with the entity, or historical information of the corresponding repository.

[0067] In some embodiments, the respective repository is included in a NoSQL database. In some embodiments, the respective repository is isolated from one or more other repositories and is not accessible by one or more other entities.

[0068] In some embodiments, the computer-implemented method further comprises dynamically adjusting resources of the network-connected server based on a number of computing devices accessing the network-connected server.

[0069] In some embodiments, the computer-implemented method further comprises: replicating data of the first data center to the second data center; and in response to determining that the first data center is inaccessible, automatically directing traffic to the second data center. In some embodiments, each of the first data center and the second data center includes at least one of a network portal accessible to an operator-side computing device, an operator application, or an application layer for data processing and data analysis.

[0070] In some embodiments, the computer-implemented method further comprises storing the same data in a plurality of data centers, wherein the data comprises application data of an entity and information associated with the eye tracking data.

[0071] In some embodiments, the computer-implemented method further includes: associating the generated assessment result with a corresponding patient in the session; and generating an assessment report for the corresponding patient.

[0072] In some embodiments, the computer-implemented method further comprises: outputting an evaluation result or an evaluation report to be presented in a user interface of the operator-side computing device. In some embodiments, the evaluation report comprises at least one of: information of the corresponding patient, information of an entity performing a session of the corresponding patient, information of calibration accuracy in the session, information of session data collection, or an evaluation result of the corresponding patient.

[0073] In some embodiments, the assessment results indicate the likelihood that the corresponding patient has a developmental, cognitive, social, or intellectual disability or ability. In some embodiments, the assessment results indicate the likelihood that the corresponding patient suffers from an Austin spectrum disorder (ASD) or non-ASD. In some embodiments, the assessment results include a corresponding score for each of one or more of a social disability index, a language ability index, a non-verbal ability, a social adaptation index, and a social interaction index.

[0074] In some embodiments, the assessment results include at least one of: a visualization of the eye tracking data overlaid on a corresponding visual stimulus still at the socially relevant moment, an animation of the visualization of the eye tracking data overlaid on a corresponding visual stimulus still at the socially relevant moment, a visualization of aggregated reference data of multiple reference patients matched to the corresponding patient on one or more patient attributes, or an annotation describing at least one of the visual stimulus content or the eye gaze pattern.

[0075] In some embodiments, the corresponding patient has an age in the range of 5 months to 7 years, including an age in the range of 5 months to 43 months or 48 months, an age in the range of 16 months to 30 months, an age in the range of 18 months to 36 months, an age in the range of 16 months to 48 months, or an age in the range of 16 months to 7 years.

[0076] Another aspect of the disclosure features a system that includes: at least one processor; and one or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a computer-implemented method as described herein.

[0077] Another aspect of the disclosure features one or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform a computer-implemented method as described herein.

[0078] One or more embodiments described herein can achieve many technical effects and advantages. In a first example, some embodiments can provide a convenient, miniaturized and effective computing system for advantageously collecting eye tracking data and subsequently transmitting these data for analysis and diagnosis results. The computing system may include at least two separate portable computing devices, such as an operator-side portable device and at least one patient-side portable device integrated with an eye tracking device. These portable devices can be equipped differently (different peripheral devices or equipment, different user interfaces, etc.) and can be wirelessly connected to each other or to a network-connected server platform (which in turn provides communication between the operator-side portable device and the patient-side portable device) without a physical connection. For example, an operator can use an operator-side portable device to calibrate the patient's eye tracking device and control the playback of a predetermined visual stimulus on the patient-side portable device to obtain the patient's eye tracking data using the eye tracking device. Once the session is completed, the patient-side portable device can send session data, such as information on the detected eye tracking data and the played visual stimulus, to a cloud server for data storage, processing and analysis. The cloud server can be remotely connected to these portable devices, for example, via a network.

[0079] In a second example, some implementations of the computing system described below can have a smaller form factor (e.g., compared to a larger chair-mounted computer device) and can be more easily moved and repositioned - thereby improving the convenience of the patient, the treatment provider, or both. The computing system can also enable greater flexibility for operators and patients, who can more easily position themselves in the appropriate position during setup and data collection. In addition, in some embodiments detailed below using two separate portable computing devices (e.g., tablet computers), the system can advantageously impose lower requirements on the computing devices themselves and provide greater flexibility in implementing different functions in the two separate computing devices. In addition, in some methods described below, a patient (e.g., a toddler or other young patient) can be held / seated by a caregiver (e.g., a parent) during the display of visual stimuli to improve comfort and accurate eye tracking data acquisition at the patient-side portable device, while the treatment provider (carrying the operator-side portable device to control the playback of the predetermined visual stimuli) can simultaneously select a position spaced apart from the patient and the caregiver, thereby reducing distraction to the patient. In addition, the technology enables the treatment provider to use the operator-side portable device to log in to the web portal of the cloud server for device management, patient management, and data management. For example, through the web portal of the cloud server, the treatment provider can use the operator-side portable device to communicate with multiple patient-side portable devices to obtain eye tracking data for multiple patients in multiple sessions, which can greatly improve work efficiency and reduce the workload of the treatment provider. In addition, the network-based operator application can be run on the cloud server and provide a user interface for the treatment provider to select the patient-side portable device for connection, perform calibration, verification, and session data acquisition. The network-based operator operation can be maintained or updated at the cloud server, which can be more convenient, more efficient, and less costly than installing the operator application in a single operator-side portable device. In addition, the operator-side portable device can be paired with multiple patient-side portable devices, which can simplify the entire system, reduce system costs, and improve system performance.

[0080] In the third example, some embodiments described in detail below can advantageously support the services of a large number of treatment providers across multiple geographical locations.In certain embodiments, cloud server is implemented in a centralized cloud environment, and can communicate with multiple computing devices or systems as described above.Cloud server can utilize software as a service (SaaS) model by providing subscription-based diagnostic services to treatment providers.Centralized cloud environment can provide greater flexibility to expand the ability of cloud server.For example, cloud server can implement multi-tenant architecture.Tenant (tenant) can be a single health care organization, including but not limited to autism center, health care institution, expert, physician or clinical research.The resources of cloud server can be dynamically managed based on the total number of tenants and expected average workload (for example, how many tenants access cloud server at a given time point).Cloud server can adopt horizontal scaling technology such as automatic scaling to handle the spike in resource workload.

[0081] In addition, in a fourth example, some versions of the technology described in detail below can provide efficient and fast diagnosis results to treatment providers. As described above, once the session is completed, the computing system (e.g., the patient-side portable device) can automatically upload the session data to the cloud server. The cloud server can also be configured to automatically receive, process and analyze session data from multiple computing systems without human intervention. In addition, the cloud server can process and analyze the session data of multiple sessions from a large number of computing systems in parallel, which can greatly improve the session processing speed and provide diagnosis results in a short time (e.g., within a 24-hour window). In some embodiments, the cloud server can deploy a corresponding container for each session, and the corresponding container can include a corresponding data processing module and a data analysis module. In this way, once the session data of the session is uploaded, the session data of the session can be processed and analyzed using its own container (e.g., its own data processing and data analysis instance). In some embodiments, when, for example, the majority of processing units (or cores) in the cloud server are used to process the session data of multiple sessions in the corresponding container, the model data for analyzing the processed session data can be preloaded into the corresponding container in parallel using, for example, the remaining or a few processing units in the cloud server. All processed session data and loaded model data can then be analyzed in parallel in corresponding containers, for example, using the total number of processing units in the cloud server. This technology can be automated and parallelized in a variety of ways, which can greatly increase the speed of session data processing and analysis and provide rapid diagnosis results in a short period of time (e.g., within a 24-hour window). Parallelization can also make cloud server resource utilization more efficient, further improving system performance.

[0082] In a fifth example, some embodiments described herein can also provide improved accuracy of diagnostic results for patients based on efficient data analysis. A large amount of model data, including data related to patients of similar age, similar background and / or similar situation, can be used together with the patient's processed session data to generate a diagnostic result for the patient, for example, using comparison or inference via a statistical model, algorithm, artificial intelligence (AI) model (such as a machine learning or artificial neural network model), which can greatly improve the accuracy of the diagnostic result.

[0083] In a sixth example, some versions of the technology described herein can achieve privacy, confidentiality, and security of treatment provider data. In some embodiments, the cloud server stores different types of data such as those related to organizations, users, sessions, patients, and devices as application data or documents in a database (e.g., a NoSQL database). Different tenants can have their own databases, which can isolate data between tenants and prevent others from accessing their own data, thereby improving data integrity and confidentiality. Tenants can share a single instance of an application in a cloud server, or each tenant can get their own application instance.

[0084] In a seventh example, some embodiments described herein can ensure high availability of services to treatment providers, so that treatment providers can access their services regardless of any interruptions to the cloud server. The cloud server can be hosted in a cloud environment with built-in high availability or configurable high availability, for example, using a high availability service level agreement or by using geographic redundancy. For example, the technology can replicate data centers in different physical locations. When a data center fails, traffic can be automatically redirected to the replicated data center. The process can be seamless, and the treatment provider may not be aware of the switching of the replicated data center. In some embodiments, the data center includes a web portal accessible by the treatment provider / user, as well as data processing and data analysis modules. In addition, these technologies can replicate databases in multiple locations. The database can include application data and raw eye tracking data, processed data, data analysis, diagnostic results and / or evaluation reports.

[0085] In an eighth example, some versions of the technology described herein can provide centralized data input (such as patient identifiers and metadata to augment eye tracking data), for example in a cloud server, which can reduce the workload of treatment providers, reduce human error, and ensure data accuracy. In some embodiments, the technology provides two-way communication between different devices or systems. Compared to one-way communication that requires multiple data inputs in separate devices or systems, two-way communication can reduce data input by treatment providers and can enable autonomous data transmission.

[0086] In a ninth example, some embodiments described herein may also provide secure communications between different devices and systems. For example, communications between a local computing system (including an operator-side portable device and a patient-side portable device) and a cloud server may use Hypertext Transfer Protocol Secure (HTTPS) communications, through which information is encrypted to protect the privacy and integrity of the information. Communications between the patient-side portable device and the operator-side portable device may be accomplished using WebSocket communications, for example, using a secure implementation of WebSocket called WebSocket Secure (WSS). WebSocket communications may also provide two-way communications between devices, which allows the operator-side portable device to control the patient-side portable device while receiving information from the patient-side portable device.

[0087] In some implementations, these technologies (based on objective data) can be used to provide diagnosis for early young patients, where such diagnosis and intervention can achieve the greatest impact. In addition, the technology described herein can be used to monitor risk severity, which can provide insight into the severity of earlier ages, and provide confirmation of severity and disorder status over time. The technology can provide an immediate and reliable baseline for children's status and progress to start individualized treatment for children. The technology can provide an objective baseline and confirmatory diagnosis to inform individualized treatment plans and minimize the inefficiency and cost associated with patient measurements and lengthy evaluations. These technologies can reduce the variability in care by providing clear standards and progress measurements, and can provide clear and quantified standards that can be managed and understood regardless of the skills of the provider / staff. The technology can create access in underserved and under-resourced populations and can be used in a variety of care environments at a price point that is both affordable and less time-consuming than current evaluation alternatives.

[0088] The technology implemented herein can be used to provide early identification and assessment of risk of developmental, cognitive, social, language or non-verbal abilities or intellectual abilities or disabilities of patients, for example, by measuring visual attention to social information in the environment relative to normative, age-specific benchmarks. The patient can have an age in the range of 5 months to 7 years, for example, 16 months to 7 years, 12 months to 48 months, 16 months to 30 months or 18 months to 36 months.

[0089] According to certain aspects, the visual fixation of the patient provides a marker of the possible development, cognition, social or intellectual ability or obstacle (such as ASD) of the patient relative to the change over time of some dynamic stimulation. Visual fixation is a kind of eye movement for stabilizing the visual information on the retina, and is usually consistent with the person who watches or "watches" the point or area on the display plane. In certain embodiments, by repeated eye tracking sessions and / or by comparing with the model data of a large number of patients based on similar age and / or background, the visual fixation of the patient is identified, monitored and tracked over time. Then the data related to visual fixation are compared with relative standards to determine the possible increased risk of this condition in the patient. Compared with the similar visual fixation data of the patient with typical development or with the previous visual fixation data of the patient himself, the change of visual fixation (specifically, the decline or increase of the visual fixation of the image of the eyes, body or other regions of interest of the person or object displayed on the visual stimulus) provides the indication of development, cognition or mental disorder. These technologies can be used to quantitatively measure and monitor the symptoms of corresponding ability or disability, and in some cases, provide more accurate and relevant treatment information to patients, families and service providers. According to additional aspects, the technology can be used to predict outcomes for individuals with autism (thereby providing treatment capabilities), while also providing similar diagnostic and treatment measures for overall developmental, cognitive, social or intellectual abilities or disabilities.

[0090] As described in detail below, the techniques described herein for detecting developmental, cognitive, social or intellectual disabilities can be applied to detecting conditions including, but not limited to, expressive and receptive language delay, non-verbal delay, intellectual disability, intellectual disability of known or unknown genetic origin, traumatic brain injury, disorder of infancy not otherwise specified (DOI-NOS), social interaction disorder, and autism spectrum disorder (ASD), as well as conditions such as attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), concussion, sports injuries, and dementia.

[0091] It should be understood that the method according to the present disclosure may include any combination of the embodiments described herein. That is, the method according to the present disclosure is not limited to the combination of the embodiments specifically described herein, but also includes any combination of the embodiments provided.

[0092] The details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the description below. Other embodiments and advantages of the present disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 is a block diagram of an exemplary environment for assessing developmental disorders through eye tracking in accordance with one or more embodiments of the present disclosure.

[0094] Figure 2Ais a block diagram of an exemplary system for assessing developmental disorders through eye tracking in accordance with one or more embodiments of the present disclosure.

[0095] Figure 2B According to one or more embodiments of the present disclosure, Figure 2A An example of managing session data in a system.

[0096] Figure 2C According to one or more embodiments of the present disclosure, Figure 2A An example of managing multiple session data in parallel in a system.

[0097] Figure 2D According to one or more embodiments of the present disclosure, Figure 2A An exemplary database in which different types of documents are stored as application data in a system.

[0098] Figure 2E The present invention shows one or more embodiments of the present invention. Figure 2A An example of a multi-tenant architecture in a system.

[0099] Figure 2F-2G The present invention shows a method for Figure 2A Example of data backup of a system.

[0100] Figure 3 is a flow chart of an exemplary process for session data acquisition according to one or more embodiments of the present disclosure.

[0101] Figures 4A-4J Shown is a series of illustrative display screens presented on an operator device (Fig. a) and a participant device (Fig. b) during session data acquisition in accordance with one or more embodiments of the present disclosure.

[0102] Figure 5 Example session data including (a) movie playlist data; and (b) eye tracking sensor data according to one or more embodiments of the present disclosure is shown.

[0103] Figure 6 is a flow chart of an exemplary process for managing session data in accordance with one or more embodiments of the present disclosure.

[0104] Figure 7A-7B A flow chart is shown of another exemplary process for managing session data in accordance with one or more embodiments of the present disclosure.

[0105] Fig. 8A An exemplary results interface displaying at least one index value based on eye tracking data according to one or more embodiments of the present disclosure is shown.

[0106] Figures 8B-8C Another example results interface showing performance-based measures of developmental assessment based on eye tracking data is shown, for example: According to one or more embodiments of the present disclosure, Figure 8B Nonverbal Communication and Gestures (A) and Joint Attention & Mutual Gaze (B) in Figure 8C Facial Affect (C) and Pointing and Social Monitoring (D) in.

[0107] Fig. 9 is a flow chart of an exemplary process for session data acquisition according to one or more embodiments of the present disclosure.

[0108] Fig.10 is a flow chart of an exemplary process for managing session data in accordance with one or more embodiments of the present disclosure.

[0109] Fig.11 The architecture of a cloud computing system according to one or more embodiments of the present disclosure is shown.

[0110] Fig.12 The architecture of a computing device according to one or more embodiments of the present disclosure is shown.

[0111] Like reference numerals and designations in the various drawings indicate like elements.It should also be understood that the various exemplary implementations shown in the figures are merely illustrative representations and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0112] The present disclosure describes systems, apparatus, and methods for detecting and assessing a patient's developmental, cognitive, social, or intellectual abilities or disabilities, including autism spectrum disorder (ASD), using eye tracking data generated in response to displaying specific predetermined visual stimuli (e.g., one or more videos) to the patient.

[0113] In order to provide a comprehensive understanding of the systems, devices and methods described herein, certain illustrative embodiments will be described. It should be understood that if these data do not indicate a measurement of an obstacle, a measurement of the typical degree of normative development can be provided, providing an indication of the variability of typical development. In addition, all components and other features outlined below can be combined with each other in any suitable manner and can be adapted and applied to systems other than medical diagnosis. For example, the interactive visual stimulation of the present disclosure is used as a therapeutic tool. In addition, the collected data can produce measurements of certain types of visual stimulation that patients prefer. The measurement of this preference has applications and does not have applications in the field of medical diagnosis and treatment, including, for example, advertising or other industries that are interested in data related to visual stimulation preferences.

[0114] Example Environment and System

[0115] Figure 1 is a block diagram of an exemplary environment 100 for assessing developmental disorders through eye tracking according to one or more embodiments of the present disclosure. The environment 100 includes a cloud server 110 and multiple computing systems 120-1, ..., 120-n (collectively referred to as computing systems 120 or individually referred to as computing systems 120) that communicate via a network 102. The cloud server 110 can provide developmental disorder assessment or diagnosis services to multiple users (e.g., treatment providers). Treatment providers can use the corresponding computing system 120 to conveniently and reliably collect data for patients in sessions (e.g., programs associated with collecting data) of any age from newborns to the elderly, and the exemplary embodiments described below are particularly suitable for toddlers or other young patients. The data collected in the session (or session data) may include eye tracking data generated in response to displaying specific predetermined visual stimuli (e.g., one or more videos) to the patient. The computing system 120 can securely send the session data to the cloud server 110, and the cloud server 110 can store, process, and analyze the session data for diagnosing the patient's ASD or other cognitive, developmental, social, or intellectual abilities or disabilities, and providing diagnostic results or reports to treatment providers in a highly secure, robust, rapid, and accurate manner, as discussed in further detail below.

[0116] A treatment provider may be a single healthcare organization, including but not limited to an autism center, a healthcare facility, a specialist, a physician, or a clinical study. A healthcare organization may provide developmental assessment and diagnosis, clinical care, and / or treatment services to a patient. Figure 1As shown, a patient (e.g., an infant or child) can be brought to a health care facility by a caregiver (e.g., a parent). An operator (e.g., a specialist, a physician, a medical assistant, a technician, or other medical professional in a health care facility) can use a computing system 120 to collect, for example, non-invasive eye tracking data of a patient while he or she views visual stimuli (e.g., dynamic visual stimuli, such as movies) depicting common social interactions (e.g., binary or ternary interactions). The stimuli displayed to the patient for the purpose of data collection can be specific to the patient (e.g., based on the patient's age and condition). The stimuli can be any suitable visual image (static or dynamic), including movies or videos, as well as still images or any other visual stimuli. It should be understood that movies or videos are cited only as examples, and any such discussion also applies to other forms of visual stimuli.

[0117] In some embodiments, Figure 1 As shown, the computing system 120 includes at least two separate computing devices 130 and 140, such as an operator-side computing device 140 and at least one patient-side computing device 130. Optionally, the two computing devices 130 and 140 can be wirelessly connected, such as via a wireless connection 131, without being physically connected. The wireless connection 131 can be through a cellular network, a wireless network, Bluetooth, near field communication (NFC), or other standard wireless network protocols. In some embodiments, the patient-side computing device 130 is configured to connect to the operator-side computing device 140 through a wired connection such as a universal serial bus (USB), for example, when the wireless connection 131 fails.

[0118] In some cases, the two computing devices 130 and 140 communicate with each other by respectively communicating with the cloud server 110 via the network 102, and the cloud server 110 in turn provides communication between the operator-side computing device 140 and the patient-side computing device 130. For example, in conjunction with Figures 4A-4B As discussed in further detail in , the operator can, for example, log into a web portal running on the cloud server 110 through a web-based operator application for device management, patient management, and data management. The operator can use an operator-side computing device 140 (e.g., a tablet computer) to communicate with multiple patient-side portable devices 130, for example, in the same medical facility, for eye tracking data acquisition of multiple patients in multiple sessions, which can greatly simplify the computing system 120, reduce system costs, improve work efficiency, and reduce the workload of the operator.

[0119] Computing devices 130, 140 may include any suitable type of device, such as a tablet computing device, a camera, a handheld computer, a portable device, a mobile device, a personal digital assistant (PDA), a cellular phone, a web device, a smart phone, an Enhanced General Packet Radio Service (EGPRS) mobile phone, or any suitable combination of any two or more of these or other data processing devices. As an example, Fig.12 The architecture of a computing device that may be implemented as computing device 130 or 140 is shown.

[0120] At least one of computing device 130 or computing device 140 may be a portable device, such as a tablet device. In some cases, computing devices 130, 140 are both portable and wirelessly connected to each other. In this way, computing system 120 can be moved and repositioned more easily, and allows the operator to more flexibly choose his or her position relative to the patient. For example, the operator (carrying operator-side computing device 140) is not physically tethered to the patient-side computing device 130, and can easily position him or herself in an optimal position (e.g., away from the patient's direct field of view) during setup and data collection. In addition, a caregiver (e.g., a parent) can carry a patient (e.g., a toddler or other child) in a more appropriate position and in a more comfortable manner, which can enable the patient to be more involved in the visual stimulation played for effective and accurate eye tracking data acquisition. Patient-side computing device 130 can be carried by a caregiver or arranged in front of the patient and the caregiver (e.g., adjustably).

[0121] like Figure 1 As shown, the patient-side computing device 130 includes a screen 132 (e.g., a display screen) for displaying or presenting visual stimuli to the patient. The patient-side computing device 130 may also include an eye tracking device 134 or be integrated with the eye tracking device 134 in the same housing 136. In some embodiments, the patient-side computing device 130 integrated with the eye tracking device 134 may be referred to as an eye tracking console or an eye tracking system.

[0122] The eye tracking device 134 can be connected to the patient-side computing device 130 via a wired connection (e.g., using a USB cable or wire or using electrical pins). In some cases, the eye tracking device 134 is configured to connect to the patient-side computing device 130 via a wireless connection (e.g., Bluetooth or NFC). The eye tracking device 134 can be arranged in an appropriate position relative to the screen 132 and / or the patient, wherein the eye tracking device 134 can obtain the patient's eye movements while viewing visual stimuli while also minimizing visual interference from the patient's field of view.

[0123] like Figure 1As shown, the eye tracking device 134 may include one or more eye tracking units (or sensors) 135 arranged below the bottom of the screen 132. The one or more eye tracking units 135 may be arranged on one or more sides of the screen 132, on the top of the screen 132, and / or around the screen 132. The one or more eye tracking units 135 may be mechanically mounted to the patient-side computing device 130 at a position adjacent to the periphery of the screen 132. For example, the patient-side computing device 130 may include the screen 132 and a screen holder structure that holds the eye tracking device 134 in a fixed predetermined position relative to the screen 132. In some embodiments, the eye tracking device 134 includes a first eye tracking unit and a second eye tracking unit, the first eye tracking unit being configured to obtain or collect eye movements of the patient's left eye, and the second eye tracking unit being configured to obtain or collect eye movements of the patient's right eye. The eye tracking device 134 may also include a third eye tracking unit configured to obtain the position of the patient's eye or an image acquisition unit (e.g., a camera) configured to obtain an image of the patient's eye.

[0124] The eye tracking unit includes a sensor that can detect the presence of a person and track what he / she is looking at or measure where a person is looking or how the eyes react to stimuli in real time. The sensor can convert the person's eye movements into a data stream containing information such as pupil position, gaze vector and / or gaze point of each eye. In some embodiments, the eye tracking unit includes a camera (e.g., an infrared sensitive camera), an illumination source (e.g., infrared (IR) illumination), and an algorithm for data collection and / or processing. The eye tracking unit can be configured to track pupil or corneal images or reflections (CR). The algorithm can be configured for pupil center and / or corneal detection and / or artifact suppression.

[0125] Since there may be variations in eye size, foveal position, and general physiology that can be adapted to each individual before using an eye tracking unit to collect eye tracking data for a participant (e.g., a patient), the eye tracking unit may first be calibrated. In calibration, the physical position of the eye is algorithmically associated with a point in space that the participant is looking at (e.g., gazing). The gaze position may be a function of the participant's perception. In some embodiments, calibration involves the participant looking at a fixed known calibration target (e.g., a point) in the field of view. The calibration may include a single centering target, or 2, 5, 9, or even 13 targets. The algorithm may create a mathematical transformation between the eye position (negative CR) and the gaze position for each target, and then create a matrix covering the entire calibration area, for example interpolating between each target. The more targets used, the higher and more uniform the accuracy across the field of view. The calibration area defines the highest accuracy portion of the eye tracking unit range, and the accuracy decreases if the eye moves at an angle greater than the target used.

[0126] In some embodiments, the eye tracking unit is able to perform self-calibration, for example by creating a model of the eyes and passively measuring features of each individual. Calibration can also be done without active cooperation from the participant by making assumptions about gaze position based on content, effectively "hiding" the calibration target among other visual information. In some embodiments, calibration is not performed on the eye tracking unit if useful data can be obtained from the raw pupil position, for example using a medical vestibular ocular reflex (VOR) system or a fatigue monitoring system.

[0127] In some cases, validation can be performed to measure the success of the calibration, for example by displaying a new target and measuring the accuracy of the calculated gaze. The tolerance for the calibration accuracy may depend on the application of the eye tracking unit. For example, an error between 0.25 and 0.5 degrees of visual angle may be considered acceptable. For some applications, more than 1 degree is considered a failed calibration and another attempt is required. Participants may improve on the second or third attempt. Participants who consistently have high validation errors may have visual or physiological problems that prevent them from participating in the experiment. The validation results can be expressed in visual angles and displayed graphically.

[0128] like Figure 2A As shown, the patient-side computing device 130 may include an eye tracking application (or software) configured to retrieve or receive raw eye tracking data collected by the eye tracking device 134. The patient-side computing device 130 may generate session data based on the raw eye tracking data, for example, by storing the raw eye tracking data and associated information (timestamp information) in a data file (e.g., in .tsv format, .idf format, or any suitable format), such as Figure 5 (b). The session data may also include information about the visual stimulus played or presented in another data file (e.g., in .tsv format or any suitable format), such as Figure 5 (a) This information may include timestamp information for each visual stimulus played.

[0129] In some embodiments, the patient-side computing device 130 stores a plurality of predetermined visual stimuli (e.g., movies or video files) grouped to correspond to patients of a particular age group and / or condition group. For example, a first list of predetermined visual stimuli may be configured for ASD assessment of patients within a first age range (e.g., 5 to 16 months old), and a second list of predetermined visual stimuli may be configured for ASD assessment of patients within a second age range (e.g., 16 to 30 months old) that is different from the first age range. In some embodiments, an operator may use the operator-side computing device 140 to control the playback of a predetermined list of visual stimuli to a particular patient based on information of the particular patient. In some embodiments, the operator application sends age information to the eye tracking application upon patient selection, and the eye tracking application then dynamically selects an appropriate preset playlist based on the age information without operator intervention or selection. In some embodiments, the number of predetermined visual stimuli may also be stored in the operator-side computing device 140.

[0130] The testing method depends on the patient being awake and viewing the screen 132 of the patient-side computing device 130. During the calibration and data collection procedures, predetermined movies and / or other visual stimuli are presented to the patient via the patient-side computing device 130. These movies and / or other visual stimuli may include people or animated actors performing hand / face / body movements. Figure 5 As discussed in further detail in , during the data collection period, the computing system 120 can periodically show the patient a calibration or fixation target (which can be animated). This data can later be used to verify accuracy.

[0131] The visual stimulation (e.g., movie or video scene) displayed to the patient can depend on the patient's age. That is, the visual stimulation can be age-specific. In some embodiments, processing session data includes measuring the amount of time the patient spends looking at the actor's eyes, mouth or body or other predetermined areas of interest, and the amount of time the patient spends looking at the background area in the video. The video scene shown to the patient via the screen 132 of the patient-side computing device 130 can depict a scene of social interaction (e.g., an actor directly looking at the camera, trying to participate in watching the patient, or a scene of a child being played). In some embodiments, the video scene can include other suitable stimulations, including, for example, animations and priority viewing tasks. The measurement of the time of looking at a specific spatial position in the video can be related to the patient's social and / or cognitive development level. For example, children aged between 12-15 months show increased mouth looking due to their developmental stage of language development, and alternate between eye and mouth looking. As another example, the decline in the visual looking of the patient's eyes over time relative to the actor's eyes in the video can be an indication of ASD or another developmental condition in the patient. Analysis of a patient's viewing patterns (during the film displayed and over multiple viewing sessions or in comparison to historical data for patients of substantially the same age and / or condition) can be performed for the purpose of diagnosing and monitoring developmental, cognitive, social, or intellectual abilities or disabilities, including ASD.

[0132] The operator-side computing device 140 is configured to run an operator application (or software). In some embodiments, the operator application is installed and runs on the operator-side computing device 140. In some embodiments, the operator application runs on the cloud server 110, and the operator can log in to the web portal to interact with the operator application through the user interface presented on the operator-side computing device 140, such as in conjunction with Figures 4A-4J. The operator application can be configured to supervise or control the steps of the eye tracking application or software in the patient-side computing device 130, for example, selecting and playing specific visual stimuli for the patient and collecting raw eye tracking data. In some embodiments, the operator application is docked with the eye tracking software via a software development kit (SDK). In some embodiments, the communication between the patient-side computing device 130 and the operator-side computing device 140 or the communication between the operator application and the eye tracking application can be completed using WebSocket communication. WebSocket communication allows two-way communication between the two devices. This two-way communication allows the operator to control the patient-side computing device 130 while receiving information from the patient-side computing device 130. WebSocket communication can be completed using a secure implementation of WebSocket called WebSocket Secure (WSS). As described above, the communication between the patient-side computing device 130 and the operator-side computing device 140 (e.g., the communication between the operator application and the eye tracking application) can be through the cloud server 110. For example, the operator may use the operator-side computing device 140 to log into a web portal running on the cloud server 110 and establish a wireless connection with the patient-side computing device 130 for eye tracking data acquisition of the patient.

[0133] The operator application can additionally be used to perform other functions, such as presenting an interface to the operator showing the patient's name, date of birth, etc., information about the stimulus (e.g., movie) shown to the patient, etc. The operator can also use the operator-side computing device 140 to log into the web portal of the cloud server 110 for device management, patient management, and data management. In some embodiments, the operator application runs on the cloud server 110 and is controlled by the operator through the web portal using the operator-side computing device. The operator can operate the computing system 120 with only minimal training.

[0134] As in Figure 3 , 4A-4J and 5, the computing system 120 can be configured for session data acquisition. In some embodiments, a session is initialized by establishing a connection between an operator-side computing device 140 and a patient-side computing device 130. After the patient's information is entered into an operator application (e.g., customized software) running on the operator-side computing device 140, the operator application can control the eye tracking application running on the patient-side computing device 130 to select age-specific stimuli and instruct the patient's operator or caregiver to position the patient-side computing device 130 in an appropriate orientation and / or position in front of the patient. The operator can use the operator-side computing device 140 to control the operator application and / or the eye tracking application or software to (a) calibrate the eye tracking device 134 for the patient, (b) verify that the calibration is accurate, and (c) collect eye tracking data from the patient while he or she views dynamic video or other visual stimuli in the session, such as collecting eye tracking data from a patient who moves his or her eyes in response to a predetermined movie or other visual stimulus. After the session is over, both the eye tracking data and the stimulus-related information (e.g., a list of stimuli viewed by the patient) can be stored as session data in two separate data files. The session data can then be automatically transmitted, for example, by the patient-side computing device 130, to a secure database in the cloud server 110, for example, via the network 102. The database can be remote from the computing system 120 and configured to accommodate and aggregate data collected from multiple computing systems 120.

[0135] The network 102 may include a large computer network that connects any number of mobile computing devices, fixed computing devices, and server systems, such as a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, or a combination thereof. Each of the computing devices 130, 140 in the computing system 120 may communicate with the cloud server 110 via the network 102.

[0136] In some embodiments, the communication between the local computing devices 130, 140 and the cloud server 110 can be completed using the Hypertext Transfer Protocol Secure (HTTP). HTTP follows a request and response model, in which the client (e.g., through a browser or desktop application) sends a request to the server and the server sends a response. The response sent from the server can contain various types of information, such as documents, structured data, or authentication information. HTTP communication can be completed using a secure implementation of HTTP called the Hypertext Transfer Protocol Secure (HTTPS). Information transmitted through HTTPS is encrypted to protect the privacy and integrity of the information.

[0137] The cloud server 110 may be a computing system hosted in a cloud environment. The cloud server 110 may include one or more computing devices and one or more machine-readable repositories or databases. In some embodiments, the cloud server 110 may be a cloud computing system that includes one or more server computers in a local or distributed network, each server computer having one or more processing cores. The cloud server 110 may be implemented in a parallel processing or peer-to-peer infrastructure or on a single device having one or more processors. As an example, Fig.11 It is an architecture of a cloud computing system that can be implemented as a cloud server 110.

[0138] like Figure 1 As shown, the cloud server 110 includes a cloud platform 112 and a data pipeline system 114. Figure 2A-2G As discussed in further detail in , the cloud platform 112 can be configured to provide a web portal, store application data associated with treatment providers or tenants, and store data such as raw eye tracking data, processed data, analysis and / or diagnosis results. The data pipeline system 114 is configured to perform data processing and data analysis.

[0139] In some embodiments, such as in combination Figure 6 and 7A -7B, the cloud server 110 is configured to automatically receive, process and analyze session data from multiple computing systems. In addition, the cloud server can process and analyze session data from multiple sessions of a large number of computing systems in parallel, which can greatly improve the session processing speed and provide diagnostic results in a short time (e.g., within a 24-hour window). For example, receiving session data by the cloud server 110 (e.g., by the cloud platform 112) can start the processing and analysis process implemented by automatic software (e.g., by the data pipeline system 114). In this process, the patient's individual data can be compared with a model of eye tracking data, which is previously generated based on historical eye tracking data of patients with substantially the same age, background and / or condition. The result of the comparison can be a diagnosis of a neurodevelopmental disorder, including but not limited to ASD, a measurement of the patient's developmental / cognitive function and / or a prescription recommendation for a treatment plan. Alternatively or additionally, the collected data is compared and / or reviewed for a given patient in multiple sessions (and in a predetermined time period) to identify potential changes in visual gaze (e.g., a decline in visual gaze). These results can be compressed into a diagnostic report for use by the patient's doctor. In some embodiments, once the diagnosis result is ready, the cloud server 110 may transmit the diagnosis result to the operator-side computing device 140, and may present the diagnosis result on the user interface of the operator-side computing device 140, for example, in combination with Figures 8A-8C discussed in further detail in .

[0140] In some embodiments, a large amount of model data (including data related to patients of similar age, similar background and / or similar conditions) can be used together with the patient's processed session data to generate a diagnosis result for the patient, for example using comparison or inference via statistical models, algorithms, artificial intelligence (AI) models such as machine learning or artificial neural network models, which can greatly improve the accuracy of the diagnosis result.

[0141] Environment 100 includes corresponding Figure 1 The three main steps of the three parts of the environment 100 shown (e.g., the computing system 120 for data acquisition, the cloud platform 112, and the data pipeline system 114). Figure 2A-2G As discussed in further detail in , these three parts can be configured together to reliably collect data from a patient, and to effectively process and analyze the collected data to diagnose ASD or other cognitive, developmental, social, or intellectual abilities or disabilities.

[0142] Figure 2A is a block diagram of an exemplary system 200 for assessing developmental disabilities through eye tracking according to one or more embodiments of the present disclosure. The system 200 may be Figure 1 The system 200 is implemented in the environment 100. According to the three steps of data processing, the system 200 includes three subsystems: a data acquisition subsystem 210, a platform subsystem 220, and a data pipeline subsystem 230. Each subsystem may be composed of corresponding hardware and software items.

[0143] The data acquisition subsystem 210 is configured to collect eye tracking data of the patient. The data acquisition subsystem 210 may be Figure 1 The computing system 120. Figure 2A As shown, the data acquisition subsystem 210 includes an eye tracking console 212 running an eye tracking application 214 and an operator-side computing device (e.g., Figure 1 140). In some embodiments, the operator application 216 is deployed in the operator-side computing device. In some embodiments, the operator application 216 is deployed in the platform subsystem 220, and the operator can use the operator-side computing device to log in to the platform subsystem 220 through the web portal 222 to run the operator application 216 on the platform subsystem 220. Deploying the operator application 216 in the platform subsystem 220 can avoid deploying the operator application 216 in one or more operator-side computing devices, which can reduce the software and hardware requirements of the operator-side computing device, and enable the operator application 216 to be conveniently maintained or updated, for example, without the need to maintain or update the operator information 216 on each of the one or more operator-side computing devices.

[0144] The eye tracking console 212 may include Figure 1 The data acquisition subsystem 210 may include an integrated device of a patient-side computing device 130 (e.g., a tablet) and an eye tracking device 134. As described above, the data acquisition subsystem 210 may include a plurality of movie files 218 stored in the eye tracking console 212 and optionally stored in the operator-side computing device. The movie files 218 may be predetermined age-specific visual stimuli for patients of different ages and / or different conditions.

[0145] like Figure 1 As described above, the platform subsystem 220 and the data pipeline subsystem 230 may be included in a cloud server (e.g., Figure 1 The platform subsystem 220 is a network-connected server of the cloud server 110 and implemented in a centralized cloud hosting environment provided by a cloud provider (e.g., Microsoft Azure). In some embodiments, the platform subsystem 220 is configured for management and orchestration of resources of the cloud hosting environment. The platform subsystem 220 may be Figure 1 Cloud platform 112. Figure 2A As shown, platform subsystem 220 includes a web portal 222 , a database 224 storing application data, and a database 226 .

[0146] The web portal 222 may be a web-based interface. Through the web portal 222, an operator (e.g., a medical professional) may log into the platform subsystem 220, for example, using an operator-side computing device, to manage (view and / or query) application data 224 stored in the database 224 and / or data in the database 226. For example, the web portal 222 allows an operator to view a diagnosis result. A pre-written course of action may be provided based on the diagnosis result (e.g., seeking further evaluation).

[0147] As an example of the database 224, Figure 2D A database 240 is shown storing different types of documents. The database 240 may be a NoSQL database such as Azure Cosmos DB. Different types of documents may be stored as application data in the database 240. Unlike a relational database, a NoSQL database does not have strong relationships between documents. Figure 2D The dashed lines in the represent embedded references and information between documents.

[0148] In some embodiments, database 240 stores corresponding application data for treatment providers (or tenants). Treatment providers can be healthcare organizations, including but not limited to autism centers, healthcare institutions, specialists, physicians, or clinical research. Organizations may differ in structure, number of patients, and lifespan. Figure 2DAs shown, the corresponding application data may include organization documents 242, user documents 244, device documents 246, patient documents 248, session documents 250, and history documents 252. The user may be an operator associated with the healthcare organization, such as a medical assistant, specialist, physician, or any other medical professional.

[0149] Organization document 242 contains the organization's settings and customizations. User document 244 contains identifier information and the user's roles and permissions. The user role indicates whether the user is an administrator or operator associated with different security levels or permissions. Device document 246 contains the device information for each eye tracking console (e.g., Figure 2A 212). Patient document 248 contains information about the patient, such as information about an infant or child who is a patient for developmental assessment. Session document 250 contains information related to the session, which may include a session identifier (session ID), a reference to the patient, a reference to the user performing the session, a pointer to eye tracking data, and the results of data processing and analysis. History document 252 can be used to maintain a version history of document changes. The document reflects the structure of its parent document and includes additional audit information. In some embodiments, database 224 allows URL-based queries (e.g., for those with administrative roles) to query across multiple variables. For example, variables can include patients / devices / sessions, adverse events, etc.

[0150] In some embodiments, a cloud server including a platform subsystem 220 and a data pipeline subsystem 230 can be implemented in a centralized cloud environment, which can provide more flexibility to expand the capabilities of the cloud server. For example, a cloud server can utilize a multi-tenant architecture to provide diagnostic services based on software as a service (SaaS) subscriptions to treatment providers. In a multi-tenant architecture, treatment providers share a single version of software in various geographic locations. The term "tenant" in a multi-tenant architecture describes a single treatment provider of the system. The resources of a cloud server can be dynamically managed based on the total number of tenants and the expected average workload (e.g., how many tenants access the cloud server at a given point in time). A cloud server can use horizontal scaling techniques such as autoscaling to handle spikes in resource workloads.

[0151] In a multi-tenant architecture where tenants share an application, it is important to isolate their tenant data and prevent other tenants from accessing their tenant data. This is called isolation. There are 3 different isolation strategies that can be implemented: shared database, database per tenant, and application per tenant. In a shared database strategy, tenants share a single instance of the application and all data is stored in a single database. In a database per tenant strategy, for example Figure 2EIn the strategy 260 shown in FIG. 2 (a), tenants share a single instance of an application in the application layer 262, but have their own databases 264 (e.g. Figure 2A Database 224 or Figure 2D 240). In the application policy of each tenant, for example Figure 2E According to the strategy 270 shown in FIG. (b), each tenant has a corresponding application layer 272 and its own database 274 (e.g. Figure 2A Database 224 or Figure 2D The cloud server can deploy the database / tenant policy 260 or the application / tenant policy 270 to the treatment provider.

[0152] Continue to refer Figure 2A , the database 226 is configured to store raw eye tracking data or session data, processed session data, analysis results, and / or diagnostic results or reports. The database 226 can be a storage platform (e.g., Azure Blob) and can be paired with tools written in any suitable programming language (e.g., Python, Matlab), thereby allowing a URL-based interface and query to the database 226. In addition, the database 226 can be compatible with the programming language (e.g., Python, Matlab) used to transfer data from the data acquisition subsystem 210 to the database 226 and from the database 226 to the data pipeline subsystem 230. For example, on a patient-side computing device (e.g., Figure 1 In the case where the 130) is located at a medical facility, data collection is performed at the facility and data is transmitted between the database 226 and the patient-side computing device. The database 226 can be secure, HIPAA-compliant, and protected by a redundant backup system.

[0153] In some embodiments, the platform subsystem 220 is configured to enable one or more operations including (a) ingestion of new patient information, (b) storage of raw data files (e.g., including eye tracking data), (c) storage of data on a data collection device (e.g., Figure 2A The eye tracking console 212 of the embodiment of the present invention is provided, wherein: (a) the eye tracking console 212 is provided, and (b) the data processing computer and the data database are automatically and securely transferred between the data processing computer and the data database; (c) the eye tracking console 212 is provided, and (d) the data is automatically and securely transferred between the data processing computer and the data database; (e) the eye tracking console 212 is provided, and (c) the eye tracking console 212 is provided, and (d) the data is automatically and securely transferred between the data processing computer and the data database; (e) the data is automatically and securely transferred between the data processing computer and the data database; (f) the data is automatically and securely transferred between the data processing computer and the data database; (g ...

[0154] Continue to refer Figure 2AThe data pipeline subsystem 230 is configured to process and analyze the patient eye tracking data and generate a diagnosis result. In some embodiments, the data pipeline subsystem 230 includes a data processing module 232, a data analysis module 234, and a model data 236. Figure 7A-7B As discussed in further detail in , the data processing module 232 is configured to process session data including eye tracking data to obtain processed session data, and the data analysis module 234 is configured to analyze the processed session data using the model data 236 to generate a diagnosis result.

[0155] In some embodiments, system 200 includes interfaces for devices and subsystems. The interfaces can be inter-subsystem. For example, system 200 can also include an interface between data acquisition subsystem 210 and cloud platform subsystem 220, and an interface from cloud platform subsystem 220 to data pipeline subsystem 230. The interfaces can be intra-subsystem. For example, system 200 can include an interface between eye tracking console hardware (e.g., tablet computer and eye tracking device) and eye tracking application software.

[0156] Figure 2B According to one or more embodiments of the present disclosure, Figure 2A 200 . As described above, after a data collection session is completed, the eye tracking console 212 can automatically transmit the session data for the session to the platform subsystem 220. The session data can include two files: one containing raw eye tracking data (e.g., gaze position coordinates, blink data, pupil size data, or a combination thereof), and the other containing stimulation-related information (e.g., a list or playlist of those movies watched by the patient). The session data can be stored in the database 226 and stored in the application data in the database 224 through the upload function module 221 implemented in the platform subsystem 220. The stored session data is then automatically transmitted from the platform subsystem 220 to the data pipeline subsystem 230 for data processing and analysis without human intervention. For example, a software script written in any suitable programming language (e.g., Python, Matlab) can be used to transmit the raw, unprocessed data file from the database 226 to the data pipeline subsystem 230 for processing. The session data is first processed by the data processing module 232 and then analyzed by the data analysis module 234, which generates diagnostic information about the patient.

[0157] In some embodiments, three files are generated, one containing processed eye tracking data, one containing a summary of eye tracking statistics, and one containing diagnostic information. The file containing the diagnostic information can be uploaded to database 224 to be associated with the patient in the application data, such as Figure 2DAs shown. These three files can then be uploaded to database 226 for storage. In some cases, the processed eye tracking data is tabulated into a session table. A summary of the eye tracking information (e.g., fixation sample / movie, etc.) can be read from the processed summary file and tabulated in database 226 for subsequent querying. Summary values ​​(e.g., fixation / movie percentage, etc.) can then be calculated in database 226.

[0158] Figure 2C According to one or more embodiments of the present disclosure, Figure 2A An example of processing multiple session data in parallel in the system 200. Figure 2C As shown, multiple eye tracking consoles 212a, 212b can send session data 213a, 213b, 213c (collectively referred to as session data 213 or individually referred to as session data 213) of multiple sessions to the platform subsystem 220. In the data pipeline subsystem 200, the data processing module 232 and the data analysis module 234 can be written in an appropriate programming language (e.g., Python), which enables the data processing module 232 and the data analysis module 234 to be deployed in containers 231a, 231b, 231c (collectively referred to as containers 231 or individually referred to as containers 231). Each session can be processed using its own instance of data processing and analysis. The use of containers allows data processing and analysis to be performed when the session data is uploaded from the data acquisition subsystem 210, which can enable the session to be returned in a short time, such as within a 24-hour window.

[0159] As combined Figure 7A-7BAs discussed in further detail in , the cloud server can process and analyze session data of multiple sessions from a large number of computing systems in parallel. First, the cloud server can deploy a corresponding container (e.g., 231) for each session, and the corresponding container can include a corresponding data processing module 232 and a corresponding data analysis module 234. In this way, once the session data of a session (e.g., 213) is uploaded by the corresponding eye tracking console 212, the session data of the session can be processed and analyzed using its own container (e.g., 231 with its own data processing and data analysis instance). Second, when the session data of multiple sessions is processed in the corresponding container, for example, using most of the processing units (or cores) in the cloud server, the model data for analyzing the processed session data can be preloaded in parallel into the corresponding container, for example, using the remaining or a few processing units in the cloud server. Third, all processed session data and loaded model data can be analyzed in parallel in the corresponding container, for example, using the total number of processing units in the cloud server. The use of parallelization in a variety of ways can greatly increase the speed of session data processing and analysis, and provide rapid diagnosis results in a short time, such as within a 24-hour window. For example, once the diagnosis results are available, the cloud server can send the diagnosis results to the corresponding operator-side computing device (e.g., Figure 1 140), and then the diagnosis results can be displayed in the result interface of the operator application 216. Parallelization can also make the resource utilization of the cloud server more efficient, thereby further improving system performance.

[0160] Figure 2F The present invention shows a method for Figure 2A An exemplary configuration 280 of data backup of the system 200. Configuration 280 can achieve high availability of services to treatment providers, so that treatment providers can access their services regardless of any interruptions in one or more specific areas of the cloud server (e.g., platform subsystem 220 and data pipeline subsystem 230).

[0161] High availability refers to the ability of the treatment provider to access its services regardless of whether the cloud service provider suffers an interruption. Availability can be achieved by replicating resources in different physical locations. The cloud server implemented herein can be provided by a cloud service provider, which can provide software as a service (PAAS) resources with high availability built-in or configurable high availability. The resources hosted in the cloud environment can use a high availability service level agreement or have high availability by using geographic redundancy.

[0162] Figure 2F An example of high availability through geographic redundancy is shown. Figure 2FAs shown in (a), the resources of the cloud server can be hosted in a first data center 282 having a web portal 222a. The resources are replicated in a second data center 284. When the first data center 282 is operating normally, the treatment provider traffic is directed to the first data center 282, and the second data center 284 is a mirror. However, if Figure 2F As shown in (b), when the first data center 282 goes down, the treatment provider traffic is redirected to the replicated resources in the second data center 284 running the replicated web portal 222b. The switching process can be seamless, and the treatment provider may not be aware of the switch to different resources in the replicated data center.

[0163] Figure 2G An exemplary data backup of the system 200 (e.g., the platform subsystem 220 and the data pipeline subsystem 230) is shown. A database 224 storing application data and a database 226 storing raw and processed eye tracking data and analysis or diagnosis results can be stored in multiple data centers. The web portal 222 in the platform subsystem 220, the data processing module 232 and the data analysis module 234 in the data pipeline subsystem 230, and optionally the operator application 216 (running on the platform subsystem 220) can be included in the active data center 282 and can be replicated in the backup data center 284.

[0164] Example procedure for session data acquisition

[0165] Figure 3 is a flow chart of an exemplary process 300 for session data acquisition according to one or more embodiments of the present disclosure. The process 300 may be performed by, for example Figure 1 The computing system 120 or Figure 2A The system executes the data acquisition subsystem 210. The system includes an operator-side computing device (e.g., Figure 1 140) and an eye tracking device associated with it (e.g., Figure 1 134) integrated with one or more patient-side computing devices (e.g., Figure 1 130). Each of the operator-side computing device and the one or more patient-side computing devices can communicate with each other via a network (e.g., Figure 1 102) with a network-based server or cloud server (e.g., Figure 1 Cloud server 110 or Figure 2A-2G The system can be associated with a treatment provider, such as to provide developmental disability assessment and / or treatment services to a patient. The cloud server can be associated with a service provider for providing services to the treatment provider, such as data processing, analysis, and diagnostic results. For illustration, Figures 4A-4JThe following illustrates a method for obtaining session data during a session data acquisition (e.g., during a session data acquisition) according to one or more embodiments of the present disclosure. Figure 3 300 ) presents a series of illustrative display screens on an operator-side computing device (a) and a patient-side computing device (b).

[0166] In step 302, a session is initiated, for example, by establishing a connection or communication between an operator-side computing device and a patient-side computing device. In some embodiments, the two computing devices 130 and 140 can be connected wirelessly, for example, via a wireless connection, without being physically connected. The wireless connection can be through a cellular network, a wireless network, Bluetooth, near field communication (NFC), or other standard wireless network protocols. In some cases, the patient-side computing device can also be configured to connect to the operator-side computing device through a wired connection such as a universal serial bus (USB), for example, when the wireless connection fails.

[0167] In some embodiments, the connection between the operator-side computing device and the patient-side computing device is established by the two computing devices communicating with a cloud server, which in turn provides communication between the operator-side computing device and the patient-side computing device. Figure 4A As shown, an operator (e.g., a medical assistant, a medical professional, or any other representative of a treatment provider) can log into a web portal running on a cloud server (e.g., Figure 2A 222) for device management, patient management, and data management. Operators may have corresponding user roles and permissions, such as Figure 2D discussed in . Figure 4A FIG. 1 shows a user interface (UI) presented on a display screen of an operator-side computing device after an operator logs into a network portal using the operator-side computing device. The UI may be an operator application (e.g., Figure 2A 216) user interface.

[0168] like Figure 4AAs shown in Figure (a), the UI includes a menu displaying buttons "Home", "Patient", "Device" and "User". By clicking the button, the corresponding information (e.g., patient information, device information or user information) can be presented in the UI. For example, when the button "Device" is clicked, the UI displays a list of names of patient-side computing devices (e.g., device 1, device 2, device 3, device 4, device 5) that can be controlled by the operator. If the patient-side computing device is connected to a cloud server, such as device 4 or device 5, an indication may be presented near the name of the patient-side computing device, such as a string showing "Connect". The operator can select one of the names, such as device 4, to connect the corresponding patient-side computing device with the operator-side computing device. Once a name is selected, the UI displays a request to enter an access code for connecting to the corresponding patient-side computing device, such as Figure 4B As shown in Figure (a).

[0169] Figure 4A FIG. (b) shows a screen (eg, Figure 1 132). For example, the UI may be presented after the operator turns on and logs into the patient-side computing device. The UI may show a button “Start” that may be clicked, for example, by the operator to start the session. After clicking the button “Start”, the patient-side computing device connects to the cloud server, for example, to a web portal. The cloud server may associate the patient-side computing device with the operator based on an identifier of the patient-side computing device, for example, Figure 2D Once the patient-side computing device successfully connects to the cloud server, the UI presented on the patient-side computing device may display information of an access code (e.g., "5678") generated by the web portal for connecting to the operator-side computing device, as shown in FIG. Figure 4B The operator may obtain the access code from the UI presented on the patient-side computing device, input the access code into the UI presented on the operator-side computing device, and then submit the access code to the network portal. After the network portal confirms that the access code input into the operator-side computing device matches the access code generator of the patient-side computing device, the network portal may establish a wireless connection between the operator-side computing device and the patient-side computing device.

[0170] Once the connection between the operator-side computing device and the patient-side computing device (e.g., device 4) is established, connection information, such as “Connected to device 4”, may be displayed on the UI of the operator-side computing device, e.g., Figure 4C At the same time, the UI may show a button for starting to display visual information (e.g., a movie) to the patient on the screen of the patient-side computing device. The patient's caregiver (e.g., a parent) may take (or carry) the patient to watch the movie presented on the screen of the patient-side computing device.

[0171] In step 304, desensitization is started, for example, by the operator clicking a button "Start Movie" on the UI of the operator-side computing device, which may cause visual desensitization information (e.g., a movie) to be displayed to the patient on the screen of the patient-side computing device, such as Figure 4C As shown in Figure (b).

[0172] During the display of the desensitization movie, data is usually not recorded. Instead, the movie is displayed to gain the patient's attention. The movie can reflexively elicit external cues from the patient without verbal mediation or instructions from the operator. For example, the operator does not need to give instructions to view the screen of the patient-side computing device because the movie itself captures the patient's attention.

[0173] When the desensitization movie is displayed on the screen of the patient-side computing device, Figure 4D As shown in FIG. 2( b ), the operator can select the patient information of the patient through the UI of the operator-side computing device, such as Figure 4D The operator can select a patient from a list of existing patients associated with the operator in the cloud server, for example, Figure 2D After confirming the patient, the process begins by setting up the eye tracking device (or patient-side computing device) relative to the patient by displaying setup information on the UI of the operator-side computing device, as shown in FIG. Figure 4E The operator can also select "pause movie" or "skip movie" on the UI of the operator-side computing device.

[0174] During the setup, the desensitization movie can be kept playing on the screen of the patient's computing device, such as Figure 4E Figure (b) and Figure 4F As shown in Figure (b). Figure 4F As shown in FIG. 1( a ), the relative position between the eye tracking device and the patient's eye is shown on the UI of the operator-side computing device, for example, by text or graphics. The relative position can be determined by obtaining image data of the patient's eye using an image acquisition device (e.g., a camera) included in or adjacent to the eye tracking device. In some embodiments, after the operator clicks a button “Start Setup” in the UI of the operator-side computing device, as shown in FIG. Figure 4EAs shown in Figure (a) of , an operator application running on a cloud server can send a command to a patient-side computing device to use an image acquisition device to obtain an image of the patient's eye. The patient-side computing device can then send the obtained image to the cloud server, and the operator application can process the image to determine the relative position between the eye tracking device and the patient's eye. The relative position may include the distance between the eye tracking device and the patient's eye, the horizontal and / or vertical deviation between the center of the eye and the center of the field of view (or detection area) of the eye tracking device. Based on the relative position, the operator application can display instructions for adjusting the position of the eye tracking device on the UI of the operator-side computing device, such as "move the console down", such as Figure 4F Once the relative position of the patient's eye and the eye tracking device is acceptable, the operator can confirm the setup, for example, by clicking a button in the UI for "Confirm Setup". In some embodiments, in response to determining that the relative position of the patient's eye and the eye tracking device is less than a predetermined threshold (e.g., a horizontal / vertical deviation of less than 0.1 cm), the operator application can determine that the setup is complete and display an indication to the operator.

[0175] In step 306, the patient is calibrated with the eye tracking device. After completing the setup, the operator application can present a "Start Calibration" button on the UI of the operator-side computing device, such as Figure 4G . In some embodiments, calibration involves the patient viewing one or more fixed, known calibration targets (e.g., dots or icons) in the field of view. The calibration or fixation target reflexively captures the patient's attention and causes eye saccades toward and fixations on known target locations. The target reliably induces fixations on limited locations; for example, a radially symmetric target spans less than 0.5 degrees of visual angle. Other examples include concentric patterns, shapes, or contracting stimuli that reliably induce fixations on fixed target locations even if the initial size is larger.

[0176] For example, once the operator clicks a button to start calibration, a plurality of calibration targets may be sequentially presented at predetermined positions (or target positions) (e.g., center, upper left corner, or lower right corner) on the screen of the patient-side computing device, for example, as Figure 4G When multiple calibration targets are presented on the screen of the patient-side computing device, the eye tracking device can be activated to obtain the patient's eye tracking calibration data, for example in response to receiving a command from an operator application. The eye tracking application (e.g., Figure 2A 214) can be run on a patient-side computing device to collect eye tracking calibration data for the patient.

[0177] In some embodiments, a patient-side computing device (e.g., an eye tracking application) is configured to determine the position of a corresponding visual gaze of a calibration target, and then compare the determined position of the patient's corresponding visual gaze with a predetermined position at which the calibration target is presented. Based on the result of the comparison, the eye tracking application can determine whether the calibration target is calibrated. If the distance between the position of the patient's corresponding visual gaze and the predetermined position of the calibration target is within a predetermined threshold, the eye tracking application can determine that the patient's corresponding visual gaze matches the predetermined position of the calibration target, or that the calibration target is calibrated. If the distance is greater than or equal to the predetermined threshold, the eye tracking application can determine that the patient's corresponding visual gaze does not match the predetermined position, or that the calibration target has failed calibration.

[0178] In some embodiments, the patient-side computing device sends information about the obtained patient's eye tracking calibration data and / or predetermined position to an operator-side computing device or a cloud server, and the operator application can determine the patient's corresponding visual gaze position, and compare the determined position with multiple predetermined positions, and / or determine whether the calibration target is calibrated based on the comparison results.

[0179] In some embodiments, the first calibration target may first be presented in the center of the screen, and calibration may continue with four or more calibration targets presented in each corner of the screen along the direction of rotation. The operator application may alert the operator to the active status of the calibration (e.g., calibration point 1, calibration point 2, calibration point 3, or calibration completed 4 / 5 targets). Between each calibration target, the desensitization movie plays for a set time before displaying a new calibration target. Each calibration target may be cycled a set number of times before determining that the calibration target has failed to be calibrated and moving to the next calibration target. If the calibration target fails calibration, the calibration target may be retried after showing all remaining calibration targets and attempting gaze collection.

[0180] At step 308, the calibration is verified. For example, verification can be performed to measure the success of the calibration by displaying new targets and measuring the accuracy of the calculated gaze. The verification can show a smaller number of calibration targets (e.g., 3) compared to the calibration targets (e.g., 5) of step 306. A desensitization movie can be played between displaying two adjacent calibration targets.

[0181] In some embodiments, based on a comparison between the determined position of the patient's corresponding visual gaze and the predetermined position of the presented calibration target, an initial verification with different levels of success (e.g., the number of calibration targets calibrated or verified) can automatically instruct the operator to (1) recalibrate the eye tracking device with the patient, (2) reverify those calibration targets that could not be verified, or (3) accept the calibration and continue data collection at step 310.

[0182] In some embodiments, the operator may have the discretion to decide whether to accept the calibration. Figure 4H As shown, on the display screen of the operator-side computing device, the calibration targets and the representations (e.g., points) of the patient's corresponding visual gaze at the determined positions of the patient's corresponding visual gaze are simultaneously presented at multiple predetermined positions. The UI may also display a first button of "Accept Confirmation" and a second button of "Recalibrate". The operator can view the matches between the multiple calibration targets and the representations of the patient's corresponding visual gaze, and determine whether to accept the verification (by clicking the first button) or to recalibrate the patient relative to the eye tracking device (by clicking the second button).

[0183] At step 310, eye tracking data from the patient is collected by presenting a playlist of predetermined visual stimuli (e.g., stimulating movies) to the patient on a screen of a patient-side computing device, for example after calibration has been verified or operator acceptance has been verified. Figure 5 As shown in (a), the predetermined visual stimulation list may include, for example, a plurality of patient-specific social stimulation videos (e.g., 0075PEER, 0076PEER, 0079PEER) based on the patient's age and / or condition. Between each social stimulation video or before presenting each social stimulation video, a centering video (e.g., centerstim video) may be displayed to simply center the patient's gaze. In some embodiments, as Figure 5 As shown in (a), in the data collection step, a calibration check is performed, for example, between displaying the centering video (e.g., similar to step 306). For example, the calibration check can include showing five calibration targets, CCTL for calibration check upper left, CCTR for calibration check upper right, CCBL for calibration check lower left, CCCC for calibration check center-to-center, and CCBR for calibration check lower right. Data related to the calibration check can be used for post-processing, for example, for recalibrating the eye tracking data and / or for determining calibration accuracy.

[0184] In a specific example, the data collection sequence at step 310 may be as follows:

[0185] 1. Centering stimulation

[0186] 2. Exciting movies

[0187] 3. Centering stimulation

[0188] 4. Stimulus movie or calibration check (e.g., show 5 calibration targets, e.g. randomly between 2 and 4 stimulus movies)

[0189] 5. Repeat steps 1-4 until the playlist of the scheduled motivational movies is complete

[0190] In some embodiments, Fig. 4I As shown, the UI on the operator-side computing device shows a button for "start collection". After the operator clicks the "start collection" button, a playlist of predetermined visual stimuli can be sequentially presented on the screen of the patient-side computing device according to a predetermined sequence. Fig. 4I As shown, the UI may textually show the status of the running playlist (eg, Playing Movie: Centering Stimulation) or display the same content as that presented on the screen of the patient-side computing device (eg, Displaying Centering Stimulation Video).

[0191] In some embodiments, Figure 4J As shown, the UI can show a running playlist of videos that have been played or are being played, such as centering stimulation, PEER1234, centering stimulation, PEER5678. The UI can also display the video being presented on the screen of the patient-side computing device. The UI can also display a progress bar indicating the percentage of the stimulation movie that has been played in the playlist of predetermined stimulation movies. The UI can also display a button for the operator to skip the movie.

[0192] In some embodiments, the collected eye tracking data (e.g., Fig. 8A The calibration accuracy of 812) can be evaluated, for example, via the presentation of visual stimuli that reflexively capture attention and cause eye saccades and fixations toward known target locations. The targets reliably elicit fixations to limited locations; for example, radially symmetric targets span less than 0.5 degrees of visual angle. Other examples include concentric patterns, shapes, or contraction stimuli that reliably elicit fixations to fixed target locations even when initially larger in size. Such stimuli can be tested under data collection with head restraint to ensure that they reliably elicit fixations under ideal testing conditions; their use can then be expanded to include non-head restrained data collection.

[0193] In some embodiments, numerical evaluation of the accuracy of the collected eye tracking data may include the following steps: (1) presenting a fixation target that reliably elicits fixation to a small area of ​​the visual display unit; (2) recording eye tracking data throughout the target presentation; (3) identifying fixations in the collected eye tracking data; (4) calculating the difference between the fixation position coordinates and the target position coordinates; and (5) storing the difference between the calculated fixation position coordinates and the target position coordinates as vector data (direction and magnitude) for as few as one target or for as many targets as possible (e.g., five or nine but may be more). In some embodiments, the fixation position coordinates may be aligned with the actual target position coordinates, for example, by applying a spatial transformation, performing a recalibration or post-processing step by methods including but not limited to (a) trilinear interpolation, (b) linear interpolation in barycentric coordinates, (c) affine transformations, and (d) piecewise polynomial transformations.

[0194] Once the playlist of predetermined visual stimuli is completely played, the session ends. The patient-side computing device may generate session data based on the raw eye tracking data collected by the eye tracking device, for example, by storing the raw eye tracking data and associated information (time stamp information) in a data file (e.g., in .tsv format, .idf format, or any suitable format), such as Figure 5 (b) The raw eye tracking data may include values ​​of multiple eye tracking parameters at different timestamps. The eye tracking parameters may include gaze coordinate information of the left eye, the right eye, the left pupil and / or the right pupil.

[0195] The session data may also include information about the visual stimuli played or presented in another data file (e.g., in .tsv format or any suitable format), such as Figure 5 (a). The information may include timestamp information and the name of each visual stimulus played. The timestamp information of the visual stimulus may be associated with the timestamp information of the eye tracking data, so that the eye tracking data for each visual stimulus (and / or calibration check) may be determined separately based on the timestamp information in the two data files.

[0196] At step 312, the session data is sent to the cloud server. Once the session data is generated by the patient-side computing device, the patient-side computing device may send the session data to the cloud server. Figure 2A-2G As discussed in Figure 6 , 7A -7B and 8, the cloud server may first store the session data in a centralized database (e.g., Figure 2A-2B The session data is then processed, the processed data is analyzed, and a diagnosis of the patient is generated, which can be accessed or viewed by an operator or medical professional.

[0197] Example Data Processing and Analysis

[0198] Figure 6 is a method for executing a command by a cloud server (e.g., Figure 1 Cloud server 110 or Figure 2A-2G Flowchart of an exemplary process 600 for managing session data (e.g., data processing and analysis) using a cloud server described in . Figure 7A-7B A method for performing a comparison of the number of packets received by a cloud server according to one or more embodiments of the present disclosure is shown. Figure 6 A flow chart of an exemplary process 700 for managing session data in greater detail.

[0199] At step 702, once the session is complete, the corresponding patient-side computing device (e.g., Figure 1 130) or an eye tracking console (e.g., Figure 2A-2G 212) sends the session data of the session to the cloud platform of the cloud server, for example, through a web portal. The cloud platform may be Figure 1 Platform 112 or Figure 2A-2G In response to receiving the session data, the cloud platform of the cloud server stores the session data in a database in the cloud platform (e.g., Figure 2A-2G The cloud platform then automatically transmits the session data to a data pipeline system (e.g., Figure 1 114 or Figure 2A-2G 230) for data processing and analysis.

[0200] At step 704, a file pointer for the session data of the session is added to a processing queue (step 704). The session data of all completed sessions are waiting for processing according to the processing queue. Once the session data of a session is uploaded and stored in the cloud server, a corresponding file pointer can be assigned to the session data of the session and added to the processing queue. The file pointer can be an identifier of the session data of the corresponding session. The session data of the corresponding session can be retrieved from the database in the cloud platform based on the file pointer.

[0201] At step 706, a corresponding container is created for the session data of each session, for example based on an automatic scaling technique that can implement session parallelization. For example, in response to adding a file pointer for a new session to the processing queue, a new container can be created for the new session. Each container (e.g., Figure 2C 231) may have its own instances of data processing modules and data analysis modules, for example, Figure 2C shown.

[0202] In each container, for example, Figure 2A-2GThe data processing module 232 performs steps 708 to 714 on the session data of the corresponding session. Note that steps 708 to 714 may be performed in parallel on the session data of multiple sessions in multiple containers.

[0203] In step 708 (corresponding to Figure 6 In step 602), the session data is obtained from the database in the cloud platform using the corresponding file pointer. As described above, the session data may include two files: an eye tracking data file (e.g., Figure 5 (b)) and playlist files (e.g., Figure 5 (as shown in (a)).

[0204] refer to Figure 6 , step 602 may correspond to step 708. At step 604, the session data is prepared for processing. Step 604 may include one or more steps as described in steps 710 to 714 of FIG. 7 .

[0205] Step 604 may include linking the eye tracking data in the eye tracking data file to the movie played in the playlist file. Fig. 7A As shown, at step 710, the eye tracking data is decomposed into separate runs, for example, based on the timestamp information in the two files. Each run can correspond to playing a corresponding movie (e.g., a centering target, a predetermined visual stimulus, or one or more calibration targets). For example, eye tracking data corresponding to timestamps within a range defined by the timestamps of two adjacent movies are included in one run. At step 712, the eye tracking data in each run is linked to the corresponding movie in the playlist based on the timestamp information in the two files. In some embodiments, the eye tracking data is not decomposed into separate runs, but is processed as a continuous stream of data samples with links to corresponding movies in the playlist.

[0206] In step 714, the eye tracking data is recalibrated to account for drift or deviation. In some embodiments, the eye tracking data collected in the calibration step during the presentation of the playlist (e.g., Figure 5 The calibrated target shown in Figure (a) can be used to calibrate or align the eye tracking data collected during the playback of each movie in different rounds. For example, using data from a time adjacent to the time when the additional calibration target is shown, any differences in gaze position are corrected. Some large differences may exclude certain data from subsequent analysis.

[0207] In step 606, the prepared session data is processed. In some embodiments, the data processing module extracts relevant information from the prepared session data, such as the patient's visual gaze and / or visual gaze on an object or area of ​​interest in the movie. In some embodiments, the data is resampled to account for any temporal changes between samples. Any appropriate interpolation and / or smoothing techniques may be used to resample the data. The data may be converted from a specified original resolution and / or coordinate system of the collected eye tracking data to an appropriate resolution and / or coordinate system for analysis. For example, the raw data may be collected at a higher resolution (e.g., 1024×768 pixels) than the resolution of the presented stimulus (e.g., rescaled to 640×480 pixels). In some embodiments, the data processing module may automatically identify basic eye movement events (unwanted gazes, eye saccades, blinks, off-screen or lost data, etc.), and may automatically identify the time when the subject is gazing (in an undesirable manner), saccades, blinks, or the time when the subject is not watching the screen. The data processing module may adjust for aberrations in the gaze position estimate as output by the eye tracking device.

[0208] In some embodiments, Figure 7B As shown in step 716, the session data of the multiple sessions of the patient are processed in parallel in the multiple session containers, and the corresponding model data of the patient is preloaded into the multiple session containers. In some examples, the session data of the multiple sessions are processed in the multiple session containers using a majority of the processing units (e.g., N processing cores) in the cloud server, while the corresponding model data is preloaded in parallel into the multiple session containers using a minority of the processing units (e.g., M processing cores) in the cloud server. The processing unit or core may be a central processing unit (CPU). Parallelization may avoid additional time waiting for uploading model data.

[0209] The cloud server can store the model data in advance in a database, e.g. Figure 2A-2G 226. The model data may include data of a large number of instances in which gaze positions of patients (e.g., infants, toddlers, or children) vary significantly at different levels of social, cognitive, or developmental function. The patient's corresponding model data may include data related to patients of similar age, similar background, and / or similar condition, which may be used with the patient's processed session data to generate a diagnosis for the patient. The patient's corresponding model data may be identified and retrieved from a database, for example, based on the patient's age, the patient's background, and / or the patient's condition. Step 608 of process 600, in which the processed data is prepared for analysis, may include obtaining the processed data in a plurality of session containers and preloading the corresponding model data in the plurality of session containers.

[0210] At step 610, the processed data is analyzed to generate an analysis result. In some embodiments, for a session, the processed data is compared with corresponding model data in a corresponding session container to obtain a comparison result. In some embodiments, the data analysis module uses the processed data and the corresponding model data to generate a result, for example, using comparison or inference via a statistical model, an algorithm, an artificial intelligence (AI) model (e.g., a machine learning or artificial neural network model). In some embodiments, as Figure 7B As shown in step 718 , in multiple session containers, the total number of processing units (eg, N+M cores) are used to analyze the processed session data and the preloaded model data in parallel.

[0211] In some embodiments, the processed session data is compared to a corresponding data model to determine the level of developmental, cognitive, social, or psychiatric condition. The resulting score is then compared to a predetermined cutoff or other value to determine the patient's diagnosis of ASD and the severity of the condition. In certain other embodiments, the patient's gaze point data (e.g., visual fixation data) is analyzed over a predetermined time period (e.g., over multiple treatment sessions spanning several months) to identify a decrease, increase, or other significant change in visual fixation (e.g., gaze point data that initially corresponds to a typically developing child changes to more unstable gaze point data corresponding to gaze point data of a child exhibiting ASD, or gaze point data that becomes more similar to that of a typically developing child in response to targeted therapy).

[0212] At step 720 (corresponding to step 612), a summary of the results is calculated. As described above, the results of the analysis can be used to determine a score for at least one index, such as a social disability index, a language ability index, a non-verbal ability index, a social adaptation index, and / or a social interaction index. Based on a comparison of the score to at least one predetermined cutoff value, an ASD diagnosis for the patient and the severity of the condition can be calculated. For example, Fig. 8A As shown, based on the analyzed results and / or any other suitable information (e.g., from other relevant analyses of the patient), a social disability index score of 6.12 is shown in the range of -50 (social disability) to 50 (social ability), indicating that social disability is not involved; a language ability index score of 85.89 is shown in the range of 0 to 100, indicating higher than average language ability; a non-verbal ability index score of 85.89 is shown in the range of 0 to 100, indicating higher than average non-verbal ability. In addition, a non-ASD diagnosis can also be calculated based on the analyzed data.

[0213] In some embodiments, the summary of the results includes a visualization of an individual's eye tracking data (e.g., gaze point data) overlaid on a film clip from a socially relevant moment, allowing clinicians and parents to better understand how the patient visually attends to social information. For example, in step 610, a film clip for which the patient has available data can be cross-referenced with a list of films that have been predetermined to elicit eye gaze behavior using information about diagnostic status (including symptom severity). The visualization can also include a visualization of aggregated reference data from typically developing children, for example, matched on patient attributes such as age, gender, etc. These visualizations can be side-by-side so that clinicians and / or parents can compare individual patient data with reference data to see how gaze patterns align or diverge. These visualizations can include annotations that explain film content, eye gaze patterns, etc.

[0214] In some embodiments, the summary of results includes an animation that visualizes the patient's eye tracking data overlaid on a film clip from a socially relevant moment. For example, the web portal may include a dashboard that allows the clinician to view a stimulus movie displayed on the patient with the eye gaze data overlaid. The dashboard may be configurable to allow the user to select which movies to display, and whether to display frames that obtain information about the social disability index, the language ability index, the non-verbal index, or any other index calculated in the report.

[0215] Continue to refer Figure 7B , in step 722, the data pipeline subsystem returns the result output to the web portal, e.g. Figure 2B In some embodiments, the result output includes three files: one containing processed eye tracking data, one containing a summary of eye tracking statistics, and one containing diagnostic information (e.g., a summary of the results). These three files can then be uploaded to a database (e.g., Figure 2A-2G 226) for storage. In some cases, the processed eye tracking data is tabulated into a session table. A summary of the eye tracking information (e.g., fixation sample / movie, etc.) can be read from the processed summary file and tabulated in a database for subsequent querying. Summary values ​​(e.g., fixation / movie percentage, etc.) can then be calculated within the database.

[0216] At step 724, the result output is reconnected with the patient information to generate a diagnostic report or result for the patient. For example, a file containing diagnostic information may be uploaded to an application data database (e.g., Figure 2A-2G 224) to be associated with the patient in the application data, e.g. Figure 2DThe diagnostic report or results may be presented in any suitable manner to a user (an operator or medical professional, such as a physician) associated with the patient in the application data database or a caregiver associated with the patient.

[0217] In some embodiments, once a patient's diagnostic report or results are generated, the user may be notified (e.g., via email or message) to log in to view the diagnostic report or results through a web portal. The diagnostic report or results may be presented on a user interface, such as Fig. 8A or Figures 8B-8C In some embodiments, once a diagnostic report or result for a patient is generated, the diagnostic report or result can be sent to an operator-side computing device for presentation to the user. The diagnostic report or result can also be sent to the operator in a secure email or message. The diagnostic report or result can be stored in an application data database (e.g., Figure 2A-2G 224) and / or databases (e.g., Figure 2A-2G 226).

[0218] Fig. 8A An exemplary result interface 800 is shown for displaying a diagnostic report or result including at least one index value based on eye tracking data according to one or more embodiments of the present disclosure. The result interface 800 shows patient information 802, requesting physician / institution information 804, device ID of the patient-side computing device 806, processing date 807 (indicating the time when the processed session data was obtained), and report release date 808.

[0219] Results interface 800 also shows collection information 810 including calibration accuracy 812, eye movement function 814, and data collection summary 816. Calibration accuracy 812 and eye movement function 814 may be represented graphically. Data collection summary 816 may include at least one of the number of videos viewed, the number of videos excluded, duration of data collection, time spent viewing videos, time spent not viewing, calibration accuracy, eye movement measurements, or quality control measurements.

[0220] The results interface 800 also shows neurodevelopmental test results 820, which may include a diagnosis result 822 (e.g., ASD or non-ASD), social disability index information 824, language ability index information 826, and non-verbal ability index information 828. The results interface 800 may graphically display these index information 824, 826, 828 with corresponding descriptions.

[0221] Figures 8B-8C Another example results interface 850 is shown showing performance-based measurements of development assessments related to: Figure 8BNonverbal Communication and Gestures (A) and Joint Attention & Mutual Gaze (B) in Figure 8C Facial Affect (C) and Pointing and Social Monitoring (D) in.

[0222] Results interface 850 shows performance-based measures of children's individual weaknesses and opportunities for skill development. Neurodevelopmental assessments via eye tracking measure how children interact with social and non-social cues that appear continuously within natural environmental contexts (left column 852, shown as still frames from test videos). With respect to these contexts, standard reference measures provide objective quantification of visual engagement for non-ASD, expected ages (middle column 854 shown as density distributions in pseudo-color format, middle column 856 shown as color-scale fades overlaid on corresponding still frames). Reference measures for expected ages can be used to measure and visualize patient comparisons, revealing individual strengths, weaknesses, and opportunities for skill building (right column 858, shown as example patient data covering circular apertures that contain the portion of the video viewed by each patient, e.g., each aperture spans the center of the patient's field of view ~5.2 degrees). Individuals with ASD demonstrate instances of not paying attention to (A) verbal and nonverbal interactions and gestures (860); (B) joint attention and mutual gaze cues (870); (C) dynamic facial effects (880); and (D) joint attention and social monitoring (890). Figures 8B-8C As shown, children with ASD engage with toys of interest (1, 3, 5, 7); color and contrast cues (2, 6, 8); objects (10, 11, 12); background elements not directly related to the social context (4, 9, 13); and recurring visual features (14, 15, 16, 17, 18). The elapsed time in the lower right corner of the still frame highlights the rapidly changing nature of social interactions: over the course of approximately 12 minutes of viewing time, hundreds of verbal and nonverbal communicative cues are presented, each eliciting age-expected patterns of engagement and providing opportunities for objective, quantitative comparisons of the patients' behaviors.

[0223] Example Process

[0224] Fig. 9 is a flow chart of an exemplary process 900 for session data acquisition according to one or more embodiments of the present disclosure. The process 900 may be performed by a system, such as Figure 1 The computing system 120 or Figure 2A-2G The data acquisition subsystem 210. The process 900 may be similar to Figure 3of the course 300 and may refer to Figures 4A to 4J to describe.

[0225] The system includes an operator-side computing device (e.g., Figure 1 140) and an eye tracking device associated with it (e.g., Figure 1 134) integrated with one or more patient-side computing devices (e.g., Figure 1 130). At least one of the operator-side computing device or the patient-side computing device may be a portable device. The operator-side computing device and each of the one or more patient-side computing devices may be connected to the network (e.g., Figure 1 102) with a network-based server or cloud server (e.g., Figure 1 Cloud server 110 or Figure 2A-2G The system may be associated with a treatment provider, such as to provide developmental disability assessment and / or treatment services to a patient. The cloud server may be associated with a service provider for providing services to the treatment provider, such as data processing, analysis, and diagnostic results. Process 900 may include multiple steps, some of which are performed by an operator-side computing device, some of which are performed by a patient-side computing device and / or an eye tracking device, and some of which are performed by a combination of an operator-side computing device and a patient-side computing device.

[0226] At step 902, a patient session is initiated by establishing communications with an operator-side computing device and a patient-side computing device. In some embodiments, establishing communications includes establishing a wireless connection between the operator-side computing device and the patient-side computing device, such as Figure 1 Wireless connection 131.

[0227] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes: accessing, by the operator-side computing device, a web portal at a network connection server (eg, Figure 2A-2G 222), and in response to receiving a selection of the patient-side computing device in the network portal, wirelessly connecting the operator-side computing device to the patient-side computing device.

[0228] In some embodiments, establishing a wireless connection between the operator-side computing device and the patient-side computing device includes, for example, Figure 4B As shown, the patient-side computing device displays connection information on a screen of the patient-side computing device, and in response to the operator-side computing device receiving an input of the connection information, a wireless connection is established between the operator-side computing device and the patient-side computing device.

[0229] In some embodiments, process 900 further includes: after establishing communication, displaying visual desensitization information to the patient on a screen of the patient-side computing device, such as Figure 4C The eye tracking device may be configured to not collect eye tracking data from the patient when the visual desensitization information is displayed.

[0230] In some embodiments, process 900 further includes: while displaying the visual desensitization information, accessing, by the operator-side computing device, a web portal at a network connection server to establish a session for the patient, such as Figure 4D and 4E In some cases, establishing the session includes one of selecting the patient in a patient list or creating a profile for the patient on a network connection server.

[0231] In some embodiments, process 900 further includes determining a relative position between the eye tracking device and at least one eye of the patient, and displaying instructions for adjusting the position of the eye tracking device or the position of the patient on a user interface of the operator-side computing device, such as Figure 4F In some cases, process 900 also includes: in response to determining that the relative position of at least one eye of the patient is at a predetermined position in the detection area of ​​the eye tracking device, determining that the patient is aligned with the eye tracking device.

[0232] At step 904, the patient is calibrated with respect to the eye tracking device by displaying one or more calibration targets to the patient on a screen of the patient-side computing device, e.g. Figure 4G As shown. Each of the one or more calibration targets can be sequentially presented at a corresponding predetermined position of a screen of a patient-side computing device while obtaining eye tracking calibration data of the patient using an eye tracking device. Process 900 may include: for each of the one or more calibration targets, processing the obtained eye tracking calibration data of the patient to determine a corresponding visual gaze position of the patient for the calibration target; comparing the corresponding visual gaze position of the patient with a corresponding predetermined position of the presented calibration target; and determining whether the calibration target is calibrated relative to the eye tracking device based on the result of the comparison.

[0233] In some embodiments, calibrating the patient relative to the eye tracking device also includes: in response to determining that a deviation between the patient's corresponding visual gaze position and the corresponding predetermined position is less than or equal to a predetermined threshold, determining that the calibration target is calibrated and displaying the next calibration target, or in response to determining that the deviation is greater than the predetermined threshold, determining that the calibration target has failed to be calibrated, and redisplaying the calibration target for calibration.

[0234] In some embodiments, process 900 further includes: after calibrating the patient relative to the eye tracking device, verifying the calibration using one or more new calibration targets. Similar to the calibration described in step 904, verifying the calibration includes: sequentially presenting each of the one or more new calibration targets at a corresponding predetermined position of the screen of the patient-side computing device while obtaining eye tracking calibration data of the patient using the eye tracking device; and processing the obtained eye tracking calibration data of the patient to determine the position of the patient's corresponding visual gaze for each of the one or more new calibration targets.

[0235] In some embodiments, for example, Figure 4H As shown, verifying the calibration includes: simultaneously presenting on the user interface of the operator-side computing device the one or more new calibration targets at one or more corresponding predetermined positions and one or more corresponding visual gaze representations of the patient at the determined one or more positions; and in response to receiving an indication of verifying the result of the calibration, determining that the calibration has been verified, or in response to receiving an indication that the calibration result is invalid, starting to recalibrate the patient relative to the eye tracking device.

[0236] In some embodiments, verifying the calibration includes: determining the number of new calibration targets that each pass the calibration based on the patient's corresponding visual gaze position and the corresponding predetermined position; and if the number or the relevant percentage is greater than or equal to a predetermined threshold, determining that the calibration is verified, or if the number or the relevant percentage is less than a predetermined threshold, determining that the calibration is invalid and starting to recalibrate the patient relative to the eye tracking device.

[0237] At step 906 , after determining that the calibration has been verified, a list of predetermined visual stimuli is sequentially presented to the patient on a screen of the patient-side computing device while eye tracking data of the patient is collected using the eye tracking device.

[0238] In some embodiments, for example, Fig. 4I , 4J As shown in FIG. 5 , before presenting each of the predetermined visual stimuli in the list, a centering target may be presented to the patient on the screen of the patient-side computing device for centering the patient's gaze.

[0239] In some embodiments, for example, Figure 5 As shown, the patient performs calibration of the eye tracking device between two adjacent visual stimuli in a playlist presenting predetermined visual stimuli. The eye tracking data collected when performing the calibration can be used to calibrate the patient's eye tracking data or to determine at least one of the calibration accuracy by the network connection server.

[0240] In some embodiments, for example, Figure 4J As shown, process 900 also includes: presenting at least one of the following items on the user interface of the operator-side computing device: a progress indicator that is kept updated during the presentation of the predetermined visual stimulation playlist; information about visual stimulation that has been presented or is being presented; information about visual stimulation to be presented; and a user interface element for skipping visual stimulation in the predetermined visual stimulation playlist.

[0241] At step 908, the patient-side computing device sends session data for the session to the network-connected server, the session data including the patient's eye tracking data collected during the session. The patient-side computing device may automatically send the session data for the session to the network-connected server in response to one of: determining completion of presenting the playlist of predetermined visual stimuli on the screen, or receiving a completion indication of the session from the operator-side computing device, such as via a web portal on the network-connected server.

[0242] In some embodiments, the session data includes information related to a playlist of the pre-determined visual stimuli presented, which may include the names of the pre-determined visual stimuli presented and associated timestamps when the pre-determined visual stimuli were presented, e.g. Figure 5 The session data is shown in Figure (a). Figure 5 As shown in FIG. (b), when eye tracking data is generated or collected, the session data may include the eye tracking data and an associated timestamp. In some embodiments, sending the session data includes sending a first file storing the patient's eye tracking data and a second file storing information related to the list of predetermined visual stimuli presented.

[0243] Fig.10 1 is a flow chart of an exemplary process 1000 for data processing and analysis according to one or more embodiments of the present disclosure. Process 1000 may be performed by a network-connected server, which may be a cloud server in a cloud environment, such as Figure 1 Cloud server 110 or Figure 2A-2G For example, a network server may include a platform such as Figure 1 112 or Figure 2A-2G 220, and data pipeline systems, such as Figure 1 114 or Figure 2A-2G 230. The platform may include a web portal (e.g., Figure 2A-2G 222), application data database (e.g., Figure 2A-2G 224) and databases (e.g. Figure 2A-2G 226). The data pipeline system may include one or more data processing modules (e.g., Figure 2A-2G 232) and one or more data analysis modules (e.g., Figure 2A-2G234). Process 1000 may be similar to Figure 6 Course 600 or Figure 7A-7B Process 700.

[0244] In step 1002, session data of a plurality of sessions is received, for example, Figure 2B As shown, the session data of each session includes eye tracking data of a corresponding patient in the session. In step 1004, the session data of the plurality of sessions are processed in parallel to generate processed session data of the plurality of sessions. In step 1006, for each session in the plurality of sessions, the processed session data of the session is analyzed based on corresponding reference data to generate an assessment result of the corresponding patient in the session.

[0245] In some embodiments, method 1000 further includes loading corresponding reference data for the plurality of sessions in parallel with processing session data for the plurality of sessions.

[0246] In some embodiments, the network connection server includes multiple processing cores. Processing session data for multiple sessions in parallel may include, for example, Figure 7B As shown in step 716 of , the session data of the plurality of sessions are processed in parallel using a first plurality of processing cores, and corresponding reference data of the plurality of sessions are loaded using a different second plurality of processing cores. The number of the first plurality of processing cores may be greater than the number of the second plurality of processing cores. In some embodiments, analyzing the processed session data of the plurality of sessions based on the loaded corresponding reference data of the plurality of sessions may include, for example, Figure 7B As shown in step 718, a plurality of processing cores including a first plurality of processing cores and a second plurality of processing cores are used.

[0247] In some embodiments, analyzing the processed session data of the multiple sessions based on the loaded corresponding reference data of the multiple sessions includes at least one of: comparing the processed session data of the sessions with corresponding reference data, using the corresponding reference data to infer assessment results of the corresponding patients from the processed session data, or using at least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model.

[0248] In some embodiments, the corresponding reference data includes historical eye tracking data or results for a patient having substantially the same age or condition as the corresponding patient. In some embodiments, method 1000 includes generating the assessment result based on previous session data for the corresponding patient.

[0249] In some embodiments, for each session in the plurality of sessions, a corresponding container is allocated to the session, e.g. Figure 2COr as shown in FIG7A . The process 1000 may include: in a corresponding container, processing session data of a session and analyzing the processed session data of the session based on corresponding model data to generate an assessment result of a corresponding patient in the session.

[0250] In some embodiments, when the eye tracking data is collected in the session, the eye tracking data is associated with a predetermined list of visual stimuli presented to the patient, and the session data includes information associated with the predetermined list of visual stimuli in the session.

[0251] In some embodiments, process 1000 further includes, for example, Fig. 7A As shown, in a corresponding container, the eye tracking data is decomposed into a plurality of parts based on information associated with the predetermined visual stimulus list, each part of the eye tracking data being associated with one of the corresponding predetermined visual stimulus or the corresponding calibration.

[0252] In some embodiments, processing the session data of the session includes processing a portion of the eye tracking data associated with the corresponding predetermined visual stimulus based on information of the corresponding predetermined visual stimulus. In some embodiments, method 1000 also includes: in the corresponding container, recalibrating a portion of the eye tracking data associated with the corresponding predetermined visual stimulus based on at least a portion of the eye tracking data associated with the corresponding calibration.

[0253] In some embodiments, method 1000 further includes determining, in the respective container, a calibration accuracy using at least a portion of the eye tracking data associated with the respective calibration and a plurality of predetermined locations at which the plurality of calibration targets are presented in the respective calibration.

[0254] In some embodiments, receiving the session data for the plurality of sessions includes, for example, Figure 2C As shown, session data for a plurality of sessions is received from a plurality of computing devices associated with a corresponding entity via a web portal.

[0255] In some embodiments, process 1000 further includes, for example, Fig. 7A As shown, in response to receiving session data for a session, a file pointer for the session data is added to a processing queue to be processed. Process 100 may also include: storing the session data for the session in a database using the file pointer for the session; and retrieving the session data for the session from the database using the file pointer for the session.

[0256] In some embodiments, method 1000 further includes, for each entity, storing session data from one or more computing devices associated with the entity in a corresponding repository in the application data database, e.g., Figure 2EAs shown. The corresponding repository may be isolated from one or more other repositories and inaccessible to one or more other entities. The application data database may be a NoSQL database.

[0257] In some embodiments, corresponding repositories for entities include, for example, Figure 2D At least one of the following items shown: information about the entity, information about one or more operators or operator-side computing devices associated with the entity, information about one or more patient-side computing devices associated with the entity, information about one or more sessions conducted in the entity, information about one or more patients associated with the entity, or historical information from a corresponding repository.

[0258] In some embodiments, method 1000 further includes: for example, Figure 2F As shown, the resources of the network connection server are dynamically adjusted based on the number of computing devices accessing the network connection server. Process 1000 may also include: replicating data from the first data center to the second data center; and in response to determining that the first data center is inaccessible, automatically directing traffic to the second data center.

[0259] In some embodiments, each of the first data center and the second data center includes a web portal accessible by an operator-side computing device, an operator application, or at least one instance of an application layer for data processing and data analysis, such as Figure 2G The process may further include: storing the same data in a plurality of data centers. The data may include: application data of the entity and information associated with the eye tracking data.

[0260] In some embodiments, the method 1000 further includes associating the generated assessment result with the corresponding patient in the session and generating an assessment report for the corresponding patient, for example, Figure 7B As shown in step 724.

[0261] In some embodiments, the method 1000 further includes: outputting the evaluation result or the evaluation report to be presented on a user interface of the operator-side computing device, such as through a web portal.

[0262] In some embodiments, for example, Fig. 8AAs shown. The assessment report includes at least one of the following: information about the corresponding patient, information about the entity that performed the session with the corresponding patient, information about the calibration accuracy in the session, information about the session data collection, or an assessment result for the corresponding patient. In some embodiments, the assessment result indicates the possibility that the corresponding patient has a developmental, cognitive, social, or intellectual disability or ability. For example, the assessment result indicates the possibility that the corresponding patient has Austin spectrum disorder (ASD) or non-ASD. In some embodiments, the assessment result includes a corresponding score for each of one or more of a social disability index, a language ability index, and non-verbal ability, for example, Fig. 8A shown.

[0263] In some embodiments, the corresponding patient has an age in the range of 5 months to 7 years, including an age in the range of 5 months to 43 months or 48 months, an age in the range of 16 months to 30 months, an age in the range of 18 months to 36 months, an age in the range of 16 months to 48 months, or an age in the range of 16 months to 7 years.

[0264] Example cloud computing system architecture

[0265] Fig.11 A cloud computing system according to one or more embodiments of the present disclosure (eg, referring to Figure 1 The cloud server 110 described or referenced Figure 2A-2G 100 is an exemplary architecture 1100 of a cloud server (described in the embodiment of the present invention). Other architectures are also possible, including architectures with more or fewer components. In some implementations, the architecture 1100 includes one or more processors 1102 (e.g., dual-core The components may include a processor, one or more network interfaces 1106, one or more storage devices 1104 (e.g., hard disk, optical disk, flash memory), and one or more computer-readable media 1108 (e.g., hard disk, optical disk, flash memory, etc.). These components may exchange communications and data over one or more communication channels 1110 (e.g., buses), which may utilize various hardware and software to facilitate the transmission of data and control signals between components.

[0266] The term "computer-readable medium" refers to any medium that participates in providing instructions to processor 1102 for execution, including but not limited to non-volatile media (such as optical or magnetic disks), volatile media (such as memory), and transmission media. Transmission media include but are not limited to coaxial cables, copper wire, and fiber optics.

[0267] The computer readable medium 1108 may further include a computer program for operating systems (e.g., Mac server, NT server, Linux server) instructions 1112, instructions 1114 for network communication module, data processing instructions 1116, and interface instructions 1118.

[0268] The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but not limited to: identifying input from and providing output to devices 1102, 1104, 1106, and 1108; tracking and managing files and directories on computer-readable media 1108 (e.g., memory or storage); controlling peripheral devices; and managing traffic on one or more communication channels 1110. The network communication module includes software for establishing and maintaining network connections (e.g., software for implementing communication protocols such as TCP / IP, HTTP, etc.) and for using, for example, Apache Kafka. TM To create various components of the distributed streaming platform. Data processing instructions 1116 include server-side or back-end software for implementing server-side operations, such as reference Figure 1 The interface instructions 1118 include software for implementing a network server and / or portal that sends and receives data to and from a user-side computing device and a service-side computing device.

[0269] The architecture 1100 can be implemented by a cloud computing system and can be included in any computer device, including one or more server computers in a local or distributed network, each server computer having one or more processing cores. The architecture 1100 can be implemented in a parallel processing or peer-to-peer infrastructure or on a single device with one or more processors. The software can include multiple software components or can be a single body of code.

[0270] Example computing device

[0271] Fig.12 The architecture of a computing device according to one or more embodiments of the present disclosure is shown. Fig.12 , showing a schematic diagram of an apparatus 1200. The apparatus 1200 includes a processor 1204, a memory 1206, a storage component 1208, an input interface 1210, an output interface 1212, a communication interface 1214, and a bus 1202. In some embodiments, the apparatus 1200 corresponds to Figure 1 At least one of the patient-side computing device 130 or the operator-side computing device 140 .

[0272] The bus 1202 includes components that allow communication between components of the device 1200. In some embodiments, the processor 1204 is implemented in hardware, software, or a combination of hardware and software. In some examples, the processor 1204 includes a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), and / or the like), a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or the like). The memory 1206 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic and / or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores data and / or instructions for use by the processor 1204.

[0273] Storage component 1208 stores data and / or software related to the operation and use of device 1200. In some examples, storage component 1208 includes a hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a tape, a CD-ROM, a RAM, a PROM, an EPROM, a FLASH-EPROM, an NV-RAM, and / or other types of computer-readable media, and corresponding drives.

[0274] The input interface 1210 includes components that allow the device 1200 to receive information, for example, via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, a microphone, a camera, etc.). Additionally or alternatively, in some embodiments, the input interface 1210 includes a sensor that senses information (e.g., a global positioning system (GPS) receiver, an accelerometer, a gyroscope, an actuator, etc.). The output interface 1212 includes components that provide output information from the device 1200 (e.g., a display, a speaker, one or more light emitting diodes (LEDs), etc.).

[0275] In some embodiments, communication interface 1214 includes transceiver-like components (e.g., transceivers, separate receivers and transmitters, etc.) that allow device 1200 to communicate with other devices via a wired connection, a wireless connection, or a combination of wired and wireless connections. In some examples, communication interface 1214 allows device 1200 to receive information from another device and / or provide information to another device. In some examples, communication interface 1214 includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, Interface, cellular network interface, etc.

[0276] In some embodiments, the device 1200 performs one or more processes described herein. The device 1200 performs these processes based on the processor 1204 executing software instructions stored by a computer-readable medium (such as a memory 1206 and / or a storage component 1208). A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes a storage space located within a single physical storage device or a storage space distributed across multiple physical storage devices.

[0277] In some embodiments, the software instructions are read into the memory 1206 and / or storage component 1208 from another computer-readable medium or from another device via the communication interface 1214. When executed, the software instructions stored in the memory 1206 and / or storage component 1208 cause the processor 1204 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry is used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any specific combination of hardware circuitry and software, unless otherwise expressly stated.

[0278] The memory 1206 and / or storage component 1208 include a data storage or at least one data structure (e.g., a database, etc.). The device 1200 can receive information from, store information in, transmit information to, or search for information stored in the memory 1206 or the data storage or at least one data structure in the storage component 1208. In some examples, the information includes network data, input data, output data, or any combination thereof.

[0279] In some embodiments, the device 1200 is configured to execute software instructions stored in the memory 1206 and / or in the memory of another device (e.g., another device that is the same or similar to the device 1200). As used herein, the term "module" refers to at least one instruction stored in the memory 1206 and / or in the memory of another device, which, when executed by the processor 1204 and / or by the processor of another device (e.g., another device that is the same or similar to the device 1200), causes the device 1200 (e.g., at least one component of the device 1200) to perform one or more processes described herein. In some embodiments, the module is implemented in software, firmware, hardware, etc.

[0280] Fig.12 The number and arrangement of components shown are provided as examples. In some embodiments, the apparatus 1200 may include additional components, fewer components, different components, or different components. Fig.12Additionally or alternatively, a set of components (eg, one or more components) of the apparatus 1200 may perform one or more functions described as being performed by another component or another set of components of the apparatus 1200.

[0281] The disclosed and other examples may be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, or a combination of one or more of them. The term "data processing device" includes all devices, means, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0282] The system may include all devices, means and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the system may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0283] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple collaborative files (e.g., files storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a computer or multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0284] The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform the functions described herein. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0285] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer may include a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or may be operatively coupled to one or more mass storage devices for storing data to receive data from one or more mass storage devices for storing data or to transfer data to one or more mass storage devices for storing data or both. However, a computer need not have such a device. Computer-readable media suitable for storing computer program instructions and data may include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks. The processor and memory may be supplemented by or incorporated into a dedicated logic circuit.

[0286] Although many details may be described herein, these details should not be interpreted as limitations on the scope of the claimed invention or the scope that may be claimed, but as descriptions of specific features of specific embodiments. Certain features described herein in the context of a separate embodiment may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described as working in certain combinations and even initially claimed as such, in some cases, one or more features from the claimed combination may be removed from the combination, and the claimed combination may involve a sub-combination or a change in a sub-combination. Similarly, although operations are depicted in the accompanying drawings in a particular order, it should not be understood that such operations need to be performed in the particular order shown or in a continuous order, or that all described operations are performed to achieve the desired result.

[0287] The foregoing only illustrates the principles of the present disclosure, and the system, device and method can be practiced by embodiments other than the embodiments, which are presented for the purpose of illustration and not limitation. Although the embodiments and features herein are specifically described for collecting and analyzing the eye tracking data of the patient to evaluate, screen, monitor or diagnose autism spectrum disorder (ASD), it should be understood that the system, device and method can also be applied to other developmental, cognitive, social or intellectual abilities or disabilities, and other conditions, including but not limited to language disorders, intellectual disabilities, developmental disabilities with or without known genetic disorders, and attention deficit hyperactivity disorder (ADHD), attention deficit disorder (ADD), post-traumatic stress disorder (PTSD), head trauma, concussion, sports injuries and dementia. It should be understood that if these data do not indicate the measurement of the disorder, the measurement of the typical degree of normative development can be provided, providing the indication of the variability of typical development. In addition, all components and other features outlined herein can be combined with each other in any suitable manner, and can be applicable to and applied to systems other than medical diagnosis. For example, the interactive visual stimulation of the present disclosure is used as a therapeutic tool. In addition, the collected data can produce the measurement of certain types of visual stimulation that the patient pays priority attention to. Measurement of such preferences has applications and non-applications in the field of medical diagnosis and treatment, including, for example, advertising or other industries where data related to visual stimulus preferences are of interest.

[0288] Many implementations have been described. However, it should be appreciated that various modifications may be made without departing from the spirit and scope of the techniques and devices described herein. For example, the phase perturbation or variation methods discussed above may be implemented in a diffraction structure to remove high frequency artifacts or intermediate frequency artifacts in an interference pattern. The features shown in each implementation may be used independently or in combination with one another. Additional features and variations may also be included in the implementation. Therefore, other implementations are within the scope of the following claims.

Claims

1. A system for developmental assessment via eye tracking, the system comprising: a patient-side mobile computing device including a screen for presenting visual stimuli to the patient; an eye tracking device mounted with the patient-side mobile computing device and oriented at a fixed position relative to the screen of the patient-side mobile computing device to collect eye tracking data of the patient while the visual stimulus is presented to the patient on the screen of the patient-side mobile computing device; as well as An operator-side mobile computing device is configured to present a user interface that controls activation of the visual stimulus presented to the patient on a screen of the patient-side mobile computing device. 2 . The system of claim 1 , wherein the operator-side mobile computing device and the patient-side mobile computing device are configured to communicate with each other via a wireless connection.

3. The system of claim 2, wherein the operator-side mobile computing device and the patient-side mobile computing device are configured to wirelessly communicate with each other via a network connection server.

4. The system of claim 3, wherein the network-connected server comprises a cloud computing system or a cloud server implemented in a cloud environment.

5. A system as described in claim 3 or 4, wherein the patient-side mobile computing device is configured to send data to the network-connected server, the data comprising eye tracking data of the patient collected in a session when visual stimuli of a predetermined list of visual stimuli are presented to the patient.

6. The system of claim 5, wherein the patient-side mobile computing device is configured to automatically send the data in response to completion of all visual stimuli in the predetermined list of visual stimuli presented in the session.

7. The system of claim 5, wherein the patient-side mobile computing device is configured to send the data in response to receiving a completion indication from the operator-side mobile computing device or the network-connected server.

8. The system of claim 7, wherein the operator-side mobile computing device or the network connection server is configured to generate the completion indication in response to determining that the session is ended or receiving input indicating that the session is complete.

9. A system as claimed in any one of claims 5 to 8, wherein the data comprises information of the predetermined list of visual stimuli and eye tracking data of the patient collected during the session.

10. The system of claim 9, wherein the patient-side mobile computing device is configured to send the data in two files, the two files comprising a first file and a second file, the first file comprising the patient's eye tracking data and associated timestamp information, the second file comprising information of the visual stimulus list.

11. The system of claim 10, wherein the timestamp information associated with the eye tracking data comprises a timestamp when the eye tracking data is generated or collected, and the information of the predetermined visual stimulus list comprises a timestamp when each visual stimulus in the list is presented.

12. The system of any one of claims 3 to 11, wherein the operator-side mobile computing device is configured to access a web portal in the web-connected server.

13. The system of claim 12, wherein the operator-side mobile computing device is configured to communicate with one or more patient-side computing devices through the web portal in the network connection server.

14. The system of claim 13, wherein the user interface of the operator-side mobile computing device includes at least one of information of one or more patients associated with the operator, information of one or more patient-side computing devices associated with the operator, or information of the operator.

15. The system of any one of claims 3 to 14, wherein the patient-side mobile computing device is configured to display connection information including an access code on a screen of the patient-side mobile computing device in response to connecting to the network connection server. 16 . The system of claim 15 , wherein the operator-side mobile computing device is configured to connect with the patient-side computing device by receiving input of the connection information including the access code at the user interface.

17. A system as described in claim 15 or 16, wherein the user interface of the operator-side mobile computing device presents a request for the connection information in response to receiving a selection of the patient-side mobile computing device among the one or more patient-side computing devices presented in the user interface.

18. The system of any one of claims 3 to 17, wherein the operator-side mobile computing device is configured to present a user interface of an operator application running on one of: the operator-side computing device, or The network is connected to the server.

19. The system of claim 18, wherein the operator application is configured to: presenting in the user interface a user interface element for initiating desensitization; and In response to selection of the user interface element, a command is sent to the patient-side computing device to play visual desensitization information.

20. The system of claim 19, wherein the patient-side mobile computing device is configured to: In response to receiving the command, playing the visual desensitization information to the patient on a screen of the patient-side mobile computing device, and When visual desensitization information is displayed on the screen, the eye tracking device is controlled not to collect eye tracking data of the patient.

21. A system as described in claim 19 or 20, wherein the operator application is configured to present the user interface so that the operator can establish a session for the patient by selecting the patient in a patient list or creating a profile for the patient, while the visual desensitization information is displayed on the screen of the patient-side mobile computing device.

22. The system of any one of claims 19 to 21, wherein the operator application is configured to display instructions in the user interface to adjust a position of the eye tracking device relative to the patient or a position of the patient relative to the patient-side mobile computing device based on a sensed position of the eye tracking device relative to at least one eye of the patient.

23. The system of claim 22, wherein the sensed position is determined based on image data of at least one eye of the patient, the image data being obtained using an image acquisition device included in or adjacent to the eye tracking device.

24. The system of claim 23, wherein the sensed location is determined by the patient-side mobile computing device or the operator application.

25. The system of any one of claims 18 to 24, wherein the operator application is configured to send a command to the patient-side mobile computing device for calibration between the patient and the eye tracking device in response to one of the following: selecting a user interface element for calibration in said user interface, or A determination is made to establish a session for the patient.

26. The system of claim 25, wherein the patient-side mobile computing device is configured to: In response to receiving the command, one or more calibration targets are sequentially presented at one or more predetermined locations of the screen of the patient-side mobile computing device while obtaining eye tracking calibration data of the patient using the eye tracking device.

27. The system of claim 26, wherein the patient-side mobile computing device is configured to: for each of the one or more calibration targets, processing the obtained eye tracking calibration data of the patient to determine a position of a corresponding visual gaze of the patient with respect to the calibration target; comparing a corresponding position of visual gaze of the patient to a corresponding predetermined position of a calibration target presented on the screen; as well as A determination is made based on a result of the comparison whether the calibration target is calibrated for the patient.

28. The system of claim 27, wherein the patient-side mobile computing device is configured to determine that the calibration is complete in response to determining that the one or more calibration targets have been calibrated.

29. The system of claim 27 or 28, wherein the patient-side mobile computing device is configured to: In response to determining that the calibration target is not calibrated, the calibration target is recalibrated.

30. The system of any one of claims 26 to 29, wherein the patient-side mobile computing device is configured to play desensitization information between presentation of two adjacent calibration targets.

31. The system of any one of claims 25 to 30, wherein the operator application is configured to: In response to receiving the indication that the calibration is complete, verifying the calibration is initiated.

32. The system of claim 31 , wherein the patient-side mobile computing device is configured to: In response to receiving a request to verify the calibration, presenting at least one additional calibration target on the screen while obtaining additional eye tracking calibration data for the patient using the eye tracking device; and The obtained additional eye tracking calibration data for the patient is processed to determine a position of a corresponding visual gaze of the patient for the at least one additional calibration target.

33. The system of claim 32, wherein the patient-side mobile computing device is configured to: comparing a position of the patient's corresponding visual gaze for the at least one additional calibration target with a corresponding predetermined position at which the at least one additional calibration target is presented on the screen; and A determination is made based on a result of the comparison whether the calibration has been verified.

34. The system of claim 32, wherein the operator application is configured to: simultaneously presenting in the user interface the at least one additional calibration target at the corresponding predetermined position and presenting at least one representation of the patient's corresponding visual gaze at the determined position of the patient's corresponding visual gaze for the at least one additional calibration target; and A first user interface element for verifying the calibration and a second user interface element for recalibrating are presented in the user interface.

35. The system of any one of claims 18 to 34, wherein the operator application is configured to send a command for data collection to the patient-side computing device in response to one of: Select a user interface element to initiate data collection, or Verify that the calibration is complete or verified.

36. The system of claim 35, wherein the patient-side mobile computing device is configured to: In response to receiving the command, a predetermined list of visual stimuli is sequentially presented to the patient on a screen of the patient-side mobile computing device while eye tracking data of the patient is obtained using the eye tracking device.

37. The system of claim 36, wherein the patient-side mobile computing device is configured to: Prior to presenting each of the list of predetermined visual stimuli, a centering target is presented to the patient on a screen of the patient-side mobile computing device.

38. The system of claim 36 or 37, wherein the patient-side mobile computing device is configured to: Calibration of the patient relative to the eye tracking device is performed between presentation of two adjacent visual stimuli in the predetermined list of visual stimuli.

39. The system of claim 38, wherein the eye tracking data collected while performing the calibration is used to at least one of recalibrate the eye tracking data of the patient or to determine calibration accuracy.

40. The system of any one of claims 36 to 39, wherein the operator application is configured to present at least one of the following in the user interface: a progress indicator that is updated during presentation of said list of predetermined visual stimuli, a user interface element for skipping a visual stimulus in a predetermined list of visual stimuli, information about the visual stimulus that has been or is being presented, or Information about the visual stimulus to be presented.

41. A system as described in any one of claims 3 to 40, wherein the network-connected server is configured to provide a diagnosis result for the patient based on the patient's eye tracking data, the diagnosis result including at least one index value associated with a developmental disorder.

42. The system of claim 41, wherein the operator-side mobile computing device is configured to present the diagnostic results in the user interface.

43. The system of any of the preceding claims, wherein the visual stimulus is predetermined based on at least the age of the patient or the condition of the patient.

44. A system as described in any of the preceding claims, wherein each of the visual stimuli comprises at least one of a static visual stimulus, a dynamic visual stimulus, a pre-recorded visual stimulus, a pre-recorded audiovisual stimulus, a live visual stimulus, a live audiovisual stimulus, a two-dimensional stimulus, or a three-dimensional stimulus.

45. A system as described in any of the preceding claims, wherein each of the visual stimuli is standardized to elicit a specific eye movement response with a statistical confidence level greater than 95%.

46. ​​A system as described in any of the preceding claims, wherein each of the visual stimuli is configured to elicit an eye movement response to a discrete spatio-temporal location with a statistical confidence level greater than 95%.

47. The system of any of the preceding claims, wherein the eye tracking device is connected to the patient-side mobile computing device via a wired connection.

48. A system as claimed in any preceding claim, wherein the eye tracking device and the screen are housed together in a holder.

49. The system of any of the preceding claims, wherein the patient-side mobile computing device comprises a screen holder that holds the screen and the eye tracking device in the fixed position relative to the screen.

50. A system as claimed in any preceding claim, wherein the eye tracking arrangement comprises one or more eye tracking units arranged in one or more locations adjacent a periphery of the screen.

51. The system of any of the preceding claims, wherein at least one of the patient-side mobile computing device or the operator-side mobile computing device is a tablet computing device.

52. The system of any of the preceding claims, wherein the operator-side computing device is configured to communicate with the patient-side mobile computing device via two-way communication.

53. An apparatus comprising: a patient-side computing device including a screen for presenting visual stimuli to the patient; as well as an eye tracking device integrated with the patient-side computing device and configured to collect eye tracking data of the patient while the visual stimulus is presented to the patient on the screen of the patient-side computing device, The patient-side computing device comprises a patient-side computing device as described in at least one of claims 1 to 52.

54. An apparatus comprising an operator-side computing device as claimed in at least one of claims 1 to 52.

55. An apparatus comprising: at least one processor; as well as at least one non-transitory memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform operations comprising: establishing a wireless connection with a patient-side computing device integrated with the eye tracking device; as well as A user interface is presented to communicate with the patient-side computing device to obtain eye tracking data of the patient.

56. The apparatus of claim 55, the operations further comprising: A web portal of a network connection server is accessed, wherein the wireless connection is established through the web portal.

57. The apparatus of claim 55 or 56, wherein the operations further comprise: A diagnosis result is presented in the user interface based on the eye tracking data of the patient.

58. A computer-implemented method of developmental assessment via eye tracking, the computer-implemented method comprising: initiating a session for the patient by establishing communications with an operator-side computing device and a patient-side computing device, the patient-side computing device being integrated with an eye tracking device; sequentially presenting a list of predetermined visual stimuli to the patient on a screen of the patient-side computing device while collecting eye tracking data of the patient using the eye tracking device; as well as Session data for the session is sent to a network-connected server, the session data including eye tracking data of the patient collected during the session.

59. The computer-implemented method of claim 58, wherein at least one of the operator-side computing device or the patient-side computing device is a portable device.

60. The computer-implemented method of claim 58 or 59, wherein establishing the communication comprises establishing a wireless connection between the operator-side computing device and the patient-side computing device.

61. The computer-implemented method of claim 60, wherein establishing the wireless connection between the operator-side computing device and the patient-side computing device comprises: Accessing the network portal of the network connection server by the operator-side computing device; as well as In response to receiving a selection of the patient-side computing device in the web portal, wirelessly connecting the operator-side computing device to the patient-side computing device.

62. The computer-implemented method of claim 61 , wherein establishing the wireless connection between the operator-side computing device and the patient-side computing device comprises: displaying, by the patient-side computing device, connection information on a screen of the patient-side computing device; as well as In response to receiving input of connection information by the operator-side computing device, the wireless connection is established between the operator-side computing device and the patient-side computing device.

63. The computer-implemented method of any one of claims 58 to 62, further comprising: After the communication is established, visual desensitization information is displayed to the patient on a screen of the patient-side computing device.

64. The computer-implemented method of claim 63, further comprising: The eye tracking device is controlled not to collect eye tracking data of the patient while displaying the visual desensitization information.

65. The computer-implemented method of claim 63 or 64, further comprising: When the visual desensitization information is displayed, the operator-side computing device accesses a web portal of the web connection server to establish the session for the patient.

66. The computer-implemented method of claim 65, wherein establishing the session comprises one of: Select the patient in the patient list, or A profile is created for the patient on the network connected server.

67. The computer-implemented method of claim 66, further comprising: determining a relative position between the eye tracking device and at least one eye of the patient; as well as Instructions for adjusting the position of the eye tracking device or the position of the patient are displayed on a user interface of the operator-side computing device.

68. The computer-implemented method of claim 67, further comprising: In response to determining that the relative position of at least one eye of the patient is at a predetermined position in a detection area of ​​the eye tracking device, it is determined that the patient is aligned with the eye tracking device.

69. The computer-implemented method of any one of claims 58 to 68, further comprising: The patient is calibrated relative to the eye tracking device by displaying one or more calibration targets to the patient on a screen of the patient-side computing device.

70. The computer-implemented method of claim 69, wherein calibrating the patient relative to the eye tracking device comprises: sequentially presenting each of the one or more calibration targets at a corresponding predetermined position of a screen of the patient-side computing device while obtaining eye tracking calibration data for the patient using the eye tracking device; For each of the one or more calibration targets, processing the obtained eye tracking calibration data of the patient to determine a location of the patient's corresponding visual gaze with respect to a calibration target; comparing the position of the patient's corresponding visual gaze to a corresponding predetermined position of a presented calibration target; as well as A determination is made based on a result of the comparison whether the calibration target is calibrated with respect to the eye tracking device.

71. The computer-implemented method of claim 70, wherein calibrating the patient relative to the eye tracking device further comprises: In response to determining that the deviation between the position of the patient's corresponding visual gaze and the corresponding predetermined position is less than or equal to a predetermined threshold, determining that the calibration target is calibrated and displaying a next calibration target, or In response to determining that the deviation is greater than the predetermined threshold, it is determined that the calibration target has failed to be calibrated, and the calibration target is redisplayed for calibration.

72. The computer-implemented method of any one of claims 69 to 71, further comprising: After calibrating the patient relative to the eye tracking device, the calibration is verified using one or more new calibration targets.

73. The computer-implemented method of claim 70, wherein verifying the calibration comprises: sequentially presenting each of the one or more new calibration targets at a corresponding predetermined position of a screen of the patient-side computing device while obtaining eye tracking calibration data for the patient using the eye tracking device; as well as The obtained eye tracking calibration data for the patient is processed to determine a corresponding position of the patient's visual gaze for each of the one or more new calibration targets.

74. The computer-implemented method of claim 73, wherein verifying the calibration comprises: simultaneously presenting on a user interface of the operator-side computing device a representation of the one or more new calibration targets at one or more corresponding predetermined locations and one or more corresponding visual fixations of the patient at the determined one or more locations; as well as In response to receiving an indication to verify the results of the calibration, determining that the calibration has been verified, or In response to receiving an indication that the calibration results are invalid, recalibration of the patient relative to the eye tracking device is initiated.

75. The computer-implemented method of claim 73, wherein verifying the calibration comprises: determining a number of new calibration targets that each pass calibration based on the corresponding visual gaze position of the patient and the corresponding predetermined position; as well as If the number or the associated percentage is greater than or equal to a predetermined threshold, then the calibration is determined to be verified, or If the number or associated percentage is less than a predetermined threshold, then the calibration is determined to be invalid and recalibration of the patient relative to the eye tracking device is initiated.

76. The computer-implemented method of any one of claims 58 to 75, wherein sequentially presenting the predetermined list of visual stimuli to the patient on a screen of the patient-side computing device comprises: Prior to presenting each of the predetermined list of visual stimuli, a centering target is presented to the patient on a screen of the patient-side computing device.

77. The computer-implemented method of any one of claims 58 to 76, wherein sequentially presenting the predetermined list of visual stimuli to the patient on a screen of the patient-side computing device comprises: performing a calibration of the patient relative to the eye tracking device between presentation of two adjacent visual stimuli in the predetermined list of visual stimuli, Wherein the eye tracking data collected when performing the calibration is used to at least one of calibrate the eye tracking data of the patient or to determine calibration accuracy.

78. The computer-implemented method of any one of claims 58 to 77, further comprising: At least one of the following items is presented on the user interface of the operator-side computing device: a progress indicator that is updated during presentation of said list of predetermined visual stimuli, Information about visual stimuli that have been presented or are being presented, information about the visual stimulus to be presented, or A user interface element for skipping a visual stimulus in a predetermined list of visual stimuli.

79. The computer-implemented method of any one of claims 58 to 78, wherein sending session data for the session to the network connection server comprises: The session data for the session is automatically sent by the patient-side computing device to the network connection server in response to one of the following: determining completion of presenting said predetermined list of visual stimuli on said screen, or A completion indication of the session is received from the operator-side computing device.

80. A computer implemented method as described in claims 58 to 79, wherein the session data includes information related to a predetermined list of visual stimuli presented.

81. A computer-implemented method as described in claim 80, wherein the information associated with the list of predetermined visual stimuli presented includes the names of the predetermined visual stimuli presented and the associated timestamps when the predetermined visual stimuli were presented.

82. The computer-implemented method of claim 80 or 81, wherein the session data comprises the eye tracking data and an associated timestamp when the eye tracking data was generated or collected.

83. A computer-implemented method as described in any one of claims 80 to 82, wherein the session data is stored in a first file and a second file, the first file storing eye tracking data for the patient and the second file storing information related to a predetermined list of visual stimuli presented.

84. A computer-implemented method for developmental assessment using eye tracking data by a network-connected server, the computer-implemented method comprising: receiving session data for a plurality of sessions, the session data for each session including eye tracking data for a corresponding patient in the session; processing the session data of the plurality of sessions in parallel to generate processed session data for the plurality of sessions; as well as For each session of the plurality of sessions, processed session data of the session is analyzed based on corresponding reference data to generate an assessment result of a corresponding patient in the session.

85. The computer-implemented method of claim 84, further comprising: Corresponding reference data for the plurality of sessions is loaded in parallel with processing session data for the plurality of sessions.

86. The computer-implemented method of claim 85, wherein the network-connected server comprises a plurality of processing cores, and The parallel processing of the session data of the plurality of sessions comprises: Session data for the plurality of sessions is processed in parallel using a first plurality of processing cores, and corresponding reference data for the plurality of sessions is loaded using a second, different plurality of processing cores, the first plurality of processing cores being greater in number than the second plurality of processing cores.

87. The computer-implemented method of claim 86, wherein analyzing the processed session data for the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions comprises: A plurality of processing cores including the first plurality of processing cores and the second plurality of processing cores are used.

88. The computer-implemented method of any one of claims 84 to 87, wherein analyzing the processed session data for the plurality of sessions based on the loaded corresponding reference data for the plurality of sessions comprises at least one of: comparing the processed session data of the session with the corresponding reference data, inferring the corresponding patient's assessment result based on the processed session data using the corresponding reference data, or At least one of a statistical model, a machine learning model, or an artificial intelligence (AI) model is used.

89. A computer-implemented method as described in any one of claims 84 to 88, wherein the corresponding reference data includes historical eye tracking data or results for a patient having substantially the same age or condition as the corresponding patient.

90. The computer-implemented method of any one of claims 84 to 89, further comprising: The assessment result is generated based on previous session data of the corresponding patient.

91. The computer-implemented method of any one of claims 84 to 89, further comprising: For each session in the plurality of sessions, a corresponding container is allocated to the session, and in the corresponding container, session data of the session is processed, and the processed session data of the session is analyzed based on the corresponding model data to generate an assessment result of a corresponding patient in the session.

92. The computer-implemented method of claim 91, wherein when the eye tracking data is collected in the session, the eye tracking data is associated with a predetermined list of visual stimuli presented to the patient, and wherein the session data includes information associated with the predetermined list of visual stimuli in the session, and The computer-implemented method further comprises: The eye tracking data for the session is linked to a list of predetermined visual stimuli in the session.

93. The computer-implemented method of claim 92, wherein linking the eye tracking data for the session with a list of predetermined visual stimuli in the session comprises: In a respective container, the eye tracking data is decomposed into a plurality of portions based on information associated with a list of predetermined visual stimuli, each portion of the eye tracking data being associated with one of a respective predetermined visual stimulus or a corresponding calibration.

94. A computer-implemented method as described in claim 93, wherein processing session data for the session includes processing a portion of the eye tracking data associated with the corresponding predetermined visual stimulus based on information of the corresponding predetermined visual stimulus.

95. The computer-implemented method of any one of claims 92 to 94, further comprising: In the respective container, a portion of the eye tracking data associated with the respective predetermined visual stimulus is recalibrated based on at least a portion of the eye tracking data associated with the corresponding calibration.

96. The computer-implemented method of any one of claims 92 to 94, further comprising: In a respective container, a calibration accuracy is determined using at least a portion of the eye tracking data associated with the corresponding calibration and a plurality of predetermined locations at which the plurality of calibration targets are presented in the corresponding calibration.

97. The computer-implemented method of any one of claims 84 to 96, wherein receiving session data for the plurality of sessions comprises: Session data for a plurality of sessions is received from a plurality of computing devices associated with a corresponding entity via a web portal.

98. The computer-implemented method of claim 97, further comprising: In response to receiving session data for a session, a file pointer for the session data for the session is added to a processing queue to be processed.

99. The computer-implemented method of claim 98, further comprising: storing session data for the session in a database using a file pointer for the session; as well as The session data for the session is retrieved from the database using the file pointer for the session.

100. The computer-implemented method of any one of claims 97 to 99, further comprising: For each entity, session data from one or more computing devices associated with the entity is stored in a corresponding repository.

101. The computer-implemented method of claim 100, wherein the corresponding repository for the entity comprises at least one of: information about the entity, information of one or more operators or operator-side computing devices associated with the entity, information of one or more patient-side computing devices associated with the entity, information about one or more sessions taking place in said entity, information about one or more patients associated with the entity, or History information for the corresponding repository.

102. The computer-implemented method of claim 100 or 101, wherein the corresponding repository is included in a NoSQL database.

103. The computer-implemented method of any one of claims 100 to 102, wherein the respective repository is isolated from one or more other repositories and is inaccessible to one or more other entities.

104. The computer-implemented method of any one of claims 84 to 103, further comprising: The resources of the network-connected server are dynamically adjusted based on the number of computing devices accessing the network-connected server.

105. The computer-implemented method of claim 104, further comprising: Replicate data from the first data center to the second data center; as well as In response to determining that the first data center is inaccessible, traffic is automatically directed to the second data center.

106. The computer-implemented method of claim 105, wherein each of the first data center and the second data center comprises at least one of: a web portal accessible by an operator-side computing device, Operator application, or Application layer for data processing and data analysis.

107. The computer-implemented method of any one of claims 104 to 106, further comprising: The same data is stored in a plurality of data centers, wherein the data includes application data of an entity and information associated with the eye tracking data.

108. The computer-implemented method of any one of claims 84 to 107, further comprising: associating the generated assessment results with corresponding patients in the session; as well as Generate an assessment report for the corresponding patient.

109. The computer-implemented method of claim 108, further comprising: The evaluation result or evaluation report is output to be presented on the user interface of the operator-side computing device.

110. The computer-implemented method of claim 108 or 109, wherein the assessment report includes at least one of the following: Corresponding patient information, information about the entity that performed the session corresponding to the patient, Information about the calibration accuracy in the session, Information collected by Session Data, or The corresponding patient's assessment results.

111. A computer-implemented method as described in any one of claims 84 to 110, wherein the assessment result indicates the likelihood that the corresponding patient has a developmental, cognitive, social, or intellectual disability or ability.

112. The computer-implemented method of claim 111, wherein the assessment result indicates a likelihood that the corresponding patient suffers from an Austin Spectrum Disorder (ASD) or non-ASD.

113. A computer-implemented method as described in any one of claims 84 to 112, wherein the assessment results include a corresponding score for each of one or more of a social disability index, a language ability index, non-verbal ability, a social adaptation index, and a social interaction index.

114. The computer-implemented method of any one of claims 84 to 113, wherein the evaluation result comprises at least one of the following: Visualization of eye-tracking data overlaid on corresponding visual stimulus stills at socially relevant moments, Animation visualizing eye-tracking data overlaid on corresponding visual stimulus stills at socially relevant moments, visualization of aggregated reference data for multiple reference patients that match the corresponding patient on one or more patient attributes, or An annotation describing at least one of visual stimulus content or eye gaze pattern.

115. A computer-implemented method as described in any one of claims 84 to 114, wherein the corresponding patient has an age in the range of 5 months to 7 years, including an age in the range of 5 months to 43 months or 48 months, an age in the range of 16 months to 30 months, an age in the range of 18 months to 36 months, an age in the range of 16 months to 48 months, or an age in the range of 16 months to 7 years.

116. A system comprising: at least one processor; as well as One or more memories storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a computer-implemented method as described in at least one of claims 84 to 115.

117. One or more non-transitory computer-readable media storing instructions, When the instructions are executed by at least one processor, the at least one processor is caused to perform a computer-implemented method as described in at least one of claims 84 to 115.

Images