Extended reality system for treating subjects associated with autism spectrum disorder

Abstract

Methods and systems for assessing autism spectrum disorder (ASD) are described herein. Data identifying one or more behaviors associated with ASD may be received. A scenario specifying a plurality of different tasks may be received. An extended reality (XR) device may present an XR environment to a subject. The XR device may present a first task of a plurality of different tasks in the XR environment. Interaction data may be calculated based on subject interactions with one or more objects in the XR environment. At least one second task may be selected from the plurality of different tasks based on the interaction data. The at least one second task may be configured to train a different skill compared to the first task. The XR environment may be modified to present the second task.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 952,804, filed December 23, 2019, entitled "DIAGNOSTIC DEVICE, SYSTEM AND METHOD FOR MONITORING AND ASSESSING SUBJECTS ASSOCIATED WITH AUTISM SPECTRUM DISORDER," the disclosure of which is incorporated herein by reference in its entirety.

[0002] Field The embodiments described herein generally relate to medical procedures and devices for improved subject testing and analysis. More specifically, the embodiments described herein provide devices, systems, and methods for addressing social, communication, and sensory limitations, such as those experienced by individuals experiencing autism spectrum disorder, in a subject by active testing, passive monitoring, and / or treatment of the subject. [Background technology]

[0003] background Autism spectrum disorder (ASD) encompasses a wide range of conditions that can negatively impair an individual's social, communication and / or behavioral abilities. Individuals experiencing ASD may experience difficult communication and interaction with others, may have particularly restricted interests, and / or may exhibit repetitive behaviors. For example, daily living tasks that involve social interaction may present particular difficulties for individuals experiencing ASD. ASD is often accompanied by sensory sensitivities and medical problems, such as gastrointestinal disorders, seizures or sleep disorders, as well as mental health challenges, such as anxiety, depression and attention problems. Thus, individuals experiencing ASD may have difficulties in school, the workplace, and other social situations.

[0004] A variety of treatment methods exist for ASD. In general, early identification of ASD symptoms in children is valuable because early intervention strategies (e.g., treatments that help children experiencing ASD to talk, walk, or generally interact with others) can be beneficial. A common approach, applied behavior analysis (ABA), involves encouraging positive behaviors (e.g., social interaction) and discouraging negative behaviors (e.g., being withdrawn or unable to communicate). Within the category of ABA, many approaches exist, including discrete trial training (e.g., testing and rewarding positive behaviors in discrete tasks), early focused behavioral intervention, pivotal response training (e.g., prompting the subject to learn and monitoring the subject's own behavior), verbal behavioral intervention, occupational therapy (e.g., helping the subject live independently by learning to dress, eat, bathe, and perform other tasks), sensory integration therapy (e.g., helping the subject process unpleasant sights, sounds, and smells), and others. Other approaches include modifying the individual's diet, using medications, and the like.

[0005] The above methods have many drawbacks. Individuals experiencing ASD may withdraw from social interactions, which means that they are less likely to seek treatment on a regular basis. Furthermore, even if an individual experiencing ASD is willing to participate in treatment regularly, such treatment sessions may not be frequent enough to improve the individual's symptoms. It may also be difficult to alleviate the severity of the sensory input of an individual experiencing ASD, which means that it may be difficult to gradually acclimate the individual to more intense sensory input. For example, it may be difficult (and cost-prohibitive) to gradually introduce increasingly complex social interactions in a treatment situation. Furthermore, having a clinician continually monitor a subject's performance throughout the training process may be extremely costly. For example, training for a subject with ASD may be time-consuming, such that the cost of such training may be prohibitive if the clinician must continually monitor and modify aspects of the training. Summary of the Invention [Means for solving the problem]

[0006] overview The following presents a simplified summary of various aspects described herein. This summary is not an extensive overview and is not intended to identify necessary or critical elements or to delineate the claims. The following summary merely presents some concepts in a simplified form as a prelude to the more detailed description provided below.

[0007] To overcome the limitations of the prior art described above, as well as other limitations that will become apparent upon reading and understanding this specification, embodiments described herein are directed to using extended reality (e.g., virtual reality, extended reality, and / or mixed reality) environments to measure and ameliorate one or more social, communication, and / or sensory deficits in individuals who exhibit such symptoms.

[0008] The computing device can be coupled to an extended reality (XR) device (either indirectly via a data connection or directly, and thus also physically a part of), which can provide an XR environment. The computing device can receive data identifying one or more behaviors associated with a symptom of ASD. The computing device can receive a scenario specifying a plurality of different tasks configured to train a skill associated with one or more behaviors associated with a symptom of ASD. The computing device can cause the XR device to present the XR environment to a subject, and can cause the XR device to present at least one first task of the plurality of different tasks specified in the scenario in the XR environment and to the subject. The at least one first task can be configured to train a first skill associated with an improvement of one or more behaviors associated with a symptom of ASD. The at least one first task can be configured to prompt the subject to interact with one or more objects in the XR environment. The computing device can detect a subject interaction with one or more objects in the XR environment. The computing device can calculate a behavior score indicative of one or more behaviors associated with a symptom of ASD based on the one or more subject interactions. The computing device can select at least one second task of the plurality of different tasks from the plurality of different tasks and based on the behavioral score indicative of the one or more behaviors associated with a symptom of the ASD. The second task can be different from the first task and can be configured to train a second skill associated with the improvement of the one or more behaviors associated with a symptom of the ASD.

[0009] Additionally and / or alternatively, existing data can be collected about the subject. The existing data can include information about intake surveys and / or clinical instruments completed by the clinician, caregiver, and / or subject, electronic medical record data, and / or the presence or absence of one or more biomarkers. An extended reality environment can be provided and the therapeutic data can be collected using the environment. Such an extended reality environment can include video and / or audio and can include one or more scenarios, such as one or more of a test scenario, an observation scenario, a learning scenario, a practical scenario, and / or a mitigation scenario. The therapeutic data can include data parameters, such as implicit non-verbal communication parameters, non-verbal communication parameters, verbal communication parameters, sensory intensity parameters, sensory complexity parameters, sensory predictability parameters, social intensity parameters, and the like. The therapeutic data can be compared to the existing data and the extended reality environment can be altered. For example, the intensity and / or difficulty of various sensory challenges within the extended reality environment can be altered.

[0010] These and additional aspects will be understood with the benefit of the disclosure that is more fully described below. The present invention provides, for example, the following items. (Item 1) 1. A method for assessing autism spectrum disorder (ASD), comprising: A computing device coupled to an extended reality (XR) device, the computing device comprising at least one processor and a memory, the computing device comprising: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment; and generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; and A method comprising: (Item 2) 2. The method of claim 1, further comprising receiving biometric data associated with the subject and collected during performance of the first task from one or more biometric tracking devices, wherein generating the interaction data comprises generating the interaction data further based on the biometric data. (Item 3) 3. The method of claim 2, wherein calculating a score associated with the one or more behaviors includes comparing the biometric data to a criteria established by the scenario. (Item 4) 4. The method of any one of claims 1 to 3, wherein calculating a score associated with the one or more behaviors comprises calculating a performance metric indicative of how well the subject performs the one or more behaviors in the XR environment corresponding to the subject interaction. (Item 5) 5. The method of any one of claims 1 to 4, wherein selecting the second task is in response to use of an input indicating that the user is unresponsive to the first task, and modifying the XR environment includes presenting the second task without human interaction. (Item 6) 6. The method of claim 1, wherein the scenario specifies a task sequence including a plurality of different tasks to be performed in a certain order, and selecting the second task is based on determining which task of the plurality of different tasks should be performed after the first task based on the order. (Item 7) 7. The method of any one of claims 1 to 6, further comprising, after modifying the XR environment to present the second task, generating updated interaction data based on further subject interactions in response to the second task. (Item 8) The first skill is: the speech patterns of the subject; the subject's line of sight, a position of the subject relative to a position of an avatar object; decisions made in said XR environment; or Movement of said object 8. The method according to any one of items 1 to 7, wherein the method corresponds to one or more of the following: (Item 9) 1. A method for assessing autism spectrum disorder (ASD), comprising: A computing device coupled to an extended reality (XR) device, the computing device comprising at least one processor and a memory, the computing device comprising: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; Collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the gaze tracking data; generating interaction data indicative of performance of the one or more behaviors associated with symptoms of the ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; and A method comprising: (Item 10) 10. The method of claim 9, further comprising generating updated interaction data after modifying the XR environment to present the second task based on further subject interactions in response to the second task. (Item 11) 11. The method of claim 9 or 10, wherein the gaze data indicates whether the subject looked at a particular region of the one or more second objects. (Item 12) Item 12. The method of item 11, wherein the one or more second objects include an avatar object and the particular region includes an eye of the avatar object. (Item 13) Item 13. The method of any one of items 9 to 12, wherein the gaze data indicates whether the subject has looked away from the one or more second objects over a period of time. (Item 14) 1. A method for assessing autism spectrum disorder (ASD), comprising: A computing device coupled to an extended reality (XR) device, the computing device comprising at least one processor and a memory, the computing device comprising: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions by the subject with the one or more objects in the XR environment; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; and A method comprising: (Item 15) 15. The method of claim 14, further comprising generating updated interaction data based on further subject interactions in response to the second task after modifying the XR environment to present the second task. (Item 16) generating said interaction data, 16. The method of claim 14 or 15, comprising calculating a confidence score associated with the subject's confidence when speaking based on the audio data. (Item 17) generating said interaction data, 17. The method of any one of claims 14 to 16, comprising calculating an intelligibility score associated with the intelligibility of the subject's speech based on the audio data. (Item 18) 1. A method for assessing autism spectrum disorder (ASD), comprising: A computing device coupled to an extended reality (XR) device, the computing device comprising at least one processor and a memory, the computing device comprising: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; causing the XR device to present a second task of the plurality of different tasks in the XR environment and to the subject, the second task configured to emulate a daily living skill; collecting subject data by monitoring performance of the first task and the second task by the subject over a period of time; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; causing the XR device to modify the XR environment to present the second task and the third task based on a template; A method comprising: (Item 19) 20. The method of claim 18, wherein causing the XR device to modify the XR environment includes causing the XR environment to present the second task for performance by the subject in parallel with the third task. (Item 20) 20. The method of claim 18 or 19, wherein causing the XR device to present the second task includes causing the XR device to present the second task in parallel with the first task. (Item 21) 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment; and generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; Device. (Item 22) 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; Collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the gaze tracking data; generating interaction data indicative of performance of the one or more behaviors associated with symptoms of the ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; Device. (Item 23) 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions by the subject with the one or more objects in the XR environment; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; Device. (Item 24) 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; causing the XR device to present a second task of the plurality of different tasks in the XR environment and to the subject, the second task configured to emulate a daily living skill; collecting subject data by monitoring performance of the first task and the second task by the subject over a period of time; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; causing the XR device to modify the XR environment to present the second task and the third task based on a template; Device. (Item 25) One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment; and generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media. (Item 26) One or more non-transitory computer-readable media containing instructions that, when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; Collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the gaze tracking data; generating interaction data indicative of performance, by the subject, of the one or more behaviors associated with symptoms of the ASD based on the one or more interactions and based on the gaze data; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media. (Item 27) One or more non-transitory computer-readable media containing instructions that, when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions by the subject with the one or more objects in the XR environment; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a second task from the distinct tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media. (Item 28) One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of the ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; causing the XR device to present a second task of the plurality of different tasks in the XR environment and to the subject, the second task configured to emulate a daily living skill; collecting subject data by monitoring performance of the first task and the second task by the subject over a period of time; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of the ASD based on the subject interactions; selecting a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; and causing the XR device to modify the XR environment to present the second task and the third task based on a template. One or more non-transitory computer-readable media. [Brief description of the drawings]

[0011] A more complete understanding of the aspects described herein and their advantages can be obtained by reference to the following description in consideration of the accompanying drawings, in which like reference numerals indicate like features, and in which:

[0012] [Figure 1]1 illustrates an example computer system architecture that can be used in accordance with one or more example aspects described herein.

[0013] [Diagram 2] 1 illustrates an exemplary extended reality device.

[0014] [Diagram 3] 1 shows a flowchart including steps that may be performed to monitor, assess, and train symptoms of ASD using an extended reality device.

[0015] [Figure 4] 1 shows a diagram illustrating how different scenarios can be presented as part of an extended reality environment.

[0016] [Diagram 5] 1 shows a flowchart including steps that may be performed to monitor, assess, and train symptoms of ASD using an extended reality device.

[0017] [Figure 6A] 1 illustrates an example of an extended reality environment showing an example where a subject is prompted to order food.

[0018] [Figure 6B] 1 shows example outputs regarding a subject's actions in an XR environment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Detailed Description In the following description of various embodiments, reference is made to the accompanying drawings, which are identified above and which form a part of the description, and which show by way of illustration various embodiments in which aspects described herein may be practiced. It is to be understood that other embodiments may be utilized and structural and functional changes may be made without departing from the scope of what is described herein. Various aspects are capable of other embodiments and of being practiced or carried out in a variety of different ways.

[0020] As a general introduction to the subject matter described in more detail below, embodiments described herein relate to using extended reality to assess and / or treat one or more symptoms of social, communication, and / or sensory deficits in an individual. The method can include identifying a subject experiencing social, communication, and / or sensory limitations. The method can further include determining at least one data parameter as a baseline value for the subject. Such determination may be during a predefined time window and / or intake session. The data parameter can include, for example, one or more parameters provided in or on an intake survey or clinical instrument completed by a clinician or subject, past electronic medical record data, and / or digital biomarker data. The method can further include delivering a set of incremental therapeutic interventions to the subject by use of an extended reality device (as used herein, unless otherwise stated, reference to any one of extended reality, virtual reality, and / or mixed reality devices may also or alternatively include the other). By comparing at least one newly determined value of the data parameter from the subject to the baseline value of the data parameter, a difference can be calculated that can indicate an improvement or worsening of the symptom. Such steps may be repeated until improvements in social and communication function and / or daily living skills are obtained, or other behavioral processes are detected. Therapeutic extended reality interventions may include one or more steps selected from the group including observational training scenarios, instrumental training scenarios, affective evaluation of performance, and a virtual "cool down" room or safe space that allows psychological and physiological arousal to return to baseline. The method may further include defining observational or instrumental training scenarios, staging such scenarios, allowing clinicians and caregivers to interact with and / or monitor the subject in extended reality, incorporating all resulting data into a scoring system, etc.

[0021] As another example, and as described herein, a computing device may be coupled to an extended reality (XR) device. The XR device may be capable of providing one or more of a virtual reality environment, an extended reality environment, or a mixed reality environment. The computing device may receive data identifying one or more behaviors associated with a symptom of ASD. For example, the one or more behaviors may include not looking at a speech participant with eyes during a conversation. The computing device may receive a scenario specifying a plurality of different tasks. The plurality of different tasks may be configured to train a skill associated with one or more behaviors associated with a symptom of ASD. For example, one of the plurality of different tasks may include training a subject to look at a speaking partner with eyes. The computing device may cause the XR device to present an XR environment to the subject. The computing device may also cause the XR device to present, in the XR environment and to the subject, at least a first task of the plurality of different tasks specified in the scenario. The at least one first task may be configured to train a first skill associated with improvement of one or more behaviors associated with a symptom of ASD. For example, as described above, the at least one first task can include training the subject to look at a speaking partner with eyes. The at least one first task can be configured to prompt the subject to interact with one or more objects in the XR environment. Such interactions can include, for example, looking at an avatar object with eyes, standing an appropriate distance away from the avatar object, using a virtual wallet to pay for items at a store, etc. The computing device can detect the subject's interaction with one or more objects in the XR environment. For example, the computing device can monitor the subject's eye movements to generate eye tracking data, and then process the eye tracking data to determine whether the subject is looking at the avatar object.The computing device can calculate a behavioral score indicative of one or more behaviors associated with a symptom of ASD based on the one or more subject interactions. For example, the computing device can provide a score for how well the subject looked into the eyes of a speaking partner. The computing device can select at least one second task of the plurality of different tasks from the plurality of different tasks and based on the behavioral score indicative of the one or more behaviors associated with a symptom of ASD. The second task can be different from the first task and can be configured to train a second skill associated with the improvement of the one or more behaviors associated with a symptom of ASD. For example, the second task can require the subject to voice interact with an avatar object. The computing device can modify the XR environment to present the second task based on the scenario.

[0022] It is to be understood that the phrases and terms used herein are for the purpose of description and should not be considered limiting. Rather, the words and terms used herein should be given their broadest interpretation and meaning. The use of "including" and "comprises" and variations thereof is meant to encompass the items listed thereafter and their equivalents, as well as additional items and their equivalents. The use of the terms "connected," "coupled," "engaged," and similar terms is meant to include both direct and indirect connections, couplings, and engagements.

[0023] Extended reality environments and computing devices FIG. 1 is a block diagram illustrating hardware elements of an exemplary computing device 100. As described in more detail below, such a computing device can be used to provide an extended reality environment. For example, a computing device such as the exemplary computing device 100 can include one or more processors that execute instructions (stored in memory) that cause the computing device to use the extended reality device 111 to provide an extended reality (e.g., virtual reality, extended reality, and / or mixed reality) environment. A computing device such as the exemplary computing device 100 can be part of and / or communicatively connected to the extended reality device 111. The extended reality device 111 can be configured to provide an extended (e.g., one or more of virtual reality, extended reality, and / or mixed reality) environment to a subject. The extended reality device 111 can use the computing device 100 for one or more portions of providing an extended reality environment to a subject, as described in more detail below with respect to FIG. 2. In some examples, a computing device as described herein can omit one or more of the elements illustrated in FIG. 1.

[0024] The computing device 100 may include one or more processors 101 capable of executing instructions of a computer program to perform any of the features described herein. Instructions may be stored in a computer-readable medium or memory to configure the operation of the processor 101. For example, the instructions may be stored in a read-only memory (ROM) 102, a random access memory (RAM) 103, a universal serial bus (USB) drive, a removable medium 104 such as a compact disc (CD) or digital versatile disc ("DVD"), a floppy disk drive, a cloud or network storage device, or another storage medium. The instructions may also be stored on an attached (or internal) hard drive 105. The computing device 100 may include one or more output devices such as a display 106 (e.g., an external television or monitor) and may include one or more output device controllers 107 such as a video processor. There may also be one or more user input devices 108 such as a remote control, a keyboard, a mouse, a touch screen, a microphone, a camera (e.g., to capture input for user gestures). The computing device 100 may also include one or more network interfaces, such as network input / output (I / O) circuitry 109 (e.g., a network card) for communicating with an external network 110. The network input / output circuitry 109 may be a wired interface, a wireless interface, or a combination thereof. Additionally, the device may include a location detection device, such as a Global Positioning System (GPS) microprocessor 111, which may be configured to receive and process global positioning signals and, with possible assistance from an external server and antenna, calculate the geographic location of the device.

[0025] FIG. 1 illustrates an example of a hardware configuration that may be used in some configurations, although in other configurations, one or more of the components may be implemented as software. Additionally, changes may be made to add, remove, combine, divide, etc., components of computing device 100. Additionally, components may be implemented using basic computing devices and components, and the same components (e.g., processor 101, ROM storage device 102, display 106, etc.) may be used to implement any of the other computing devices and components described herein. For example, one or more of the various components described herein may be implemented using a computing device having components such as a processor that executes computer-executable instructions stored on a computer-readable medium, as shown in FIG. 1. Some or all of the entities described herein may be software-based and may coexist on a common physical platform (e.g., a requesting entity may be a separate software process and program from a subordinate entity, both of which may be executed as software on a common computing device).

[0026] FIG. 2 illustrates an example of an extended reality (XR) device 111, which may be the same as or similar to the extended reality device 202. The XR device 202 may be communicatively connected to an on-board computing device 204, which may be the same as or similar to the computing device 100. The XR device 202 may include multiple different elements, such as a display device 203a, an audio device 203b, a motion-sensing device 203c, a camera 203d, a position tracking element 203e, and an input / output device 203f. Such elements may additionally and / or alternatively be referred to as sensors. Other such elements not illustrated may include in-ear electroencephalogram (EEG) and / or heart rate variability (HRV) measuring devices, scalp and / or forehead-based EEG and / or HRV measuring devices, eye tracking devices (e.g., using infrared), and the like. The XR device 202 may further include an assistive computing device 201, which may be the same as or similar to the computing device 100. Not all elements shown in FIG. 2 need to be present for operation of the XR device 202. For example, the XR device 202 may not have a camera 203d. As another example, the XR device 202 may lack any of the assistive computing device 201 such that the on-board computing device 204 directly interfaces with the display device 203a, the audio device 203b, the motion sensitive device 203c, the camera 203d, the position tracking element 203e, and / or the input / output 203f to provide an extended reality environment. As yet another example, the assistive computing device 201 may be powerful enough such that the on-board computing device 204 may be omitted. The assistive computing device may be communicatively coupled to the biometric tracking device 205 and the database 206 (e.g., via a network such as the network 110).

[0027] In some cases, the on-board computing device 204 and / or the assisting computing device 201 may not have the specific processing capabilities or functionality to provide an extended reality environment. The on-board computing device 204 and / or the assisting computing device 201 may include, for example, a relatively underpowered processor that provides basic video and / or audio. In other examples, the on-board computing device 204 and / or the assisting computing device 201 may include, for example, a relatively powerful processor that provides highly realistic video and / or audio.

[0028] The XR device 202 may provide an extended reality (e.g., virtual reality, extended reality, and / or mixed reality) environment to a user, such as a subject. Generally, a virtual reality environment provides an entirely virtual world, while an extended reality and / or mixed reality environment mixes elements of the real and virtual worlds. The XR device 202 may be a device (e.g., an extended reality headset) that is specifically configured to provide an extended reality environment, or may be a combination of devices (e.g., a smartphone inserted into a headset and / or communicatively coupled to a headset) that provides an extended reality environment when operated in a particular manner. The XR device 202 may lack a physical connection to another device (e.g., may be battery powered), and thus may be said to be at least partially untethered. The XR device 202 may be said to be tethered if it is connected to another device (e.g., an assistive computing device 201, a power source, etc.). Examples of XR devices 202 can include the VALVE INDEX virtual reality device developed by Valve Corporation of Bellevue, Washington, the OCULUS QUEST virtual reality device sold by Facebook Technologies, LLC of Menlo Park, Calif., and the HTC VIVE virtual reality device sold by HTC Corporation of New Taipei City, Taiwan. Examples of XR devices 202 can also include smartphones that can be placed in a headset for extended reality purposes, such as the GEAR VR product sold by Samsung Group of Seoul, South Korea.Examples of XR devices 202 may also include, among others, an extended reality headset sold by Magic Leap, Inc. of Plantation, Fla., a HOLOLENS mixed reality headset sold by Microsoft Corporation of Redmond, Wash., and an NREAL LIGHT headset sold by Hangzhou Tairuo Technology Co., Ltd. of Beijing, China. Examples of XR devices 202 may also include audio-based devices such as ECHO FRAMES sold by Amazon, Inc. of Seattle, Wash. All such extended reality devices may have different specifications. For example, some extended reality devices may have a camera, while others may not. These are merely examples, and other AR / VR systems may also or alternatively be used. Additionally, as described in more detail below (e.g., with respect to the steps shown in FIG. 3), either or both of the on-board computing device 204 and / or the assistive computing device 201 may perform the steps described herein. Thus, the disclosure herein may be performed exclusively by the on-board computing device 204 (e.g., such that the XR device 202 is untethered), the assisting computing device 201 (e.g., such that the XR device 202 is tethered to a computing device in a laboratory environment, etc.), and / or a combination thereof (e.g., such that the on-board computing device 204 performs some steps described herein, the assisting computing device 201 performs other steps described herein, and the devices collectively perform all steps described herein).

[0029] The assistive computing device 201 can provide all or a portion of an extended reality environment to the XR device 202, for example, as used by a tethered OCULUS RIFT. For example, the assistive computing device 201 can provide the XR device 202 with a video data stream that shows a virtual world when displayed by the XR device 202 (e.g., via the display device 203a). Such a configuration can be advantageous when the XR device 202 (e.g., the on-board computing device 204 that is part of the XR device 202) is not powerful enough to display a complete extended reality environment. The assistive computing device 201 does not need to be present for the XR device 202 to provide an extended reality, extended reality, and / or mixed reality environment. For example, if the on-board computing device 204 is powerful enough, the assistive computing device 201 may be omitted (or should be considered to be implemented within the on-board computing device 204), for example, an untethered OCULUS QUEST.

[0030] The display device 203a may be any device configured to display all or a portion of an extended reality environment. Such a display device 203a may include, for example, a flat panel display, such as one or more liquid crystal display (LCD) panels. The display device 203a may be the same as or similar to the display 106. The display device 203a may be singular or multiple and may be configured to display different images to different eyes of a user. For example, the display device 203a may include one or more display devices coupled to a lens (e.g., a Fresnel lens) that isolates all or a portion of the display for viewing by different eyes of a user.

[0031] The audio device 203b may be any device capable of receiving and / or outputting audio associated with the extended reality environment. For example, the audio device 203b may include a speaker that directs audio to the user's ears. As another example, the audio device 203b may include one or more microphones that receive voice input from the user. The audio device 203b may be used to provide an audio-based extended reality environment to a user of the XR device 202.

[0032] The motion sensitive device 203c can be any element that receives input associated with the movement of the user of the XR device 202. For example, the motion sensitive device 203c can include one or more accelerometers that can detect when the user of the extended reality device is moving (e.g., leaning, moving forward, moving backward, turning, etc.). A three-dimensional accelerometer and / or gyroscope can be used to detect the complete movement of the XR device 202. The optional outward-facing camera 203d can be used for three-dimensional orientation as well. The motion sensitive device 203c can enable the XR device 202 to present an extended reality environment that changes based on the user's movements.

[0033] Camera 203d can be used to aid in the safety of the user as well as the presentation of the extended reality environment. Camera 203d may be used to monitor the user's surroundings to avoid the user inadvertently contacting real-world elements (e.g., walls). Camera 203d can additionally and / or alternatively monitor the user (e.g., the user's eyes, the user's eye focus, the user's pupil dilation, etc.) to determine which elements of the extended reality environment to render, the user's movements in such environment, etc.

[0034] The position tracking element 203e may be any element configured to help track the position and / or movement of the XR device 202. The position tracking element 203e may be all or part of a system of infrared emitters that, when monitored by a sensor, indicates the position of the XR device 202 (e.g., the position of the XR device 202 in a room). The position tracking element 203e may be configured to enable "inside-out" tracking, where the XR device 202 tracks the position of one or more elements (e.g., the XR device 202 itself, the user's hands, an external controller, etc.), or to enable "outside-in" tracking, where an external device assists in tracking the position of one or more elements. The position tracking element 203e can help determine the user's position in both the real world (e.g., in a room) and the extended reality environment. For example, the user may be simultaneously moving away from a real world object (e.g., a wall) and an extended reality object (e.g., an avatar object).

[0035] The input / output 203f can be configured to receive and transmit data associated with the extended reality environment. For example, the input / output 203f can be configured to communicate data associated with the user's movements to the assistive computing device 201. As another example, the input / output 203f may be configured to receive information associated with other users of the massively multiplayer extended reality environment.

[0036] The on-board computing device 204 can be configured to provide an extended reality environment via the display device 203a, the audio device 203b, the motion-sensitive device 203c, the camera 203d, the position tracking element 203e, and / or the input / output 203f. The on-board computing device 204 may include one or more processors (e.g., a graphics processor), storage (e.g., storing an extended reality program), and the like. In one or more configurations, the on-board computing device 204 may be powerful enough to provide an extended reality environment without the use of the assistive computing device 201, such that the assistive computing device 201 is not needed and may not be connected to the XR device 202. In other configurations, the assistive computing device 201 and the on-board computing device 204 may work in conjunction to provide an extended reality environment. In other configurations, the XR device 202 may not have an on-board computing device 204, such that the assistive computing device 201 directly interfaces with the display device 203a, audio device 203b, motion sensitive device 203c, camera 203d, position tracking element 203e, and / or input / output 203f.

[0037] The above elements of the XR device 202 are merely examples. The XR device 202 may have more or similar elements. For example, the XR device 202 may include an in-ear EEG and / or HRV measuring device, a scalp and / or forehead-based EEG and / or HRV measuring device, an eye tracking device (e.g., using a camera pointed at the user's eye, pupil tracking, infrared, etc.), etc.

[0038] ASD Assessment and Treatment in Extended Reality Environments The discussion now turns to the use of virtual reality, extended reality, and / or mixed reality to monitor, assess, and / or ameliorate symptoms of ASD.

[0039] 3 shows a flow chart including steps that may be performed using an extended reality device, such as the XR device 202, to monitor, assess, and treat symptoms of ASD. The steps shown in FIG. 3 may be performed by a computing device, such as the computing device 100, the on-board computing device 204, and / or the assistive computing device 201, and may be all or part of an algorithm. The steps shown in FIG. 3 may be implemented to monitor, assess, and / or treat skills associated with one or more symptoms of social, communication, and / or sensory deficits in a tested subject.

[0040] In step 301, the on-board computing device 204 and / or the assistive computing device 201 can collect existing data associated with the subject. The existing data can be received via a database, such as the database 206. The existing data can include personal data about the subject. The existing data can include information provided via intake surveys and / or clinical instruments completed by one or more of the clinician, the caregiver, and the subject. The existing data can include electronic medical record data. The existing data can include any other similar form of historical information about the subject and need not originate from a clinical site. For example, the existing data can include information reflecting feedback from a parent or spouse of the subject. The existing data associated with the subject can be used to calculate a baseline for the subject (e.g., by the XR device 202). The baseline can be associated with a period of time, such as a previous treatment session. The on-board computing device 204 and / or the assistive computing device 201 can use the existing data to calculate a current or baseline state of the subject.

[0041] The existing data may include information indicating the presence and / or absence of one or more biomarkers for the presence, absence, and / or severity of ASD. For example, some differences in the subject's gastrointestinal system may indicate the presence of ASD. As another example, certain differences in the subject's immune system, nervous system, and / or toxicological system may indicate the presence of ASD. Digital biomarkers from computerized intake systems may also be used.

[0042] Determining existing data associated with the subject can include receiving existing data from one or more external sources, such as database 206. The on-board computing device 204 and / or the assistive computing device 201 can store the existing data, and determining the existing data can include obtaining the existing data from those computing devices. For example, determining the external data can include obtaining one or more portions of the external data from an external database (e.g., database 206). Determining existing data associated with the subject can additionally and / or alternatively include receiving input from the subject. For example, determining the external data can include obtaining the external data from a caregiver, a clinician, and / or the subject via a user interface.

[0043] In step 302, the XR device 202 can provide an extended reality environment (XR environment). The extended reality environment can be configured to determine one or more data parameters, as described in more detail below. The extended reality environment can be presented as part of one or more therapeutic interventions (e.g., a series of tasks) for the subject. For example, the extended reality environment may be presented as part of a therapeutic intervention and / or a series of therapeutic interventions. As a particular example, the extended reality environment can be configured to monitor, assess, and / or train skills associated with at least one symptom associated with ASD (e.g., social communication and social interaction difficulties, restricted, repetitive behavior patterns, interests or activities, etc.).

[0044] The extended reality environment (e.g., may be provided by the XR device 202 in step 302) may present and / or otherwise include one or more tasks (which may be part of a scenario and may include one or more tasks) to the user. The one or more tasks may include one or more observation tasks, where one or more social, communication and / or sensory skills are demonstrated for the subject. Additionally and / or alternatively, the extended reality environment may present one or more tasks, such as daily living skills tasks (e.g., interacting with a bank teller, waiter / waitress, going through security at an airport), personal hygiene tasks, etc. The one or more tasks may include one or more learning tasks, where the subject is caused to demonstrate learning of one or more social, communication and / or sensory skills by interacting with objects and / or avatars in the extended reality environment. The one or more tasks may include one or more practical tasks in which the subject may learn to schedule, plan, evaluate and / or recall one or more learned social, communication and / or sensory skills in the world outside the extended reality environment. The one or more tasks may include one or more calming tasks that allow the subject to escape from one or more of the observation, learning, and actual tasks while still using the extended reality device. For example, the one or more calming tasks may include meditation, a personal area associated with the subject, and the like. All such tasks may be associated with training materials (e.g., pamphlets, smartphone apps, and the like) that help educate the subject regarding one or more social, communication, and / or sensory skills to be tested in the extended reality environment. The one or more calming tasks may be alternated with other tasks to allow the subject to return to baseline and / or to allow the subject to rest and recover from other tasks, which may overwhelm the subject as a whole. Such tasks are described in more detail in FIG. 4, which is further described below.

[0045] As an example, the one or more tasks can include presenting, via an XR environment presented using the XR device 202 and to the subject, one or more welcoming or unwelcoming groups of avatar objects, where the welcoming or unwelcoming behavior is evidenced by verbal and non-verbal behaviors including gaze, physical proximity, skin tone, pupil dilation, posture, arm position, physical orientation towards or away from the subject, mutual gaze, eye socket width due to orbicularis or frontalis muscle contraction, and rate, pitch, prosody and semantic content of verbal utterances. As another example, the one or more tasks can include presenting, via an XR environment presented using the XR device 202 and to the subject, one or more avatar objects, where the behavior towards each other and towards the subject is characterized by time-delayed verbal and non-verbal mimicry between the avatars with respect to gaze, physical proximity, skin tone, pupil dilation, posture, arm position, physical orientation towards or away from the subject, mutual gaze, eye socket width due to orbicularis or frontalis muscle contraction, and rate, pitch, prosody and semantic content of verbal utterances.

[0046] The extended reality environment can be managed by a computing device external to the extended reality device, such as the assistive computing device 201. For example, the extended reality environment may be presented by an extended reality device, such as the XR device 202, but control of the environment (e.g., determining which scenarios to present, determining which sensory inputs to emulate) may be controlled by a separate computing device, such as an application running on a tablet managed by the clinician. The clinician can control the environment in various situations, such as when monitoring a subject's activity in the extended reality environment.

[0047] The extended reality environment can be augmented with video, text, or other learning materials. For example, providing the extended reality environment can include the XR device 202 first providing the subject with instructions (e.g., in the environment, such as on a tablet, a brochure, or elsewhere) that indicate the goals of the environment. In this manner, the subject can be informed of one or more social, communication, and / or sensory skills to be tested in the extended reality environment.

[0048] In step 303, the on-board computing device 204 and / or the assistive computing device 201 may collect therapy data including one or more data parameters. The extended reality environment presented by the XR device 202 may be configured to collect (e.g., periodically) one or more data parameters associated with the subject in various scenarios (e.g., performing one or more tasks). As a simple example, the extended reality environment may include a scenario emulating a virtual store, where the subject is tasked with using verbal communication skills to purchase an item, and the subject's speech volume is monitored to see if the subject speaks at an appropriate volume. The scenario may, but need not, emulate a real-world situation in which a subject experiencing an ASD may experience difficulties. For example, the scenario may mimic a crowded party and present a situation in which a subject experiencing an ASD may have particular difficulties and the subject's ability to ignore sudden attention deficits may be monitored. The scenario may test one or more such parameters. For example, scenarios can be constructed to test both the subject's ability to speak with others and the subject's ability to ignore attention deficits (eg, unexpected loud noises).

[0049] The data parameters can include any information associated with the subject, such as the extended reality environment, the scenario, and the like. The data parameters can correspond to, for example, biomarker data received from the biotracking device 205, such that the digital parameters can include, for example, the subject's heart rate. The digital parameters need not come from a sensor or device associated with the extended reality device. For example, the biomarker data can be collected using an external device, such as the extended reality device (which may be, for example, a health device or other wearable device or monitor of the subject) and / or the biotracking device 205. As another example, the heart rate data may be collected from the biotracking device 205 (e.g., a smartwatch or smartphone of the subject). As yet another example, such heart rate data may be obtained from an electroencephalogram (EEG) (which may be one of the biotracking devices 205). As yet another example, the digital parameters can include heart rate data superimposed on brain rhythm data acquired using an EEG (which may be part of the biotracking device 205).

[0050] The data parameters can be subjective or objective. For example, some data parameters can be associated with a subjective assessment of the subject's word use, while other data parameters can correspond to an objective measurement of the subject's pupil dilation. The data parameters can include clinician-derived scores of the subject's social, communication, and / or sensory functioning. Additionally and / or alternatively, such data parameters can include clinician-derived scores of the subject's daily living skills. Such clinician-derived scores can be subjective or objective. Such clinician-derived scores can be derived from a device, such as a laptop or tablet, operated by the clinician. Similarly, the data parameters can include caregiver-derived scores of the subject's social, communication, and / or sensory functioning. Additionally and / or alternatively, such data parameters can include caregiver-derived scores of the subject's daily living skills. Such caregiver-derived scores can be subjective or objective. Such caregiver-derived scores can be derived from a device, such as a smartphone, operated by the caregiver.

[0051] During the presentation of a scenario in the extended reality environment, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including the physical proximity of the subject to other individuals (e.g., avatars representing the individuals). Individuals experiencing ASD may tend to keep the distance between themselves and others too wide or too narrow. Thus, the physical proximity of the subject to individuals (e.g., avatars displayed in the scenario, whether such avatars represent actual individuals or not) can be collected as one such data parameter. Such data parameters can be executed on a computing device such as the on-board computing device 204 and / or the assistive computing device 201 using software providing the extended reality environment and collected using sensors such as the motion-sensing device 203c and / or the position tracking element 203e. The physical proximity can indicate whether the subject is too close or too far from the individual (e.g., avatar). In this manner, the collected data parameters indicative of the subject's physical proximity to other individuals can indicate whether the data parameters meet a threshold associated with standing too close and / or a threshold associated with standing too far. Such thresholds can be modified to account for, for example, cultural differences, contextual differences, etc. For example, a crowded party may have a different threshold compared to an airport, and the socially appropriate distance in France may be slightly different from the socially appropriate distance in China. In this way, subjects can be assessed as to whether they are standing an acceptable distance from other subjects during an interaction.

[0052] During the presentation of a scenario in the extended reality environment, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters, including implicit non-verbal communication parameters, such as the subject's pupil dilation and / or body posture. Such non-verbal communication parameters can indicate the subject's emotions, including whether the subject is stressed (and / or undergoing a so-called "fight or flight" response). For example, a subject undergoing stress can exhibit pupil dilation. Monitoring of such data parameters can be performed by an extended reality device, such as the XR device 202, using sensors, such as the motion-sensing device 203c, the camera 203d, and / or the position tracking element 203e. As a particular example, one or more of the cameras 203d may be configured to monitor the subject's pupil size.

[0053] During the presentation of a scenario in an extended reality environment, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including non-verbal communication parameters such as head nodding, whether the subject is staring or looking away from another human and / or avatar object, whether the user is staring or looking away from the face of another human and / or avatar object, whether the user is staring or looking away from the eyes of another human and / or avatar object, etc. Such data parameters can be collected during the subject's participation in one or more tasks of the scenario, such as when prompted to look in a particular direction. A subject experiencing ASD may be less likely to look at someone they are speaking to. In particular, the likelihood of looking at an avatar with their eyes can indicate comfort when performing a particular scenario and / or a particular task, as looking at someone else with their eyes can make the subject feel vulnerable. Monitoring of such data parameters can be performed by an extended reality device such as the XR device 202 using sensors such as the motion sensitive device 203c, the camera 203d, and / or the position tracking element 203e.

[0054] During presentation of a scenario in an extended reality environment by the XR device 202, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including verbal communication parameters such as the subject's speech rate, speech volume, speech fundamental frequency, speech shimmer, speech jitter, and speech frequency trajectory. Such data parameters can be collected during the subject's participation in one or more tasks of the scenario, such as when prompted to speak with an avatar. Such data parameters can indicate symptoms of ASD, as a subject experiencing ASD may modulate their voice undesirably quickly, quietly, and / or in an unusual manner, especially when nervous. Additionally, the scenario can test data parameters including verbal communication parameters that require analysis of the subject's voice. Such analysis can, for example, provide information regarding the subject's directness and empathy. For example, the data parameters may include the subject's speech vectorized word vectors. As another example, the data parameters can reflect a semantic analysis of the subject's voice, including a sentiment analysis of such voice. Monitoring of such data parameters can be performed via an extended reality device, such as XR device 202, while the subject is in a scenario including one or more tasks, using sensors such as audio device 203b. Analysis of the subject's voice may be performed while the scenario is being presented or after. For example, the subject's voice may be recorded in text form (e.g., using a speech recognition algorithm) and then analyzed to evaluate word choice.

[0055] During the presentation of a scenario in an extended reality environment by the XR device 202, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including the presence of other real or simulated objects, the number of such real or simulated objects, the temporal coordination of those real or simulated objects relative to each other and to the subject, the number of real or simulated other objects engaged in joint as opposed to isolated activities, the number of subjects participating in each such joint activity, the relative rewards for social versus non-social activities, and / or social intensity parameters such as the presence of shared versus non-shared social goals between real or simulated others. Such data parameters can be collected during the subject's participation in one or more tasks of the scenario, such as when prompted to walk across a simulated crowded street or walk through a simulated crowded store. Such data parameters can indicate the overall difficulty of the scenario presented to the subject. For example, a crowded party with loud noises may present a particularly challenging scenario for the subject, whereas a quiet room with a single virtual avatar may present a relatively less challenging scenario.

[0056] During presentation of a scenario in an extended reality environment by the XR device 202, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including sensory intensity parameters such as auditory and spatial frequency content and auditory and spatial power at various frequencies. Such data parameters can be collected during the subject's participation in one or more tasks of the scenario, such as when prompted to perform a task in a loud or otherwise uncomfortable environment. Such data parameters can indicate the intensity of sensory information received by the subject (e.g., how loud the noise is around the subject in the scenario), which, as described above, can indicate the overall difficulty of the scenario for an individual experiencing ASD. Monitoring of such data parameters may additionally and / or alternatively be performed by an extended reality device, such as the XR device 202, and using sensors, such as the audio device 203b.

[0057] During presentation of a scenario in the extended reality environment by the XR device 202, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters, including sensory complexity parameters, such as the number of auditory and visual objects in the virtual scenario. Similar to the above, the amount of clutter and / or unwelcome elements in a scenario (e.g., the number of bright lights in a particular scenario) can affect the difficulty of such a scenario for an individual experiencing ASD. Also, the number of people (whether avatars or real people) in a scenario can affect the difficulty of such a scenario for an individual experiencing ASD.

[0058] During the presentation of a scenario in an extended reality environment by the XR device 202, the on-board computing device 204 and / or the assistive computing device 201 can collect data parameters including sensory predictability parameters, such as determining the time series entropy of sensory intensity and complexity. Such data parameters can be collected during the subject's participation in one or more tasks of the scenario, such as when prompted to perform a task in a critical environment where a sudden, unexpected problem occurs. Individuals experiencing ASD may have a more difficult time dealing with unpredictable sensory inputs, such as sudden loud noises. Thus, the data parameters can be indicative of the overall difficulty of the scenario for the subject, such as some of the other data parameters described above. Such data parameters can be collected using software that, when executed on a computing device such as the on-board computing device 204 and / or the assistive computing device 201, provides a scenario in an extended reality environment.

[0059] The on-board computing device 204 and / or the assistive computing device 201 may collect and evaluate all data parameters based on the time and / or conditions of the scenario, including comparing a first data parameter to a second data parameter. For one or more of the data parameters, the on-board computing device 204 and / or the assistive computing device 201 may calculate a time lag. For example, a maximum inter-agent correlation and / or a time lag of a maximum inter-agent correlation may be calculated. The on-board computing device 204 and / or the assistive computing device 201 may normalize one or more of the data parameters based on the conditions of the scenario. For example, if the scenario is also loud, the subject's speech volume may be considered normal, even if it is high. The data parameters may include a weighted sum of values ​​(e.g., one or more of the above data parameters) under appropriate linear or nonlinear transformations. For example, the data parameters may include a weighted sum of the results of caregiver reports, clinician reports, and self-reports under linear or nonlinear transformations. The data parameters may be for a particular time window and / or session. The on-board computing device 204 and / or the assistive computing device 201 may collect data parameters at predetermined intervals, which may be based, for example, on the number of data parameters collected, the intensity of the scenario, the desired fidelity in the data parameters, etc.

[0060] In step 304, the on-board computing device 204 and / or the assistive computing device 201 may evaluate the treatment data (including one or more data parameters). The treatment data may be compared to existing data as described with respect to step 301. Step 304 may include the on-board computing device 204 and / or the assistive computing device 201 determining whether the subject has improved or regressed with respect to symptoms of ASD.

[0061] Determining whether the subject has improved, remained stationary in the achievement of his / her skill, or regressed can include the on-board computing device 204 and / or the assistive computing device 201 comparing the difference between the treatment data and the existing data with a predetermined threshold. The predetermined threshold may be set to avoid small measurement variations from falsely indicating improvement or regression. That said, the treatment data does not necessarily have to show improvement in one or more dimensions to prove improvement by the subject. For example, the scenario presented in the virtual, augmented, and / or composite environment presented in step 302 may be more complex (e.g., including more avatars, including more unexpected sensory information) than previous scenarios, such that the subject may perform worse in the scenario but may nevertheless improve compared to past behavior. Thus, as part of step 304, the on-board computing device 204 and / or the assistive computing device 201 may normalize certain parameters (e.g., implicit non-verbal communication parameters, non-verbal communication parameters, verbal communication parameters, etc.) based on information about the provided scenario (e.g., sensory intensity parameters, sensory complexity parameters, sensory predictability parameters, social intensity parameters, etc.).

[0062] In step 305, based on the evaluation, the on-board computing device 204 and / or the assistive computing device 201 may determine whether to modify the scenario presented in the extended reality environment. In particular, based on the evaluation of the treatment data (e.g., whether the subject is improving or regressing with respect to one or more skills), it may be desirable to modify the extended reality environment to encourage the subject to continue improving one or more skills, for example, in different scenarios. It may be advantageous to increase the difficulty of the scenario, especially if the subject has developed proficiency in an easier scenario. As an example, after determining that the subject has mastered a simple social scenario (e.g., having dinner with friends), it may be beneficial for the on-board computing device 204 and / or the assistive computing device 201 to modify the extended reality environment to present a relatively complex social scenario (e.g., a crowded party with strangers). If the answer in step 305 is "yes," the flowchart proceeds to step 306. If not, the flowchart proceeds to step 307.

[0063] In step 306, based on determining to modify the extended reality environment (e.g., modify one or more scenarios presented in the extended reality environment), the on-board computing device 204 and / or the assistive computing device 201 can modify the environment presented by the XR device based on the treatment data. The on-board computing device 204 and / or the assistive computing device 201 can increase or decrease the intensity of the sensory input of the extended reality environment, for example, by modifying an existing scenario or presenting a new scenario. The on-board computing device 204 and / or the assistive computing device 201 can increase or decrease the complexity of the extended reality environment, for example, by modifying an existing scenario or presenting a new scenario. The context of the extended reality environment (e.g., the location, identity, etc. of an avatar) can be modified by the on-board computing device 204 and / or the assistive computing device 201.

[0064] Modification of the extended reality environment by the on-board computing device 204 and / or the assistive computing device 201 may include changing the intensity or difficulty of various sensory challenges within one or more scenarios, such as changing the visual and / or auditory and / or tactile components of the frequency content (e.g., tactile, spatial and auditory frequency spectra), intensity (e.g., amplitude and / or power at each frequency), complexity (e.g., number of visual objects and / or distinct audio or tactile sources in the environment) and / or predictability (e.g., time series entropy) of these sensory features over time.

[0065] Modification of the extended reality environment by the on-board computing device 204 and / or the assistive computing device 201 may include altering the intensity or difficulty of various social challenges within one or more scenarios, such as the presence of other real or simulated objects, the number of these other real or simulated objects, the temporal coordination of those objects' actions toward each other and toward the object, the number of other real or simulated objects engaged in joint as opposed to isolated activities, the number of objects participating in each such joint activity, the relative rewards for social versus non-social activities, the presence of shared versus non-shared social goals between real or simulated others, the predictability of these social features over time, etc.

[0066] Modification of the extended reality environment by the on-board computing device 204 and / or the assistive computing device 201 can include altering the intensity or difficulty of various communication challenges of one or more scenarios, such as the average and intensity of non-verbal behaviors (e.g., body language, facial expressions, and physical proximity) between real and / or simulated users, the average, intensity and match of verbal behaviors (e.g., acoustic features such as volume, pitch, prosody, and tone, as well as semantic features including semantic term embedding vectors), the number of real or simulated others communicating simultaneously, the existence of shared vs. unshared communication goals between real or simulated others, the predictability of these communication features over time, etc. The use of such semantic term embedding vectors can be based on models such as word2vec and / or alternative embedding algorithms such as GloVe.

[0067] As part of step 306, the on-board computing device 204 and / or the assistive computing device 201 may display the results of the evaluation of step 304. The results may be displayed to the subject or to another individual, such as a clinician. The subject may be rewarded for improvement (e.g., using a points system). Such rewards may, for example, allow the subject to decorate a space in the scenario.

[0068] In step 307, the on-board computing device 204 and / or the assistive computing device 201 may determine whether to terminate the presentation of the extended reality environment. The presentation may end at the end of a particular time period, when the subject completes a particular scenario in the extended reality environment, when the subject generates a particular value in the treatment data, etc. If the answer is yes, the flowchart ends. If not, the flowchart returns to step 302.

[0069] Examples of different scenarios Figure 4 shows a diagram representing how different tasks can be presented as part of the extended reality environment presented in step 302 of Figure 3. Shown are a test scenario 401, an observation scenario 402, a learning scenario 403, an application scenario 404, and a relaxation scenario 405. As shown by the arrows in Figure 4, the different tasks may be interrelated such that, for example, a therapeutic intervention may include moving a subject from the learning scenario 403 to the test scenario 401, from the test scenario 401 to the relaxation scenario 405, etc.

[0070] In the test scenarios 401, the on-board computing device 204 and / or the assistive computing device 201 can determine one or more skills to be trained. For example, the test subject can be presented with a rapid series of scenarios in which skills may be particularly weak and / or may otherwise require attention. As another example, the subject, clinician, and / or caregiver may be presented with a list of skills to be trained. The test scenarios 401 can be referred to as a "skills thermometer" module.

[0071] In an observation scenario 402, the on-board computing device 204 and / or the assistive computing device 201 can provide an opportunity to observe symptom-related behaviors via an XR environment presented by the XR device 202. For example, the subject may be shown examples of skills to be mastered, examples of interactions where skills were not properly utilized, etc. The observation scenario 402 may be referred to as a "See One" module.

[0072] In the learning scenario 403, the on-board computing device 204 and / or the assistive computing device 201 can provide the subject with learning opportunities in an extended reality environment via the XR environment presented by the XR device 202. Such opportunities can include having the subject test social interactions, practicing looking at an avatar with their eyes, practicing speaking, etc. The learning scenario 403 can be referred to as a "Do One" module.

[0073] In the practical scenario 404, the on-board computing device 204 and / or the assistive computing device 201 can provide a subject strategy for transferring skills learned in the extended reality environment to the real world via the XR environment presented by the XR device 202. As part of the practical scenario 404, the subject's ability to transfer such skills to the real world may be evaluated. The practical scenario 404 may be referred to as the "Live One" module.

[0074] The application scenario 404 can include providing the subject with strategies for implementing the learned skill in the extended reality environment. For example, the application scenario 404 can include adding data corresponding to the skill to the subject's calendar to remind the subject to practice the skill at a specific time. As another example, the application scenario 404 can include sending a message (e.g., text message, email) to the subject to remind the subject of the learned skill in the extended reality environment. As yet another example, the application scenario 404 can include checking with the subject before and after a specific task associated with the skill is actually performed, which can both prepare the subject for the task (e.g., by asking "Are you ready?") as well as receive feedback from the subject regarding the skill (e.g., by asking "How was it?").

[0075] In a relaxation scenario 405, the on-board computing device 204 and / or the assistive computing device 201 can provide physiological and / or psychological relaxation to a subject in a virtual environment via an XR environment presented by the XR device 202. Such a scenario can include providing a room to a subject with elements selected and / or modified by the subject so that the subject can enjoy familiarity with the relaxation scenario 405. The relaxation scenario 405 can be referred to as a "My Room" module.

[0076] All of the scenarios shown in Figure 4 can be repeated. The intensity of any scenario can be increased based on the results of any other scenario. For example, the intensity of the learning scenario 403 can be increased based on determining that the results from the practical scenario 404 indicate that the subject is not ready to transfer one or more skills to the real world.

[0077] Figure 5 shows a flow chart for presenting an extended reality environment. The steps shown in Figure 3 can be performed, for example, to monitor, assess and / or treat one or more symptoms of social, communication and / or sensory deficits in a tested subject.

[0078] In step 501, the on-board computing device 204 and / or the assistive computing device 201 can load behavioral data. For example, the on-board computing device 204 and / or the assistive computing device 201 can receive data identifying one or more behaviors associated with symptoms of ASD. The behavioral data can identify and / or indicate behaviors associated with symptoms of ASD. The data may be received by one or more servers or other sources, such as the database 206. For example, the data can be received from a database maintained by a clinician and specifying behaviors associated with symptoms of ASD. The behavioral data can be generated based on analyzing a raw dataset associated with ASD. For example, a dataset of subject data from subjects having ASD can be collected and such data can be processed to determine commonalities between subjects.

[0079] In step 502, the on-board computing device 204 and / or the assistive computing device 201 may load scenario data. Such scenario data may be loaded from a database, such as database 206. The scenario data may be data defining a scenario (including any tasks performed in the scenario), including instructions for the scenario environment (e.g., a location such as a convenience store), one or more objects in the environment (e.g., an avatar object, a chair, a wall, etc.), one or more tasks to be performed in the environment (e.g., finding and purchasing an item at a convenience store), including whether those tasks should be performed in a particular sequence (e.g., the order of tasks for purchasing an item: locating the item, locating the cash register, withdrawing money, paying for the item, exiting the store), and / or data to be measured to assess whether and to what extent the task was completed, and any logic for calculating a behavior score that assesses the level / degree of task completion. For example, the on-board computing device 204 and / or the assistive computing device 201 may receive scenario data specifying a number of different tasks, the number of different tasks configured to train a skill associated with one or more behaviors associated with symptoms of ASD. The scenario data can include a training scenario. The scenario data can indicate parameters for training one or more symptoms of ASD. For example, the scenario data can indicate an appropriate amount of time for an individual without ASD to look into the eyes of another human being when talking to them. The scenario data can include a list of activities (e.g., tasks) to complete. For example, the scenario data can indicate that a subject with ASD should practice clear speech to increase speaking confidence. The scenario data can include data indicating one or more thresholds. For example, the scenario data can include data specifying an acceptable distance (e.g., both a minimum and maximum distance) that a subject should stand from another human being.

[0080] The scenario data may include a task sequence. A task sequence may be any sequence of a number of tasks for execution by a subject. For example, a task sequence for a product purchase activity may include first entering a virtual store, then finding a product, then receiving the product, then taking the product to a counter, then speaking to a store clerk, then paying for the product, and then leaving.

[0081] The scenario data can include all or a portion of the existing data described with respect to step 301 of Figure 3. For example, the scenario data may include information indicative of a history of performance by a particular subject, information indicative of which tasks the subject is skilled at, etc. Such information can be used to help determine subsequent tasks for performance by the subject, as described further below.

[0082] In step 503, the XR device 202 can provide an extended reality environment to the subject. For example, the on-board computing device 204 and / or the assistive computing device 201 may cause the XR device to present the XR environment to the subject. The XR environment can include a scenario (e.g., a scenario defined by scenario data) having one or more tasks. This step can be the same as or similar to step 302 of FIG. 3.

[0083] In step 504, the XR device 202 may present an object in a scenario in the extended reality environment. For example, the on-board computing device 204 and / or the assistive computing device 201 may cause the XR device to present a scenario in the XR environment and to the subject, the scenario including at least one first task of a plurality of different tasks specified in the scenario, the at least one first task configured to train a first skill associated with the improvement of one or more behaviors associated with symptoms of ASD, and the at least one first task configured to prompt the subject to interact with one or more objects in the XR environment. For example, the scenario data may correspond to training the subject in social interaction, and the scenario presented in the XR environment based on the scenario data may include one or more tasks associated with purchasing an item from a store clerk. Presenting the object in the extended reality environment may include displaying one or more visual representations of the object. For example, in the aforementioned scenario, various visual representations of objects such as a store clerk, store shelves, a counter, virtual money, etc. may be described. For example, presenting an object in the extended reality environment may include the on-board computing device 204 and / or the support computing device 201 determining a three-dimensional model corresponding to the object and displaying it in the extended reality environment. Returning to the example above, such a three-dimensional model may correspond to the object described above such that the XR environment is configured to appear as an actual three-dimensional store. Presenting an object does not necessarily mean that the object is always presented to the subject. For example, if the subject looks away from an object in the extended reality environment, the object may be removed from the environment for purposes of processing efficiency. Thus, for example, returning to the example scenario above, moving away from the store clerk may cause the XR environment to stop rendering the store clerk to conserve processing resources.

[0084] Presenting the object can be part of presenting a task, such as the tasks described with respect to Figures 3 and 4, by the XR device 202 in a scenario in an extended reality environment. The task can be one of a number of different tasks specified, for example, by the scenario data. The task can be configured to train a skill associated with improving one or more behaviors associated with symptoms of ASD. The task can prompt the subject to interact with one or more objects. The skill can correspond to one or more of a subject's speech pattern, a subject's gaze, a subject's location, and / or a subject's movement.

[0085] The XR device 202 can cause the extended reality environment to present multiple tasks at once in a scenario. For example, presenting an object can be part of presenting multiple tasks, such as a first task (e.g., one for training a symptom of ASD) and a daily living task (e.g., one that the subject must complete during the first task). In this way, the steps described herein can include multiple tasks that the subject can be tasked with performing in sequence or in parallel. As described in further detail below, this can advantageously allow for the testing of one task (e.g., one associated with training a symptom of ASD) while challenging the subject to perform a series of other simulated tasks (e.g., daily living tasks).

[0086] As an example of an object, the XR device 202 can present an avatar object through the extended reality environment (and, e.g., in a scenario). A subject can be prompted to interact with the avatar object, e.g., by talking to the avatar object, positioning themselves around the avatar object, etc. The performance of such interactions (e.g., success of the conversation, success of the avatar's hand movements) can be monitored by the XR device 202.

[0087] In step 505, the on-board computing device 204 and / or the assistive computing device 201 may receive interaction data. For example, the on-board computing device 204 and / or the assistive computing device 201 may detect a subject interaction with one or more objects in a scenario presented in the XR environment. The interaction data may include any data indicative of a subject's interaction with any part of the extended reality environment. The interaction data may additionally and / or alternatively be referred to as subject data. The interaction data may include, for example, where a subject is standing or moving, where a subject is looking (e.g., line of sight), voice or other sounds made by a subject, etc.

[0088] The interaction data can include data the same as or similar to the treatment data described with respect to step 303 of Figure 3. For example, the interaction data can include various information about the subjects, such as their speech patterns, where their eyes are looking, their heart rate, etc.

[0089] The interaction data can be based on detection of a subject interacting with one or more objects within the extended reality environment (e.g., within a scenario). The subject can, for example, move the object, talk to the object, look at the object, or perform any action or inaction with respect to the object.

[0090] Interaction data need not include information about a subject positively interacting with any object. For example, a subject not interacting with an object can be valuable data in itself. For example, a subject staring at the floor and not interacting with an avatar object can be a valuable data point in itself for purposes of determining whether the subject is accomplishing an assigned task in a scenario (e.g., forming and maintaining eye contact with a conversation partner).

[0091] As briefly described with respect to step 303 of FIG. 3, the on-board computing device 204 and / or the assistive computing device 201 receiving the interaction data may additionally and / or alternatively include collecting eye-tracking data. Such eye-tracking data may be collected using one or more eye-tracking systems of the extended reality device. For example, the eye-tracking data may be collected by the camera 203d and / or the position tracking element 203e. The eye-tracking data may be used to generate gaze data by identifying one or more second objects in the scenario viewed by the subject during performance of the task. For example, the eye-tracking data may be compared to the position of the object in the scenario presented by the extended reality environment to determine whether the subject was looking at the object. Thus, the gaze data may indicate whether the subject looked at a particular area (e.g., eyes, feet) of one or more second objects (e.g., avatar objects). In this manner, the gaze data may be used to determine whether the subject looked at a socially appropriate portion of the object (e.g., eyes of an avatar object) or a socially inappropriate portion of the object (e.g., chest of an avatar object). Similarly, the gaze data can indicate whether the subject has looked away from one or more objects for a period of time. In this manner, the gaze data can indicate that the subject has avoided looking at a conversation partner. Thus, any of this information can be input to a scenario (e.g., the scenario received in step 502) that can be used to determine whether and / or how to modify the scenario (e.g., the XR environment presented by the XR device 202). For example, the on-board computing device 204 and / or the assistive computing device 201 can use the scenario to determine whether the gaze tracking data indicates symptoms of ASD.

[0092] As briefly described in connection with step 303 of FIG. 3, the on-board computing device 204 and / or the assistive computing device 201 receiving the interaction data may additionally and / or alternatively include collecting audio data. The audio data may correspond to an audio interaction by the subject with one or more objects in a scenario presented in the extended reality environment. For example, the audio data may correspond to an utterance by the subject directed at an avatar object. The audio data may be received by a microphone, such as part of the audio device 203b. Based on the audio data, a confidence score may be calculated. The confidence score may be associated with the subject's confidence when speaking. For example, excessive pauses or words that evoke uncertainty may be associated with a relatively low confidence score, while a clear, concise, and sufficiently loud voice may be associated with a relatively high confidence score. Based on the audio data, an intelligibility score may be calculated. The intelligibility score may be associated with the subject's clarity when speaking. For example, a quiet voice may be provided with a relatively low intelligibility score, while a sufficiently loud and clear voice may be provided with a relatively high intelligibility score. Thus, any of this information can be input to a scenario (e.g., the scenario received in step 502) that can be used to determine whether and / or how to modify the scenario (e.g., the XR environment presented by the XR device 202). For example, the on-board computing device 204 and / or the assistive computing device 201 can use the scenario to determine whether the audio data is indicative of symptoms of an ASD.

[0093] In step 506, the on-board computing device 204 and / or the assistive computing device 201 may determine whether additional sensors are present. The additional sensors may include any sensor device that can provide details in addition to the interaction data received in step 505. For example, the additional sensors may include a biometric sensor, such as a biometric tracking device 205. If there are additional sensors, the flowchart proceeds to step 507. If not, the flowchart proceeds to step 508.

[0094] In step 507, if additional sensors are present, the on-board computing device 204 and / or the assisting computing device 201 may receive sensor data. For example, the on-board computing device 204 and / or the assisting computing device 201 may receive biometric data associated with the subject and collected during performance of the at least one first task from one or more biometric tracking devices, and calculating the behavioral score includes calculating the behavioral score further based on the biometric data. The sensor data can be associated with the subject. For example, the sensor data can include an indication of the subject's heart rate. The collected sensor data can be the same as or similar to all or a portion of the treatment data described with respect to step 303 of FIG. 3. Thus, any of this sensor data can be an input of a scenario (e.g., the scenario received in step 502) that can be used to determine whether and / or how to modify a scenario (e.g., an XR environment presented by the XR device 202). For example, the on-board computing device 204 and / or the assisting computing device 201 can use the scenario to determine whether the scenario data indicates symptoms of an ASD.

[0095] The sensor data may include biometric data received from one or more biometric tracking devices, corresponding to the subject, and collected during performance of the task. The biometric data may include, for example, the subject's heart rate, the subject's oxygen level, etc. Such information may indicate, for example, the subject's tension or the subject's engagement.

[0096] In step 508, the on-board computing device 204 and / or the assistive computing device 201 can execute the scenario data to identify modifications to the extended reality environment (e.g., a scenario presented in the extended reality environment). This step can be the same as or similar to step 305 of FIG. 3. The modifications can be based on the interaction data, the sensor data, and / or the scenario data. The scenario data can include a task sequence to be performed by the subject. For example, the task sequence can specify that to train the subject's interaction in a store, the subject can be prompted to first enter the store and then find a product, then take the product to a cash register, and then pay for the product. In other words, based on the task sequence, a next step in the task sequence can be identified, and a modification can be determined based on the next step in the task sequence.

[0097] As part of identifying changes to the extended reality environment, the on-board computing device 204 and / or the assistive computing device 201 can calculate a behavior score indicative of one or more behaviors associated with ASD symptoms (e.g., generated based on data indicative of the behavior). For example, the on-board computing device 204 and / or the assistive computing device 201 can calculate a behavior score indicative of one or more behaviors associated with ASD symptoms (e.g., providing an objective measure of the behavior) based on one or more subject interactions. Calculating the score can include calculating a performance metric corresponding to the subject interactions. For example, the performance metric can correspond to how quickly subjects responded to a question, how close or far they stood from an avatar object, etc. As another example, the computing device can calculate a behavior score including a ranking of how frequently a particular subject potentially exhibits a behavior associated with ASD symptoms, and the performance metric can be associated with those behaviors. If biometric data is received, calculating the behavior score can include the on-board computing device 204 and / or the assistive computing device 201 comparing the biometric data to a standard established by the scenario data. For example, the scenario data may include an indication that a heart rate above 100 beats per minute indicates irritability, and the heart rate from the biometric data may be compared to this 100 beats per minute threshold.

[0098] The on-board computing device 204 and / or the assistive computing device 201 identifying a modification to the extended reality environment (e.g., a scenario presented in the extended reality environment) can include selecting a second task. The task may be selected from a number of different tasks specified by the scenario data. Selecting the task can be based on a behavioral score. The second task can be the same or different from the task initially presented to the subject. For example, the subject can initially be presented with two tasks (e.g., talking to a cashier and opening a wallet to find a credit card), and the modification can include presenting a second task that replaces one of the two tasks (e.g., such that the subject is tasked with further talking to the cashier while paying for an item using a point of sale system). Thus, selecting the second task can include replacing one task of the scenario while also continuing another task of the same scenario.

[0099] The second task may be selected based on the subject's performance of one or more previous tasks. The subject may perform one or more tasks well or poorly, for example, as indicated by the interaction data described above. In general, the mere fact that the subject completes a task need not indicate that the subject performed the task well, and need not indicate that the subject is addressing symptoms associated with ASD. Thus, the second task may be selected based on the quality of the performance of one or more previous tasks. For example, if the subject has performed particularly well in past tasks, the second task may be selected to increase the scenario difficulty. Such scenario difficulty may be the sum of the difficulties of the tasks presented in the scenario. For example, for a scenario involving walking into a party, the task of opening the door may be relatively easy, but the next task (e.g., greeting someone at the party) may be significantly more difficult for an individual experiencing ASD, and thus the scenario difficulty may be relatively high. Conversely, if the subject has performed poorly in past tasks, the second task may be selected to decrease the scenario difficulty. It may also be the case that one or more previous tasks have been completed, but the quality of the subject's performance in those previous tasks was average and / or expected. In such a situation, the second task can be selected based on a logical sequence of tasks that can be defined by the scenario data. For example, if the scenario data indicates a logical sequence of three tasks and the subject performs the first task well (but not particularly well or poorly), the next task in the sequence can be selected accordingly.

[0100] The second task may additionally and / or alternatively be selected based on interaction data, such as behavioral scores. The subject's interaction data (e.g., with respect to one or more symptoms of ASD) can be used to select the second task so that the second task trains, improves, and / or otherwise addresses the symptoms of ASD. For example, if the subject completes the task well and has a positive behavioral score (e.g., indicating that the subject did not behave in a manner that reflects the symptoms of ASD), the second task can be selected to be slightly more difficult to train the subject. In contrast, if the subject completes the task well but has a negative behavioral score (e.g., indicating that the subject behaved in a manner that reflects difficulty in dealing with the symptoms of ASD), the second task can be selected to be slightly easier to allow the subject to take a break.

[0101] The second task may be selected based on feedback from the subject. The subject may provide feedback regarding the scenario and / or one or more tasks of the scenario. For example, the feedback may indicate that the subject is confident, anxious, overwhelmed, etc. Such feedback may be received via the on-board computing device 204 and / or the assistive computing device 201, for example, by presenting a prompt in the XR environment, receiving verbal feedback from the subject, etc. Based on the feedback, the second task may be selected. For example, if the subject is confident, a relatively difficult second task may be selected. As another example, if the subject is anxious and / or overwhelmed, a relatively easy task may be selected.

[0102] In step 509, the on-board computing device 204 and / or the assistive computing device 201 may determine whether to modify the extended reality environment (e.g., by modifying a scenario presented by the extended reality environment). For example, the on-board computing device 204 and / or the assistive computing device 201 may select at least one second task of the plurality of different tasks from the plurality of different tasks and based on a behavior score indicative of one or more behaviors associated with a symptom of ASD, the at least one second task configured to train a second skill associated with the improvement of one or more behaviors associated with a symptom of ASD, the second skill being different from the first skill. Determining whether to modify the extended reality environment may be based on the scenario data. For example, if the scenario data specifies a task sequence, determining whether to modify the extended reality environment may include determining that one task is completed such that a next task (e.g., a next task in a task sequence) is presented to the subject. If it is determined to modify the extended reality environment, the flowchart proceeds to step 510. If not, the flowchart returns to step 508.

[0103] In step 510, the on-board computing device 204 and / or the assistive computing device 201 may modify the extended reality environment presented by the XR device 202. For example, the on-board computing device 204 and / or the assistive computing device 201 may modify the XR environment based on the scenario to present a second task in the scenario. The extended reality environment may be modified based on the modification identified in step 508. This step may be the same as or similar to step 306 of FIG. 3. For example, if the modification includes presenting a second task to the subject, the modification may include modifying the extended reality environment to present the second task in the same or a different scenario. As another example, if the modification includes making a current task easier, the extended reality environment may be modified to reduce the difficulty of one or more portions of the current task.

[0104] In step 511, the XR device may provide a modified extended reality environment to the subject. This step may be the same as or similar to step 503 and / or step 302 of Figure 3. As part of presenting the modified extended reality environment, an updated behavioral score may be determined. Such updated behavioral score may include the same as or similar steps as described with respect to step 508 of Figure 5.

[0105] Steps 508 to 511 may be performed without human involvement. In other words, the steps may be performed automatically and based on logic provided by the scenario data. In this way, the clinician does not need to be involved in the process of modifying the training program for the subject. This can advantageously allow the subject to proceed through multiple tasks (e.g., multiple tasks in parallel, task sequences) without positive monitoring and intervention by another human.

[0106] In step 512, the on-board computing device 204 and / or the assistive computing device 201 may determine whether to terminate the presentation of the extended reality environment. The on-board computing device 204 and / or the assistive computing device 201 may determine to terminate the presentation of the extended reality environment, for example, if the subject completes one or more tasks of one or more scenarios. For example, if the modified extended reality environment prompts the subject to complete two tasks in parallel, the on-board computing device 204 and / or the assistive computing device 201 may determine to terminate the presentation of the extended reality environment based on a determination that both tasks are completed. Additionally and / or alternatively, the on-board computing device 204 and / or the assistive computing device 201 may determine to terminate the presentation of the extended reality environment based on interaction data and / or sensor data. For example, if the subject appears tired, inattentive, or otherwise fatigued, the on-board computing device 204 and / or the assistive computing device 201 may determine to terminate the presentation of the extended reality environment. If it is determined to exit the extended reality environment, the flow chart ends, otherwise the flow chart returns to step 511.

[0107] FIG. 6A illustrates an example of an extended reality environment that can be generated, presented, and / or provided by the XR device 202. In particular, FIG. 6A illustrates an exemplary extended reality environment in which a subject is prompted to order food. The extended reality environment illustrated in FIG. 6A includes a human-like avatar object 601 that asks, in a text bubble 602, what the subject would like to order. The subject is presented with various options 603, which the subject can speak by pressing an "A" button. The XR environment also includes various distractions, such as a poster behind the avatar object and another avatar object sitting in a booth. In the example illustrated in FIG. 6A, the subject may be tasked with speaking one of the options 603, for example, rather than looking away or moving away from the avatar object.

[0108] The extended reality environment shown in FIG. 6A may be all or a portion of the environment presented in step 302 of FIG. 3 and / or the environment provided in step 503 of FIG. 5. For example, the extended reality environment shown in FIG. 6A may be part of an extended reality environment of a food shopping scenario. In such an example, step 503 of FIG. 5 associated with presenting objects in an XR environment may include rendering a wall shown in FIG. 6A, a booth shown in FIG. 6A, an avatar object 601 shown in FIG. 6A, etc. Then, in that example, as part of step 505 (which includes receiving interaction data), the system may receive interaction data indicative of an interaction with any of those objects: for example, the subject may stand near a wall, talk to the avatar object 601, etc.

[0109] FIG. 6B illustrates an example of an output regarding a subject's behavior in an XR environment. In particular, the output illustrated in FIG. 6B (which may be presented and / or provided by a computing device such as the XR device 202, for example) illustrates a series of scores 604 regarding various strategies for improving symptoms of ASD. The scores 604 are scored from 0 to 3 stars, and the total number of stars is displayed at the bottom of the output. On the left side of the output, a list 605 of tasks completed by the subject is displayed. For example, as illustrated in the example of FIG. 6B, the subject completed tasks including meeting parents, toasting toast, and playing an emoji game, but was unable to participate in the slideshow task.

[0110] The output shown in Figure 6B may be part of step 508 of Figure 5 in that it may be associated with running a training scenario and identifying changes to be made to the extended reality environment. For example, the output shown in Figure 6B indicates that the subject performed somewhat poorly with respect to "building communication" and "using life skills." Thus, the second task selected for performance by the subject may include, for example, a task associated with training skills for "building communication" and / or "using life skills."

[0111] Paragraphs (M1) through (M20) below describe examples of methods that can be implemented in accordance with the present disclosure.

[0112] (M1) A method for assessing autism spectrum disorder (ASD), comprising: receiving, in a computing device having at least one processor and memory and coupled to an extended reality (XR) device, data identifying one or more behaviors associated with a symptom of ASD in a subject; receiving a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task being associated with an improvement in at least one of the behaviors associated with the symptom of ASD. presenting a first task, the first task configured to prompt the subject to interact with an object within the XR environment; detecting a subject interaction with an object within the XR environment; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interaction; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with an improvement of another one or more of the behaviors associated with the symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0113] 2. The method of claim 1, further comprising: (M2) receiving biometric data from one or more biometric tracking devices associated with the subject and collected during performance of the first task; and generating the interaction data comprises generating the interaction data further based on the biometric data.

[0114] The method of claim 2 , wherein (M3) calculating a score associated with the one or more behaviors includes comparing the biometric data to a criteria established by the scenario.

[0115] (M4) The method of any one of claims 1 to 3, wherein calculating a score associated with the one or more behaviors includes calculating a performance metric indicative of how well the subject performs the one or more behaviors in the XR environment corresponding to the subject interaction.

[0116] (M5) The method of any one of claims 1 to 4, wherein selecting the second task is responsive to use of an input indicating that the user is unresponsive to the first task, and modifying the XR environment includes presenting the second task without human interaction.

[0117] (M6) The method of any one of claims 1 to 5, wherein the scenario specifies a task sequence including a plurality of different tasks to be performed in a certain order, and selecting the second task is based on determining a task of the plurality of different tasks to be performed after the first task based on the order.

[0118] (M7) After modifying the XR environment to present the second task, generating updated interaction data based on further subject interactions in response to the second task. The method of any one of claims 1 to 6, further comprising:

[0119] (M8) The method of any one of claims 1 to 7, wherein the first skill corresponds to one or more of a target's speech pattern, a target's gaze, a target's position compared to a position of an avatar object, a decision made in an XR environment, or a target's movement.

[0120] (M9) A method for assessing autism spectrum disorder (ASD), comprising: receiving data in a computing device coupled to an extended reality (XR) device, the computing device having at least one processor and memory, the computing device including: receiving data identifying one or more behaviors associated with a symptom of ASD in a subject; receiving a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD, the first task configured to prompt the subject to interact with an object within the XR environment; and presenting the first task in the XR environment. detecting one or more interactions with one or more objects in the XR environment, where the one or more interactions are associated with the subject; collecting eye tracking data by monitoring eye movements of the subject using an eye tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the eye tracking data; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with a symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0121] (M10) after modifying the XR environment to present the second task, generating updated interaction data based on further subject interactions in response to the second task.

[0122] (M11) The method of claim 9 or 10, wherein the gaze data indicates whether the subject looked at a particular region of one or more second objects.

[0123] The method of claim 11, wherein (M12) the one or more second objects include an avatar object, and the particular region includes an eye of the avatar object.

[0124] (M13) The method of any one of claims 9 to 12, wherein the gaze data indicates whether the subject has looked away from one or more second objects over a period of time.

[0125] (M14) A method for assessing autism spectrum disorder (ASD), comprising: receiving data in a computing device coupled to an extended reality (XR) device, the computing device having at least one processor and memory, the computing device receiving the data including: receiving data identifying one or more behaviors associated with a symptom of ASD in a subject; receiving a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task being configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD; and causing the subject to interact with an object in the XR environment. detecting one or more interactions with one or more objects in the XR environment, the one or more interactions being associated with a subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions with the one or more objects in the XR environment by the subject; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with a symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0126] (M15) The method of claim 14, further comprising, after modifying the XR environment to present the second task, generating updated interaction data based on further subject interactions in response to the second task.

[0127] (M16) The method of claim 14 or 15, wherein generating the interaction data includes calculating a confidence score associated with a subject's confidence when speaking based on the audio data.

[0128] (M17) The method of any one of claims 14 to 16, wherein generating the interaction data includes calculating an intelligibility score associated with the intelligibility of the target's speech based on the audio data.

[0129] (M18) A method for assessing autism spectrum disorder (ASD), comprising: receiving data in a computing device coupled to an extended reality (XR) device, the computing device having at least one processor and memory, and receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with the ASD symptom; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task being configured to train a first skill associated with improvement of at least one of the behaviors associated with the ASD symptom. causing the XR device to present a second task of a plurality of different tasks in the XR environment and to the subject, the second task configured to emulate a daily living skill; collecting subject data by monitoring performance of the first task and the second task over a period of time by the subject; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; selecting a third task from the plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with improvement of at least one of the behaviors associated with a symptom of ASD; and modifying the XR device based on the template to present the second task and the third task.

[0130] (M19) The method of claim 18, wherein causing the XR device to modify the XR environment includes causing the XR environment to present a second task for performance by the subject in parallel with a third task.

[0131] (M20) The method of claim 18 or 19, wherein causing the XR device to present a second task includes causing the XR device to present the second task in parallel with the first task.

[0132] Paragraphs (A1) through (A20) below describe examples of apparatus that can be implemented in accordance with the present disclosure.

[0133] (A1) An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, the apparatus comprising: one or more processors; and a memory storing instructions, the instructions, when executed by the one or more processors, causing the apparatus to: receive data identifying one or more behaviors associated with a symptom of ASD in a subject, the instructions being configured to receive a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with the symptom of ASD; receiving the data; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first of the different tasks specified in the scenario, the first task being one of the behaviors associated with the symptom of ASD. 1. An apparatus for detecting a subject's interaction with an object in an XR environment, the object being configured to prompt the subject to interact with an object in the XR environment; detecting a subject interaction with an object in the XR environment; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interaction; selecting a second task from the different tasks and based on the interaction data, the second task being configured to train a second skill associated with another improvement of one or more of the behaviors associated with the symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0134] (A2) The apparatus described in (A1), wherein the instructions, when executed by the one or more processors, cause the apparatus to receive biometric data associated with the subject and collected during performance of a first task from one or more biometric tracking devices, and the instructions, when executed by the one or more processors, cause the apparatus to generate interaction data further based on the biometric data.

[0135] (A3) The apparatus of (A2), wherein the instructions, when executed by the one or more processors, cause the apparatus to calculate a score associated with one or more behaviors based on comparing the biometric data to a standard established by the scenario.

[0136] (A4) The apparatus of any one of (A1) to (A3), wherein the instructions, when executed by the one or more processors, cause the apparatus to calculate a score associated with one or more actions based on calculating a performance metric indicative of how well the subject performs one or more actions within the XR environment corresponding to the subject interaction.

[0137] (A5) An apparatus as described in any one of (A1) to (A4), wherein the instructions, when executed by the one or more processors, cause the apparatus to select a second task in response to using an input indicating that a user is not responding to the first task, and the instructions, when executed by the one or more processors, cause the apparatus to modify the XR environment by presenting the second task without human interaction.

[0138] (A6) An apparatus as described in any one of (A1) to (A5), wherein the scenario specifies a task sequence including a plurality of different tasks to be performed in a certain order, and the instructions, when executed by one or more processors, cause the apparatus to select a second task based on determining a task to be performed after a first task among the plurality of different tasks and based on the order.

[0139] (A7) The apparatus of any one of (A1) to (A6), wherein the instructions, when executed by the one or more processors, cause the apparatus to generate updated interaction data based on further object interactions in response to the second task after modifying the XR environment to present the second task.

[0140] (A8) The device of any one of (A1) to (A7), wherein the first skill corresponds to one or more of a speech pattern of a subject, a gaze of the subject, a position of the subject compared to a position of an avatar object, a judgment made in an XR environment, or a movement of the subject.

[0141] (A9) An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, the apparatus comprising: one or more processors; and a memory storing instructions, the instructions, when executed by the one or more processors, cause the apparatus to: receive data identifying one or more behaviors associated with a symptom of ASD in a subject; receive a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; receive the scenario, causing the XR device to present an XR environment based on the scenario; and cause the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD, the first task configured to prompt the subject to interact with an object within the XR environment. detecting one or more interactions with one or more objects in the XR environment, the one or more interactions being associated with a subject; collecting eye tracking data by monitoring eye movements of the subject using an eye tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the eye tracking data; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with a symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0142] (A10) The apparatus of (A9), wherein the instructions, when executed by the one or more processors, cause the apparatus to generate updated interaction data based on further object interactions in response to the second task after modifying the XR environment to present the second task.

[0143] (A11) The apparatus of (A9) or (A10), wherein the gaze data indicates whether the subject looked at a particular region of one or more second objects.

[0144] (A12) The apparatus described in (A11), wherein the one or more second objects include an avatar object and the specific region includes an eye of the avatar object.

[0145] (A13) The apparatus of any one of (A9) to (A12), wherein the gaze data indicates whether the subject has looked away from one or more second objects over a period of time.

[0146] (A14) An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory that stores instructions, which when executed by the one or more processors, cause the apparatus to: receive data identifying one or more behaviors associated with a symptom of ASD in a subject; receive a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of ASD; receive the scenario; cause the XR device to present an XR environment based on the scenario; and cause the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task being configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD; and cause the subject to interact with an object in the XR environment. detecting one or more interactions with one or more objects in the XR environment, the one or more interactions being associated with a subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions with the one or more objects in the XR environment by the subject; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with the symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0147] (A15) The apparatus of (A14), wherein the instructions, when executed by the one or more processors, cause the apparatus to generate updated interaction data based on further object interactions in response to the second task after modifying the XR environment to present the second task.

[0148] (A16) The apparatus of (A14) or (A15), wherein the instructions, when executed by one or more processors, cause the apparatus to generate interaction data by calculating, based on the voice data, a confidence score associated with the subject's confidence when speaking.

[0149] (A17) An apparatus described in any one of (A14) to (A16), wherein the instructions, when executed by one or more processors, cause the apparatus to generate interaction data by calculating an intelligibility score associated with the intelligibility of a target's speech based on the audio data.

[0150] (A18) An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions, which when executed by the one or more processors, cause the apparatus to: receive data identifying one or more behaviors associated with a symptom of ASD in a subject; receive a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with the symptom of ASD; receive the scenario; cause the XR device to present an XR environment based on the scenario; and cause the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD. present a first task; cause the XR device to present a second task of a plurality of different tasks in an XR environment and to a subject, the second task configured to emulate a daily living skill; collect subject data by monitoring performance of the first and second tasks by the subject over a period of time; generate interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; select a third task from the plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with improvement of at least one of the behaviors associated with a symptom of ASD; and modify the XR device to present the second and third tasks based on the template.

[0151] (A19) The apparatus of (A18), wherein the instructions, when executed by one or more processors, cause the apparatus to modify the XR environment by causing the XR environment to present a second task for performance by the subject in parallel with a third task.

[0152] (A20) The apparatus of (A18) or (A19), wherein the instructions, when executed by one or more processors, cause the apparatus to cause the XR device to present a second task in parallel with the first task.

[0153] The following paragraphs (CRM1) through (CRM20) describe examples of computer-readable media that may be implemented in accordance with the present disclosure.

[0154] (CRM1) One or more non-transitory computer-readable media including instructions, which when executed by at least one processor of a computer-readable medium coupled to an extended reality (XR) device for assessing autism spectrum disorder (CRMSD), can include: receiving, on the computer-readable medium, data identifying one or more behaviors associated with a symptom of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of ASD; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first of the different tasks specified in the scenario, the first task being one of the behaviors associated with a symptom of ASD. one or more non-transitory computer-readable media that cause a subject to: present a first task configured to train a first skill associated with at least one improvement, the first task configured to prompt the subject to interact with an object within the XR environment; detect a subject interaction with an object within the XR environment; generate interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interaction; select a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with another one or more of the behaviors associated with the symptom of ASD, the second skill being different from the first skill; and modify the XR environment to present the second task based on the scenario.

[0155] (CRM2) A computer-readable medium as described in (CRM1), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to receive biometric data from one or more biometric tracking devices associated with the subject and collected during performance of a first task, and the instructions, when executed by the one or more processors, cause the computer-readable medium to generate interaction data further based on the biometric data.

[0156] (CRM3) A computer-readable medium as described in (CRM2), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to calculate a score associated with one or more behaviors based on comparing the biometric data to a standard established by the scenario.

[0157] (CRM4) The computer-readable medium of any one of (CRM1) to (CRM3), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to calculate a score associated with one or more actions based on calculating a performance metric indicative of how well the subject performs one or more actions within the XR environment corresponding to the subject interaction.

[0158] (CRM5) A computer-readable medium according to any one of (CRM1) to (CRM4), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to: select a second task in response to using an input indicating that a user is not responding to the first task; and, when executed by the one or more processors, cause the computer-readable medium to modify the XR environment by presenting the second task without human interaction.

[0159] (CRM6) A computer-readable medium according to any one of (CRM1) to (CRM5), wherein the scenario specifies a task sequence including a plurality of different tasks to be performed in a certain order, and the instructions, when executed by one or more processors, cause the computer-readable medium to select a second task based on determining a task to be performed after a first task among the plurality of different tasks and based on an order.

[0160] (CRM7) The computer-readable medium of any one of (CRM1) to (CRM6), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to generate updated interaction data based on further subject interactions in response to the second task after modifying the XR environment to present the second task.

[0161] (CRM8) A computer-readable medium according to any one of (CRM1) to (CRM7), wherein the first skill corresponds to one or more of a subject's speech pattern, a subject's gaze, a subject's position compared to a position of an avatar object, a judgment made in an XR environment, or a subject's movement.

[0162] (CRM9) One or more non-transitory computer-readable media including instructions, which when executed by at least one processor of a computer-readable medium coupled to an extended reality (XR) device for assessing autism spectrum disorder (CRMSD), can include: receiving, on the computer-readable medium, data identifying one or more behaviors associated with a symptom of ASD for a subject; receiving a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; receiving the scenario; causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD; and the first task configured to prompt the subject to interact with an object within the XR environment. detecting one or more interactions with one or more objects in the XR environment, the one or more interactions being associated with the subject; collecting eye tracking data by monitoring eye movements of the subject using an eye tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the eye tracking data; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with a symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.one or more non-transitory computer-readable media;

[0163] (CRM10) The computer-readable medium of (CRM9), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to generate updated interaction data based on further subject interactions in response to the second task after modifying the XR environment to present the second task.

[0164] (CRM11) The computer-readable medium of (CRM9) or (CRM10), wherein the gaze data indicates whether the subject looked at a particular region of one or more second objects.

[0165] (CRM12) The computer-readable medium of (CRM11), wherein the one or more second objects include an avatar object and the particular region includes an eye of the avatar object.

[0166] (CRM13) A computer-readable medium according to any one of (CRM9) to (CRM12), wherein the gaze data indicates whether the subject has looked away from one or more second objects over a period of time.

[0167] (CRM14) One or more non-transitory computer-readable media including instructions, which when executed by at least one processor of a computer-readable medium coupled to an extended reality (XR) device for assessing autism spectrum disorder (CRMSD), can include: receiving, on the computer-readable medium, data identifying one or more behaviors associated with a symptom of ASD for a subject; receiving a scenario specifying a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; receiving the scenario, causing the XR device to present an XR environment based on the scenario; and causing the XR device to present, in the XR environment and to the subject, a first task of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD, the first task configured to cause the subject to interact with an object within the XR environment. detecting one or more interactions with one or more objects in the XR environment, the one or more interactions being associated with a subject; receiving, via one or more microphones of the XR device, audio data corresponding to audio interactions with the one or more objects in the XR environment by the subject; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; selecting a second task from the different tasks and based on the interaction data, the second task configured to train a second skill associated with improvement of another one or more of the behaviors associated with the symptom of ASD, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario.

[0168] (CRM15) The computer-readable medium of (CRM14), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to generate updated interaction data based on further subject interactions in response to the second task after modifying the XR environment to present the second task.

[0169] (CRM16) A computer-readable medium as described in (CRM14) or (CRM15), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to generate interaction data by calculating a confidence score associated with the subject's confidence when speaking based on the voice data.

[0170] (CRM17) A computer-readable medium described in any one of (CRM14) to (CRM16), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to generate interaction data by calculating an intelligibility score associated with the intelligibility of a target's speech based on the audio data.

[0171] (CRM18) One or more non-transitory computer-readable media including instructions, which when executed by at least one processor of a computer-readable medium coupled to an extended reality (XR) device for assessing autism spectrum disorder (CRMSD), cause the computer-readable medium to: receive data identifying one or more behaviors associated with a symptom of ASD for a subject; receive a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with the symptom of ASD; receive the scenario, causing the XR device to present an XR environment based on the scenario; and cause the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with the symptom of ASD. one or more non-transitory computer-readable media that cause: presenting a first task; causing an XR device to present a second task of a plurality of different tasks in an XR environment and to a subject, the second task configured to emulate a daily living skill; collecting subject data by monitoring performance of the first task and the second task over a period of time by the subject; generating interaction data indicative of performance of one or more behaviors associated with a symptom of ASD by the subject based on the subject interactions; selecting a third task from the plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with improvement of at least one of the behaviors associated with a symptom of ASD; and modifying the XR device to present the second task and the third task based on the template.

[0172] (CRM19) The computer-readable medium of (CRM18), wherein the instructions, when executed by one or more processors, cause the computer-readable medium to modify the XR environment by causing the XR environment to present a second task for performance by the subject in parallel with a third task.

[0173] (CRM20) A computer readable medium as described in (CRM18) or (CRM19), wherein the instructions, when executed by one or more processors, cause the XR device to present a second task in parallel with the first task, thereby causing the computer readable medium to cause the XR device to present a second task. Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are described as example implementations of the following claims.

Claims

1. 1. A method of operating a computing device for assessing an autism spectrum disorder (ASD), the computing device comprising at least one processor and memory and coupled to an extended reality (XR) device, the method comprising: receiving, by the at least one processor, data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving, by the at least one processor, a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of the ASD; and causing the at least one processor to present an XR environment on the XR device based on the scenario; and The at least one processor causes the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, and the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment, the at least one processor; generating, based on the subject interactions, interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD; and wherein the at least one processor selects a second one of the different tasks from the different tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptoms, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; and A method comprising:

2. The method of claim 1, further comprising: at least one processor receiving biometric data from one or more biometric tracking devices associated with the subject and collected during performance of the first task; and generating the interaction data includes generating the interaction data further based on the biometric data.

3. The method of claim 2, further comprising: at least one processor calculating a score associated with the one or more behaviors, wherein calculating the score comprises comparing the biometric data to a criteria established by the scenario.

4. The method of any one of claims 1 to 2, further comprising: the at least one processor calculating a score associated with the one or more behaviors, wherein calculating the score comprises calculating a performance metric indicative of how well the subject performs the one or more behaviors in the XR environment corresponding to the subject interaction.

5. 5. The method of claim 1, wherein selecting the second task is responsive to use of an input indicating a user is unresponsive to the first task, and modifying the XR environment includes presenting the second task without human interaction.

6. 6. The method of claim 1, wherein the scenario specifies a task sequence including a plurality of different tasks to be performed in a certain order, and selecting the second task is based on determining which task of the plurality of different tasks should be performed after the first task based on the order.

7. 7. The method of claim 1, further comprising: after modifying the XR environment to present the second task, the at least one processor generating updated interaction data based on further object interactions in response to the second task.

8. The first skill is the speech patterns of the subject; the subject's line of sight, a position of the subject relative to a position of an avatar object; a judgment made in the XR environment; or Movement of said object The method according to claim 1 , wherein the method corresponds to one or more of the following:

9. 1. A method of operating a computing device for assessing an autism spectrum disorder (ASD), the computing device comprising at least one processor and memory and coupled to an extended reality (XR) device, the method comprising: receiving, by the at least one processor, data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving, by the at least one processor, a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of the ASD; and causing the at least one processor to present an XR environment on the XR device based on the scenario; and The at least one processor causes the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, and the first task configured to prompt the subject to interact with an object within the XR environment; the at least one processor detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and said at least one processor collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of said XR device; generating gaze data by identifying, based on the gaze tracking data, one or more second objects in the XR environment viewed by the subject while performing the task; generating, by the at least one processor, interaction data indicative of a performance by the subject of the one or more behaviors associated with symptoms of ASD based on the one or more interactions and based on the gaze data; and wherein the at least one processor selects a second one of the different tasks from the different tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptoms, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; and A method comprising:

10. 10. The method of claim 9, further comprising, after modifying the XR environment to present the second task, the at least one processor generating updated interaction data based on further object interactions in response to the second task.

11. The method of claim 9 or 10, wherein the gaze data indicates whether the subject looked at a particular region of the one or more second objects.

12. The method of claim 11 , wherein the one or more second objects include an avatar object, and the particular region includes an eye of the avatar object.

13. The method of claim 9 , wherein the gaze data indicates whether the subject has looked away from the one or more second objects over a period of time.

14. 1. A method of operating a computing device for assessing an autism spectrum disorder (ASD), the computing device comprising at least one processor and memory and coupled to an extended reality (XR) device, the method comprising: receiving, by the at least one processor, data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving, by the at least one processor, a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of the ASD; and causing the at least one processor to present an XR environment on the XR device based on the scenario; and The at least one processor causes the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, and the first task configured to prompt the subject to interact with an object within the XR environment; the at least one processor detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and receiving, by the at least one processor, via one or more microphones of the XR device, audio data corresponding to audio interactions by the subject with the one or more objects in the XR environment; generating, based on the one or more interactions, interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD; and and wherein the at least one processor selects a second one of the different tasks from the different tasks and based on the interaction data, the second task being configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptoms, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; and A method comprising:

15. 15. The method of claim 14, further comprising, after modifying the XR environment to present the second task, the at least one processor generating updated interaction data based on further object interactions in response to the second task.

16. generating said interaction data, 16. The method of claim 14 or 15, comprising calculating a confidence score based on the audio data, the confidence score being associated with the subject's confidence when speaking.

17. generating said interaction data, 17. The method of any one of claims 14 to 16, comprising calculating an intelligibility score associated with the intelligibility of the subject's speech based on the audio data.

18. 1. A method of operating a computing device for assessing an autism spectrum disorder (ASD), the computing device comprising at least one processor and memory and coupled to an extended reality (XR) device, the method comprising: receiving, by the at least one processor, data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving, by the at least one processor, a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks being configured to train a skill associated with a symptom of the ASD; and causing the at least one processor to present an XR environment on the XR device based on the scenario; and The at least one processor causes the XR device to present a first task of the plurality of different tasks in the XR environment and to the subject, the first task being configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; and the at least one processor causes the XR device to present, in the XR environment and to the subject, a second task of the plurality of different tasks, the second task being configured to emulate a daily living skill; and said at least one processor collecting subject data by monitoring performance of said first task and said second task by said subject over a period of time; generating, based on the subject data, interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD; selecting, by the at least one processor, a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; causing the at least one processor to modify the XR environment to present the second task and the third task based on a template; A method comprising:

19. 20. The method of claim 18, wherein causing the XR device to modify the XR environment includes causing the XR environment to present the second task for performance by the subject in parallel with the third task.

20. 20. The method of claim 18 or 19, wherein causing the XR device to present the second task comprises causing the XR device to present the second task in parallel with the first task.

21. 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment; and generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the subject interactions; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; Device.

22. 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the gaze tracking data; generating interaction data indicative of performance of the one or more behaviors associated with symptoms of ASD by the subject based on the one or more interactions and based on the gaze data; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; Device.

23. 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and receiving, via one or more microphones of the XR device, audio data corresponding to a voice interaction by the subject with the one or more objects in the XR environment; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the one or more interactions; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; Device.

24. 1. An apparatus for assessing autism spectrum disorder (ASD) coupled to an extended reality (XR) device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first task of the plurality of different tasks, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; causing the XR device to present, in the XR environment and to the subject, a second task of the plurality of different tasks, the second task configured to emulate a daily living skill; and collecting subject data by monitoring performance of the first task and the second task by the subject over a period of time; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the subject data; selecting a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; causing the XR device to modify the XR environment to present the second task and the third task based on a template. Device.

25. One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting a subject interaction with the object in the XR environment; and generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the subject interactions; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media.

26. One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and collecting eye-tracking data by monitoring eye movements of the subject using an eye-tracking system of the XR device; generating gaze data by identifying one or more second objects in the XR environment viewed by the subject while performing the task based on the gaze tracking data; generating interaction data indicative of performance, by the subject, of the one or more behaviors associated with symptoms of ASD based on the one or more interactions and based on the gaze data; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media.

27. One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first one of the different tasks specified in the scenario, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD, the first task configured to prompt the subject to interact with an object within the XR environment; Detecting one or more interactions with one or more of the objects in the XR environment, the one or more interactions being associated with the subject; and receiving, via one or more microphones of the XR device, audio data corresponding to a voice interaction by the subject with the one or more objects in the XR environment; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the one or more interactions; selecting a second task from the distinct tasks based on the interaction data, the second task configured to train a second skill associated with an improvement in another one or more of the behaviors associated with the ASD symptom, the second skill being different from the first skill; and modifying the XR environment to present the second task based on the scenario; One or more non-transitory computer-readable media.

28. One or more non-transitory computer-readable media containing instructions, which when executed by at least one processor of an apparatus for assessing autism spectrum disorder (ASD), coupled to an extended reality (XR) device, cause the apparatus to: Receiving data identifying one or more behaviors associated with symptoms of ASD for a subject; receiving a scenario that specifies a plurality of different tasks based on the one or more behaviors, the plurality of different tasks configured to train a skill associated with a symptom of ASD; and causing the XR device to present an XR environment based on the scenario; causing the XR device to present, in the XR environment and to the subject, a first task of the plurality of different tasks, the first task configured to train a first skill associated with improvement of at least one of the behaviors associated with a symptom of the ASD; causing the XR device to present, in the XR environment and to the subject, a second task of the plurality of different tasks, the second task configured to emulate a daily living skill; and collecting subject data by monitoring performance of the first task and the second task by the subject over a period of time; generating interaction data indicative of performance by the subject of the one or more behaviors associated with symptoms of ASD based on the subject data; selecting a third task from a plurality of different tasks and based on the interaction data, the third task configured to train a second skill associated with an improvement in at least one of the behaviors associated with a symptom of the ASD; and causing the XR device to modify the XR environment to present the second task and the third task based on a template. One or more non-transitory computer-readable media.

Images