Extended reality-based brain disorder assessment systems and methods

Abstract

An extended reality (XR)-based platform integrated with real-time eye-tracking technology that assesses brain functions by leveraging artificial intelligence (AI) models is provided. In a first aspect, a method includes: receiving sensor data from an extended reality (XR) apparatus, and generating, using an artificial intelligence (AI) model, a functional map of a brain of a subject based on the sensor data. The sensor data includes eye-tracking data and is collected by the XR apparatus while the subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus. The one or more tasks stimulate a plurality of neurotransmitters of the subject. The functional map indicates neurological function of a plurality of regions of the brain. Other aspects and features are also claimed and described.

Description

Attorney Docket No. CRNU.P0037WOTITLEEXTENDED REALITY-BASED BRAIN DISORDER ASSESSMENT SYSTEMS AND METHODSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to and the benefit of U.S. Provisional Application 63 / 708,609, filed October 17, 2024; U.S. Provisional Application 63 / 767,899, filed March 6, 2025; and U.S. Provisional Application 63 / 768,762, filed March 7, 2025. The entirety of each is herein incorporated by reference.TECHNICAL FIELD

[0002] The present application relates generally to assessing brain health and function. More specifically, the present application provides techniques using extended reality for virtual brain imaging and detecting brain function (e.g., brain disorders).BACKGROUND

[0003] Understanding the functional abnormity of each brain region is important across all neurological and psychiatric disorders. These conditions are fundamentally defined by disruptions in how brain regions function and communicate, leading to the cognitive, behavioral, or motor symptoms observed clinically. For example, Alzheimer’s disease is characterized by the accumulation of amyloid-beta plaques and tau protein neurofibrillary tangles in the brain. Tau pathology, in particular, follows a spatiotemporal progression classified by Braak stages (I- VI) corresponding to the spreading of tau tangles from the brainstem and limbic regions to the neocortex. Early in this progression, the locus coeruleus (LC) - a small brainstem nucleus (~2 mm in diameter) that is the brain’s primary source of norepinephrine - is one of the first regions to accumulate hyperphosphorylated tau. Degeneration of the LC due to tau deposition leads to a loss of norepinephrine signaling, disrupting multiple cognitive and arousal processes (including attention and memory) that depend on this neuromodulator.

[0004] The current standard-of-care for Alzheimer’s disease diagnosis includes brain positron emission tomography (PET) imaging and lumbar puncture for cerebrospinal fluid (CSF) analysis. PET imaging, however, involves radiation exposure and a lumbar puncture is invasive. In addition, these tools are expensive and not scalable, meaning patients are generallyAttorney Docket No. CRNU.P0037WO diagnosed after symptoms appear. Another traditional diagnosis method includes using blood biomarkers, which are more scalable. The blood concentrations of Alzheimer’ s-associated proteins are up to 10-fold lower than in CSF, however, which result in low positive predictive values and necessitates use of other clinical markers for screening and confirmation of diagnosis.SUMMARY

[0005] The following summarizes some aspects of the present disclosure to provide a basic understanding of the discussed technology. This summary is not an extensive overview of all contemplated features of the disclosure and is intended neither to identify key or critical elements of all aspects of the disclosure nor to delineate the scope of any or all aspects of the disclosure. Its sole purpose is to present some concepts of one or more aspects of the disclosure in summary form as a prelude to the more detailed description that is presented later.

[0006] The present disclosure provides systems and methods for an extended reality (XR)- based platform integrated with real-time eye-tracking technology that assesses brain functions by leveraging artificial intelligence (Al) models trained on data from people with brain pathology and healthy controls. The platform enables accessible, non-invasive, and scalable brain function assessment through digital functional brain mapping without the use of traditional imaging modalities, such as PET imaging or functional magnetic resonance imaging (fMRI). In this way, the platform provides a non-invasive alternative to fMRI or PET -based functional brain mapping, eliminating the need for radiation exposure, radioactive tracers, or costly scanning infrastructure. The system utilizes an extending reality (XR) device (e.g., an XR headset or glasses) for immersive XR tasks (e.g., neurocognitive or sensorimotor tasks), and captures key behavioral and physiological biometric parameters during the tasks, such as eye-tracking parameters using embedded eye-tracking in the XR device. The system may additionally or alternatively capture one or more of tactile feedback, such as from a handheld controller of the XR device, or electrical activity (e.g., electroencephalography (EEG) or electrocardiogram (EKG)) from electrodes. The engaging nature of XR tasks improves the experience of the subject compared to traditional neuropsychological testing.

[0007] With the Al models, the platform can also identify learned patterns in behavioral and physiological biometrics (e.g., eye movement behavior, tactile feedback, or electrical activity) that are predictive of brain disorders (e.g., cognitive impairment, stroke, traumatic brain injury, etc.). These Al models enable the platform to provide early diagnosis by detecting subtle changes in the behavioral and physiological biometrics that may signify the onset of a brainAttorney Docket No. CRNU.P0037WO disorder (e.g., neurodegenerative disease, such as Alzheimer's disease). The platform can also deliver task-specific functional readouts that provide actionable insights across various brain function domains, enhancing its utility in both diagnosis and treatment planning.

[0008] Beyond diagnosing disease, the platform also enables objective profiling of neurological function across multiple neural domains affected in brain disorders, providing insights into brain health that go beyond symptom observation. For example, the platform may compute region-specific functional indices for the brain. The platform can be used to establish neural activity baselines prior to treatment and to monitor changes in brain function over time, supporting longitudinal assessments. Additionally, the platform can assist in rehabilitation and neurofeedback by identifying target brain networks for therapy or brain stimulation.

[0009] The platform’s scalability and ease of use make the platform suitable for repeatable assessments and widespread deployment in healthcare settings or home settings, providing both clinicians and other users with a powerful tool to monitor and track brain health over time.

[0010] Shortcomings mentioned here are only representative and are included to highlight problems that the inventors have identified with respect to existing devices and sought to improve upon. Aspects of devices described below may address some or all of the shortcomings as well as others known in the art. Aspects of the improved devices described herein may present other benefits than, and be used in other applications than, those described above.

[0011] In an example, a system includes a memory storing processor-readable code, and at least one processor coupled to the memory. The at least one processor is configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving sensor data from an extended reality (XR) apparatus, and generating, using an artificial intelligence (Al) model, a functional map of a brain of a subject based on the sensor data. The sensor data includes eye-tracking data and is collected by the XR apparatus while the subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus. The one or more tasks stimulate a plurality of neurotransmitters of the subject. The functional map indicates neural function of a plurality of regions of the brain.

[0012] In an example, a method includes receiving sensor data from an extended reality (XR) apparatus, and generating, using an artificial intelligence (Al) model, a functional map of a brain of a subject based on the sensor data. The sensor data includes eye-tracking data and is collected by the XR apparatus while the subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus. The one or more tasks stimulate a plurality of neurotransmitters of the subject. The functional map indicates neurological function of a plurality of regions of the brain.Attorney Docket No. CRNU.P0037WO

[0013] In an example, a non-transitory, computer-readable medium stores instructions. The instructions, when performed by a processor, cause the processor to perform operations including: receiving sensor data from an extended reality (XR) apparatus, and generating, using an artificial intelligence (Al) model, a functional map of a brain of a subject based on the sensor data. The sensor data includes eye-tracking data and is collected by the XR apparatus while the subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus. The one or more tasks stimulate a plurality of neurotransmitters of the subject. The functional map indicates neurological function of a plurality of regions of the brain.

[0014] In an example, a system comprises an extended reality (XR) apparatus and a computing device. The XR apparatus comprises: one or more sensors including an eye-tracking sensor; a first memory storing processor-readable code; and at least one first processor coupled to the first memory. The at least one first processor is configured to execute the processor-readable code to cause the at least one first processor to perform operations including: presenting one or more tasks in a virtual environment to a subject, the one or more tasks stimulating a plurality of neurotransmitters of the subject; and detecting, via the one or more sensors, sensor data while the subject performs the one or more tasks, the sensor data including eye-tracking data. The computing device comprises: a second memory storing processor-readable code; and at least one second processor coupled to the second memory. The at least one second processor is configured to execute the processor-readable code to cause the at least one second processor to perform operations including: receiving the sensor data from the XR apparatus; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

[0015] Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.

[0016] The terms “comprise” and any form thereof such as “comprises” and “comprising,” “have” and any form thereof such as “has” and “having,” and “include” and any form thereof such as “includes” and “including” are open-ended linking verbs. As a result, an apparatus or system that “comprises,” “has,” or “includes” one or more elements possesses those one or more elements but is not limited to possessing only those elements. Likewise, a method thatAttorney Docket No. CRNU.P0037WO“comprises,” “has,” or “includes” one or more steps possesses those one or more steps but is not limited to possessing only those one or more steps.

[0017] Any embodiment of any of the apparatuses, systems, and methods can consist of or consist essentially of — rather than comprise / have / include — any of the described steps, elements, and / or features. Thus, in any of the claims, the term “consisting of’ or “consisting essentially of’ can be substituted for any of the open-ended linking verbs recited above in order to change the scope of a given claim from what it would otherwise be using the open-ended linking verb.

[0018] An apparatus or system that is configured in a certain way is configured in at least that way, but it can also be configured in other ways than those specifically described. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative.

[0019] The feature or features of one embodiment may be applied to other implementations, even though not described or illustrated, unless expressly prohibited by this disclosure or the nature of the implementations.

[0020] Some details associated with the implementations are described above and others are described below.

[0021] Additional features and advantages of the disclosed method and apparatus are described in, and will be apparent from, the following Detailed Description and the Figures. The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the figures and description. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and not to limit the scope of the inventive subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A further understanding of the nature and advantages of the present disclosure may be realized by reference to the following drawings. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label with a second label that distinguishesAttorney Docket No. CRNU.P0037WO among the similar components. If just the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label. Like reference numbers and designations in the various drawings indicate like elements.

[0023] FIG. 1 is a system for brain function assessment of a subject, according to an aspect of the present disclosure.

[0024] FIG. 2 is a block diagram of an extended reality apparatus in communication with a handheld controller, according to an aspect of the present disclosure.

[0025] FIG. 3 is a block diagram of a computing device for brain function assessment of a subject, according to an aspect of the present disclosure.

[0026] FIG. 4 is a block diagram of an example artificial intelligence (Al) model represented by an artificial neural network (ANN), according to an aspect of the present disclosure.

[0027] FIG. 5 is a flow diagram illustrating an example pipeline for brain function assessment of a subject, according to an aspect of the present disclosure.

[0028] FIG. 6 is a chart showing connections between example tasks and the neurotransmitters that each task targets, according to an aspect of the present disclosure.

[0029] FIG. 7 is a flow diagram illustrating an example pipeline for generating a digital brain model, according to an aspect of the present disclosure.

[0030] FIG. 8 is a flow chart illustrating an example method for brain function assessment of a subject, according to an aspect of the present disclosure

[0031] FIG. 9 is a block diagram of a computing environment, according to an aspect of the present disclosure.

[0032] FIG. 10A illustrates representative regional Braak standardized uptake value ratios (SUVR) maps from processed MK-6240 tracer tau-PET images across four diagnostic groups visualized in the standard template space.

[0033] FIG. 10B illustrates a Linear Discriminant Analysis (LDA) chart of features extracted from subject responses during cognitive tasks, according to an aspect of the present disclosure.DETAILED DESCRIPTION

[0034] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to limit the scope of the disclosure to solely that described explicitly herein. Rather, the detailed description includes specific details for the purpose of providing a thorough understanding of the inventive subject matter. It will be apparent to those skilled in the art that these specific details are not requiredAttorney Docket No. CRNU.P0037WO in every case and that, in some instances, well-known structures and components are shown in block diagram form for clarity of presentation.

[0035] An extended reality (XR)-based approach leveraging Al models for assessing brain function is described herein. Instead of directly capturing anatomical images or biochemical tracers, the described techniques generate quantitative, region-specific functional indices for the brain. Stated differently, the techniques produce a functional activity profile across key brain regions, that is analogous to what might be obtained from PET or fMRI, but without the need for physical scanners or invasive procedures.

[0036] The functional mapping of the brain can also be disease-diagnostic, since the indices reflect how well each brain region is performing (e.g., locus coeruleus (LC) functions of the LC-norepinephrine (LC-NE) system, memory encoding in the hippocampus, emotional processing in the amygdala, motor control in basal ganglia), independent of the underlying disease etiology or pathology. Whatever a subject’s impairment is a result of, the techniques focus on the functional output of brain circuits rather than a specific biochemical hallmark. This broad applicability makes the provided techniques a universal neurofunctional mapping tool, and one that can be applied across a wide range of neurologic and psychiatric conditions to characterize brain function in a consistent manner.

[0037] A wide range of clinical and research applications are supported by the brain function assessment techniques. The techniques enable objective profiling of neurological function across multiple neural domains affected in brain disorders, providing insights into brain health that go beyond symptom observation. The techniques can be used to establish neural activity baselines prior to treatment and to monitor changes in brain function over time, supporting longitudinal assessments. By generating individualized functional brain profiles over time for a subject, the techniques also enable the creation of digital models that replicate the subject’s brain, which supports predictive analytics and clinical decision-making tailored to the subject.

[0038] FIG. 1 illustrates a system 10 (e.g., a telecommunications network) that may be used to implement various aspects of the present application. Generally, the system 10 includes various devices communicating and functioning together in the functional assessment of a subject’s brain. As illustrated, a communications network 20 allows for communication in the system 10. The communications network 20 may include one or more wireless networks such as, but not limited to one or more of a Local Area Network (LAN), Wireless Local Area Network (WLAN), a Personal Area Network (PAN), Campus Area Network (CAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a Wireless Wide Area Network (WWAN), Global System for Mobile Communications (GSM), PersonalAttorney Docket No. CRNU.P0037WOCommunications Service (PCS), Digital Advanced Mobile Phone Service (D-Amps), Bluetooth, Wi-Fi, Fixed Wireless Data, 2G, 2.5G, 3G, 4G, LTE networks, enhanced data rates for GSM evolution (EDGE), General packet radio service (GPRS), enhanced GPRS, messaging protocols such as, TCP / IP, SMS, MMS, extensible messaging and presence protocol (XMPP), real time messaging protocol (RTMP), instant messaging and presence protocol (IMPP), instant messaging, USSD, IRC, or any other wireless data networks or messaging protocols. The communications network 20 may also include wired networks.

[0039] System 10 includes an XR apparatus 100 that is capable of presenting one or more tasks to a subject 702 (FIG. 7) in a virtual XR environment, and detecting, with one or more sensors, sensor data 170 (FIG. 2) that captures the responses of the subject 702 to the task stimulus. In some implementations, XR apparatus 100 may be at least partially worn by the subject 702. For example, in the illustrated implementation, XR apparatus 100 includes an XR headset 102 that may be worn on the head of the subject. In other implementations, XR apparatus 100 may include XR glasses or another suitable XR device worn on a head of the subject 702. XR apparatus 100 is depicted also including a handheld controller 200. The handheld controller 200 may be utilized as an input device to XR headset 102. In some implementations, the handheld controller 200 may be omitted.

[0040] In some implementations, XR apparatus 100 may present a virtual XR environment to the subject 702 without being worn by the subject 702. For example, the XR apparatus 100 may enclose the subject 702 within a space within which the virtual XR environment is presented. In this example, the portability advantages of system 10 may be lessened compared to the implementations in which XR apparatus 100 is worn by the subject 702.

[0041] As is known, XR encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR). In this same manner, XR apparatus 100 may be a VR apparatus, an AR apparatus, or a MR apparatus. For the remaining description, the XR apparatus 100 will be considered to be an XR headset 102 and a handheld controller 200 for ease of description.

[0042] XR apparatus 100 may transmit the sensor data 170 to a computing system 300. As will be described more below, computing system 300 leverages one or more Al models to process sensor data 170 and generate a brain function report 310 (FIG. 3) providing one or more insight into the brain function of the subject 702. Computing system 300 may transmit the brain function report 310 as appropriate. For example, computing system 300 may transmit the brain function report 310 to one or more clinician devices 30 or to one or more subject devices 40, or both.

[0043] Further detail regarding the components of system 10 will now be provided.Attorney Docket No. CRNU.P0037WOXR Apparatus

[0044] FIG. 2 shows a block diagram of XR apparatus 100. While main components of XR apparatus 100 are shown, it will be appreciated that well-known components of XR headset 102 or handheld controller 200 may have been omitted for brevity. The illustrated implementation of XR headset 102 includes one or more processors 110, a memory 120, a display 130, one or more sensors 140, an electroencephalography (EEG) unit 150a, an electrocardiogram (EKG) unit 150b, and one or more input / output (I / O) devices 160. In other implementations, one or more of the illustrated components of XR apparatus 100 may be omitted.

[0045] Processor(s) 110 may include, for example, one or more central processing units (CPUs) or graphics processing units (GPUs), for rendering and computing. Processor(s) 110 is coupled to memory 120 that stores processor-executable code for processor(s) 110 to perform the rendering and computing. In other implementations, XR headset 100 may rely on external computing systems or consoles for processing power, and the processor(s) 110 may be omitted.

[0046] Display 130 may include one or more displays to deliver immersive visuals of a virtual environment. For example, the one or more displays of display 130 may include high-resolution organic light-emitting diode (OLED) or liquid crystal display (LCD) panels. In some implementations, XR headset 100 includes one or more displays (e.g., one display) for each eye of the subject 702. Display 130 is integrated with XR headset 100. In some instances, XR headset 100 may include optic components, such as lenses (e.g., Fresnel or pancake lenses) that magnify or focus the one or more displays 130 for the subject 702.

[0047] The one or more sensors 140 are integrated with or otherwise connected to XR headset 100. For example, the sensor(s) 140 include one or more eye-tracking sensors. The one or more eye-tracking sensors may be an infrared-based system including one or more infrared cameras that detect corneal reflections and pupil movements for monitoring gaze and pupil response of the subject 702. In other implementations, the one or more eye-tracking sensors may be a videobased system including visible-light cameras to track eye movements by analyzing pupil position and facial landmarks, a scleral search coil system that includes tracking a tiny coil embedded in a contact lens worn by the subject 702, or an electrooculography system that includes electrodes placed on the skin of the subject 702 to measure electrical potential differences around the eyes. Eye-tracking data (e.g., oculomotor data) detected by the eyetracking sensors includes pupil size, eye position coordinates, eye rotation angles, or blinking.Attorney Docket No. CRNU.P0037WOThe eye-tracking data may be sampled at a rate of, for example, 60 Hz or higher for capture of rapid eye movements.

[0048] The sensor(s) 140 may include other sensors in addition to the one or more eye-tracking sensors. For example, the sensor(s) 140 may include positional tracking sensors (e.g., an inertial measurement unit (IMU)), such as accelerometers, gyroscopes, and magnetometers, that work together to detect head orientation and movement in real time. The sensor(s) 140 may include hand-tracking sensors that utilize computer vision or infrared depth cameras to track hand gestures of the subject 702.

[0049] The electroencephalography (EEG) unit 150a of XR headset 100 includes electrodes that record brain electrical activity. The EEG electrodes may be arranged so as to be placed on the scalp of the subject 702. The EEG unit 150a may be part of a dry -EEG system. In an example, the EEG unit 150a may include channels positioned over frontal and parietal regions (important for capturing P300 and other task-related potentials). The EEG unit 150a may sample electrical activity at a rate of, for example, 500 Hz or more to accurately capture fast neural events.

[0050] In some implementations, the EEG unit 150a may be provided in the system 10 separate from the XR headset 100, such as being embedded in a close-fitting EEG cap worn with the XR headset 100. In some implementations, the EEG unit 150a may be omitted.

[0051] The electrocardiogram (EKG) unit 150b of XR headset 100 includes electrodes that record heart rate. The EKG electrodes are placed on the body of the subject 702. In some implementations the EKG unit 150b may be provided in the system 10 separate from the XR headset 100. In some implementations, the EKG unit 150b may be omitted.

[0052] The one or more I / O devices 160 of XR headset 100 may contribute to creating an immersive virtual XR environment. The I / O device(s) 160 may include, for example, audio input devices (e.g., a microphone for voice recognition), audio output devices (e.g., speakers for emitting auditory stimuli), haptic devices (e.g., vibration motors), or electrodes for applying electrical neurostimulation. I / O device(s) 160 may be integrated with XR headset 100 or may be separate from XR headset 100. For example, while shown separately in FIG. 2, I / O device(s) 160 may include handheld controller 200.

[0053] Handheld controller 200 may include a microcontroller 210, an IMU 220, and one or more I / O devices 230. The IMU 200 tracks orientation and movement of the handheld controller 200, allowing the subject 702 to point, grab, throw, or manipulate virtual objects with natural gestures. The I / O device(s) 230 may include a combination of buttons, triggers, thumbsticks, and touch-sensitive surfaces to enable precise interaction within the virtualAttorney Docket No. CRNU.P0037WO environment. In some implementations, the I / O device(s) 230 may include haptic feedback mechanisms that provide tactile sensations, such as vibrations or pulses, to simulate the feeling of impact, texture, or resistance.

[0054] In the system 10, XR apparatus 100 is used to present stimuli to the subject 702, in the form of cognitive or sensorimotor tasks 104, within an interactive virtual three-dimensional (3D) environment. The XR presentation ensures immersive engagement that reduces external distractions for the subject 702. The system 10 supports a range of diverse cognitive and sensorimotor targeted tasks 104 using the XR apparatus 100. For example, the tasks 104 may include one or more of emotional classification paradigm tasks (e.g., for probing the function and dysfunction of the amygdala), N-back working memory tasks (e.g., for probing the function and dysfunction of cortical layers (e.g., dorsolateral prefrontal cortex, premotor cortex, parietal cortex)), associative memory tasks with interference (e.g., probing prefrontal-temporal- hippocampal network), cross-modal binding tasks (e.g., for probing the function and dysfunction of the superior temporal sulcus or fusiform gyrus), episodic encoding and retrieval tasks (e.g., for probing the function and dysfunction of the prefrontal cortex or parietal cortex. FIG. 6 is a chart showing example associations between neurotransmitters and example tasks 104 by indicating which neurotransmitters 600 of glutamate, dopamine, serotonin, norepinephrine, acetylcholine, and gamma-aminobutyric acid (GABA) are targeted by which example tasks 104.

[0055] Some of the tasks 104 are specifically designed to activate single, distinct brain circuits. These targeted tasks include, for example: visual oddball tasks, anti-saccade tasks, pro-saccade tasks, smooth pursuit task, emotional reactivity and regulation tasks, or procedural learning and short / 1 ong-term memory tasks.

[0056] The system 10 also supports complex, integrative cognitive tasks 104 designed to engage multiple brain networks simultaneously. These include, for example: executive function tasks, attention and inhibition tasks, language processing tasks, memory recall tasks, or sensorimotor tasks. Additional detail regarding example tasks is provided in the following paragraphs.

[0057] A visual oddball task targets Locus Coeruleus-Norepinephrine (LC-NE) system and attention functions. The visual oddball task involves presenting a series of repetitive visual stimuli interspersed with infrequent "oddball" images that differ in some noticeable way, such as shape, color, or orientation. The subject is asked to detect or respond to these rare stimuli.Attorney Docket No. CRNU.P0037WO

[0058] An anti-saccade task tests inhibitory control and targets the Basel Ganglia, eye fields, superior colliculus, and oculomotor nuclei. The anti-saccade task instructs the subject to look away from a suddenly appearing visual stimulus, rather than toward it.

[0059] A pro-saccade task tests basic oculomotor functions and targets eye fields, Superior Colliculus, and oculomotor nuclei. The anti-saccade task instructs the subject to quickly look toward a visual stimulus that appears on the screen.

[0060] A reward-modulated saccade task examines how anticipated rewards influence eye movement behavior, particularly the speed and accuracy of saccades toward visual targets. Subjects are cued to make rapid eye movements, with varying reward values linked to different targets or response times.

[0061] An express saccade task measures rapid eye movements in response to visual stimuli, typically involving a target that appears suddenly in the peripheral visual field. The subject is instructed to shift their gaze to the target as quickly as possible, allowing assessment of visual attention, reaction time, and inhibitory control.

[0062] A saccadic adaptation task evaluates the brain's ability to adjust eye movement accuracy when visual feedback is altered. During the task, the target shifts slightly during a saccade, prompting the brain to recalibrate future eye movements to maintain precision.

[0063] A smooth pursuit task tests basic oculomotor functions, such as smooth pursuit eye movement and attention, and targets eye fields, brainstem, and ventrolateral thalamus. In a smooth pursuit task, the subject must track a slowly moving visual target with their eyes, requiring continuous and coordinated eye movement.

[0064] A free viewing or visual search task assesses how a subject scan and interpret visual scenes without specific instructions, often tracking eye movements to study attention and perception. In cognitive testing, participants may be asked to locate a target among distractors or simply observe images while researchers analyze gaze patterns, fixation durations, and search strategies.

[0065] The Monetary Incentive Delay (MID) task assesses reward processing and motivation by measuring how quickly subjects respond to cues signaling potential monetary gain or loss. During the task, subjects see a cue, wait briefly, then respond to a target as fast as possible to earn or avoid money. Pupil dilation may be measured during the task.

[0066] A contextual cueing task evaluates implicit learning and visual attention by presenting repeated spatial configurations where a target is embedded among distractors. Over time, subjects respond faster to familiar layouts, revealing how context guides search efficiency without conscious awareness.Attorney Docket No. CRNU.P0037WO

[0067] A visual paired associate task assesses associative memory by presenting subjects with pairs of visual stimuli, such as objects or symbols, and later testing their ability to recall one item when cued with its pair.

[0068] A blink rate task measures spontaneous or task-related eye blinking to assess cognitive load, attention, and neurological function. Subjects perform activities while their blink frequency is recorded, revealing changes linked to mental effort or emotional states.

[0069] An oculomotor delayed response task tests working memory and spatial attention by requiring subjects to remember the location of a briefly presented visual target. After a delay period with no visual cues, they must make a saccade to the remembered location.

[0070] A fixation stability task evaluates an ability of a subject to maintain steady gaze on a single point over time.

[0071] A go / no-go task assesses response inhibition and attention by requiring subjects to perform an action (e.g., press a button) when presented with a "go" stimulus and withhold the response when a "no-go" stimulus appears.

[0072] A flanker task assesses attention and cognitive control by presenting a central target stimulus flanked by distracting stimuli that may be congruent or incongruent. Subjects must respond based on the central target while ignoring the flankers, revealing how well they can suppress interference.

[0073] A reward anticipation (pupil) task measures changes in pupil size as subjects anticipate potential rewards, reflecting arousal and motivational states. Typically, cues signal varying reward levels before a response task, and pupil dilation is tracked during the anticipation phase.

[0074] An emotional reactivity and regulation task targets the limbic system, including the Amygdala, Anterior Cingulate Cortex, and other emotion processing circuits. The subject views images or videos designed to elicit emotions, and the response of the subject is tracked.

[0075] A procedural learning and short / long-term memory task targets Hippocampus and Entorhinal Cortex function. For example, a procedural learning and short / long-term memory task may include remembering and finding objects in the virtual scene.

[0076] Executive function tasks (e.g., working memory, task-switching, or attention) engage prefrontal networks. Executive function tasks assess higher-order processes like planning, decision-making, problem-solving, and impulse control. These tasks may require the subject to shift attention, inhibit automatic responses, or manage multiple rules — such as in the Stroop test or the Wisconsin Card Sorting Test.

[0077] Attention and inhibition tasks (e.g., anti-saccade, oddball) probe fronto-parietal control systems. Attention and inhibition tasks evaluate the ability of the subject to focus on relevantAttorney Docket No. CRNU.P0037WO stimuli while suppressing distractions or impulsive responses. These tasks often involve identifying target items amid distractors or resisting automatic reactions — like in the Stroop test or go / no-go paradigms.

[0078] Language processing tasks (e.g., word-finding, verbal fluency) engage language networks, such as the frontal and temporal cortex. Language processing tasks assess the ability of the subject to understand, interpret, and produce spoken or written language. These tasks may involve naming objects, following verbal instructions, reading comprehension, or generating coherent speech.

[0079] Memory recall tasks (e.g., associative learning) probe medial temporal lobe and hippocampal regions. Memory recall tasks assess the ability of the subject to retrieve previously learned information without cues. These tasks may involve recalling word lists, stories, or visual patterns after short or long delays.

[0080] Sensorimotor tasks (e.g., visual tracking, motor coordination) assess sensorimotor and occipital networks. Sensorimotor tasks assess the coordination of the subject between sensory input and motor output, such as responding to visual or auditory cues with precise movements. These tasks often involve tracking, tapping, or manipulating objects to evaluate reaction time, fine motor skills, and perceptual-motor integration.

[0081] As shown in FIG. 2, one or more tasks 104 (e.g., processor-executable code) for the subj ect 702 to perform may be input to XR headset 102, or to XR apparatus 100 more generally. For example, the one or more tasks 104 may be selected on a clinician device 30 or a subject device 40, and a signal indicative of the one or more tasks 104 selected may be transmitted to the XR headset 102. In some implementations, the signal may be transmitted to the XR headset 102 from the clinician device 30 or a subject device 40 via the computing system 300. In another example, the computing system 300 may determine the one or more tasks 104 for the subject 702 to perform and transmit a signal indicative of the one or more tasks 104 determined to the XR headset 102. In another example, the handheld controller 200 may be used to input one or more tasks 104 to XR headset 102.

[0082] In some instances, the tasks 104 selected include visual oddball, anti-saccade, and smooth pursuit. For instance, as shown in FIG. 6, these three tasks may be considered foundational as the combination of the visual oddball, anti-saccade, and smooth pursuit tasks targets all the neurotransmitters. In other instances, tasks 104 may be selected to localize the pathology for a brain disorder (e.g., neurocognitive impairment or neurodegenerative disorder).

[0083] XR headset 102 thereafter presents the one or more tasks 104 that are input to the subj ect 702. As the subject 702 performs the one or more tasks 104, the XR headset 102 continuouslyAttorney Docket No. CRNU.P0037WO detects and records sensor data 170. For example, the XR headset 102 continuously detects and records eye-tracking data (e.g., pupil size, eye position coordinates, saccade velocities), such as with one or more infrared cameras, in synchrony with the task events. In another example, the XR headset 102 continuously detects and records tactile data (e.g., tactile response time, tactile error rate) received from the I / O device(s) 230 of handheld controller 200. In another example, the XR headset 102 continuously detects and records EEG signals with the EEG unit 150a. In another example, the XR headset 102 continuously detects and records EKG signals with the EKG unit 150b. In another example, XR headset 102 may continuously detect and record head motion data. In some examples, the XR headset 102 may continuously and simultaneously detect and record two or more of eye-tracking data, EEG signals, EKG signals, or head motion data. The hardware of XR headset 102 is time-synchronized so that each stimulus onset (or other event marker) of a task is logged alongside the physiological data streams occurring.

[0084] XR headset 102 outputs sensor data 170 capturing the results of the one or more tasks 104 presented. Sensor data 170 includes the raw data detected and recorded by the sensor(s) 140, EEG unit 150a, or EKG unit 150b of XR headset 102. The data streams in sensor data 170 include precise timestamp alignment, enabling correlation of neural responses with specific stimuli or eye movement events. Sensor data 170 may be transmitted to computing system 300.Computing System

[0085] FIG. 3 is a block diagram of an example of computing system 300. Computing system 300 receives, as input, sensor data 170 and outputs a brain function report 310. Computing system 300 includes a processor 302 coupled to a memory 304. In various aspects, processor 302 may include more than one processor. For example, processor 302 may include a first processor 302 A (not shown) and a second processor 302B (not shown) that are each coupled to the memory 304. The first processor 302A may be in communication with the second processor 302B. The first processor 302 A and the second processor 302B may each perform all of the operations performed by processor 302, or alternatively, the first processor 302 A may only perform a first portion of the operations and the second processor 302B may only perform a second portion of the operations.

[0086] In various aspects, memory 304 may include more than one memory. For example, memory 304 may include a first memory 304 A (not shown) and a second memory 304B (not shown) that are each coupled to processor 302. The first memory 304A and the second memory 304B may each store all of the processor-executable code for all of the operations of processorAttorney Docket No. CRNU.P0037WO302, or alternatively, the first memory 304 A may only store a first portion of the processorexecutable code and the second memory 304B may only store a second portion of the processor-executable code. In another example, processor 302 may include the first processor 302 A and the second processor 302B that are each coupled to a first memory 304 A (not shown) and a second memory 304B (not shown) of memory 304. In another example, processor 302 may include the first processor 302 A coupled to the first memory 304 A of memory 304, but not to the second memory 304B of memory 304, and the second processor 302B coupled to the second memory 304B, but not to the first memory 304B.

[0087] In aspects in which processor 302 includes two or more processors, the two or more processors may be included with the same computing system 300, or may be suitably separated among two or more computing systems 300. In aspects in which memory 304 includes two or more memories, the two or more memories may be included with the same computing system 300, or may be suitably separated among two or more computing devices 300. The computing system(s) 300 with which the two or more memories of memory 304 are included may be the same computing system(s) 300 with which the at least one processor of processor 302 is included or may be different. For example, a processor 302 may be included with a first computing system 300 A (not shown) and a memory 304 may be included with a second computing system 300B (not shown), e.g., a server, in communication with the first computing system 300 A over a network.

[0088] The at least one memory 304 stores an Al model 306. Al model 306 is trained to output the brain function report 310. In this way, Al model 306 predicts one or more of a virtual functional brain map, a brain function diagnosis, a treatment plan, or performance information on specific performed tasks. For example, Al model 306 may be implemented as one or more artificial intelligence models, including supervised learning models, unsupervised learning models, other types of artificial intelligence models, and / or other types of predictive models. For example, Al model 306 may be implemented as one or more of a neural network, a transformer model, a decision tree model, a support vector machine, a Bayesian network, a classifier model, a regression model, and the like. Al model 306 may be trained based on training data to assess brain function of a subject. For example, one or more training datasets may be used that contain task-based fMRI activation patterns, fluorodeoxyglucose (FDG)-PET metabolic maps indicating region-specific brain activity, or structural MRI or diffusion tensor imaging (DTI) data indicating network connectivity. The training data sets may specify one or more expected outputs. For example, spatially resolved virtual functional brain region activation maps. Parameters of Al model 306 may be updated based on whether Al model 306Attorney Docket No. CRNU.P0037WO generates correct outputs when compared to the expected outputs. In particular, Al model 306 may receive one or more pieces of input data from the training data sets that are associated with a plurality of expected outputs. Al model 306 may generate predicted outputs based on a current configuration of Al model 306. The predicted outputs may be compared to the expected outputs and one or more parameter updates may be computed based on differences between the predicted outputs and the expected outputs. In particular, the parameters may include weights (e.g., priorities) for different features and combinations of features (e.g., saccadic reaction time, saccade peak velocity, multiple step saccades incidence, square wave jerks amplitude, or change in pupil diameter to cognitive stimuli). The parameter updates to Al model 306 may include updating one or more of the features analyzed and / or the weights assigned to different features or combinations of features (e.g., relative to the current configuration of Al model 306).

[0089] The at least one memory 304 may, in some implementations, store a database 308. In other implementations, the database 308 may be stored in a separate computing system in communication with computing system 300.

[0090] FIG. 4 is an illustrative block diagram of an example artificial intelligence (Al) model, that may be implemented as Al model 306, represented by an artificial neural network (ANN) 400. ANN 400 may receive input data 406 which may include one or more bits of data 402, pre-processed data output from pre-processor 404 (optional), or some combination thereof. Here, data 402 may include training data, verification data, application-related data, or the like, e.g., depending on the stage of deployment of ANN 400. Pre-processor 404 may be included within ANN 400 in some other implementations. Pre-processor 404 may, for example, process all or a portion of data 402 which may result in some of data 402 being changed, replaced, deleted, etc. In some implementations, pre-processor 404 may add additional data to data 402. In some implementations, the pre-processor 404 may be a Al model, such as an ANN.

[0091] ANN 400 includes at least one first layer 408 of artificial neurons 410 to process input data 406 and provide resulting first layer data via edges 412 to at least a portion of at least one second layer 414. Second layer 414 processes data received via edges 412 and provides second layer output data via edges 416 to at least a portion of at least one third layer 418. Third layer 418 processes data received via edges 416 and provides third layer output data via edges 420 to at least a portion of a final layer 422 including one or more neurons to provide output data 424. All or part of output data 424 may be further processed in some manner by (optional) postprocessor 426. Thus, in certain examples, ANN 400 may provide output data 428 that is based on output data 424, post-processed data output from post-processor 426, or some combinationAttorney Docket No. CRNU.P0037WO thereof. Post-processor 426 may be included within ANN 400 in some other implementations. Post-processor 426 may, for example, process all or a portion of output data 424 which may result in output data 428 being different, at least in part, to output data 424, e.g., as result of data being changed, replaced, deleted, etc. In some implementations, post-processor 426 may be configured to add additional data to output data 424. In this example, second layer 414 and third layer 418 represent intermediate or hidden layers that may be arranged in a hierarchical or other like structure. Although not explicitly shown, there may be one or more further intermediate layers between the second layer 414 and the third layer 418. In some implementations, the post-processor 426 may be a Al model, such as an ANN.

[0092] The structure and training of artificial neurons 410 in the various layers may be tailored to specific requirements of an application. Within a given layer of an ANN, some or all of the neurons may be configured to process information provided to the layer and output corresponding transformed information from the layer. For example, transformed information from a layer may represent a weighted sum of the input information associated with or otherwise based on a non-linear activation function or other activation function used to “activate” artificial neurons of a next layer. Artificial neurons in such a layer may be activated by or be responsive to weights and biases that may be adjusted during a training process or during operation of the Al model. Weights of the various artificial neurons may act as parameters to control a strength of connections between layers or artificial neurons, while biases may act as parameters to control a direction of connections between the layers or artificial neurons. An activation function may select or determine whether an artificial neuron transmits its output to the next layer or not in response to its received data. Different activation functions may be used to model different types of non-linear relationships. By introducing nonlinearity into an Al model, an activation function allows the configuration for the Al model to change in response to identifying complex patterns and relationships in the input data 406 and determinations that should be made when those complex patterns and relationships are identified in the input data. Some non-exhaustive example activation functions include a sigmoid based activation function, a hyperbolic tangent (tanh) based activation function, a convolutional activation function, up-sampling, pooling, and a rectified linear unit (ReLU) based activation function.Brain Function Report Generation

[0093] FIG. 5 is a flow diagram of an example pipeline 500 for generating a brain function report 310, that includes information reflecting the brain function of the subject 702, based onAttorney Docket No. CRNU.P0037WO the sensor data 170 collected by XR apparatus 100. Pipeline 500 includes a preprocessing stage 502 at which the sensor data 170 undergoes comprehensive signal processing to reduce noise and extract quantitative features.

[0094] Referring to the eye-tracking data as an example, signal processing includes blink detection and interpolation. For example, the sensor data 170 may include eye rotation angles, pupil dilation, two-dimensional (2D) gaze coordinates, and blinking. In this example, corrupted frames in the sensor data 170 may be identified by computing the derivative of the pupil dilation (PD) time series and applying a threshold to detect sudden changes associated with lost pupil tracking and blinks. These identified corrupted frames can be removed and subsequently interpolated (e.g., using Akima interpolation) to reconstruct the missing data due to the lost pupil tracking and blinks. In some aspects, to reduce noise while preserving the shape of the signal, the interpolated time series may be processed using a median filter.

[0095] Preprocessing stage 502 may include, at diamond 504, determining whether any of the tasks 104 need to be repeated based on the sensor data 170. For example, it may be determined that a task 104 needs to be repeated if a quantity of corrupted frames exceeds a threshold, if the interpolation has a confidence score below a threshold, or it isn’t possible to perform the interpolation. When it is determined that one or more tasks 104 need to be repeated, instructions 506 are determined indicating which tasks 104 those are.

[0096] The instructions 506 may alert a human user, such as in the form of an alert on a clinician device 30 or a subject device 40, so that the clinician or subject may set up the XR apparatus 100 to repeat the indicated tasks 104. Additionally or alternatively, the instructions 506 may include processor-executable code that can be executed by a processor to perform the indicated tasks 104. For example, the XR apparatus 100 may be controlled to automatically configure into a state to present the indicated tasks 104 based on receiving the instructions 506. If one or more tasks 104 are to be repeated, pipeline 500 includes repeating the preprocessing stage 502 when new sensor data 170 is received corresponding to the repeated one or more tasks 104.

[0097] In some implementations, the decision of whether a task 104 needs to be repeated at diamond 504 is performed in real-time. In such implementations, the XR apparatus 100 may transmit the sensor data 170 to computing system 300 or another suitable computing system, which outputs the instructions 506, or an indication that no task repeating is necessary, within a time window on a scale of seconds or minutes, rather than an hour or more.

[0098] When it is determined at diamond 504 that no tasks 104 need to be repeated, a feature set 508 is generated by extracting features from the sensor data 170. Feature set 508 includes aAttorney Docket No. CRNU.P0037WO set of multimodal features that characterize the cognitive-physiological response profile of the subject 702. For example, pupil size time-series in sensor data 170 may be analyzed to compute pupillary reflex metrics. Task-evoked pupillary responses are characterized by features like peak dilation to novel stimuli and sustained pupil diameter changes under cognitive load. In another example, changes in pupil size in response to different stimulus types (standard, target, and novel faces) may be identified from sensor data 170 and averaged to calculate mean responses. In another example, features such as target oddball accuracy (the proportion of correctly identified target stimuli), false-positive rate (the proportion of incorrect button presses for non-target stimuli), and reaction time may be extracted from sensor data 170.

[0099] In another example, saccadic performance metrics may be extracted as features from gaze data of sensor data 170. For instance, in an anti-saccade task, the XR apparatus 100 measures the latency of correct anti-saccades, error rates (frequency of incorrect reflexive glances toward the stimulus), and metrics like saccade velocity or trajectory. In another example, multiple step saccades (MSS) and square wave jerks (SWJs) may be identified and quantified from sensor data 170. MSS’s are defined as a sequence of small-amplitude saccades that occur instead of a single ballistic movement, often reflecting real-time error correction and impaired motor planning. These patterns are detected using eye-tracking signal segmentation and classified using task-specific thresholds. Similarly, SWJs, defined as small, involuntary, back-and-forth saccades around a fixation point, may be extracted as biomarkers of impaired inhibitory control and brainstem dysfunction. The frequency, duration, and amplitude of MSS’s and SWJs may be used as candidate features for Al model 306 that are aimed at distinguishing between neurodegenerative conditions.

[0100] In another example, from EEG data of sensor data 170, both time-domain and frequency-domain features may be extracted, such as key event-related potentials (e.g., the P300 wave amplitude and latency elicited by oddball targets). Other potential EEG features include theta and beta band power changes during task performance, signals reflecting sensory processing (e.g., visual evoked potentials), or measures of functional connectivity between electrodes.

[0101] Example features in feature set 508 include, but are not limited to: saccadic reaction time, saccade amplitude, saccade peak velocity, saccade duration, direction error rate, anticipatory saccades, MSS incidence, MSS step count, inter-saccadic interval (ISI), total time to target, SWJ Frequency, SWJ Amplitude, SWJ duration time, fixation count before target number fixation duration, pupil dynamic change, and baseline pupil size. Further description of these example features is as follows.Attorney Docket No. CRNU.P0037WO

[0102] Saccadic reaction time is a time from stimulus onset to saccade initiation.

[0103] Saccade amplitude is a distance covered by a saccade.

[0104] Saccade peak velocity is a maximum speed of the saccade.

[0105] Saccade duration is a time duration of the saccade.

[0106] Direction error rate is a percentage of trials where initial saccade is toward the stimulus.

[0107] Anticipatory Saccades are saccades occurring before stimulus presentation or instruction.

[0108] MSS incidence is a percentage of trials with MSS (2+ consecutive hypometric saccades).

[0109] MSS Step Count is a number of steps within an MSS sequence.

[0110] Inter-saccadic Interval (ISI) is a time between steps in an MSS sequence.

[0111] Total time to target is a duration from a first saccade to a final fixation on the target.

[0112] SWJ Frequency is a number of square wave jerks per minute during fixation tasks.

[0113] SWJ Amplitude is an angular displacement per jerk (degrees).

[0114] SWJ Duration Time is a time duration between onset and return of the jerk.

[0115] Fixation Count Before Target Number is a count of mini fixations before reaching the target (e.g., in anti -saccade).

[0116] Fixation Duration is an average duration of fixations during task.

[0117] Pupil Dynamic Change is a change in pupil diameter to cognitive stimuli.

[0118] Baseline Pupil Size is an average pupil size at rest.

[0119] In some instance, the feature set 508 may be high-dimensional once extracted. In such instances, dimensionality reduction techniques (e.g., principal component analysis or feature selection algorithms) can be applied to distill the most informative biomarkers from the combined data. For example, principal component analysis (PCA) may be used to reduce the dimensionality of the feature set 508. The end result of the dimensionality reduction is a compact set of multimodal features that characterize the cognitive-physiological response profile of the subject 702.

[0120] With the features extracted, pipeline 500 includes providing the feature set 508 to Al model 306. In some implementations, the preprocessing stage 502 is performed by processor 302 (e.g., pre-processor 904 of FIG. 9) of computing system 300. In other implementations, the preprocessing stage 502 is performed by a computing system separate from computing system 300. In such other implementations, the extracted feature set 508 may be transmitted to computing system 300 for input into Al model 306.Attorney Docket No. CRNU.P0037WO

[0121] Al model 306 includes one or more Al models implementing a report generator 510. The feature set 508 is provided as input to the report generator 510. At diamond 512, the report generator 510 determines whether any additional tasks 104 need to be performed by the subject 702 in order to generate the brain function report 310. For example, the feature set 508 may be missing certain features that the report generator 510 needs to make a diagnosis. When it is determined that one or more additional tasks 104 need to be performed, instructions 506 are determined indicating which tasks 104 those are, though in this case, instructions 506 indicate additional task rather than tasks to be repeated. Otherwise, the description of instructions 506 remains the same and will not be repeated. If one or more additional tasks 104 are instructed to be performed, pipeline 500 includes repeating the preprocessing stage 502, including feature extraction, when new sensor data 170 is received corresponding to the one or more additional tasks 104.

[0122] In some implementations, the decision of whether one or more additional tasks 104 need to be performed at diamond 504 is performed in real-time. In such implementations, the XR apparatus 100 may transmit the sensor data 170 to computing system 300 or another suitable computing system, which extracts the feature set 508, and the report generator 510 determines the instructions 506, or an indication that no additional tasks need to be performed, within a time window on a scale of seconds or minutes.

[0123] When it is determined at diamond 512 that no additional tasks 104 need to be performed, the report generator 510 generates the brain function report 310.

[0124] In some implementations, Al model 306 may include a plurality of task encoders 514a, 514b. Each encoder of the plurality of task encoders 514a, 514b is associated with a different, respective task of the one or more tasks 104. For example, the task encoder 514a may be trained to output, based on the input feature set 508, task information 526a corresponding to the performance of the subject 702 on the visual oddball task, whereas the task encoder 514b may be trained to output, based on the input feature set 508, task information 526b corresponding to the performance of the subject 702 on the anti-saccade task. In this way, a clinician or subject may be made aware of the performance of the subject on a specific task.

[0125] Brain function report 310 may include various information useful for an assessment of the brain health of the subject 702. For example, the brain function report 310 may include a virtual functional brain map 520. A functional brain map is a representation of how different regions of the brain contribute to various cognitive, sensory, motor, and emotional functions. Unlike anatomical brain maps, which focus on physical structures, functional brain maps highlight activity patterns and neural networks involved in specific tasks or behaviors. In aAttorney Docket No. CRNU.P0037WO similar manner, the virtual functional brain map 520 generated by Al model 306 displays the relative engagement or dysfunction in predefined brain regions or networks. For example, the virtual functional brain map 520 may be one or more images where each pixel value in the one or more images directly represents brain functional engagement or impairment. For example, the pixels of the virtual functional brain map 520 may be color-coded to indicate relative functional strength or weakness for different brain regions or networks. In some instances, the virtual functional brain map 520 may be accompanied by network-level scores (e.g., frontal executive network efficiency, temporal memory network performance). The virtual functional brain map 520 may be used with clinical reports for neurological activity profiling or treatment planning.

[0126] In another example, the brain function report 310 may include a brain function diagnosis 522. In some instances, the brain function diagnosis 522 may include a diagnosis of one or more neurological impairments (e.g., neurocognitive impairments) of the subject 702. For example, report generator 510 may generate a diagnosis of a neurological impairment by comparing one or more features of feature set 508 to one or more first thresholds. For example, an anti-saccade error rate that meets a first threshold may indicate partial impairment in executive oculomotor control. In another example, a pupil dynamic response to light changes below a first threshold may suggest impairment in the parasympathetic and sympathetic pathways regulating pupil dynamics to luminance changes. In another example, impairments in saccadic eye movement control may be indicated by (1) a mean saccade reaction time that meets a first threshold and a standard deviation of reaction times that meets a first threshold, indicating delayed initiation of voluntary eye movement, or (2) a saccade amplitude below a first threshold with both a mean amplitude below a threshold and variability that meets a first threshold.

[0127] In some instances, the brain function diagnosis 522 may include a diagnosis of at least one brain disorder (e.g., a neurodegenerative disorder such as Alzheimer's disease). For example, report generator 510 may generate a diagnosis of a brain disorder by comparing one or more features of feature set 508 to one or more second thresholds that indicate greater neurological decline than the first thresholds indicating neurological impairment. For example, an anti-saccade error rate that meets the first threshold but is below the second threshold may indicate mild cognitive impairment, and an anti-saccade error rate that meets the second threshold may indicate a neurodegenerative disorder.Attorney Docket No. CRNU.P0037WO

[0128] Whether a neurological impairment or a brain disorder, the system 10 generates the brain function diagnosis 522 without the use of a radioactive tracer or a positron emission tomography (PET) scanner.

[0129] In another example, the brain function report 310 may include a treatment plan 524 for the one or more neurological impairments or brain disorders. For example, the treatment plan 524 may include a series of next steps (e.g., exercises or tasks to perform, appointment frequency, test frequency, etc.) for the subject 702 to do to slow or reverse the one or more neurological impairments or brain disorders. In some implementations, the treatment plan 524 may include processor-executable code with which to control XR apparatus 100.

[0130] In another example, the brain function report 310 may include the task information 526a, 526b. It should be appreciated that brain function report 310 may include two or more of the virtual functional brain map 520, the brain function diagnosis 522, the treatment plan 524, the task information 526a, or the task information 526a. Any portion of the brain function report 310 may be presented on a display, such as a display of the clinician device 30 or the subject device 40. After generation, the brain function report 310 may be stored in the database 308.

[0131] The brain function report 310 enables objective profiling of neurological function across multiple neural domains affected in brain disorders, providing insights into brain health that go beyond symptom observation. The system 10 can be used to establish neural activity baselines prior to treatment and to monitor changes in brain function over time, supporting longitudinal assessments. Additionally, the brain function report 310 assists in rehabilitation and neurofeedback by identifying target brain networks for therapy or brain stimulation.Digital Cognitive Twin Modeling

[0132] As described above, system 10 generates individualized functional brain profiles in the form of a brain function report 310. As shown in FIG. 7, new instances of the brain function report 310 (e.g., virtual functional brain map 520) may be generated for a particular subject 702 over time, such as at each subsequent appointment with the clinician of the particular subject 702. Each of these instances of the brain function report 310 may be stored in the database 308. In some implementations, Al model 306 may generate a digital brain model 704 based on the plurality of instances of the brain function report 310. Stated differently, system 10 may be capable of digital cognitive twin modeling of the brain of a subject. Digital brain model 704 replicates the brain of the particular subject 702, which supports predictive analytics and clinical decision-making tailored to each subject.Attorney Docket No. CRNU.P0037WO

[0133] For example, the Al model 306 may generate a predicted progression of one or more neurological impairments or brain disorders based on the digital brain model 704. After a subsequent test using the system 10, the digital brain model 704 may be updated.Method

[0134] One method of performing a functional assessment of a user’s brain according to implementations described above is shown in FIG. 8. FIG. 8 is a flow chart illustrating an example method 800 for the functional assessment of a subject’s brain. Method 800 includes, at block 802, receiving sensor data (e.g., sensor data 170) from an XR apparatus (e.g., XR apparatus 100). The sensor data 170 includes eye-tracking data. The sensor data 170 is collected by the XR apparatus 100 while a subject (e.g., subject 702) performed one or more tasks (e.g., one or more tasks 104) presented to the subject 702 in a virtual environment by the XR apparatus 100. The one or more tasks 104 stimulate a plurality of neurotransmitters of the subject 702. In some aspects of method 800, the sensor data 170 collection is time- synchronized with performance of the one or more tasks 104 by the subject 702.

[0135] At block 804, a brain function report (e.g., brain function report 310) is generated based on the sensor data 170 and using an Al model (e.g., Al model 306). For example, a virtual functional map (e.g., virtual functional brain map 520) of a brain of a subject may be generated based on the sensor data 170.

[0136] In some aspects of method 800, generating the brain function report 310 may include extracting a plurality of features (e.g., feature set 508) from the sensor data 170, the brain function report 310 being generated based on the plurality of features 508.

[0137] In some aspects of method 800, generating the brain function report 310 may include generating, based on the sensor data 170, instructions (e.g., instructions 506) indicating that a task of the one or more tasks 104 performed by the subject 702 needs to be repeated before generating the brain function report 310. In some aspects, generating that the task 104 needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data 170. In some aspects, generating that the task needs to be repeated is performed in real-time. In some aspects, method 800 includes controlling the XR apparatus 100 to present the task 104 that needs to be repeated.

[0138] In some aspects of method 800, generating the brain function report 310 may include generating, based on the sensor data 170, instructions (e.g., instructions 506) indicating that one or more additional tasks 104 need to be by the subject 702 before generating the brain function report 310. In some aspects, generating that the one or more additional tasks need toAttorney Docket No. CRNU.P0037WO be performed by the subject is performed in real-time. In some aspects, method 800 includes controlling the XR apparatus 100 to present the one or more additional tasks 104.

[0139] In some aspects, the method 800 may include repeating the receiving sensor data 170 and the generating the functional map 520 of the brain function report 310 a plurality of instances over a period of time so as to generate a plurality of functional maps 520. In such aspects, method 800 may include generating, using the Al model 306 and based on the plurality of functional maps 520, a digital model (e.g., digital brain model 704) replicating the brain of the subject 702. In such aspects, method 800 may include updating the digital model 704 at a time after the digital model 704 is generated. In such aspects, method 800 may include storing, in a database (e.g., database 308), each functional map 520 of the plurality of functional maps when each functional map 520 is generated over the period of time.Example Use Case for Alzheimer’s Disease

[0140] Alzheimer’s disease (AD) is closely linked to tau pathology, which involves the abnormal modification and accumulation of tau protein in the brain, leading to neurodegeneration and cognitive decline. Tau pathology is one of the two hallmark features of AD, alongside amyloid-beta plaques. Tau tangles, however, correlate more strongly with cognitive decline than amyloid plaques. An AD diagnosis and staging may therefore be based on a distribution of tau across brain regions of a subject. The system 10 can provide a digital replacement for tau-PET imaging.

[0141] For instance, in the example use case of AD, the system 10 can probe the activity and dysfunction of the locus coeruleus (LC), which is the earliest site for AD. The LC is buried deep in the brainstem and cannot be imaged with diagnostic accuracy. However, a special neurocircuitry of the sympathetic pathway connects the LC to the pupil, allowing the system 10 to perform dynamic measurements of eye movement patterns and pupil dynamics as a non- invasive readout of LC activity, and consequently, cognitive health and AD severity. The system 10 may also measure EEG signals when assessing AD. From these eye and EEG measurements, the system 10 can generate a virtual biomarker for tau pathology, derived entirely from non-invasive digital sensor data.

[0142] In a testing session for AD in subject 702, the testing session with XR apparatus 100 might start with a simple pupillary light reflex test, in which the virtual scene may alternate between dark and bright backgrounds to induce pupil constriction and dilation, establishing baseline autonomic response metrics. Next, tasks 104 may commence, including one or moreAttorney Docket No. CRNU.P0037WO of (e.g., all), but not limited to: visual oddball, anti-saccade, pro-saccade, smooth pursuit, emotional reactivity and regulation, or procedural learning and short / long-term memory.

[0143] After the testing session, the recorded sensor data 170 is processed by the pipeline 500 as described earlier. Specifically, by the Al model 306 analyzing physiological responses to these tasks — such as eye movement patterns, pupil dynamics, and EEG features — the Al model 306 generates virtual tau burden maps that approximate the spatial distribution of tau pathology, without requiring radioactive tracers or PET scanners. In particular, the virtual functional brain map 520 that is generated includes estimates of tau protein deposition across brain regions. For example, regions of the virtual function brain map 520 may be colored according to predicted tau burden. In some aspects, Al model 306 may be trained to predict regional tau standardized uptake value ratios (SUVRs), the normalized tracer uptake values that atau-PET scan would show in specific brain regions of interest. In such aspects, the virtual function brain map 520 may be colored according to the SUVR levels.

[0144] Al model 306, in some implementations, may generate a brain function diagnosis 522 that indicates AD along with the virtual functional brain map 520 based on one or more AD indicators. For example, tau levels meeting a threshold may indicate a diagnosis of AD. In another example, a quantity of brain regions that include tau may be determined and that quantity meeting a threshold may indicate a diagnosis of AD. In another example, a quantity of brain regions that have a threshold-meeting tau level may be determined and that quantity meeting a threshold may indicate a diagnosis of AD. In another example, a total concentration of tau meeting a threshold may indicate a diagnosis of AD. In another example, SUVRs meeting a threshold may indicate a diagnosis of AD. In another example, neurological impairments in one or more (e.g., two or more, three or more, or all four) of pupil reflex task responses, oddball task responses, anti-saccade task responses, or pro-saccade task responses may indicate a diagnosis of AD. In another example, a combination of two or more of these example AD indicators may indicate a diagnosis of AD.

[0145] Tau pathology, in particular, follows a spatiotemporal progression classified by Braak stages (I- VI) corresponding to the spreading of tau tangles from the brainstem and limbic regions to the neocortex. The different Braak stages classify the progression of tau pathology in the brain. In some implementations, the brain function diagnosis 522 may include a Braak stage estimate, indicating a stage of the AD. For example, the brain function diagnosis 522 may include one or more bar graphs of predicted tau levels in each Braak region (VII, III / IV, V / VI) that highlights the most likely disease stage.Attorney Docket No. CRNU.P0037WO

[0146] For example, if the subject is healthy with no significant tau, the Al model 306 would output a virtual functional brain map 520 with low tau estimates across all regions (akin to a negative tau-PET scan) and label the case as Braak 0 (no AD). In an example of the subject being in an early stage (preclinical AD), the Al model 306 may output a virtual functional brain map 520 with predicted elevated tau in Braak I / II regions (e.g., LC and entorhinal cortex ) but not in higher stages, corresponding to early AD pathology. In an example of the subject being a mild cognitive impairment (MCI) case, moderate tau levels are predicted in Braak III / IV regions (e.g., limbic regions). In an example of an AD case, high tau is predicted in Braak V / VI regions (e.g., neocortical regions). The Al model 306 may output a confidence measure or likelihood for each stage classification.

[0147] With respect to method 800 and Alzheimer’s disease, generating the brain function report 310 may include generating, based on the sensor data 170, a diagnosis of a neurodegenerative disease (e.g., brain function diagnosis 522). In some instances, that neurodegenerative disease is Alzheimer's disease. In these aspects of method 800, generating the brain function report 310 may include generating, using the Al model 306, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks. In such aspects, the sensor data 170 is indicative of activity of the locus coeruleus of the subject 702. In such aspects, the functional map 520 indicates a distribution of tau protein across a plurality of regions of the brain of the subject 702. In such aspects, the diagnosis 522 is generated based on the distribution of tau protein. In some aspects, the diagnosis 522 includes a Braak region indicating a stage of the Alzheimer's disease.Conducted Studies

[0148] The inventors have conducted multiple studies validating the predictions generated by system 10. One of these studies validated AD diagnosis with system 10, including Braak stage estimates. In the study, a total of 15 individuals participated: 10 healthy controls (HC) (cognitively normal older adults), 1 preclinical (PC) AD subject (cognitively normal but PET- positive for tau, corresponding to Braak I— II pathology), 2 mild cognitive impairment (MCI) patients (Braak III-IV tau pathology), and 2 mild AD patients (Braak V-VI pathology). All participants underwent conventional tau-PET imaging with the tracer MK-6240 as part of an ongoing clinical study, providing ground truth measures of their tau deposition. The results of the tau-PET imaging are shown in FIG. 10A.Attorney Docket No. CRNU.P0037WO

[0149] During a single session with the XR apparatus 100, each subject completed a suite of XR tasks (including visual oddball and antisaccade paradigms), while eye-tracking and EEG data were collected. The XR apparatus 100 successfully recorded high-quality signals in both the clinic and home environments (subjects even repeated tests at home without supervision, after an initial guided session), demonstrating the portability and robustness of the XR apparatus. Even before applying the Al model 306, clear group-wise differences were observable in the raw features. For example, the AD patients exhibited much smaller pupil dilation responses to novel oddball stimuli compared to controls, which is a finding consistent with LC-mediated autonomic dysfunction.

[0150] When all the extracted features were fed into a classifier using linear discriminant analysis (LDA), the resulting feature space separation was striking. As shown in the chart of FIG. 10B, the four diagnostic groups clustered distinctly with minimal overlap. The first two discriminant components captured the variance due to disease severity, effectively ordering the subjects from healthy control (HC) to preclinical to MCI to AD in the projected space. This provides a proof-of-concept that the system 10 can distinguish stages of tau pathology from non-invasive sensor data.General Computing System

[0151] FIG. 9 illustrates an example computing system 900 that may be utilized to implement one or more of the devices and / or components of the disclosed system, such as the one or more clinician devices 30 or the one or more subject devices 40. One or more components of computing system 900 may be utilized to implement similar components of XR apparatus 100 or computing system 300. For example, processor 904 of computing system 900 may implement processor 302 of computing system 300. In particular implementations, one or more computer systems 900 perform one or more steps of one or more methods described or illustrated herein. In particular implementations, one or more computer systems 900 provide the functionalities described or illustrated herein. In particular implementations, software running on one or more computer systems 900 performs one or more steps of one or more methods described or illustrated herein or provides the functionalities described or illustrated herein. Particular implementations include one or more portions of one or more computer systems 900. Herein, a reference to a computing system may encompass a computing device, and vice versa, where appropriate. Moreover, a reference to a computing system may encompass one or more computer systems, where appropriate.Attorney Docket No. CRNU.P0037WO

[0152] This disclosure contemplates any suitable number of computer systems 900. This disclosure contemplates the computing system 900 taking any suitable physical form. As example and not by way of limitation, the computing system 900 may be an embedded computer system, a system-on-chip (SOC), a single-board computing system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, the computing system 900 may include one or more computer systems 900; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 900 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example and not by way of limitation, one or more computer systems 900 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 900 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0153] In particular implementations, computing system 900 includes a processor 904, memory 902, storage 906, an input / output (I / O) interface 908, and a communication interface 910. Although this disclosure describes and illustrates a particular computing system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computing system having any suitable number of any suitable components in any suitable arrangement.

[0154] In particular implementations, the processor 904 includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, the processor 904 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 902, or storage 906; decode and execute the instructions; and then write one or more results to an internal register, internal cache, memory 902, or storage 906. In particular implementations, the processor 904 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates the processor 904 including any suitable number of any suitable internal caches, where appropriate. As an example and not by way of limitation, the processor 904 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffersAttorney Docket No. CRNU.P0037WO(TLBs). Instructions in the instruction caches may be copies of instructions in memory 902 or storage 906, and the instruction caches may speed up retrieval of those instructions by the processor 904. Data in the data caches may be copies of data in memory 902 or storage 906 that are to be operated on by computer instructions; the results of previous instructions executed by the processor 904 that are accessible to subsequent instructions or for writing to memory 902 or storage 906; or any other suitable data. The data caches may speed up read or write operations by the processor 904. The TLBs may speed up virtual-address translation for the processor 904. In particular implementations, processor 904 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates the processor 904 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, the processor 904 may include one or more arithmetic logic units (ALUs), be a multi-core processor, or include one or more processors 904. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0155] In particular implementations, the memory 902 includes main memory for storing instructions for the processor 904 to execute or data for processor 904 to operate on. As an example, and not by way of limitation, computing system 900 may load instructions from storage 906 or another source (such as another computing system 900) to the memory 902. The processor 904 may then load the instructions from the memory 902 to an internal register or internal cache. To execute the instructions, the processor 904 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, the processor 904 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. The processor 904 may then write one or more of those results to the memory 902. In particular implementations, the processor 904 executes only instructions in one or more internal registers or internal caches or in memory 902 (as opposed to storage 906 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 902 (as opposed to storage 906 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple the processor 904 to the memory 902. The bus may include one or more memory buses, as described in further detail below. In particular implementations, one or more memory management units (MMUs) reside between the processor 904 and memory 902 and facilitate accesses to the memory 902 requested by the processor 904. In particular implementations, the memory 902 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-portedAttorney Docket No. CRNU.P0037WORAM. This disclosure contemplates any suitable RAM. The memory 902 may include one or more memories 902, where appropriate. Although this disclosure describes and illustrates particular memory implementations, this disclosure contemplates any suitable memory implementation.

[0156] In particular implementations, the storage 906 includes mass storage for data or instructions. As an example and not by way of limitation, the storage 906 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The storage 906 may include removable or non-removable (or fixed) media, where appropriate. The storage 906 may be internal or external to computing system 900, where appropriate. In particular implementations, the storage 906 is non-volatile, solid-state memory. In particular implementations, the storage 906 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 906 taking any suitable physical form. The storage 906 may include one or more storage control units facilitating communication between processor 904 and storage 906, where appropriate. Where appropriate, the storage 906 may include one or more storages 906. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0157] In particular implementations, the I / O Interface 908 includes hardware, software, or both, providing one or more interfaces for communication between computing system 900 and one or more VO devices. The computing system 900 may include one or more of these I / O devices, where appropriate. One or more of these I / O devices may enable communication between a person and computing system 900. As an example and not by way of limitation, an I / O device may include a keyboard, keypad, microphone, monitor, screen, display panel, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I / O device or a combination of two or more of these. An I / O device may include one or more sensors. Where appropriate, the VO Interface 908 may include one or more device or software drivers enabling processor 904 to drive one or more of these I / O devices. The I / O interface 908 may include one or more VO interfaces 908, where appropriate. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface or combination of I / O interfaces.Attorney Docket No. CRNU.P0037WO

[0158] In particular implementations, communication interface 910 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computing system 900 and one or more other computer systems 900 or one or more communications networks 20. As an example and not by way of limitation, communication interface 910 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or any other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a Wi-Fi network. This disclosure contemplates any suitable communications network 20 and any suitable communication interface 910 for it. Computing system 900 may include any suitable communication interface 910 for any of the provided example networks for communications network 20, where appropriate. Communication interface 910 may include one or more communication interfaces 910, where appropriate. Although this disclosure describes and illustrates a particular communication interface implementations, this disclosure contemplates any suitable communication interface implementation.

[0159] The computing system 900 may also include a bus. The bus may include hardware, software, or both and may communicatively couple the components of the computing system 900 to each other. As an example and not by way of limitation, the bus may include an Accelerated Graphics Port (AGP) or any other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. The bus may include one or more buses, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0160] Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other types of integrated circuits (ICs) (e.g., field- programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or moreAttorney Docket No. CRNU.P0037WO of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.Example Aspects

[0161] It is noted that the order of one or more blocks (or operations) described with reference to FIGs. 1-3, 5, or 7-9 may be changed, certain blocks may be combined with other blocks, additional blocks may be added, and some of the block may be omitted. It is also noted that one or more blocks (or operations) described with reference to FIGs. 1-3, 5, or 7-9 may be combined with one or more blocks (or operations) described with reference to another of the figures. For example, one or more blocks (or operations) of FIG. 1 may be combined with one or more blocks (or operations) of FIGs. 2 or 3. As another example, one or more blocks associated with FIG. 9 may be combined with one or more blocks associated with FIGs. 2 or 3.

[0162] In one or more aspects, the present system may include additional aspects, such as any single aspect or any combination of aspects described below or in connection with one or more other processes or devices described elsewhere herein. In a first aspect, a system includes a memory storing processor-readable code, and at least one processor coupled to the memory. The at least one processor is configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving sensor data from an extended reality (XR) apparatus, and generating, using an artificial intelligence (Al) model, a functional map of a brain of a subject based on the sensor data. The sensor data includes eye-tracking data and is collected by the XR apparatus while the subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus. The one or more tasks stimulate a plurality of neurotransmitters of the subject. The functional map indicates neurological function of a plurality of regions of the brain.

[0163] In a second aspect, in combination with the first aspect, the functional map comprises a plurality of pixels, and each pixel of the plurality of pixels represents brain functional engagement or impairment.

[0164] In a third aspect, in combination with one or more of the first aspect or the second aspect, the XR apparatus is worn by the subject during presentation and performance of the one or more tasks.

[0165] In a fourth aspect, in combination with one or more of the first aspect through the third aspect, the XR apparatus includes at least one infrared camera that collects the eye-tracking data.Attorney Docket No. CRNU.P0037WO

[0166] In a fifth aspect, in combination with one or more of the first aspect through the fourth aspect, the XR apparatus is a virtual reality apparatus, an augmented reality apparatus, or a mixed reality apparatus.

[0167] In a sixth aspect, in combination with one or more of the first aspect through the fifth aspect, the sensor data includes electroencephalography data of the subject.

[0168] In a seventh aspect, in combination with one or more of the first aspect through the sixth aspect, the sensor data includes electrocardiogram data of the subject.

[0169] In an eighth aspect, in combination with one or more of the first aspect through the seventh aspect, the sensor data includes motion data associated with a handheld controller in communication with the XR apparatus.

[0170] In a ninth aspect, in combination with one or more of the first aspect through the eighth aspect, the XR apparatus includes a plurality of electrodes.

[0171] In a tenth aspect, in combination with one or more of the first aspect through the ninth aspect, the eye-tracking data includes pupil size, eye position coordinates, eye rotation angles, or blinking.

[0172] In an eleventh aspect, in combination with one or more of the first aspect through the tenth aspect, at least one task of the one or more tasks activate a single, distinct brain circuit of the subject.

[0173] In a twelfth aspect, in combination with one or more of the first aspect through the eleventh aspect, at least one task of the one or more tasks activate a plurality of brain circuits of the subject.

[0174] In a thirteenth aspect, in combination with one or more of the first aspect through the twelfth aspect, the one or more tasks include an oddball task, an anti-saccade task, or a smooth pursuit task.

[0175] In a fourteenth aspect, in combination with the thirteenth aspect, the one or more tasks include a pro-saccade task, an emotional reactivity and regulation task, a procedural learning and short / long-term memory task, an executive function task, an attention and inhibition task, a language processing task, a memory recall task, or a sensorimotor task.

[0176] In a fifteenth aspect, in combination with one or more of the first aspect through the fourteenth aspect, the sensor data collection is time-synchronized with performance of the one or more tasks by the subject.

[0177] In a sixteenth aspect, in combination with one or more of the first aspect through the fifteenth aspect, the system comprises the XR apparatus, which includes one or more sensors, a second memory storing processor-readable code, and at least one second processor coupledAttorney Docket No. CRNU.P0037WO to the second memory. The at least one second processor is configured to execute the processor- readable code to cause the at least one second processor to perform second operations including: presenting the one or more tasks in the virtual environment to the subject; detecting, via the one or more sensors, sensor data while the subject performs the one or more tasks; and transmitting the sensor data to the processor.

[0178] In a seventeenth aspect, in combination with the sixteenth aspect, the second operations including applying electrical neurostimulation to the subject.

[0179] In an eighteenth aspect, in combination with one or more of the first aspect through the seventeenth aspect, the operations include extracting a plurality of features from the sensor data, and the functional map is generated based on the plurality of features. In the eighteenth aspect, the plurality of features include two or more: a time from stimulus onset to saccade initiation, a distance covered by a saccade, a maximum speed of a saccade, a time duration of a saccade, a percentage of trials where initial saccade is toward the stimulus, a quantity of saccades occurring before stimulus presentation or instruction, a percentage of trials with multiple step saccades (MSS), a number of steps within an MSS sequence, a time between steps in an MSS sequence, a duration from a first saccade to a final fixation on the target, a number of square wave jerks (SWJ) per minute during fixation tasks, an angular displacement per jerk, a time duration between onset and return of the jerk, a count of mini fixations before reaching the target, an average duration of fixations during task, a change in pupil diameter to cognitive stimuli, or an average pupil size at rest.

[0180] In a nineteenth aspect, in combination with one or more of the first aspect through the eighteenth aspect, the operations include: generating, based on the sensor data, instructions indicating that a task of the one or more tasks performed by the subject needs to be repeated before generating the functional map.

[0181] In a twentieth aspect, in combination with the nineteenth aspect, the generating that the task needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data.

[0182] In a twenty-first aspect, in combination with one or more of the nineteenth aspect through the twentieth aspect, the generating that the task needs to be repeated is performed in real-time.

[0183] In a twenty-second aspect, in combination with one or more of the nineteenth aspect through the twenty-first aspect, the operations include: controlling the XR apparatus to present the task that needs to be repeated.Attorney Docket No. CRNU.P0037WO

[0184] In a twenty-third aspect, in combination with one or more of the first aspect through the twenty-second aspect, the operations include: generating, using the Al model, that one or more additional tasks need to be performed by the subject before generating the functional map.

[0185] In a twenty -fourth aspect, in combination with the twenty -third aspect, generating that the one or more additional tasks need to be performed by the subject is performed in real-time.

[0186] In a twenty-fifth aspect, in combination with one or more of the twenty-third aspect through the twenty-fourth aspect, the operations include: controlling the XR apparatus to present the one or more additional tasks.

[0187] In a twenty-sixth aspect, in combination with one or more of the first aspect through the twenty-fifth aspect, the operations include: generating, using the Al model and based on the sensor data, one or more neurological impairments of the subject.

[0188] In a twenty-seventh aspect, in combination with the twenty-sixth aspect, the operations include: generating, using the Al model, a treatment plan for the one or more neurological impairments.

[0189] In a twenty-eighth aspect, in combination with one or more of the first aspect through the twenty- seventh aspect, the operations include: repeating the receiving sensor data and the generating the functional map a plurality of instances over a period of time so as to generate a plurality of functional maps; and generating, using the Al model and based on the plurality of functional maps, a digital model replicating the brain of the subject.

[0190] In a twenty -ninth aspect, in combination with the twenty-eighth aspect, the operations include: generating, using the Al model, a predicted progression of one or more neurological impairments based on the digital model.

[0191] In a thirtieth aspect, in combination with one or more of the twenty-eighth aspect through the twenty-ninth aspect, the operations include: updating the digital model at a time after the digital model is generated.

[0192] In a thirty-first aspect, in combination with one or more of the twenty-eighth aspect through the thirtieth aspect, the operations include: storing, in a database in the memory, each functional map of the plurality of functional maps when each functional map is generated over the period of time.

[0193] In a thirty-second aspect, in combination with one or more of the first aspect through the thirty-first aspect, the operations include: generating, using the Al model and based on the sensor data, a diagnosis of a neurodegenerative disorder.Attorney Docket No. CRNU.P0037WO

[0194] In a thirty-third aspect, in combination with the thirty-second aspect, the neurodegenerative disorder is Alzheimer's disease.

[0195] In a thirty-fourth aspect, in combination with the thirty-third aspect, the operations include: generating, using the Al model, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks.

[0196] In a thirty-fifth aspect, in combination with one or more of the thirty-third aspect through the thirty-fourth aspect, the sensor data is indicative of activity of the locus coeruleus of the subject.

[0197] In a thirty-sixth aspect, in combination with one or more of the thirty-third aspect through the thirty-fifth aspect, the functional map indicates a distribution of tau protein across a plurality of regions of the brain of the subject.

[0198] In a thirty-seventh aspect, in combination with the thirty-sixth aspect, the diagnosis is generated based on the distribution of tau protein.

[0199] In a thirty-eighth aspect, in combination with one or more of the thirty-third aspect through the thirty-seventh aspect, the diagnosis includes a Braak region indicating a stage of the Alzheimer's disease.

[0200] In a thirty-ninth aspect, in combination with one or more of the first aspect through the thirty-eighth aspect, the Al model includes a plurality of encoders that are trained such that each encoder of the plurality of encoders is associated with a different, respective task of the one or more tasks.

[0201] In a fortieth aspect, in combination with the thirty-ninth aspect, the operations include: generating, using the Al model, information indicative of the performance of the subject on a task of the one or more tasks based on the sensor data and the respective encoder that is associated with the task.

[0202] In a forty-first aspect, in combination with one or more of the first aspect through the fortieth aspect, the diagnosis is generated without the use of a radioactive tracer and without the use of a positron emission tomography (PET) scanner.

[0203] In a forty-second aspect, in combination with one or more of the first aspect through the forty-first aspect, the eye-tracking data is collected by the XR apparatus with a sampling frequency of at least 60 Hertz (Hz).

[0204] In a forty -third aspect, in combination with one or more of the second aspect through the fifteenth aspect or the eighteenth aspect through the forty-second aspect, a systemAttorney Docket No. CRNU.P0037WO comprises an extended reality (XR) apparatus and a computing device. The XR apparatus comprises: one or more sensors including an eye-tracking sensor; a first memory storing processor-readable code; and at least one first processor coupled to the first memory. The at least one first processor is configured to execute the processor-readable code to cause the at least one first processor to perform operations including: presenting one or more tasks in a virtual environment to a subject, the one or more tasks stimulating a plurality of neurotransmitters of the subject; and detecting, via the one or more sensors, sensor data while the subject performs the one or more tasks, the sensor data including eye-tracking data. The computing device comprises: a second memory storing processor-readable code; and at least one second processor coupled to the second memory. The at least one second processor is configured to execute the processor-readable code to cause the at least one second processor to perform operations including: receiving the sensor data from the XR apparatus; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

[0205] The above specification and examples provide a complete description of the structure and use of illustrative implementations. Although certain implementations have been described above with a certain degree of particularity, or with reference to one or more individual implementations, those skilled in the art could make numerous alterations to the disclosed implementations without departing from the scope of this invention. As such, the various illustrative implementations of the products, systems, and methods are not intended to be limited to the particular forms disclosed. Rather, they include all modifications and alternatives falling within the scope of the claims, and implementations other than the one shown may include some or all of the features of the depicted embodiment. For example, elements may be omitted or combined as a unitary structure, and / or connections may be substituted. Further, where appropriate, aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples having comparable or different properties and / or functions, and addressing the same or different problems. Similarly, it will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several implementations.

[0206] The claims are not intended to include, and should not be interpreted to include, means-plus- or step-plus-function limitations, unless such a limitation is explicitly recited in a given claim using the phrase(s) “means for” or “step for,” respectively.

Claims

Attorney Docket No. CRNU.P0037WOCLAIMSThe invention is claimed as follows:

1. A system comprising: a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving sensor data from an extended reality (XR) apparatus, the sensor data collected by the XR apparatus while a subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus, the sensor data including eye-tracking data, and the one or more tasks stimulating a plurality of neurotransmitters of the subject; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

2. The system of claim 1, wherein the functional map comprises a plurality of pixels, and wherein each pixel of the plurality of pixels represents brain functional engagement or impairment.

3. The system of any one of claims 1 to 2, wherein the XR apparatus is worn by the subject during presentation and performance of the one or more tasks.

4. The system of any one of claims 1 to 3, wherein the XR apparatus includes at least one infrared camera that collects the eye-tracking data.

5. The system of any one of claims 1 to 4, wherein the XR apparatus is a virtual reality apparatus, an augmented reality apparatus, or a mixed reality apparatus.

6. The system of any one of claims 1 to 5, wherein the sensor data includes electroencephalography data of the subject.Attorney Docket No. CRNU.P0037WO7. The system of any one of claims 1 to 6, wherein the sensor data includes electrocardiogram data of the subject.

8. The system of any one of claims 1 to 7, wherein the sensor data includes motion data associated with a handheld controller in communication with the XR apparatus.

9. The system of any one of claims 1 to 8, wherein the XR apparatus includes a plurality of electrodes.

10. The system of any one of claims 1 to 9, wherein the eye-tracking data includes pupil size, eye position coordinates, eye rotation angles, or blinking.

11. The system of any one of claims 1 to 10, wherein at least one task of the one or more tasks activate a single, distinct brain circuit of the subject.

12. The system of any one of claims 1 to 11, wherein at least one task of the one or more tasks activate a plurality of brain circuits of the subject.

13. The system of any one of claims 1 to 12, wherein the one or more tasks include an oddball task, an anti-saccade task, or a smooth pursuit task.

14. The system of claim 13, wherein the one or more tasks include a pro-saccade task, an emotional reactivity and regulation task, a procedural learning and short / 1 ong-term memory task, an executive function task, an attention and inhibition task, a language processing task, a memory recall task, or a sensorimotor task.

15. The system of any one of claims 1 to 14, wherein the sensor data collection is time-synchronized with performance of the one or more tasks by the subject.

16. The system of any one of claims 1 to 15, comprising the XR apparatus, wherein the XR apparatus comprises: one or more sensors; a second memory storing processor-readable code; andAttorney Docket No. CRNU.P0037WO at least one second processor coupled to the second memory, the at least one second processor configured to execute the processor-readable code to cause the at least one second processor to perform second operations including: presenting the one or more tasks in the virtual environment to the subject; detecting, via the one or more sensors, sensor data while the subject performs the one or more tasks; and transmitting the sensor data to the processor.

17. The system of claim 16, the second operations including applying electrical neurostimulation to the subject.

18. The system of any one of claims 1 to 17, wherein the operations include: extracting a plurality of features from the sensor data, the functional map generated based on the plurality of features, wherein the plurality of features include two or more: a time from stimulus onset to saccade initiation, a distance covered by a saccade, a maximum speed of a saccade, a time duration of a saccade, a percentage of trials where initial saccade is toward the stimulus, a quantity of saccades occurring before stimulus presentation or instruction, a percentage of trials with multiple step saccades (MSS), a number of steps within an MSS sequence, a time between steps in an MSS sequence, a duration from a first saccade to a final fixation on a target, a number of square wave jerks (SWJ) per minute during fixation tasks, an angular displacement per jerk, a time duration between onset and return of the jerk, a count of mini fixations before reaching a target, an average duration of fixations during task, a change in pupil diameter to cognitive stimuli, or an average pupil size at rest.

19. The system of any one of claims 1 to 18, wherein the operations include: generating, based on the sensor data, instructions indicating that a task of the one or more tasks performed by the subject needs to be repeated before generating the functional map.

20. The system of claim 19, wherein the generating that the task needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data.

21. The system of claim 19, wherein the generating that the task needs to be repeated is performed in real-time.Attorney Docket No. CRNU.P0037WO22. The system of claim 19, wherein the operations include: controlling the XR apparatus to present the task that needs to be repeated.

23. The system of any one of claims 1 to 22, wherein the operations include: generating, using the Al model, that one or more additional tasks need to be performed by the subject before generating the functional map.

24. The system of claim 23, wherein the generating that the one or more additional tasks need to be performed by the subject is performed in real-time.

25. The system of claim 23, wherein the operations include: controlling the XR apparatus to present the one or more additional tasks.

26. The system of any one of claims 1 to 25, wherein the operations include: generating, using the Al model and based on the sensor data, one or more neurological impairments of the subject.

27. The system of claim 26, wherein the operations include: generating, using the Al model, a treatment plan for the one or more neurological impairments.

28. The system of any one of claims 1 to 27, wherein the operations include: repeating the receiving sensor data and the generating the functional map a plurality of instances over a period of time so as to generate a plurality of functional maps; and generating, using the Al model and based on the plurality of functional maps, a digital model replicating the brain of the subject.

29. The system of claim 28, wherein the operations include: generating, using the Al model, a predicted progression of one or more neurological impairments based on the digital model.

30. The system of claim 28, wherein the operations include: updating the digital model at a time after the digital model is generated.Attorney Docket No. CRNU.P0037WO31. The system of claim 28, wherein the operations include: storing, in a database in the memory, each functional map of the plurality of functional maps when each functional map is generated over the period of time.

32. The system of any one of claims 1 to 31, wherein the operations include: generating, using the Al model and based on the sensor data, a diagnosis of a brain disorder.

33. The system of claim 32, wherein the brain disorder is Alzheimer's disease.

34. The system of claim 33, wherein the operations include: generating, using the Al model, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks.

35. The system of claim 33, wherein the sensor data is indicative of activity of the locus coeruleus of the subject.

36. The system of claim 33, wherein the functional map indicates a distribution of tau protein across a plurality of regions of the brain of the subject.

37. The system of claim 36, wherein the diagnosis is generated based on the distribution of tau protein.

38. The system of claim 33, wherein the diagnosis includes a Braak region indicating a stage of the Alzheimer's disease.

39. The system of any one of claims 1 to 38, wherein the Al model includes a plurality of encoders that are trained such that each encoder of the plurality of encoders is associated with a different, respective task of the one or more tasks.

40. The system of claim 39, wherein the operations include:Attorney Docket No. CRNU.P0037WO generating, using the Al model, information indicative of the performance of the subject on a task of the one or more tasks based on the sensor data and the respective encoder that is associated with the task.

41. The system of any one of claims 1 to 40, wherein the functional brain map is generated without the use of a radioactive tracer and without the use of a positron emission tomography (PET) scanner.

42. The system of any one of claims 1 to 41, wherein the eye-tracking data is collected by the XR apparatus with a sampling frequency of at least 60 Hertz (Hz).

43. A method comprising: receiving sensor data from an extended reality (XR) apparatus, the sensor data collected by the XR apparatus while a subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus, the sensor data including eye-tracking data, and the one or more tasks stimulating a plurality of neurotransmitters of the subject; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

44. The method of claim 43, wherein the functional map comprises a plurality of pixels, and wherein each pixel of the plurality of pixels represents brain functional engagement or impairment.

45. The method of any one of claims 43 to 44, wherein the XR apparatus is worn by the subject during presentation and performance of the one or more tasks.

46. The method of any one of claims 43 to 45, wherein the XR apparatus includes at least one infrared camera that collects the eye-tracking data.

47. The method of any one of claims 43 to 46, wherein the XR apparatus is a virtual reality apparatus, an augmented reality apparatus, or a mixed reality apparatus.Attorney Docket No. CRNU.P0037WO48. The method of any one of claims 43 to 47, wherein the sensor data includes electroencephalography data of the subject.

49. The method of any one of claims 43 to 48, wherein the sensor data includes electrocardiogram data of the subject.

50. The method of any one of claims 43 to 49, wherein the sensor data includes motion data associated with a handheld controller in communication with the XR apparatus.

51. The method of any one of claims 43 to 50, wherein the XR apparatus includes a plurality of electrodes.

52. The method of any one of claims 43 to 51, wherein the eye-tracking data includes pupil size, eye position coordinates, eye rotation angles, or blinking.

53. The method of any one of claims 43 to 52, wherein at least one task of the one or more tasks activate a single, distinct brain circuit of the subject.

54. The method of any one of claims 43 to 53, wherein at least one task of the one or more tasks activate a plurality of brain circuits of the subject.

55. The method of any one of claims 43 to 54, wherein the one or more tasks include an oddball task, an anti-saccade task, or a smooth pursuit task.

56. The method of claim 55, wherein the one or more tasks include a pro-saccade task, an emotional reactivity and regulation task, a procedural learning and short / 1 ong-term memory task, an executive function task, an attention and inhibition task, a language processing task, a memory recall task, or a sensorimotor task.

57. The method of any one of claims 43 to 56, wherein the sensor data collection is time-synchronized with performance of the one or more tasks by the subject.

58. The method of any one of claims 43 to 57, comprising:Attorney Docket No. CRNU.P0037WO presenting, by the XR apparatus, the one or more tasks in the virtual environment to the subject; detecting, via one or more sensors of the XR apparatus, sensor data while the subject performs the one or more tasks; and transmitting the sensor data to the processor.

59. The method of claim 58, comprising applying, by the XR apparatus, electrical neurostimulation to the subject.

60. The method of any one of claims 43 to 59, comprising: extracting a plurality of features from the sensor data, the functional map generated based on the plurality of features, wherein the plurality of features include two or more: a time from stimulus onset to saccade initiation, a distance covered by a saccade, a maximum speed of a saccade, a time duration of a saccade, a percentage of trials where initial saccade is toward the stimulus, a quantity of saccades occurring before stimulus presentation or instruction, a percentage of trials with multiple step saccades (MSS), a number of steps within an MSS sequence, a time between steps in an MSS sequence, a duration from a first saccade to a final fixation on a target, a number of square wave jerks (SWJ) per minute during fixation tasks, an angular displacement per jerk, a time duration between onset and return of the jerk, a count of mini fixations before reaching a target, an average duration of fixations during task, a change in pupil diameter to cognitive stimuli, or an average pupil size at rest.

61. The method of any one of claims 43 to 60, comprising: generating, based on the sensor data, instructions indicating that a task of the one or more tasks performed by the subject needs to be repeated before generating the functional map.

62. The method of claim 61, wherein the generating that the task needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data.

63. The method of claim 61, wherein the generating that the task needs to be repeated is performed in real-time.Attorney Docket No. CRNU.P0037WO64. The method of claim 61, comprising: controlling the XR apparatus to present the task that needs to be repeated.

65. The method of any one of claims 43 to 64, comprising: generating, using the Al model, that one or more additional tasks need to be performed by the subject before generating the functional map.

66. The method of claim 65, wherein the generating that the one or more additional tasks need to be performed by the subject is performed in real-time.

67. The method of claim 65, comprising: controlling the XR apparatus to present the one or more additional tasks.

68. The method of any one of claims 43 to 67, comprising: generating, using the Al model and based on the sensor data, one or more neurological impairments of the subject.

69. The method of claim 68, comprising: generating, using the Al model, a treatment plan for the one or more neurological impairments.

70. The method of any one of claims 43 to 69, comprising: repeating the receiving sensor data and the generating the functional map a plurality of instances over a period of time so as to generate a plurality of functional maps; and generating, using the Al model and based on the plurality of functional maps, a digital model replicating the brain of the subject.

71. The method of claim 70, comprising: generating, using the Al model, a predicted progression of one or more neurological impairments based on the digital model.

72. The method of claim 70, comprising: updating the digital model at a time after the digital model is generated.Attorney Docket No. CRNU.P0037WO73. The method of claim 70, comprising: storing, in a database in the memory, each functional map of the plurality of functional maps when each functional map is generated over the period of time.

74. The method of any one of claims 43 to 73, comprising: generating, using the Al model and based on the sensor data, a diagnosis of a brain disorder.

75. The method of claim 74, wherein the brain disorder is Alzheimer's disease.

76. The method of claim 75, comprising: generating, using the Al model, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks.

77. The method of claim 75, wherein the sensor data is indicative of activity of the locus coeruleus of the subject.

78. The method of claim 75, wherein the functional map indicates a distribution of tau protein across a plurality of regions of the brain of the subject.

79. The method of claim 78, wherein the diagnosis is generated based on the distribution of tau protein.

80. The method of claim 75, wherein the diagnosis includes a Braak region indicating a stage of the Alzheimer's disease.

81. The method of any one of claims 43 to 80, wherein the Al model includes a plurality of encoders that are trained such that each encoder of the plurality of encoders is associated with a different, respective task of the one or more tasks.

82. The method of claim 81, comprising:Attorney Docket No. CRNU.P0037WO generating, using the Al model, information indicative of the performance of the subject on a task of the one or more tasks based on the sensor data and the respective encoder that is associated with the task.

83. The method of any one of claims 43 to 82, wherein the functional brain map is generated without the use of a radioactive tracer and without the use of a positron emission tomography (PET) scanner.

84. The method of any one of claims 43 to 83, wherein the eye-tracking data is collected by the XR apparatus with a sampling frequency of at least 60 Hertz (Hz).

85. A non-transitory, computer-readable medium storing processor-executable code, which when executed by one or more processors, causes the one or more processors to perform operations comprising: receiving sensor data from an extended reality (XR) apparatus, the sensor data collected by the XR apparatus while a subject performed one or more tasks presented to the subject in a virtual environment by the XR apparatus, the sensor data including eye-tracking data, and the one or more tasks stimulating a plurality of neurotransmitters of the subject; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

86. The non-transitory, computer-readable medium of claim 85, wherein the functional map comprises a plurality of pixels, and wherein each pixel of the plurality of pixels represents brain functional engagement or impairment.

87. The non-transitory, computer-readable medium of any one of claims 85 to 86, wherein the XR apparatus is worn by the subject during presentation and performance of the one or more tasks.

88. The non-transitory, computer-readable medium of any one of claims 85 to 87, wherein the XR apparatus includes at least one infrared camera that collects the eye-tracking data.Attorney Docket No. CRNU.P0037WO89. The non-transitory, computer-readable medium of any one of claims 85 to 88, wherein the XR apparatus is a virtual reality apparatus, an augmented reality apparatus, or a mixed reality apparatus.

90. The non-transitory, computer-readable medium of any one of claims 85 to 89, wherein the sensor data includes electroencephalography data of the subject.

91. The non-transitory, computer-readable medium of any one of claims 85 to 90, wherein the sensor data includes electrocardiogram data of the subject.

92. The non-transitory, computer-readable medium of any one of claims 85 to 91, wherein the sensor data includes motion data associated with a handheld controller in communication with the XR apparatus.

93. The non-transitory, computer-readable medium of any one of claims 85 to 92, wherein the XR apparatus includes a plurality of electrodes.

94. The non-transitory, computer-readable medium of any one of claims 85 to 93, wherein the eye-tracking data includes pupil size, eye position coordinates, eye rotation angles, or blinking.

95. The non-transitory, computer-readable medium of any one of claims 85 to 94, wherein at least one task of the one or more tasks activate a single, distinct brain circuit of the subject.

96. The non-transitory, computer-readable medium of any one of claims 85 to 95, wherein at least one task of the one or more tasks activate a plurality of brain circuits of the subject.

97. The non-transitory, computer-readable medium of any one of claims 85 to 96, wherein the one or more tasks include an oddball task, an anti-saccade task, or a smooth pursuit task.Attorney Docket No. CRNU.P0037WO98. The non-transitory, computer-readable medium of claim 97, wherein the one or more tasks include a pro-saccade task, an emotional reactivity and regulation task, a procedural learning and short / 1 ong-term memory task, an executive function task, an attention and inhibition task, a language processing task, a memory recall task, or a sensorimotor task.

99. The non-transitory, computer-readable medium of any one of claims 85 to 98, wherein the sensor data collection is time-synchronized with performance of the one or more tasks by the subject.

100. The non-transitory, computer-readable medium of any one of claims 85 to 99, wherein the operations include: extracting a plurality of features from the sensor data, the functional map generated based on the plurality of features, wherein the plurality of features include two or more: a time from stimulus onset to saccade initiation, a distance covered by a saccade, a maximum speed of a saccade, a time duration of a saccade, a percentage of trials where initial saccade is toward the stimulus, a quantity of saccades occurring before stimulus presentation or instruction, a percentage of trials with multiple step saccades (MSS), a number of steps within an MSS sequence, a time between steps in an MSS sequence, a duration from a first saccade to a final fixation on a target, a number of square wave jerks (SWJ) per minute during fixation tasks, an angular displacement per jerk, a time duration between onset and return of the jerk, a count of mini fixations before reaching a target, an average duration of fixations during task, a change in pupil diameter to cognitive stimuli, or an average pupil size at rest.

101. The non-transitory, computer-readable medium of any one of claims 85 to 100, wherein the operations include: generating, based on the sensor data, instructions indicating that a task of the one or more tasks performed by the subject needs to be repeated before generating the functional map.

102. The non-transitory, computer-readable medium of claim 101, wherein the generating that the task needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data.Attorney Docket No. CRNU.P0037WO103. The non-transitory, computer-readable medium of claim 101, wherein the generating that the task needs to be repeated is performed in real-time.

104. The non-transitory, computer-readable medium of claim 101, wherein the operations include: controlling the XR apparatus to present the task that needs to be repeated.

105. The non-transitory, computer-readable medium of any one of claims 85 to 104, wherein the operations include: generating, using the Al model, that one or more additional tasks need to be performed by the subject before generating the functional map.

106. The non-transitory, computer-readable medium of claim 105, wherein the generating that the one or more additional tasks need to be performed by the subject is performed in real-time.

107. The non-transitory, computer-readable medium of claim 105, wherein the operations include: controlling the XR apparatus to present the one or more additional tasks.

108. The non-transitory, computer-readable medium of any one of claims 85 to 107, wherein the operations include: generating, using the Al model and based on the sensor data, one or more neurological impairments of the subject.

109. The non-transitory, computer-readable medium of claim 108, wherein the operations include: generating, using the Al model, a treatment plan for the one or more neurological impairments.

110. The non-transitory, computer-readable medium of any one of claims 85 to 109, wherein the operations include: repeating the receiving sensor data and the generating the functional map a plurality of instances over a period of time so as to generate a plurality of functional maps; andAttorney Docket No. CRNU.P0037WO generating, using the Al model and based on the plurality of functional maps, a digital model replicating the brain of the subject.

111. The non-transitory, computer-readable medium of claim 110, wherein the operations include: generating, using the Al model, a predicted progression of one or more neurological impairments based on the digital model.

112. The non-transitory, computer-readable medium of claim 110, wherein the operations include: updating the digital model at a time after the digital model is generated.

113. The non-transitory, computer-readable medium of claim 110, wherein the operations include: storing, in a database, each functional map of the plurality of functional maps when each functional map is generated over the period of time.

114. The non-transitory, computer-readable medium of any one of claims 85 to 113, wherein the operations include: generating, using the Al model and based on the sensor data, a diagnosis of a brain disorder.

115. The non-transitory, computer-readable medium of claim 114, wherein the brain disorder is Alzheimer's disease.

116. The non-transitory, computer-readable medium of claim 115, wherein the operations include: generating, using the Al model, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks.

117. The non-transitory, computer-readable medium of claim 115, wherein the sensor data is indicative of activity of the locus coeruleus of the subject.Attorney Docket No. CRNU.P0037WO118. The non-transitory, computer-readable medium of claim 115, wherein the functional map indicates a distribution of tau protein across a plurality of regions of the brain of the subject.

119. The non-transitory, computer-readable medium of claim 118, wherein the diagnosis is generated based on the distribution of tau protein.

120. The non-transitory, computer-readable medium of claim 115, wherein the diagnosis includes a Braak region indicating a stage of the Alzheimer's disease.

121. The non-transitory, computer-readable medium of any one of claims 85 to 120, wherein the Al model includes a plurality of encoders that are trained such that each encoder of the plurality of encoders is associated with a different, respective task of the one or more tasks.

122. The non-transitory, computer-readable medium of claim 121, wherein the operations include: generating, using the Al model, information indicative of the performance of the subject on a task of the one or more tasks based on the sensor data and the respective encoder that is associated with the task.

123. The non-transitory, computer-readable medium of any one of claims 85 to 122, wherein the functional brain map is generated without the use of a radioactive tracer and without the use of a positron emission tomography (PET) scanner.

124. The non-transitory, computer-readable medium of any one of claims 85 to 123, wherein the eye-tracking data is collected by the XR apparatus with a sampling frequency of at least 60 Hertz (Hz).

125. A system comprising: an extended reality (XR) apparatus comprising: one or more sensors including an eye-tracking sensor; a first memory storing processor-readable code; andAttorney Docket No. CRNU.P0037WO at least one first processor coupled to the first memory, the at least one first processor configured to execute the processor-readable code to cause the at least one first processor to perform first operations including: presenting one or more tasks in a virtual environment to a subject, the one or more tasks stimulating a plurality of neurotransmitters of the subject; and detecting, via the one or more sensors, sensor data while the subject performs the one or more tasks, the sensor data including eye-tracking data; and a computing device comprising: a second memory storing processor-readable code; and at least one second processor coupled to the second memory, the at least one second processor configured to execute the processor-readable code to cause the at least one second processor to perform second operations including: receiving the sensor data from the XR apparatus; and generating, using an artificial intelligence (Al) model, a functional map of a brain of the subject based on the sensor data, the functional map indicating neurological function of a plurality of regions of the brain.

126. The system of claim 125, wherein the functional map comprises a plurality of pixels, and wherein each pixel of the plurality of pixels represents brain functional engagement or impairment.

127. The system of any one of claims 125 to 126, wherein the XR apparatus is worn by the subject during presentation and performance of the one or more tasks.

128. The system of any one of claims 125 to 127, wherein the XR apparatus includes at least one infrared camera that collects the eye-tracking data.

129. The system of any one of claims 125 to 128, wherein the XR apparatus is a virtual reality apparatus, an augmented reality apparatus, or a mixed reality apparatus.

130. The system of any one of claims 125 to 129, wherein the sensor data includes electroencephalography data of the subject.Attorney Docket No. CRNU.P0037WO131. The system of any one of claims 125 to 130, wherein the sensor data includes electrocardiogram data of the subject.

132. The system of any one of claims 125 to 131, wherein the sensor data includes motion data associated with a handheld controller in communication with the XR apparatus.

133. The system of any one of claims 125 to 132, wherein the XR apparatus includes a plurality of electrodes.

134. The system of any one of claims 125 to 133, wherein the eye-tracking data includes pupil size, eye position coordinates, eye rotation angles, or blinking.

135. The system of any one of claims 125 to 134, wherein at least one task of the one or more tasks activate a single, distinct brain circuit of the subject.

136. The system of any one of claims 125 to 135, wherein at least one task of the one or more tasks activate a plurality of brain circuits of the subject.

137. The system of any one of claims 125 to 136, wherein the one or more tasks include an oddball task, an anti-saccade task, or a smooth pursuit task.

138. The system of claim 137, wherein the one or more tasks include a pro-saccade task, an emotional reactivity and regulation task, a procedural learning and short / 1 ong-term memory task, an executive function task, an attention and inhibition task, a language processing task, a memory recall task, or a sensorimotor task.

139. The system of any one of claims 125 to 138, wherein the sensor data collection is time-synchronized with performance of the one or more tasks by the subject.

140. The system of claim any one of claims 125 to 139, the first operations including applying electrical neurostimulation to the subject.

141. The system of any one of claims 125 to 140, wherein the second operations include:Attorney Docket No. CRNU.P0037WO extracting a plurality of features from the sensor data, the functional map generated based on the plurality of features, wherein the plurality of features include two or more: a time from stimulus onset to saccade initiation, a distance covered by a saccade, a maximum speed of a saccade, a time duration of a saccade, a percentage of trials where initial saccade is toward the stimulus, a quantity of saccades occurring before stimulus presentation or instruction, a percentage of trials with multiple step saccades (MSS), a number of steps within an MSS sequence, a time between steps in an MSS sequence, a duration from a first saccade to a final fixation on a target, a number of square wave jerks (SWJ) per minute during fixation tasks, an angular displacement per jerk, a time duration between onset and return of the jerk, a count of mini fixations before reaching a target, an average duration of fixations during task, a change in pupil diameter to cognitive stimuli, or an average pupil size at rest.

142. The system of any one of claims 125 to 141, wherein the second operations include: generating, based on the sensor data, instructions indicating that a task of the one or more tasks performed by the subject needs to be repeated before generating the functional map.

143. The system of claim 142, wherein the generating that the task needs to be repeated is based on detecting lost pupil tracking and eye blinks in the sensor data.

144. The system of claim 142, wherein the generating that the task needs to be repeated is performed in real-time.

145. The system of claim 142, wherein the second operations include: controlling the XR apparatus to present the task that needs to be repeated.

146. The system of any one of claims 125 to 145, wherein the second operations include: generating, using the Al model, that one or more additional tasks need to be performed by the subject before generating the functional map.

147. The system of claim 146, wherein the generating that the one or more additional tasks need to be performed by the subject is performed in real-time.Attorney Docket No. CRNU.P0037WO148. The system of claim 146, wherein the second operations include: controlling the XR apparatus to present the one or more additional tasks.

149. The system of any one of claims 125 to 148, wherein the second operations include: generating, using the Al model and based on the sensor data, one or more neurological impairments of the subject.

150. The system of claim 149, wherein the second operations include: generating, using the Al model, a treatment plan for the one or more neurological impairments.

151. The system of any one of claims 125 to 150, wherein the second operations include: repeating the receiving sensor data and the generating the functional map a plurality of instances over a period of time so as to generate a plurality of functional maps; and generating, using the Al model and based on the plurality of functional maps, a digital model replicating the brain of the subject.

152. The system of claim 151, wherein the second operations include: generating, using the Al model, a predicted progression of one or more neurological impairments based on the digital model.

153. The system of claim 151, wherein the second operations include: updating the digital model at a time after the digital model is generated.

154. The system of claim 151, wherein the second operations include: storing, in a database in the memory, each functional map of the plurality of functional maps when each functional map is generated over the period of time.

155. The system of any one of claims 125 to 154, wherein the second operations include:Attorney Docket No. CRNU.P0037WO generating, using the Al model and based on the sensor data, a diagnosis of a brain disorder.

156. The system of claim 155, wherein the brain disorder is Alzheimer's disease.

157. The system of claim 156, wherein the second operations include: generating, using the Al model, one or more neurological impairments of the subject, the one or more neurological impairments including impairments in: pupil reflex, responses to an oddball task of the one or more tasks, responses to an anti-saccade task of the one or more tasks, or responses to a pro-saccade task of the one or more tasks.

158. The system of claim 156, wherein the sensor data is indicative of activity of the locus coeruleus of the subject.

159. The system of claim 156, wherein the functional map indicates a distribution of tau protein across a plurality of regions of the brain of the subject.

160. The system of claim 159, wherein the diagnosis is generated based on the distribution of tau protein.

161. The system of claim 156, wherein the diagnosis includes a Braak region indicating a stage of the Alzheimer's disease.

162. The system of any one of claims 125 to 161, wherein the Al model includes a plurality of encoders that are trained such that each encoder of the plurality of encoders is associated with a different, respective task of the one or more tasks.

163. The system of claim 162, wherein the second operations include: generating, using the Al model, information indicative of the performance of the subject on a task of the one or more tasks based on the sensor data and the respective encoder that is associated with the task.Attorney Docket No. CRNU.P0037WO164. The system of any one of claims 125 to 163, wherein the functional brain map is generated without the use of a radioactive tracer and without the use of a positron emission tomography (PET) scanner.

165. The system of any one of claims 125 to 164, wherein the eye-tracking data is collected by the XR apparatus with a sampling frequency of at least 60 Hertz (Hz).

Images