A method for training and quantifying a person's specific motor skills and cognitive processing by analyzing eye movement patterns using a 3D virtual reality device incorporating eye-tracking technology, specific visual stimuli, and sensors indicating limb movements.

Abstract

The system and method combines virtual reality (VR), eye-tracking (ET), and motion sensors on extremities such as hands and feet to assess cognitive and motor skills in both healthy and unhealthy individuals using well-defined exercises. Applying VR and ET to motion sensors in cognitive exercises can improve the effectiveness of interventions and the ability to quantify cognitive and motor skills, thereby enhancing the effectiveness of training for individuals.

Description

[Background technology]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 373,228, filed August 23, 2022, and further claims the benefit of U.S. Application No. 18 / 227,577, filed July 28, 2023, and U.S. Application No. 18 / 217,688, filed July 0, 2023, the contents of which are incorporated herein by reference.

[0002] Eye-tracking systems have been used as diagnostic tools. For example, pending U.S. Application No. 18 / 217,688 [Docket No. 9901 / 1c1], the entire contents of which are incorporated herein by reference, describes a system for detecting one or more neurological disorders in a subject by measuring eye movements. Examples of neurological disorders that can be detected include multiple sclerosis (MS), attention-deficit hyperactivity disorder (ADHD), Parkinson's disease (PD), and Alzheimer's disease (AD).

[0003] Furthermore, virtual reality (VR) has been used to train cognitive abilities. Human cognitive abilities can be broadly categorized into working memory, attention, perception, reasoning and judgment, and decision-making. However, VR studies are mostly conducted by collecting partial data on behavioral feedback, such as the number of correct responses when performing an activity or the total time required to complete a task. More comprehensive and detailed VR studies are needed to assess, train, and improve cognitive abilities. Summary of the Invention

[0004] It is an object of the present invention to provide a system for detecting one or more psychiatric disorders in a subject by measuring eye movements, the eye movements being performed while the subject is reading, the system comprising: a. An eye-tracking device

[10] configured to monitor the eye movements of a subject [5] while the subject reads a text

[15] ; b. A processor

[20] configured to receive data from an eye-tracking device

[10] while a subject [5] reads a text

[15] ; and c. display means

[40] ; The processor

[20] is further configured to analyze the eye-tracking data for evidence of one or more neurological disorders or general cognitive function and report the detection of the one or more neurological disorders or the measurement of the cognitive function of the subject [5] in a test report

[50] .

[0005] Another object of the present invention is to provide a method for detecting an eye movement by a processor, the processor receiving eye-tracking data from an eye-tracking device, comprising: a. Counting the total number of eye fixations of the subject while reading the text; and b. If the subject's total number of eye fixations while reading is greater than that of the control group, report in the test report that impairment of attentional processing has been detected; The method is further configured to:

[0006] Another object of the present invention is to provide a method for detecting an eye movement by a processor, the method comprising: a. Counting the number of forward gaze fixations of the subject while reading the text, and b. If the subject's number of forward gaze fixations is less than that of the control group and the subject's number of gaze fixations while reading is more than that of the control group, a working memory impairment is detected and reported in the test report

[50] . The method is further configured to:

[0007] As mentioned above, another object of the present invention is to provide a method for processing an image of a subject, the method comprising: a. Counting the number of words that the subject fixated only once while reading the text; b. If the number of words the subject discontinued only once was greater than the control group, the test report would indicate that a decline in retrieval memory was detected; The method is further configured to:

[0008] Another object of the present invention is to provide a method for detecting an eye movement by a processor, the method comprising: a. Counting the number of multiple eye fixations by the subject while reading the text; b. If the number of multiple fixations is greater than the control group, report in the test report that a decline in executive processing has been detected; The method is further configured to:

[0009] Another object of the invention as described above is to detect one or more neurological disorders in a patient by measuring eye gaze movements, the processor receiving eye tracking data from an eye tracking device comprising: a. Calculating the average saccade amplitude from one gaze fixation to the next; and b. If the mean saccade amplitude is smaller than that of the control group, report in the test report that a decline in executive processing has been detected; The method is further configured to:

[0010] Another object of the present invention is to provide a method for measuring a pupil diameter of a subject, the method further comprising: a. Tracking the subject's pupil diameter while reading the text; and b. If the subject's pupil diameter does not constrict as the text progresses, the test report will state that a decline in executive function has been detected; The method is further configured to:

[0011] The object of the present invention is to provide a system for detecting one or more neurological disorders and checking the cognitive ability of a subject by measuring eye movements and pupil behavior and applying intelligent algorithms, the measurement of eye movements being carried out while the subject is reading, the system comprising: a. An eye-tracking device

[10] configured to monitor the eye movements and pupil behavior of a subject [5] while the subject [5] reads a text

[15] ; b. A processor

[20] configured to receive data from an eye-tracking device

[10] while a subject [5] reads a text

[15] ; c. Intelligent algorithms for learning, identifying, categorizing, and classifying eye movement features in or within a pathological condition; d. A display means

[40] that is a display that displays the output of the intelligent algorithm in a test report

[50] received by the processor

[20] ; wherein the processor

[20] is further configured to analyze and model the eye-tracking data from the cognitive performance for evidence of one or more neurological disorders, and report the detection and classification of the one or more neurological disorders of the subject [5] both across and within pathologies in a test report

[50] . That is the thing.

[0012] Another object of the above invention is that the processor, upon receiving the eye-tracking data from the eye-tracking device, is further configured to identify and classify the eye-movement characteristics and pupil behavior while reading the text so as to provide a classifier output for reporting in a test report the subject's cognitive ability and / or pathological classification (i.e., the pathological condition corresponding to the subject because of the subject's eye-movement characteristics) and intra-pathological assessment (i.e., the cognitive, behavioral and biological impairments that the subject exhibits within the particular pathological condition).

[0013] As mentioned above, another object of the present invention is to provide a method for detecting a malfunction of an intelligent algorithm, the method being configured to read at least one input, the input being: a. An index of the total number of gaze fixations of the subject while reading the text; b. An index of subjects' forward gaze fixations while reading the text; c. Index of words that the subject fixated only once while reading the text; d. An index of the subject's multiple eye fixations while reading the text; e. Mean saccade amplitude from one gaze fixation to the next, and f. The subject's pupil diameter while reading the text; g. Blink index from left eye, right eye, or both eyes; h. Elements of microsaccade formation (FF) i. HEWI: Shows the relationship between microsaccade height and width. ii. AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade. iii.LONG: The length of the horizontal-vertical trajectory of the microsaccade. iv. ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade. v. AANG: Sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. The last two FFs give an estimate of the regularity of the microsaccade trajectory. vi. MOD and THETA: The polar modulus and angle of the Cartesian sum. These give the spatial orientation of the microsaccade relative to the center of fixation. vii. TIME: The duration of the microsaccade in milliseconds. viii. VMIN and VMAX: The minimum and maximum microsaccade velocities expressed in degrees per second. ix. Microsaccade rate: The instantaneous rate of change over each time interval. x. Directional congruency: The congruency between the microsaccade direction and the stimulus location. and, i. Gaze position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes while reading the text; j. A sequence of fixations (i.e., eye movements) while reading the text, available from an image, matrix, etc.; k. The interval between eye fixations while reading the text; l. Subject's philia information (i.e., age, years of education, gender, ethnicity, occupation, hours of physical activity per week, etc.) m. Total reading time (i.e., the amount of time the subject spent reading the text); It is composed of a group consisting of:

[0014] The object of the present invention is a method for assessing the decline in neurological function associated with multiple sclerosis (MS), the method comprising: a. Providing a system for assessing neurological decline associated with MS

[0305] ; b. Requiring the subject to fixate on a reference target on the chart; c. repeatedly presenting stimulus images to the subject in one of the zones, and requiring the subject to remember the zone and order in which the stimulus images appeared; d. Presenting the subject with a cue corresponding to one of the presented stimulus images; e. Measuring the subject's saccades in response to the cue presentation step, wherein the subject is required to look at the zone in which the stimulus image is presented in response to the cue; f. Repeating the step of presenting the cue and measuring the saccades; g. Repeat steps b to f the number of times you want to try. h. A calculating step, i. WM effect (i.e., WM effect is a measure that increases with increasing WM demands. For each cue number, WM effect is expressed as the ratio between the number of errors reported by the subject across all trials and the number of trials), and ii. Saccade latency is defined as the time to initiate a saccade into the zone, with a mean saccade latency of 345; and calculating one or more of: i. reporting, i. The degree of working memory impairment accompanied by an increase in WM effects

[0350] , and ii. Degree of impairment of executive processing accompanied by increased saccade latency

[0355] reporting one or more of the following: Including, j. The method further includes an additional step including a measurement performed during the step of presenting the stimulus image, during which the subject is further required to view the stimulus image, and the measurement includes: i. the width of the subject's pupil dilation; ii. The number of fixations the subject made on the stimulus image

[0365] , and iii. Subject's gaze duration on the stimulus image

[0370] measuring one or more of: k. Additional steps include: i. The degree of impairment of subcortical processing accompanied by an increase in pupil dilation

[0375] ; ii. A level of impairment in executive processing accompanied by an increase in the number of fixations (380); iii. The degree of impairment in executive processing and working memory function, accompanied by increased gaze time

[0385] ; and further calculating or reporting one or more of:

[0015] As mentioned above, it is another object of the present invention that the reference target be in a central location on the chart and that the multiple zones be positioned around the reference target.

[0016] As noted above, another object of the present invention is for the cue to be placed at the location of the reference target.

[0017] As mentioned above, another object of the present invention is to define an error as an eye movement outside the correct zone and / or failure to initiate a saccade within the time limit.

[0018] As noted above, another object of the present invention is that the cue corresponding to the first presented stimulus is excluded from the presented cue number.

[0019] As mentioned above, another object of the present invention is that a saccade is included in the step of calculating WM effect and saccade latency only if the saccade is initiated after the step of presenting the cue number and beyond the minimum saccade latency.

[0020] Another object of the present invention is to provide a method for calculating WM that calculates saccades that are excluded from the calculation of WM. Saccades to one zone are excluded from WM calculations if they are not made within the time limit, if fixation on the reference target cannot be maintained before the initiation of a saccade to one angular zone, and if blinks obscure the gaze shift. a. An object of the present invention is to provide a system for detecting one or more neurological disorders in a subject by measuring eye movements, the measurement of eye movements being performed while the subject is performing a visual test, the system comprising: An eye-tracking device

[10] configured to monitor the eye movements of a subject [5] while the subject [5] is performing a visual test

[15] ; b. Requiring the subject to sequentially fixate targets that are part of a group of targets (e.g., dots) presented together in the same image (i.e., maze or labyrinth); and c. Requiring the subject to follow the direction of the labyrinth or maze (i.e., enter the labyrinth or maze from the bottom and exit from the top) and fixate only one target at a time until all targets have been viewed through the image; and d. A processor

[20] configured to receive data from the eye-tracking device

[10] while the subject [5] is performing a visual test

[15] ; e. a display means

[40] configured to display the test report

[50] received from the processor

[20] ; wherein the processor

[20] is further configured to analyze the eye-tracking data for evidence of neurological and attentional disorders and report the detection of one or more neurological and attentional disorders in the subject [5] in a test report

[50] .

[0021] As mentioned above, another object of the present invention is to provide a method for processing a visual signal from a subject's eye tracking device while the subject is viewing, recognizing, maintaining, matching, inhibiting and sequencing a target, the method comprising: a. counting the total number of gaze fixations of subject

[0615] while administering the qualification test; b. reporting in a test report that an impairment in attentional processing has been detected if the subject's total number of visual fixations while viewing the target is greater than that of a control group; The method is further configured to:

[0022] As mentioned above, another object of the present invention is to provide a method for processing an image of a subject, the method comprising: a. Measuring the average saccade velocity while the subject moves from one target to another; b. If the subject's average saccade velocity

[0620] is lower than that of the control group, reporting in the test report that a decline in executive function has been detected; The method is further configured to:

[0023] As mentioned above, another object of the present invention is to provide a method for processing an image of a subject, the method comprising: a. Counting the number of correct target recognitions

[0625] ; b. If the number of correct target recognitions of the subject is less than that of the control group, reporting in the test report that a decline in working memory function has been detected; The method is further configured to:

[0024] As mentioned above, another object of the present invention is to provide a method for processing an image of a subject, the method comprising: a. Calculating the average saccade amplitude

[0630] ; b. If the mean saccade amplitude

[0630] is smaller than that of the control group, reporting in the test report that a functional impairment of executive processing has been detected; The method is further configured to:

[0025] As mentioned above, another object of the present invention is to provide a method for processing an image of a subject, the method comprising: a. Measuring the total time spent performing the visual test; b. If the total time spent performing the visual test is greater than that of the control group, reporting in the test report that an impairment in attentional processing has been detected; The method is further configured to:

[0026] As mentioned above, another object of the present invention is to provide a method for measuring a pupil diameter of a subject, further comprising: means for measuring a pupil diameter of the subject

[17] , the processor: a. Tracking the pupil diameter of a subject performing a visual test; b. If the subject's pupil diameter does not increase as the task progresses, reporting in the test report that a decline in attentional processing has been detected; The method is further configured to:

[0027] As noted above, another object of the present invention is to further include means for measuring the subject's pupil diameter

[17] , wherein the processor is further configured to calculate the person's duration of fixation on a target while administering the visual test, and to report impaired concentration and executive processing in the test report if the subject's duration of fixation on the target

[0645] is less than that of a control group.

[0028] The object of the present invention is to provide a method for detecting the presence of one or more neurological disorders or for measuring the general cognitive function of a subject by detecting the eye movements of the subject, the measurement of the eye movements being carried out while the subject is reading, the method comprising: a. providing a system for detecting one or more neurological disorders according to claim 1 or claim 18; b. receiving eye-tracking data and / or pupil diameter data of the subject while the subject reads the text; Including, Here, the method further includes a step

[0417] of analyzing the eye tracking data and / or pupil diameter data for evidence of one or more neurological disorders, and a step

[0499] of displaying a report of the detection of the neurological disorder.

[0029] As mentioned above, another object of the present invention is to a. Calculating the total number of eye fixations of the subject while the subject is reading the text

[0420] ; b. If the total number of eye fixations of the subject while reading the text is greater than that of the control group, reporting that impairment of attentional processing has been detected

[0460] ; It also includes:

[0030] As mentioned above, another object of the present invention is to a. Calculating the total number of eye fixations of the subject while the subject is reading the text

[0420] ; b. counting the number of forward gaze fixations of the subject while the subject is reading the text; c. reporting that a working memory impairment has been detected if the subject's number of forward gaze fixations while reading the text is lower than the control group and the subject's number of forward gaze fixations while reading is higher than the control group; The further inclusion of

[0031] As mentioned above, another object of the present invention is to a. counting the number of gaze fixations by the subject on each word of the text while the subject is reading the text; b. Counting the number of words that the subject fixates only once while reading the text

[0445] ; c. reporting that a retrieval memory impairment has been detected if the number of words on which the subject fixates only once while reading the text is less than that of a control group; The further inclusion of

[0032] As mentioned above, another object of the present invention is to a. counting the number of multiple eye fixations of the subject while reading the text; b. If the number of multiple eye fixations by the subject while reading the text

[0450] is greater than that of the control group, reporting that a decline in executive processing has been detected

[0490] ; The further inclusion of

[0033] As mentioned above, another object of the present invention is to a. Calculating the subject's average saccade amplitude from one gaze fixation to the next while reading the text

[0454] ; b. reporting in the test report that impairment of executive processing has been detected if the subject's mean saccade amplitude from one fixation to the next while reading the text is less than that of the control group; The further inclusion of

[0034] As mentioned above, another object of the present invention is to a. Tracking the pupil diameter of the subject reading the text

[0456] ; b. If the pupil diameter

[0456] of the subject reading the text does not show a decrease as the text is read, reporting in the test report that a performance degradation has been detected

[0492] ; The further inclusion of

[0035] The object of the present invention is to provide a system for detecting disorders of memory binding function, which system comprises: a. Eye tracking device

[10] ; b. A means for measuring pupil diameter; c. A processor

[20] , i. receiving eye-tracking data of a subject [5] from an eye-tracking device

[10] ; ii. receiving pupil diameter data of the subject [5] from a means for measuring pupil diameter; a processor

[20] configured to implement the d. A display means

[40] configured to display the test report

[50] received from the processor

[20] ; wherein the processor

[20] is further configured to analyze the eye-tracking data and pupil diameter data and report the detection of one or more disorders of the memory binding function of the subject [5] in a test report

[50] .

[0036] As mentioned above, another object of the present invention is to provide a method for a processor

[20] , upon receiving gaze tracking data from an eye tracking device

[10] , a. measuring one or more gaze durations of the subject [5] at each of one or more targets that the subject [5] is looking at; b. Calculating the average gaze duration of the subject [5]; c. If the average fixation time of the subject [5] is longer than the average fixation time of the control group, reporting in the test report

[50] that the subject [5] has a functional impairment in encoding and recognizing the target; The method is further configured to:

[0037] As mentioned above, another object of the present invention is to provide a method for a processor

[20] , upon receiving gaze tracking data from an eye tracking device

[10] , a. counting the number of gaze fixations made by the subject [5] during one or more targets; b. If the number of fixations made by the subject [5] while looking at the target is greater than that of the control group, reporting in the test report

[50] that an impairment in attentional processing has been detected in the subject [5]; The method is further configured to:

[0038] As mentioned above, another object of the present invention is that the processor

[20] is further configured to apply an intelligent algorithm; a. receiving a pupil diameter of the subject [5] from a means for measuring pupil diameter while the subject [5] is performing an activity requiring a lower cognitive effort; b. receiving a pupil diameter of the subject [5] from the means for measuring pupil diameter while the subject [5] is performing an activity requiring increased cognitive effort; c. If the pupil diameter of the subject [5] during the activity requiring a stronger cognitive effort does not increase compared to the pupil diameter of the subject [5] during the activity requiring a reduced / minimal cognitive effort, reporting in the test report

[50] that a cognitive resource impairment has been detected in the subject [5]; The method is further configured to:

[0039] As mentioned above, another object of the present invention is for the processor

[20] to further report results in the test report

[50] for any disorders of memory binding function not detected by the system

[0100] in the subject [5].

[0040] The object of the present invention is to provide a method for detecting disorders of memory binding function in a subject, the method comprising: a. Providing a system according to claim 1 or claim 33; b. Step

[0510] of presenting the goal; c. A step of requesting the subject to fixate on the target and memorize it (encoding)

[0515] ; d. Step

[0520] of presenting a blank screen; e. Presenting a target and asking the subject to identify whether the target is identical to one seen before (recognition); If the target is exactly the same, the answer must be "same," and if it is not exactly the same, the answer must be "different." Both answers must be collected using a keyboard or similar support

[0525] . Steps [510-525] are repeated for the number of trials

[0530] . f. Repeating steps [510 to 525] for a number of trials

[0530] ; g. receiving eye tracking data; h. The subject views one or more targets

[0540] ; i. Measuring the time the subject spends looking at each target

[0545] ; j. Calculating the average time spent looking at the target by the subject

[0550] ; k. Measuring the subject's pupil diameter while performing an activity requiring lower cognitive effort; l. Calculating the number of gaze fixations performed by the subject while looking at the target

[0560] ; m.Furthermore, i. reporting that impairment in goal encoding and cognitive processing has been detected if the subject's mean fixation duration is longer than the control's mean fixation duration; ii. reporting that a decline in cognitive resource function has been detected if the subject's pupil diameter during the activity requiring greater cognitive effort does not decrease more than the subject's pupil diameter during the activity requiring less cognitive effort; and iii. If the number of gaze fixations performed by the subject while looking at the target is greater than that of the control group, reporting that an impairment in attentional processing has been detected in the subject

[0575] ; and Includes:

[0041] As mentioned above, another object of the present invention is for the intelligent algorithm to read at least one input, the input being: a. The total number of gaze fixations by the subject while performing each combined task; b. Cohesion assessment task, i.e., "cohesive colors" and "uncohesive colors"; c. Identification number of the combined task; d. The correct behavioral answer for the task (i.e., "same" or "different"); e. the subject's behavioral response; f. Part of the task, i.e., encoding or retrieval; g. Subject's pupil diameter while performing the connectivity assessment; h. Number of blinks from the left eye, right eye, or both eyes; i. Microsaccades: Elements of Formation (FF) i. HEWI: Shows the relationship between microsaccade height and width. ii. AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade. LONG: The length of the microsaccade trajectory in the horizontal-vertical plane. iv. ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade. v. AANG: The sum of the absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. These last two FFs estimate the regularity of the microsaccade trajectory. vi. MOD and THETA: Cartesian polar modulus and angle. These give the spatial orientation of the microsaccade relative to the center of fixation. vii. TIME: The duration of the microsaccade in milliseconds. viii. VMIN and VMAX: The minimum and maximum microsaccade velocities in degrees per second. ix. Microsaccade rate: The instantaneous rate of change over each time interval. x. Directional congruency: The congruency between the microsaccade direction and the stimulus location. j. Gaze position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes while performing the combined assessment; k. saccade amplitude while processing the target; l. A fixation sequence (i.e., eye movements) while processing a target, the sequence being available from an image, a matrix, etc. m. The distance between the fixation points of the right and left eyes while performing the binding assessment; n. Subject's philia information (i.e., age, years of education, gender, ethnicity, occupation, hours of physical activity per week, etc.); o. fixation duration while processing the target, p. fixation duration while processing the target, q. Number of fixations on each target, r. the number of fixations on each off-target, and s. Number of fixations on each target is selected from the group consisting of:

[0042] An object of the present invention is to provide a method for detecting neurological and attention disorders in a subject, the method comprising: a. Eye tracking device

[10] ; b. a means for measuring pupil diameter; c. A processor

[20] , i. receiving eye-tracking data of a subject [5] from an eye-tracking device

[10] ; ii. receiving pupil diameter data of the subject [5] from a means for measuring pupil diameter; iii. a display means

[40] configured to display the test report

[50] received from the processor

[20] ; wherein the processor

[20] is further configured to analyze the eye-tracking data and pupil diameter data and report the detection of one or more neurological and attention disorders of the subject [5] in a test report

[50] .

[0043] As mentioned above, another object of the present invention is to provide a method for a processor

[20] , upon receiving gaze tracking data from an eye tracking device

[10] , a. measuring one or more fixation times of the subject [5] for each of one or more targets that the subject [5] is looking at; b. Calculating the average saccade amplitude from each target to other targets by the subject [5]; c. If the average saccade amplitude of the subject [5] is shorter than the average saccade amplitude of the control group, reporting in the test report

[50] that the subject [5] has a functional impairment in target viewing, recognition, maintenance, matching, inhibition, and ordering; The method is further configured to:

[0044] As mentioned above, another object of the present invention is to provide a method for a processor

[20] , upon receiving gaze tracking data from an eye tracking device

[10] , a. Counting the number of gaze fixations performed by a subject [5] while looking at one or more targets; b. If the number of gaze fixations performed by the test vehicle [5] while looking at the target is greater than that of the control group, reporting in the test report

[50] that a functional impairment in attention processing has been detected in the subject [5]; The method is further configured to:

[0045] As mentioned above, another object of the present invention is to provide a processor

[20] a. receiving a pupil diameter of the subject [5] from a means for measuring pupil diameter while the subject [5] is performing an activity requiring significant attentional resources; b. receiving a pupil diameter of the subject [5] from a means for measuring pupil diameter while the subject [5] is performing an activity requiring significant attention; c. If the pupil diameter of the subject [5] during the activity requiring greater attention does not increase more than the pupil diameter of the subject [5] during the activity requiring less attention, reporting in the test report

[50] that a cognitive resource impairment has been detected in the subject [5]; The method is further configured to:

[0046] An object of the present invention is to provide a method for detecting neurological and executive disorders in a subject, the method comprising: a. Providing a system as described above; b. receiving eye-tracking data; c. The subject views one or more targets [605-610]; d. Calculating the mean saccade amplitude of the target by the subject

[0630] ; e. Measuring the subject's pupil diameter while performing an activity requiring significant attention; f. Measuring the subject's pupil diameter while performing an activity requiring greater than minimal attention; g. Counting the number of gaze fixations performed by the subject while looking at the target

[0615] ; where: h. The method is i. reporting that impairments in target visualization, recognition, maintenance, matching, inhibition, and sequencing have been detected in the subject if the subject's mean saccade amplitude is shorter than the mean saccade amplitude of the control group; ii. reporting that a decrease in cognitive and functional resources has been detected in the subject if the pupil diameter of the subject's pupil while performing the high attention demanding activity does not increase compared to the pupil diameter of the subject while performing the low attention demanding activity, and reporting that a decrease in attentional processing has been detected in the subject if the number of gaze fixations performed by the subject while looking at the target is greater than that of a control group; iii. reporting that an impairment in executive processing has been detected in the subject if the subject's mean saccade latency (velocity) is shorter than the control's mean saccade latency; Further includes:

[0047] As mentioned above, another object of the present invention is that the method is configured to report that an impairment in executive processing has been detected in a subject if the subject's mean saccade time is shorter than the mean saccade time of a control group.

[0048] As mentioned above, another object of the present invention is that neurological disorders including Parkinson's disease or attention deficit hyperactivity disorder are selected.

[0049] Another object of the present invention is to provide a system and method for assessing a person's performance, motor skills, or cognitive abilities. The system includes a three-dimensional (3D) virtual reality device configured to establish a 3D virtual reality environment in which a plurality of virtual objects are presented to a person, the objects having at least one characteristic that differs from one another, and the objects move toward or away from the person at a predetermined speed, acceleration, and direction. The system further includes an eye-tracking device configured to measure the person's eye movement while the person views the virtual objects and performs a required task, the required task including a plurality of requests requiring the person to visually touch designated virtual objects, each having one designated characteristic. One or more motion sensors are configured to measure the movement of the person's limbs while the person performs the required task. The processor is configured to receive data from the 3D virtual reality device, the eye tracking device, and the one or more motion sensors while the person is performing the required task, and is further configured to: (i) identify selected ones of the measured gaze and limb movements associated with the person's performance, motor skills, and cognitive abilities; (ii) determine expected gaze and limb movements of the person while the person is viewing the virtual object and performing the required task, and compare the expected gaze and limb movements with the selected ones of the measured gaze and limb movements to determine a deviation therebetween; and (iii) evaluate the person's performance, motor skills, and cognitive abilities based on the deviation. [Brief explanation of the drawings]

[0050] [Figure 1] 1 illustrates a system for detecting one or more neurological disorders in a subject, according to some embodiments of the present invention. [Figure 2] 1 illustrates a system for detecting one or more neurological disorders in a subject, according to some embodiments of the present invention. [Figure 3A] 1 illustrates a method for assessing MS-associated neurological decline, according to some embodiments of the present invention. [Figure 3B] 1 illustrates a method for assessing MS-associated neurological decline, according to some embodiments of the present invention. [Figure 4A] 1 illustrates a method for detecting one or more neurological disorders in a subject taking a reading, according to some embodiments of the present invention. [Figure 4B] 1 illustrates a method for detecting one or more neurological disorders in a subject taking a reading, according to some embodiments of the present invention. [Figure 5] 1 illustrates a method for detecting disorders of memory binding function according to some embodiments of the present invention. [Figure 5B] Test results using the 5A evaluation method are shown, with corrected recognition (error bars = standard error of the mean) for the control group and AD patients in the two experimental conditions. [Figure 5C] Results from the 5A assessment method are shown for the effect of the binding task on gaze duration during encoding and recognition in controls and patients with Alzheimer's disease (AD). Panels show the partial effect of the LMM (i.e., after removing other fixed effects and variance components). Shaded regions indicate 95% confidence intervals. Gaze durations are plotted on a logarithmic scale to correspond with the LMM. [Figure 6A] A method for detecting Parkinson's disease and attention deficit hyperactivity disorder is presented. [Figure 6B] A method for detecting Parkinson's disease and attention deficit hyperactivity disorder is presented. [Figure 7] The effect of dimethyl fumarate on saccade amplitude is shown in patients with multiple sclerosis who have been taking the drug for 4 years. [Figure 8] 1 is a flowchart illustrating a method for analyzing eye movement patterns when using specific visual stimuli to identify specific changes in subjects with a given disease where specific drugs or treatments would improve visual processing, cognitive function, and related brain activity. [Figure 9]This shows a conceptual diagram of a system used to assess a person's performance by applying a 3D virtual reality (3DVR) environment in combination with embedded eye-tracking technology (ET) and motion sensors to track the movements of limbs such as hands and feet while the person performs well-defined activities. [Figure 10] 1 shows an example of how an object can be positioned and how the object and a person's hand can be displayed. [Figure 11] 1 is a graphical heatmap representation of an example of the density of eye movements recorded from the right and left eyes while a person performed a task in which they were required to touch an object. [Figure 12] 1 is a graphical representation, shown as a heat map, of the density of recorded motor actions of right and left hand movements while a person performs a task in which they are required to touch an object. [Figure 13] 1 is a flowchart describing an example of a method for assessing a person's performance, motor skills, and cognitive abilities as described herein. DETAILED DESCRIPTION OF THE INVENTION

[0051] The term "cognitive effort" reflects the total amount of mental effort required by a subject to perform a task. In this application, "lower cognitive effort" refers to a reduced working memory demand when performing a task.

[0052] In this application, the term "microsaccade," also known as a "flick," refers to a small saccade made during a fixation period. It is the largest and fastest fixational gaze movement. In this application, the term "saccade" refers to a rapid, simultaneous movement of both eyes during two or more phases of fixation.

[0053] In this application, "ocular drift" is a fixational gaze movement characterized by smoother, slower, circular movements of the eyes when fixed on an object.

[0054] In this application, the term "ocular motion" (OMT) refers to small, rapid, synchronous eye vibrations occurring in the range of 40-100 Hz, but typically occurring at about 90 Hz in the average healthy individual. It is characterized by a high frequency and a very small amplitude of only a few arc seconds.

[0055] In this application, the term "stimulus image" refers to a particular visual pattern or target for the subject on the display, and the term "visual task" or "visual test" refers to the activity the subject performs while processing each stimulus image.

[0056] Referring now to FIG. 1, there is shown a system

[0100] for detecting neurological disease or function in a subject [5], according to some embodiments of the present invention.

[0057] The system

[0100] includes an eye-tracking device

[10] , a means for measuring pupil diameter

[17] , a processor “20”, and a display means

[40] .

[0058] The eye-tracking device

[10] can be of any type known in the art, such as an eye-worn tracker, an optical eye-tracking device, or an electro-oculographic eye-tracking device.

[0059] The means for measuring pupil diameter

[17] may, for example, include a camera configured to acquire an image of the eye and a processing unit for measuring the pupil diameter from the image. Alternatively to a processing unit, the means for measuring pupil diameter

[17] may include a display of the image, with manual measurements being made while viewing the display.

[0060] The eye-tracking device

[10] and the means for measuring pupil diameter

[17] are communicatively coupled to a processor

[20] . The communication connection can be of any form known in the art and can be either wired (e.g., USB, parallel port, or the like) or wireless (e.g., WiFi, Bluetooth, or the like).

[0061] The processor

[20] receives and executes instructions stored in one or more memories, such as RAM, CD / DVD, HDD, flash memory, and / or any suitable medium. The instructions instruct the processor

[20] to: 1) receive eye-tracking data from the eye-tracking device

[10] ; 2) receive pupil diameter data from the means for measuring pupil diameter

[17] ; 3) analyze the eye-tracking data and pupil diameter data (as further described herein); and 4) report the detection or non-detection of one or more disorders of memory binding function in the subject [5] in a test report 50 for display on a display means

[40] . The display means

[40] can be a monitor, a screen of a mobile device such as a smartphone, a printed copy, or any suitable means for displaying the test report

[50] . The processor

[20] can store any of the received eye-tracking data, intermediate results at any stage of the analysis, and / or the test report

[50] in a memory medium

[60] .

[0062] Neurological disorders detected by the system can include reading functions such as impairments in target encoding and recognition, impairments in attentional processing, impairments in cognitive resources, or any combination thereof. In other embodiments, the detected disorders can include multiple sclerosis (MS), attention deficit hyperactivity disorder (ADHD), Parkinson's disease (PD), Alzheimer's disease (AD), etc.

[0063] In some embodiments, the processor

[20] receives eye-tracking data from the eye-tracking device

[10] while the subject [5] looks at each of one or more targets

[30] . The processor

[20] measures the subject [5]'s gaze duration at each target

[30] that the subject [5] looks at. The processor

[20] calculates the subject [5]'s average gaze duration at each target

[30] . If the average gaze duration at the target

[30] is longer than the average gaze duration of the control group, the processor

[20] reports in the test report

[50] that impairments in target encoding and cognitive processing have been detected in the subject [5].

[0064] In some embodiments, the processor

[20] additionally or alternatively counts the number of gaze fixations made by the subject [5] while viewing each of the targets

[30] . If the number of gaze fixations made by the subject [5] while viewing the targets

[30] is greater than that of a control group, the processor

[20] reports in the test report

[50] that an impairment in attentional processing has been detected in the subject [5].

[0065] In some embodiments, the processor

[20] receives pupil diameter data from the means for measuring pupil diameter

[17] while the subject [5] is performing an activity requiring less cognitive effort. The processor

[20] further receives pupil diameter data from the means for measuring pupil diameter while the subject [5] is performing an activity requiring more cognitive effort than the activity requiring less cognitive effort. If the mean pupil diameter of the subject [5] while performing the activity requiring more cognitive effort does not show an increase compared to the mean pupil diameter of the subject [5] while performing the activity requiring less cognitive effort, the processor

[20] reports in the test report

[50] that a decline in cognitive resource function has been detected in the subject [5].

[0066] The control group may include a statistically representative cross-section of the same demographic sector as the subject [5] (e.g., the same gender, race, national culture, age group, and / or other demographic characteristics as the subject [5]). Eye-tracking data for the control group may be acquired by the system

[0100] or otherwise collected from previous research and / or clinical studies. If the subject [5]'s mean gaze duration or number of gaze fixations is within a selected margin of the control group mean, which is approximately one standard deviation of the distribution of the corresponding numerical value in the control group, the system

[0100] may treat the subject [5]'s mean gaze duration or number of gaze fixations as equal to the corresponding numerical value in the control group mean.

[0067] It will be understood that the eye-tracking data received from the processor

[20] may be a series of eye positions measured by the eye-tracking device

[10] , which the processor

[20] analyzes to find the gaze duration and gaze fixations of the subject [5]. Alternatively, the processor

[20] may receive a series of pre-processed signals from the eye-tracking device

[10] , each signal indicating that a gaze duration or gaze fixation occurred. The signals may optionally be accompanied by metadata (e.g., eye position, time, and / or gaze fixation length).

[0068] (Multiple Sclerosis) Reference is now made to Figures 3A and 3B, which illustrate a method for assessing multiple sclerosis (MS)-associated neurological decline, according to some embodiments of the present invention. a. Providing a system for assessing neurological decline associated with MS

[0305] ; b. Requiring the subject to fixate on a reference target on the chart

[0310] ; c. repeatedly presenting stimulus images to the subject in one of a number of zones on the chart, the subject being asked to remember in which zone each stimulus image appeared and in what order; d. Presenting the subject with a cue corresponding to one of the presented stimulus images; e. Measuring the subject's saccades in response to the cue presentation step, wherein the subject is asked to look at a zone where the stimulus image presented in response to the cue was located; f. Repeating the steps of presenting a cue and measuring saccades

[0330] ; g. Step

[0335] of repeating steps b to f for the number of trials; h. A calculating step, ia. WM effect (i.e., WM effect is a measure that increases with increasing WM demand. For each cue number, WM effect is represented by the ratio between the number of errors reported by the subject across all trials and the number of trials), and ii. Mean saccade latency, where saccade latency is defined as the time it takes a subject to initiate a saccade into a zone. Mean saccade latency and calculating one or more of: i. reporting, ia. The degree of decline in working memory function due to an increase in WM effect

[0350] , and iia. The degree of impairment of executive processing associated with increased saccade latency

[0355] reporting one or more of: Including, wherein the method further comprises an additional step performed during the step of presenting the stimulus image, during which the subject is further requested to view the stimulus image; j. Additional steps include: i. one or more pupil dilation amplitudes

[0360] ; iia. The number of fixations by the subject on the stimulus image

[0365] , and iiia. The subject's gaze duration on the stimulus image

[0370] , and k. Additional steps include: ia. Pupil diameter width does not change, degree of functional decline of subcortical processing

[0375] , iia. Degree of functional decline in executive processes, accompanied by increased frequency of lower back pain

[0380] , and iiia. Degree of impairment in executive processing and working memory function, accompanied by increased gaze duration

[0385] and reporting the step of calculating one or more of:

[0069] The method utilizes an intelligent algorithm that utilizes the following variables to analyze the subject: Total number of gaze fixations by subjects while performing the an-Back task. b The identification number of the n-Back task trial (i.e., if there are 20 n-Back task trials, the 5th trial is identified by the number 5, the 20th trial is identified by the number 20, etc.). c. Trial parts, i.e., 1, 2 and 3. d. Parts of a trial, i.e., encoding, retrieval. Subjects' pupil diameter while performing the en-Black task. f. Number of blinks of the left eye, right eye, or both eyes. g. Microsaccades: Elements of formation (FF) i. HEWI: Showing the relationship between height and width of microsaccades ii. AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade iii. LONG: The length of the microsaccade trajectory in the horizontal-vertical plane iv. ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade v. AANG: The sum of the absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. These last two FFs estimate the regularity of the microsaccade trajectory. vi. MOD and THETA: Cartesian polar modulus and angle. These give the spatial orientation of the microsaccade relative to the center of fixation. vii. TIME: The duration of the microsaccade in milliseconds. viii. VMIN and VMAX: The minimum and maximum microsaccade velocities in degrees per second. ix. Microsaccade rate: The instantaneous rate of change over each time interval. x. Directional congruency: The congruency between the microsaccade direction and the stimulus location. Gaze position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes while performing the hn-Back task. i. Saccade amplitude while processing the target. j. Saccade latency. k. The fixation sequence (i.e., eye movements) while processing the target. This sequence is available from an image, matrix, etc. l. The distance between the fixation points of the right and left eyes during target processing. m. Subject's philia information (i.e., age, years of education, gender, ethnicity, occupation, hours of physical activity per week, etc.). n.The duration of fixation while processing a target. o. Fixation duration while processing the target. p. Number of fixations on each target. q. Number of fixations outside each target.

[0070] Measurements taken during presentation of the stimulus image (feature j of method

[0300] ) provide information during encoding, which occurs while the subject is initially identifying the location of the visual stimulus. In a pilot study by the inventors, we found that subjects with MS performed poorly when encoding visual information (e.g., subjects made many fixations on the display). Measurements taken during encoding are added to measurements taken during recognition, when cues are presented after the visual stimulus is presented, as in the Fielding et al. study (step ai of method

[0300] ). Taking a subject's performance during encoding and recognition together can help identify additional deficits (i.e., the degree of impairment in subcortical processing, executive processing, and / or supervisory processing) and provide greater insight into the subject's condition than performance during recognition alone.

[0071] (reading) Referring now to Figures 4A and 4B, there is shown a method for measuring general cognitive function and detecting one or more neurological disorders in a subject by measuring eye movements and / or pupil diameter while the subject is reading, according to some embodiments of the present invention.

[0072] The method

[0400] includes the steps of providing a system for measuring general cognitive function and detecting the presence of one or more neurological disorders by measuring eye movements and / or pupil diameter; receiving eye tracking data and / or pupil diameter data of a subject reading text; analyzing the eye tracking data for evidence of one or more neurological disorders; and displaying a report of the detection of the neurological disorders.

[0073] In some embodiments, the method

[0400] includes a step

[0405] of counting the total number of eye fixations of the subject while the subject is reading the text, and a step

[0460] of reporting that impaired attentional processing has been detected if the total number of fresh fixations of the subject while reading the text is greater than that of a control group.

[0074] In some embodiments, the method

[0400] further includes the steps of counting the total number of eye fixations of the subject while reading the text

[0405] , counting the number of ahead-looking eye fixations of the subject while reading the text

[0430] , and reporting that a decline in working memory function has been detected if the number of ahead-looking eye fixations of the subject while reading is greater than that of a control group

[0470] .

[0075] Physiologically, working memory decline correlates with deterioration of the frontal lobe. In some embodiments, reports of working memory decline can be used in further treatment. For example, if neurosurgery is indicated, brain imaging of the subject's frontal lobe can be performed according to the method.

[0076] In some embodiments, the method

[0400] includes the steps of counting the number of visual fixations by the subject on each word in the text while the subject is reading the text

[0440] , counting the number of words on which the subject fixates only once

[0445] , and reporting that a retrieval memory impairment has been detected if the number of words on which the subject fixates only once is lower than that of a control group

[0480] .

[0077] Physiologically, retrieval memory decline correlates with temporal lobe deterioration. In some embodiments, reports of retrieval memory decline can be used to guide further treatment. For example, brain imaging of the subject's frontal lobe can be performed according to method 100 if neurosurgery is indicated.

[0078] In some embodiments, the method

[0400] includes counting the number of multiple eye fixations by the subject while reading the text

[0450] and reporting that a decline in executive processing has been detected if the number of multiple eye fixations is greater than that of a control group.

[0079] In some embodiments, the method

[0400] includes a step

[0454] of calculating the subject's average saccade amplitude from one gaze fixation to the next while reading the text, and a step

[0491] of reporting that impairment of executive processing has been detected if the average saccade width is smaller than that of a control group.

[0080] In some embodiments, the method

[0400] includes a step

[0456] of tracking the subject's pupil diameter while reading the text, and a step

[0492] of reporting a degradation in executive processing performance if the subject's pupil diameter does not exhibit a decrease as the text is read.

[0081] Physiologically, declines in executive processing correlate with deterioration of the frontal, temporal, and / or parietal lobes. In some embodiments, reports of declines in executive processing [490-491-492] may be used in further treatment. For example, if neurosurgery is indicated, brain imaging of the subject's frontal, temporal, and / or parietal lobes can be examined according to method

[0400] .

[0082] The system and method

[0400] was tested on 50 healthy controls and 50 patients with mild AD. Both groups read 40 normal sentences. [Table 1]

[0083] (References) The above rules are based in part on the following research: 1.Fernandez G, Mandolesi P, Rotstein NP, Colombo O, Agamennoni O, Politi LE. (2013) Eye movement alterations during reading in patients with early Alzheimer disease. Invest Ophthalmol Vis Sci. pii: iovs.13-12877v1. doi: 10.1167 / iovs.13-12877. 2.Fernandez G., Manes F., Politi L., Orozco D., Schumacher M., Castro L., Agamennoni O., Rotstein N. (2016). Patients with Mild Alzheimer Disease Fail When Using Their Working Memory: Evidence from the Eye Tracking Technique. Journal of Alzheimer Disease; 50, 827 - 828. 3.Fernandez, G., Laubrock, J., Mandolesi P., Colombo O., Agamennoni O. (2014) Registering eye movements during reading in Alzheimer disease: difficulties in predicting upcoming words. Journal of Clinical and Experimental Neuropsychology; 36, 302-16. 4.Fernandez G., Sapognikoff M., Guinjoan S., Orozco D., Agamennoni O. (2016). Word processing during reading sentences in patients with schizophrenia: evidences from the eyetracking technique. COMPREHENSIVE PSYCHIATRY; 68, 193-200. 5.Fernandez G, Manes F, Rotstein N, Colombo O, Mandolesi P, Politi L, Agamennoni O. (2014) Lack of contextual-word predictability during reading in patients with mild Alzheimer disease. Neuropsychologia; 62, 143-51. 6.Fernandez G., Schumacher M., Castro L., Orozco D., Agamennoni O., (2015). Patients with Alzheimer disease produced shorter outgoing saccades when reading sentences. Psychiatry Research, 229, 470-478. 7.Fernandez G., Biondi J., Castro S., Agamennoni O. (2017). Pupil size behavior during online processing of sentences. Journal of Integrative Neurosciences 15(4) 485- 496

[0084] (Memory combination) Non-limiting embodiments of the present invention will now be described in detail.

[0085] Reference is now made to FIG. 5, which illustrates a method for detecting disorders of memory binding function in a subject, according to some embodiments of the present invention.

[0086] The method includes a step

[0505] of providing a system for detecting a disorder of a subject's memory binding function.

[0087] In some embodiments, the method

[0500] includes the steps of having the subject view one or more targets [510-535], measuring the subject's gaze time at each of the targets

[0545] , calculating the subject's average gaze time on the targets

[0550] , and reporting that impairment of target encoding and cognitive processing has been detected in the subject [5] if the subject's average gaze time is longer than the average gaze time of a control group

[0565] .

[0088] In some embodiments, the method

[0500] includes a step

[0555] of measuring one or more pupil diameters of the subject while performing an activity requiring less cognitive effort (e.g., recognizing three targets or distinguishing between targets), and a step

[0570] of reporting that a decline in cognitive resource function has been detected in the subject [5] if the mean pupil diameter of the subject [5] while performing the activity requiring more cognitive effort does not show an increase compared to the mean pupil diameter of the subject [5] while performing the activity requiring less cognitive effort.

[0089] In some embodiments, the method

[0500] includes the step

[0560] of counting the number of gaze fixations made by the subject [5] while looking at the target

[30] , and the step

[0575] of reporting that attentional impairment has been detected in the subject [5] if the number of gaze fixations made by the subject [5] while looking at the target

[30] is greater than that of a control group.

[0090] (References) The above rules are based in part on the following research: 1.Fernandez G, Mandolesi P, Rotstein NP, Colombo O, Agamennoni O, Politi LE. (2013) Eye movement alterations during reading in patients with early Alzheimer disease. Invest Ophthalmol Vis Sci. pii: iovs.13-12877v1. doi: 10.1167 / iovs.13-12877. 2.Fernandez G., Manes F., Politi L., Orozco D., Schumacher M., Castro L., Agamennoni O., Rotstein N. (2016). Patients with Mild Alzheimer Disease Fail When Using Their Working Memory: Evidence from the Eye Tracking Technique. Journal of Alzheimer Disease; 50, 827 - 828. 3.Fernandez, G., Laubrock, J., Mandolesi P., Colombo O., Agamennoni O. (2014) Registering eye movements during reading in Alzheimer disease: difficulties in predicting upcoming words. Journal of Clinical and Experimental Neuropsychology; 36, 302-16. 4.Fernandez G., Sapognikoff M., Guinjoan S., Orozco D., Agamennoni O. (2016). Word processing during reading sentences in patients with schizophrenia: evidences from the eyetracking technique. COMPREHENSIVE PSYCHIATRY; 68, 193-200. 5.Fernandez G, Manes F, Rotstein N, Colombo O, Mandolesi P, Politi L, Agamennoni O. (2014) Lack of contextual-word predictability during reading in patients with mild Alzheimer disease. Neuropsychologia; 62, 143-51. 6.Fernandez G., Schumacher M., Castro L., Orozco D., Agamennoni O., (2015). Patients with Alzheimer disease produced shorter outgoing saccades when reading sentences. Psychiatry Research, 229, 470-478. 7.Fernandez G., Biondi J., Castro S., Agamennoni O. (2017). Pupil size behavior during online processing of sentences. Journal of Integrative Neurosciences 15(4) 485- 496. 8.Biondi J., Fernandez G., Castro S., Agamennoni O. (2018). Eye-movement behavior identification for Alzheimer Disease diagnosis. Journal of Integrative Neurosciences (in Press). 9.Fernandez, Orozco, Agamennoni, Schumacher, Sanudo, Biondi, Parra. (2018). Visual Processing during Short-Term Memory Binding in Mild Alzheimer's Disease. J Alzheimers Dis.;63(1):185-194. doi: 10.3233 / JAD-170728.

[0091] (Parkinson's disease (PD) and attention deficit hyperactivity disorder (ADHD)) Reference is now made to Figures 6A and 6B, which illustrate methods for detecting one or more cognitive, neurological, and behavioral disorders in a person by measuring the person's eye movements and / or pupil diameter while the person performs a visual test, according to some embodiments of the present invention.

[0092] The method

[0600] includes the steps of providing a system for detecting the presence of one or more cognitive disorders and neurological disorders by measuring eye movements while a person is viewing, recognizing, maintaining, matching, limiting, and ordering targets; receiving eye-tracking data of the person viewing, recognizing, maintaining, matching, limiting, and ordering the targets; analyzing the eye-tracking data for evidence of the one or more cognitive disorders and neurological disorders; and displaying a report of the detection of the cognitive disorders and neurological disorders.

[0093] In some embodiments, the method

[0600] includes the step of counting the number of gaze fixations of the person while administering the visual test

[0615] , and reporting that impaired attention, executive, and inhibitory processing has been detected if the number of gaze fixations of the person is greater than that of a control group.

[0094] In some embodiments, the method

[0600] includes a step

[0620] for calculating the average saccade velocity

[0620] of the subject [5] from one target to another while the subject [5] is performing a visual test, and a step for reporting that a decline in executive function has been detected if the average saccade velocity performed by the person is less than that of a control group.

[0095] Physiologically, lower saccade velocity correlates with deterioration of the frontal eye field, basal ganglia, and superior colliculus. In some embodiments, reports of functional decline in saccade velocity can be used in further treatment.

[0096] In some embodiments, the method

[0600] includes the steps of counting the number of correct target recognitions by the person while performing the visual test

[0625] and reporting that working memory impairment has been detected if the number of correct target recognitions is lower than that of a control group.

[0097] Physiologically, working memory decline correlates with deterioration of the prefrontal cortex and posterior parietal lobes. In some embodiments, reports of declines in working memory, inhibitory processing, and mental flexibility can be used in further treatment.

[0098] In some embodiments, the method

[0600] includes a step

[0630] of calculating the average saccade amplitude from one gaze fixation to the next, and a step of reporting that a decline in executive processing has been detected if the average saccade amplitude is smaller than that of a control group.

[0099] In some embodiments, the method

[0600] includes a step

[0640] of tracking the person's pupil diameter while administering the visual test, and a step of reporting that a decline in attentional processing has been detected if the person's pupil diameter does not increase as administration of the visual test progresses.

[0100] Physiologically, impairment of attentional processing correlates with deterioration of the locus coeruleus, noradrenergic system, and superior colliculus. In some embodiments, reports of impairment of executive processing can be used in further treatment.

[0101] In some embodiments, the method

[0600] includes a step

[0635] of calculating the total time spent by the person while performing the visual test, and a step of reporting that an impairment in attentional processing has been detected if the total time required to perform the trials is greater than the time reported for a control group.

[0102] Physiologically, impairments in attentional and inhibitory processes and mental flexibility are correlated with deterioration of the prefrontal cortex, posterior parietal cortex, prefrontal-striatal-cerebellar, and prefrontal-striatal-thalamic circuits. In some embodiments, reports of impairments in executive processes can be used in further treatment.

[0103] In some embodiments, the method

[0600] includes a step

[0645] of calculating the person's time fixating on the target while performing the visual test, and a step of reporting that a working memory impairment has been detected if the time fixating on the target is less than that of a control group.

[0104] Physiologically, impairments in attention and inhibitory processing, and impairments in mental flexibility are correlated with deterioration of the prefrontal cortex, frontal eye fields, and dorsolateral parietal cortex. In some embodiments, reports of impairments in executive processing can be used for further treatment.

[0105] The method utilizes the following variables to analyze the subject using an intelligent algorithm: a. The total number of eye fixations by the subject while performing the visual test b. Identification number of each target by its location in the maze or labyrinth c. The subject's pupil diameter while performing the visual test d. Number of blinks from the left eye, right eye, or both eyes e. Microsaccades: Elements of formation (FF) i. HEWI: Showing the relationship between height and width of microsaccades ii. AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade iii. LONG: The length of the microsaccade trajectory in the horizontal-vertical plane iv. ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade v. AANG: The sum of the absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. These last two FFs estimate the regularity of the microsaccade trajectory. vi. MOD and THETA: Cartesian polar modulus and angle. These give the spatial orientation of the microsaccade relative to the center of fixation. vii. TIME: The duration of the microsaccade in milliseconds. viii. VMIN and VMAX: The minimum and maximum microsaccade velocities in degrees per second. ix. Microsaccade rate: The instantaneous rate of change over each time interval. x. Directional congruency: The congruency between the microsaccade direction and the stimulus location. f. Gaze position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes while performing the visual task g. Saccade amplitude while processing the target h. Saccade latency i. The fixation sequence (i.e., eye movements) while processing the target. This sequence is made available from an image, matrix, etc. j. Distance between the fixation points of the right and left eyes during target processing k. Subject's philia information (i.e., age, years of education, gender, ethnicity, occupation, hours of physical activity per week, etc.) l. Fixation duration while processing the target m. Number of fixations on each target n. Number of fixations outside each target o. Total visual task time (i.e., the time the subject spent performing the entire trial)

[0106] Methods

[0600] was tested in patient subjects with PD and ADHD and compared with healthy controls. Parkinson's disease [Table 2] ADHD [Table 3]

[0107] (Evaluation of treatment regimens) Described herein are methods of using the above systems and techniques to evaluate a treatment regimen (e.g., a pharmaceutical product, such as a drug, pharmaceutical product, etc.) being administered by a patient as directed by a healthcare professional. In this manner, healthcare professionals and / or pharmaceutical companies can better track the effectiveness of a treatment regimen in a patient and modify and supplement the regimen as needed based on evaluations throughout the course of the disease. For example, drugs or other pharmaceutical products that can be evaluated include neurological and / or psychiatric drugs that have neurological and / or psychiatric effects.

[0108] By way of illustration only and not limitation of the methods described herein, examples are provided in which eye movements are modeled in MS patients receiving or treated with different drugs (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab) to (a) reduce inflammation and prevent cellular damage that can lead to the symptoms of multiple sclerosis, (b) test sphingosine-1-phosphate receptor modulators that sequester lymphocytes in lymph nodes and prevent them from participating in an autoimmune response, (c) check immunosuppressants that act on lymphocyte pathways, and (d) analyze the effect of monoclonal antibodies in inhibiting lymphocyte B activation. Additionally, methods are described that allow healthcare professionals, pharmaceutical companies, and others to assess the effects of these drugs and other treatments on cognitive function and higher motor skills.

[0109] Understanding the effects of various medications (e.g., drugs) on the central nervous system (CNS) and peripheral nervous system (PNS) through the analysis of eye movements during well-defined activities (e.g., Go-No-Go and N-Back tests) allows healthcare professionals to test the level and efficacy of medications or treatments on patients' disease progression. In this context, healthcare professionals have access to a new tool for testing the effects of medications on cognitive and fine motor changes in subjects. Furthermore, pharmaceutical companies will gain objective and quantitative measurements of well-defined drug effects, opening new avenues for analyzing who should receive new medications (including dosages) and which patients will better absorb their medications.

[0110] Some embodiments of the methods described herein can perform one or more of calculating, modeling, and reporting the effect of one or more drugs (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or therapies to test for (a) a reduction in inflammation and neuronal damage that can contribute to the symptoms of multiple sclerosis; (b) damage to sphingosine-L-phosphate modulator receptors, which sequester lymphocytes in lymph nodes and prevent their involvement in autoimmunity; (c) damage caused by immunosuppressants that act on lymphocyte pathways; and / or (d) the therapeutic effect of monoclonal antibodies that inhibit the activity of lymphocyte B on several well-defined neurological processes and associated cognitive activities.

[0111] We apply mathematical models, where possible dependent variables can be, for example, saccade amplitude, fixation duration, pupil behavior, etc., and predictors can be motor scales, cognitive scales, years of disease diagnosis, and treatments (i.e., medications). We obtain regression coefficients, standard errors, and t-values ​​from each model to understand how treatments affect specific eye movements (e.g., saccade amplitude), eye movement combinations, related cognitive functions, and related brain regions. We perform a first measurement (baseline) and repeat the measurements (if necessary) to check whether the treatment is working properly.

[0112] The following example illustrates the use of saccade amplitude as a dependent variable. Saccade amplitude depends on the plan the individual develops to scan a figure while performing a particular test. When the test is an n-back test, by its very nature, better individuals perform longer saccades. Longer saccades suggest better working memory functioning, while shorter saccades suggest poorer performance (as previously shown in this patent). In this case, the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus must be spared for longer saccade amplitudes. The underlying argument is that the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus are key to determining the location of different fixations (thus influencing saccade amplitude) (Fielding et al., 2015). Furthermore, saccades should be longer when individuals demonstrate better performance on cognitive scales (e.g., The Symbol Digit Modalities Test) and motor scales (e.g., The Expanded Disability Status Scale). In this context, better cognitive scale results are positively correlated with preserved working memory, and better motor comprehension is positively correlated with preserved high levels of motor function. Therefore, the Symbol Digit Modalities test and the Expanded Disability Status Scale can be used as predictors.

[0113] If a person is treated with dimethyl fumarate (which may result in a reduction in inflammation and neuronal damage that can contribute to the symptoms of multiple sclerosis) and has longer saccade amplitudes while performing the N-Back task, it can be inferred that the treatment is having a positive effect on working memory, the dorsolateral prefrontal cortex, the basal ganglia, and the superior colliculus. In Figure 7, a positive effect can be seen in patients taking dimethyl fumarate, who show longer saccade amplitudes as treatment progresses (4 years of treatment).

[0114] The following example illustrates how pupil size can be used. A patient's pupil behavior changes based on cognitive effort. When the task becomes more demanding, the pupil size increases (as previously described in this patent). When performing the N-Back task, the pupil size must increase given the complexity of the test. This specific behavior suggests that the noradrenergic system and the locus coeruleus are responding appropriately as cognitive load increases. This description is pupil size and cognitive load (Fernandez et al., 2012).

[0115] If a person is treated with interferon-β (which may attenuate the damage of immunosuppressants acting on lymphocyte pathways), and if pupil size increases as cognitive load increases, one might expect the treatment to have a positive impact on the amount of working memory resources used ( Sweller et al., 2011 ), as well as on the noradrenergic system and associated brain activity.

[0116] FIG. 8 is a flow chart illustrating a method for identifying specific changes in subjects with a given disease by analyzing eye movement patterns when using specific visual stimuli, where specific drugs or treatments would improve visual processing, cognitive function, and related brain activity.

[0117] (N-back task) In one example, a method is presented for assessing neurological impairments, fine motor skills, executive processing, decision-making, processing speed, and cognitive abilities associated with multiple sclerosis (MS) to monitor a particular medication (e.g., drug, medication, etc.) or treatment, the method comprising: a. providing a system for assessing neurological disorders, decline in fine motor skills, executive processing, decision-making, fine motor skills, and cognitive abilities associated with MS; b. Requiring the subject to fixate a reference target on a chart, the chart including multiple regions (e.g., rectangles) arranged in different zones; c. repeatedly presenting stimulus images in one zone to the subject, the subject being required to remember the zone in which each stimulus image appeared and the order in which it appeared; d. presenting to the subject a chart that does not include the stimulus image presented in step c, wherein the subject is required to fixate on the zone in which the stimulus image of step c appeared; e. measuring the saccades of the subject who is required to look at the zone where the stimulus image presented in step c is presented in response to the presentation of step d; f. repeating steps d and e of presenting the chart and measuring saccades; g. Repeating steps b to f for the number of trials while changing the time for which the stimulus image is displayed; h. A calculating step, i. WM effect, a measure that increases as WM demands increase, presented as the number of errors reported by the subject across all trials for the fixated stimulus image, and the number and proportion of trials; and ii. Saccade latency is defined as the time at which the subject initiates a saccade into the zone. Mean saccade latency and calculating one or more of: iii. The degree of impairment in working memory, with an increase in WM effects; and i. A degree of impairment in executive processing, accompanied by an increase in saccade latency; reporting one or more of: Including, j. wherein the method includes a further step including a measurement performed during the step of presenting the stimulus image, during which the subject is further required to view the stimulus image, and the measurement includes: a. The subject's pupil diameter and b. The number of fixations the subject made on the stimulus image and c. The time spent by the subject looking at the stimulus image, d. Binocular disparity between visual search and target viewing; e. The target on which the subject's gaze was fixed, where a visual stimulus had previously been present. f. When reviewing trials, count the number of consecutive hits on the target by the subject and g. Number of blinks from the right eye, left eye, or both eyes; h. an intelligent algorithm that takes eye behavior as input to classify a person's performance; i. Elements of microsaccade formation (FF) i) HEWI: indicates the height / width of the microsaccade. ii) AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade. ii) LONG: The length of the horizontal-vertical plane trajectory of the microsaccade. iii) ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade. iv) AANG: The sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. These last two FFs estimate the regularity of the microsaccade trajectory. v) MOD and THETA: the polar modulus and angle of the sum of the polar coordinates, which gives the spatial directionality of the microsaccade relative to the center of fixation. vi) TIME: The duration of the microsaccade in milliseconds. vii) VMIN and VMAX: The minimum and maximum velocity of the microsaccade in degrees per second. viii) Microsaccade rate: The instantaneous rate of change at each time interval. ix) Directional congruency: The congruency between the microsaccade direction and the stimulus location. measuring one or more of Gaze position information (ie, abscissa and ordinate) from the left eye, right eye, or both eyes during visual search is acquired.

[0118] Any one or more (or all) of the items calculated in step h may be omitted. Similarly, any one or more (or all) of the additional steps described in step i may be omitted.

[0119] Furthermore, in some embodiments, additional steps may further include calculating, modeling, and repeating the effects of one or more drugs (dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or treatments: (a) testing sphingosine-1-phosphate receptor modulators that reduce inflammation and prevent nerve damage that can lead to symptoms of multiple sclerosis; (b) testing sphingosine-1-phosphate receptor modulators that sequester lymphocytes in lymph nodes and prevent their participation in autoimmune responses; (c) checking immunosuppressants that act on lymphocyte pathways; and / or (d) analyzing the effects of monoclonal antibodies that inhibit lymphocyte B activation.

[0120] (Go-No-Go Task) In another example, methods (Go-No-Go) and systems are provided for assessing neurological disorders associated with multiple sclerosis, declines in fine motor skills, processing speed, decision-making, and cognitive processing.

[0121] In one particular example, a system and method are provided for detecting one or more neurological disorders and / or measuring fine motor skills, processing speed, decision-making, and cognitive processing in a subject by measuring gaze movements, eye movement characteristics, or pupil behavior while the subject is viewing (i.e., picturing something in their mind to imagine or remember), recognizing (i.e., identifying something from previously encountered), maintaining (i.e., retaining in existing memory), matching (i.e., inhibiting or pointing), inhibiting (i.e., preventing or refraining from doing something), and analyzing a goal. The system: a. an eye-tracking device configured to monitor the eye movements of a subject while the subject views, recognizes, maintains, matches, fixates, and analyzes a target; b. a processor configured to receive data from the eye-tracking device while the subject views, recognizes, maintains, matches, fixates, and analyzes the target; c. a display configured to display the test report received from the processor; and wherein the processor is further configured to analyze the eye-tracking data for evidence of one or more neurological disorders or general cognitive function and report the detection of the one or more neurological disorders or the measurement of the subject's cognitive function in a test report.

[0122] In one particular implementation, the processor, in receiving the eye tracking data from the eye tracking device, a. counting the total number of subject's gaze fixations while viewing, recognizing, maintaining, matching, fixating, and analyzing the target; b. reporting in the test report that attentional processing impairment has been detected if the subject's total number of gaze fixations while viewing, recognizing, maintaining, matching, fixating, and analyzing the target is greater than that of the control group; c. counting the number of correct fixation positions of the subject while viewing, recognizing, maintaining, matching, fixating, and analyzing the target; d. If the number of correct fixations of the subject is less than that of the control group, reporting in the test report that a functional impairment of executive processing has been detected; e. Counting the number of outward saccades in the direction of the correct cue while attempting to see, recognize, maintain, match, fixate, track, and analyze the target; f. If the subject makes a lower rate of outward saccades in the direction of the correct cue (e.g., the direction of the arrow) than the control group, reporting in the test report that a functional impairment of executive processing has been detected; g. Counting the number of outward saccades made by the subject that are directed opposite to the direction of the cue while attempting to view, recognize, maintain, match, fixate, track, and analyze the target; h. If the proportion of outward saccades that are opposite to the cue direction is greater than that of the control group, reporting in the test report that a functional impairment of inhibitory processing has been detected; i. calculating the average saccade amplitude from one gaze fixation to the next while looking at, maintaining, matching, fixating, tracking, and analyzing the target; j. if the average saccade amplitude is smaller than that of the control group, reporting in the test report that a functional impairment of executive processing has been detected; k. counting the subject's saccade latency lengths during gazes for viewing, recognizing, maintaining, matching, inhibiting, fixating, tracking, and analyzing the object; l. If the saccade latency length (time) is longer than that of the control group, reporting in the test report that a functional impairment in speed processing has been detected; m. tracking the subject's pupil diameter when viewing, recognizing, maintaining, matching, inhibiting, fixating, tracking, and analyzing the target; n. If the subject's pupil diameter does not change as the target is sighted, recognized, maintained, matched, matched, fixated, tracked, and analyzed, reporting in the test report that noradrenergic system hypofunction has been detected; o. examining the length of the subject's gaze fixation while attempting to view, recognize, maintain, match, fixate, track, and analyze the target; p. If the length of the gaze fixation is longer than that of the control group, reporting in the test report that a functional impairment of online processing has been detected; q. observing the subject's gaze duration while attempting to view, recognize, maintain, match, fixate, track, and analyze the target; r. If the gaze time is longer than the control group, reporting in the test report that a degradation in online processing has been detected; s. counting the number of correct targets recognized during the target sighting, recognition, maintenance, matching, fixation, tracking, and analysis; t. If the number of recognized correct targets is less than that of the control group, reporting in the test report that impairment of executive processing and working memory processing has been detected; u. counting blinks from the left eye, the right eye, or both eyes when viewing, recognizing, maintaining, matching, inhibiting, fixating, tracking, and analyzing a target; v. Applying intelligent algorithms that use eye behavior as input to recognize human performance v. Elements of microsaccade formation (FF) i. HEWI: Shows the relationship between microsaccade height and width. ii. AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade. iii. LONG: The length of the horizontal-vertical plane trajectory of the microsaccade. iv. ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade. v. AANG: The sum of the absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. vi. FF estimates the regularity of microsaccade trajectories. vii. MOD and THETA: Cartesian polar modulus and angle. These give the spatial orientation of the microsaccade relative to the center of fixation. viii.TIME: The duration of the microsaccade in milliseconds. ix. VMIN and VMAX: The minimum and maximum microsaccade velocities in degrees per second. x. Microsaccade rate: The instantaneous rate of change over each time interval. xi. Directional congruency: The congruency between the microsaccade direction and the stimulus location. measuring w. measuring the gaze positions (i.e., abscissa and ordinate) of the left eye, right eye, and both eyes while viewing, recognizing, maintaining, matching, ordering, and analyzing the target; x. Measuring total target viewing, recognition, maintenance, matching, fixation, tracking, and analysis times (i.e., the time the subject spends viewing the target throughout the trial); y. Counting the number of correct targets recognized during target viewing, recognition, maintenance, matching, inhibition, fixation, tracking, and analysis; and further configured to implement one or more (or all) of

[0123] The processor may be further configured to perform additional steps including calculating, modeling, and reporting the effects of one or more of drugs (dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or treatments: (a) reducing inflammation and preventing nerve damage that can cause symptoms of multiple sclerosis; (b) testing sphingosine-1-phosphate receptor modulators that sequester lymphocytes in lymph nodes and prevent their participation in autoimmune responses; (c) checking immunosuppressants that act on lymphocyte pathways; and / or (d) analyzing the effects of monoclonal antibodies that inhibit lymphocyte B activation.

[0124] (Assessing performance, motor skills, and cognitive abilities using virtual reality environments) Described herein are systems and methods for assessing changes in cognitive and motor skills using well-defined exercises in both healthy and unhealthy individuals, combining virtual reality (VR), eye-tracking (ET), and motion sensors for the hands and feet. The application of VR and ET in cognitive exercises using motion sensors can improve the effectiveness of interventions and the ability to quantify cognitive and motor skills, enhancing the effectiveness of training. For example, the combination of VR, visual scanning, and arm and hand movements can provide new information about a person's decision-making process and the integrity of their brain circuits (e.g., what a person does when visually inspecting a shape in a VR environment and deciding to move their hand to touch something). Such methodologies will enhance medical professionals' ability to analyze, quantify, and train cognitive and fine motor skills.

[0125] The eye-tracking system described herein can be integrated into a conventional head-mounted display (HMD), where the VR world is rendered and viewed by the user. Each manufacturer of commercially available HMD / VR devices implements eye-tracking in a slightly different way. However, the interface provided to developers allows them to access a vector (three numbers) that indicates the direction of gaze in the 3D virtual space created by the VR application. In this manner, the integration between the eye-tracking system and the VR application is seamless. This interface is typically available in multiple languages / game development environments. Examples of commercial VR devices that can be used include the HTC Vive Eye Pro, HP G2 Reverb Omnicept Edition, Varjo Aero, and Fove 0. Controllers provided with VR devices typically function to track arm movements. Some of them (such as Valve controllers) are compatible with a base independent of the VR headset used. Virtual reality (VR) controllers play a key role in unlocking immersive experiences in virtual environments. These tools allow users to interact with and manipulate digital realms, resulting in highly engaging and immersive experiences. Essentially, VR input devices act as a conduit for transmitting hand movement data to a computer system. This information is then processed and used to match objects present in the simulated world. Currently, there are two main categories of input devices in use: motion controllers and game controllers. Motion controllers utilize accelerometers and gyroscopes to detect changes in movement and direction. They can incorporate buttons, analog sticks, and a variety of input mechanisms depending on the specific device. Motion controllers excel in scenarios where direct interaction with the virtual environment is paramount, such as exploratory first-person perspectives. Game controllers, on the other hand, are more commonly associated with traditional gaming encounters.These controllers typically offer a variety of input options, including dual analog sticks and numerous buttons. Recent iterations of input devices add an additional layer of immersion by including haptic feedback. This enhancement enhances the virtual experience by providing tangible responses when interacting with elements within the virtual realm.

[0126] Mathematical models are applied where the dependent variables can be, for example, saccade amplitude, fixation duration, pupil behavior, hand reaction time, tracking accuracy, etc., and the independent variables can be motor measures, cognitive measures, years since disease diagnosis, treatment, etc. Regression coefficients, standard errors, and t-values ​​are calculated from each model to understand the effect of disease stage, treatment, and training on specific eye movements (e.g., saccade amplitude) or limb movements (e.g., hand reaction time), eye movement combinations, associated cognitive functions, fine motor pathways, and associated brain regions.

[0127] The following illustrative example describes how to use saccade amplitude as the dependent coefficient.

[0128] Saccade amplitude depends on the strategy deployed by the individual being assessed to gaze at a particular shape during a particular test. If the test is a go-no-go 3D test, due to the nature of the test, better performers will make longer saccades. Longer saccades suggest a well-functioning working memory, whereas shorter saccades indicate poorer performance (as previously discussed herein). For longer saccade amplitudes to occur, the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus must be spared. The rationale behind this assertion is that the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus are important in determining where different fixations occur (thus influencing saccade amplitudes) (see Fielding, J. et al. Nat. Rev. Neurol. 11, 637–645 (2015); doi:10.1038 / nrneurol.2015.174). For example, if a study is conducted in individuals diagnosed with multiple sclerosis, saccades will become shorter and less accurate as the impairment increases, and it can be inferred that the impairment results in significant impairments in working memory and in the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus.

[0129] The following illustrative example describes how to use the reaction of a person's hand as a dependent variable.

[0130] A person's hand reaction time assesses the average time it takes to initiate a manual response after visually perceiving a target. This can reflect a person's motor response time and coordination. This measurement can provide insight into the speed at which a person can translate visual information into motor actions. Hand reaction time assesses the potential efficiency of sensorimotor processing and a person's ability to quickly initiate a manual response. When performing Go-No-Go 3D, hand reaction time decreases as the complexity of the test increases. This specific behavior suggests that the primary motor cortex and cerebellum are responding appropriately as the test difficulty increases. For example, when individuals diagnosed with Parkinson's disease perform the survey, hand reaction time becomes slower and flexibility decreases as disability increases, indicating that the disease has a significant impact on hand speed, fine motor flexibility, and the primary motor cortex and cerebellum. A stepwise approach can also be taken to apply artificial intelligence algorithms and combine eye movement and limb variables. We begin by converting the input information into a tabular dataset using a variety of biological feature extraction methods and standardization and aggregation methods. When selecting AI algorithms, we prioritize robustness and choose models that inherently address overfitting and class imbalance. In the data processing pipeline or the model itself, we attempt to white-box and explainable processes, allowing for the detection and analysis of unexpected patterns and behaviors. We also conduct a series of statistical analyses (as described above) to ensure cross-device compatibility of the input data as well as the model results. Depending on the size of the dataset, out-of-bag cross-validation or random sample test set approaches are used to evaluate model performance. We use a variety of evaluation metrics that take into account not only general model performance but also inherent class imbalance and the difference between Type I and Type II error costs seen in the health domain.

[0131] Figure 9 shows a conceptual illustration of a system for assessing a person's performance [5] by applying a three-dimensional virtual reality (3DVR) environment combined with embedded eye-tracking technology (ET)

[10] and motion sensors for tracking the movement of extremities

[0110] such as hands and feet while the person performs a well-defined activity. The system shown in Figure 9 may be used in the manner described below.

[0132] (3D virtual reality and eye-tracking technology in the Go-No-Go task) A method for assessing performance in both healthy and unhealthy individuals [5] is presented to quantify specific motor and cognitive abilities. The method utilizes a three-dimensional virtual reality (3DVR) environment, as described above and in Figure 9, which can be used to assess a person's performance, motor skills, and cognitive abilities by applying 3DVR, ET, and limb motion tracking.

[0133] According to the method, a person is required to view objects on a VR screen while their gaze movements are registered. Each object has a defined characteristic, such as color, and moves toward the person at a defined speed, acceleration, and direction. In multiple iterations, the objects are presented to the person in different zones, and the person is required to visually observe the objects on the screen. The presented objects appear to move toward the subject in the 3D virtual environment, and when the person effectively virtually touches the correct object by moving a limb toward the shape, the object's apparent speed increases (or decreases). Similarly, when the person virtually touches the incorrect object by moving a limb, the object's speed decreases (or increases).

[0134] Figure 10 shows an example of how objects may be positioned through a virtual environment and how the objects and a person's hands may appear. In this example, the objects may be presented in two different colors and may appear to come from the screen background. The person is asked to touch the objects having one or more colors. The person can use their right hand or left foot to touch the appropriate object, depending on whether the object is presented at hand or foot level.

[0135] The method continues by measuring the saccades of a person in response to the presentation of the object. Specifically, the person looks at the object and touches it (or does not touch it), and the saccades are measured. These steps of requiring the person to visually observe the object and measuring the saccades may be repeated multiple times. While the gaze movements are registered, the person can be asked to view and virtually touch (or not touch) objects on the VR screen. For example, the person may be asked to virtually touch objects with a particular characteristic (e.g., red in color) and not virtually touch objects with another characteristic (e.g., green in color). The above steps may be repeated for any number of instances in which the objects are presented, and they may be presented at different speeds.

[0136] FIG. 11 is a graphical representation displayed as a heat map of an example of the density of eye movements recorded from the right and left eyes while a person performed a task in which they were required to touch an object.

[0137] FIG. 12 is a graphical representation displayed as a heat map of movement density recorded from right and left hand movements while a person performed a task in which they were required to touch an object.

[0138] Based on measurements of gaze and limb movements in response to presented demands, i. Inhibitory processing errors (i.e., how many times the person touched the incorrect shape) and ii. Saccade Latency: the average saccade latency, which is the amount of time it takes the person to initiate a saccade to a new shape; and may calculate any one or more metrics of iii. The degree of degradation in processing speed with increasing changes in the rate at which successive shapes are presented to the subject; and iv. The degree of impairment of executive processing due to increased inhibitory errors; Report one or more of the following. In some cases, the method includes additional steps, including taking additional measurements during the steps of presenting the object to the person and requesting the person to look at and touch (or not touch) the object. Illustrative examples of such additional measurements include: i. the width of a person's pupil dilation; ii. The number of fixations made by the person on the stimulus image; iii. The time spent by a person looking at the stimulus image; iv. Human binocular disparity while performing visual search and object viewing; v. Fixations on objects touched by the person and on the location of previous visual stimuli; vi. The number of consecutive objects touched by the person during the trial vii. The number of blinks from the left eye, the right eye, or both eyes; viii. The time taken to visually detect the object (see Figure 3 for a heatmap of eye movements) and ix. The time from seeing the object to starting to move the hands and / or feet; x. The time from when the person begins to move their hands and / or feet until they attempt to touch or dislocate the object; xi. The number of times the subject touches or does not touch the virtual object (see Figure 4 for a heat map of hand movements); xii. Optimal locations for target visualisation and locations where target visualisation is inefficient; xiii. Reaching depth towards the object during the touching action; xiv. The maximum hand velocity reached during the movement towards the touched green object; and xv. Hand dominance ratio of preferred hand during manual interaction tasks; xvi. Anticipatory time, which measures the average time it takes a person to anticipate and initiate a response following a visual cue; xvii. Next microsaccade: Elements of formation (FF) and 1) HEWI: Shows the relationship between the height and width of microsaccades 2) AREA: Indicates the area of ​​the rectangle inscribed by the microsaccade. 4) ANG: The sum of all angles in the horizontal-vertical plane of the microsaccade. 3) LONG: The length of the microsaccade trajectory in the horizontal-vertical plane. 5) AANG: The sum of the absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccade. These last two FFs estimate the regularity of the microsaccade trajectory. 6) MOD and THETA: The polar modulus and angle of the Cartesian coordinate system. These give the spatial orientation of the microsaccade relative to the center of fixation. 7) TIME: The duration of the microsaccade in milliseconds. 8) VMIN and VMAX: The minimum and maximum microsaccade velocities in degrees per second. 9) Microsaccade rate: The instantaneous rate of change at each time interval. 10) Directional congruency: The congruency between the microsaccade direction and the stimulus location. It may include one or more of the following: Other additional steps that may be taken while presenting an object to a person and requesting that they look at and touch (or not touch) the object may include measuring the person's gaze position (i.e., abscissa and ordinate) from their left eye, right eye, or both eyes while performing a visual inspection. The measurement is l. Eye movements while looking at and touching the target; ll. Total time to complete all tests; m. How many seconds it takes a person to visually identify the correct object and the incorrect object; may be considered.

[0139] Yet another additional step that may be performed includes quantifying neurological processes and associated cognitive activity when considering pupil size behavior, binocular disparity, microsaccade characteristics, saccade behavior, target touch rate, hand and foot movements, gaze, fixation duration, number of fixations, and / or executive function performance in 3DVR.

[0140] FIG. 13 is a flowchart describing one example of the methods described herein for assessing a person's performance, motor skills, and cognitive abilities.

Claims

1. 1. A method for assessing a person's performance, motor skills, and cognitive abilities, comprising: requesting a person to perform a task, the task requiring the person to virtually touch designated virtual objects, each having different designated characteristics, the designated virtual objects being presented in a three-dimensional (3D) virtual reality environment, the virtual objects moving towards or away from the person at a predetermined speed, acceleration and direction; repeating the requesting by requesting the person to perform the task multiple times for different ones of the virtual objects having different specified characteristics; measuring eye gaze and limb movements of the subject while the subject is viewing the virtual object and performing the task; identifying selected ones of the measured eye movements and limb movements that are associated with the performance, motor skills, and cognitive abilities of the person; determining expected gaze and limb movements of the person while the person is viewing the virtual object and performing the task, and comparing the expected gaze and limb movements with the selected ones of the measured gaze and limb movements to determine a deviation therebetween; assessing the performance, motor and cognitive abilities of the person based on the deviations; A method comprising:

2. The method of claim 1 , wherein the measured gaze movements include at least one of saccade amplitude, fixation duration, and pupil behavior.

3. The method of claim 1 , wherein the limb movements measured include limb reaction times required to perform a required task.

4. The method of claim 1 , wherein the different designated characteristics of the virtual objects are, but are not limited to, colors.

5. The evaluating step includes: i. Inhibition of processing errors (i.e., how many times the person touched the incorrect object); and ii. Saccade latency represents the amount of time the person needs to initiate a saccade to view successively viewed objects. The method of claim 1 , wherein the method further comprises determining a metric comprising:

6. 10. The method of claim 1, wherein the evaluating further comprises determining (i) the degree of degradation in processing speed with increasing changes in the rate at which successive objects are presented to the person, and (ii) the degree of degradation in executive processing with increasing inhibition errors.

7. acquiring one or more additional measurements while the person is viewing the virtual object and performing the task, the one or more additional measurements comprising: i. the width of the person's pupil dilation; ii. The number of fixations made by the person on the stimulus image; iii. The duration of the person's gaze on the stimulus image; iv. the person's binocular disparity while performing visual search and object viewing; v. Fixation on the target touched by the person and on the location of the previous stimulus image; vi. The number of consecutive objects touched by the person while performing the trial; and vii. The number of blinks from the left eye, the right eye, or both; viii. The time taken to visually detect the object; x) the time from when the person begins to move the hand and / or the foot until the person touches or attempts to touch the object; xi. The number of times the subject touched or did not touch the virtual object; xii. Optimal locations for target viewing and locations of low target viewing efficiency; xii. Tracking accuracy in maintaining visual focus on a moving object; xiii. The depth of the hand's reach towards the object during the touching action; xiv. the maximum hand velocity achieved by the hand during the movement towards the touched green object; xv. The proportion of preferred hand used during manual interaction; and xvi. Anticipation time, which measures the average time it takes the person to anticipate and initiate a response following a visual cue; xii. Microsaccades; 2. The method of claim 1, wherein the ion exchange rate is selected from the group consisting of:

8. A system for assessing a person's performance, motor skills, or cognitive abilities, comprising: a three-dimensional (3D) virtual reality device configured to establish a 3D virtual reality environment in which a plurality of virtual objects are presented to the person, the objects having at least one characteristic that differs from one another, and the objects moving towards or away from the person with a defined speed, acceleration, and direction; an eye-tracking device configured to measure eye movement of the person while the person is looking at the virtual object and performing a requested task, the requested task including a plurality of requests requiring the person to virtually touch designated virtual objects each having one of the designated characteristics; and one or more motion sensors configured to measure limb movements of the person while the person is performing the required task; a processor configured to receive data from the 3D virtual reality device, the eye tracking device, and the one or more motion sensors while the person is performing the required task, the processor further configured to: (i) identify selected ones of the measured eye movements and limb movements that are associated with the performance, motor skills, and cognitive abilities of the person; (ii) determine expected eye movements and limb movements of the person while the person is viewing the virtual object and performing the required task, and compare the expected eye movements and limb movements with the selected ones of the measured eye movements and limb movements to determine a deviation therebetween; and (iii) evaluate the performance, motor skills, and cognitive abilities of the person based on the deviation. Including, the system.

Images