Method for training and quantifying specific motorizing skils and cognition procedure of a human by analyzing eye motion pattern w using 3D virtual reality device and eye tracking

Abstract

Systems and methods are provided that combine virtual reality (VR), eye tracking (ET), and motion sensors on limbs, such as hands and feet, to evaluate changes in cognitive and athletic capabilities of healthy and non-healthy persons using explicitly defined exercises. The application of VR, ET and motion sensors in cognitive practice can improve the efficacy of intervention and the ability to quantify cognitive ability and athletic ability, and enhance the effectiveness of human training.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application also claims the benefit of U.S. Provisional Application No. 63 / 373,228, filed on August 23, 2022, and also claims the benefit of U.S. Application No. 18 / 227,577, filed on July 28, 2023, and U.S. Serial No. 18 / 217,688, filed on July 3, 2023. The contents of the above applications are incorporated herein by reference. Background Art

[0002] Eye tracking systems have been used as diagnostic tools. For example, co-pending U.S. application Ser. No. 18 / 217,688 [Case No. 9901 / 1c1] (which is incorporated herein by reference in its entirety) shows a system for detecting one or more neurological disorders in a subject by measuring eye movements. Examples of neurological disorders that can be detected include multiple sclerosis (MS), attention deficit hyperactivity disorder (ADHD), Parkinson's disease (PD), Alzheimer's disease (AD), etc.

[0003] In addition, virtual reality (VR) has been used to train cognitive abilities. Human cognitive abilities can be roughly divided into working memory, attention, perception, reasoning and judgment, decision-making, etc. However, VR research is mainly conducted by collecting partial data related to behavioral feedback (for example, how many correct responses are there when performing an activity? Or how much total time is required to complete a task?). More comprehensive and in-depth VR research is needed to evaluate, train, and improve cognitive abilities. Summary of the invention

[0004] The object of the present invention is to provide a system for detecting one or more neurological disorders in a subject by measuring eye movements; the measurement of eye movements is performed while the subject is reading; the system comprises a. an eye tracker

[10] configured to monitor eye movements of the subject [5] while the subject [5] reads a text

[15] ; b. a processor

[20] configured to receive data from the eye tracker

[10] as the subject [5] reads the text

[15] ; and c. a display device

[40] configured to display a test report

[50] received from the processor

[20] ; wherein the processor

[20] is further configured to analyze the eye tracking data for evidence of one or more neurological disorders or general cognitive performance, and is configured to report the detection of one or more neurological disorders or measurement of cognitive performance of the subject [5] in the test report

[50] .

[0005] Another object of the present invention is as stated, the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. counting the total number of eye fixations made by the subject while reading the text; and b. If the total number of eye fixations of the subject while reading is higher than that of the control group, then it is reported in the test report that an impairment of attentional processes is detected.

[0006] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. counting the number of positive eye fixations of the subject while reading the text; and b. If the subject's number of positive eye fixations is lower than that of the control group; and the subject's number of eye fixations while reading is higher than that of the control group, then report in the test report

[50] that working memory impairment has been detected.

[0007] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. counting the number of words that the subject fixated on only once while reading the text; and b. If the number of words that the subject fixated only once is lower than the control group, reporting in the test report that an impairment in retrieval memory was detected.

[0008] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. counting the number of multiple eye fixations of the subject while reading the text; and b. If the number of said multiple eye fixations is higher than said control group, reporting in said test report that an impairment of executive process is detected.

[0009] It is another object of the present invention to detect one or more neurological disorders in a subject by measuring eye movements, wherein the processor is further configured to, upon receiving the eye tracking data from the eye tracker: a. Calculate the average saccade amplitude from one eye fixation to the next; and b. If the mean saccade amplitude is lower than the control group, reporting in the test report that an impairment in executive processing was detected.

[0010] Another object of the present invention is as described above, further comprising a device for measuring the pupil diameter of the subject

[17] , wherein the processor is further configured to: a. tracking the pupil diameter of the subject reading the text; and b. If the subject's pupil diameter does not show a decrease as reading of the text progresses, then reporting in the test report that an impairment of executive processing was detected.

[0011] The object of the present invention is to provide a system for detecting one or more neurological disorders and examining cognitive performance in a subject by measuring eye movements and pupil behavior and applying intelligent algorithms; the measurement of eye movements is performed while the subject is reading; the system comprises a. an eye tracker

[10] configured to monitor eye movements and pupil behavior of the subject [5] while the subject [5] reads a text

[15] ; b. a processor

[20] configured to receive data from the eye tracker

[10] while the subject [5] reads the text

[15] ; c. Intelligent algorithms for learning, identifying, typifying and classifying eye movement features in and within pathologies; and d. a display device

[40] , the display being configured to display the output of the intelligent algorithm on a test report

[50] received from the processor

[20] ; Wherein the processor

[20] is further configured to analyze and model the eye tracking data to obtain evidence of one or more neurological disorders and cognitive performance, and is configured to report in the test report

[50] the detection and classification of one or more neurological disorders of the subject [5] between and within pathologies.

[0012] Another object of the present invention is as described above, wherein the processor is further configured to identify and classify eye movement characteristics and pupil behavior during reading of the text after receiving the eye tracking data from the eye tracker, and provide the output of the classifier for reporting the subject's cognitive performance and / or pathological classification (i.e., corresponding to the subject's condition due to his / her eye movement characteristics) in the test report; and values ​​within the condition (i.e., the level of cognitive impairment, behavioral impairment, and biological impairment displayed by the subject within a specific condition).

[0013] Another object of the present invention is as described above, wherein the intelligent algorithm is configured to read at least one input selected from the group consisting of: a. Index of the total number of eye fixations made by the subject while reading the text. b. An index of the subject's positive eye fixation while reading the text. c. Index of words that the subject fixated on only once while reading the text d. Index of multiple eye fixations of the subject while reading the text e. The average amplitude of the saccade from one eye fixation to the next f. The pupil diameter of the subject reading the text g. Index of blinks from the left eye, right eye, or both eyes. h. Form factor of microsaccades (FF): i. HEWI: shows the height / width relationship of microsaccades. ii. AREA: shows the area of ​​the rectangle inscribed within the microsaccade. iii.LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade. iv.ANG: is the sum of all angles in the horizontal-vertical plane of the microsaccade. v.AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. These last two FFs give an estimate of the regularity of the microsaccade trajectory. vi.MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation. vii.TIME: is the duration of the microsaccade in milliseconds. viii. VMIN and VMAX: are the minimum and maximum speeds of microsaccades in degrees per second. ix. Microsaccade rate: It is the instantaneous rate in each time period. x. Direction consistency: It is the consistency between microsaccade direction and stimulus position. i. Eye position (ie, abscissa and ordinate) from the left eye, right eye, or both eyes during reading of the text. j. The sequence of fixations (ie eye behavior) during reading of the text. The sequence will be obtainable from an image, matrix, etc. k. The separation distance between eye fixations during reading of the text. 1. Filia information of the subject (ie, age; years of education; gender; ethnicity; occupation; hours of physical activity per week). m. Total reading time (ie, the time the subject spent while reading the text).

[0014] The present invention aims to provide a method for evaluating impairment of nervous system function associated with multiple sclerosis [MS]

[300] , the method comprising a. Provide a system for evaluating the impairment of neurological function associated with MS

[305] ; b. Asking the subject to look at a reference target on the chart

[310] ; c. performing multiple repetitions, presenting stimulus images to the subject in one of the partitions

[315] ; requiring the subject to remember in which partition each stimulus image appeared and in what order; d. presenting to the subject a prompt corresponding to one of the presented stimulus images

[320] ; e. measuring the subject's saccades in response to the step of presenting the prompt

[325] ; requiring the subject to look at the partition of the stimulus image corresponding to the prompt; f. Repeat the steps of presenting the cue and measuring saccades

[330] ; g. Repeat steps bf for multiple tests

[335] ; h. Calculate one or more of the following: i. WM effect

[340] (i.e., the WM effect is a measure that increases when WM demands increase. For each cue digit, the WM effect is represented by the ratio between the number of errors reported by the subject across all trials and the number of trials); and ii. mean saccade latency

[345] , where saccade latency is defined as the amount of time it takes for the subject to elicit a saccade to the partition; and i. Report one or more of the following: i. the extent of working memory impairment as WM effects increase

[350] ; and ii. the extent to which executive processing is impaired as saccadic latency increases

[355] ; j. wherein the method further comprises an additional step, the additional step comprising measurements performed during the step of presenting the stimulus image

[315] , during which the subject is further asked to look at the stimulus image; the measurements comprising measuring one or more of the following: i. the subject's pupil dilation magnitude

[360] ; ii. the number of fixations made by the subject to the stimulus image

[365] ; and iii. the subject's gaze duration on the stimulus image

[370] ; and k. The additional steps further include calculating and reporting one or more of the following: i. the extent to which subcortical processes are impaired as the magnitude of pupil dilation increases

[375] ; ii. the extent to which executive processing is impaired as the number of fixations increases

[380] ; and iii. The extent to which executive processes and working memory are impaired as the duration of the gaze increases

[385] .

[0015] Another object of the present invention is as described above, wherein the reference target is at a central position of the graph, and the plurality of partitions are arranged around the reference target.

[0016] Another object of the present invention is as described above, wherein the prompt is arranged at the position of the reference target.

[0017] Another object of the present invention is as described above, wherein the error is defined as an eye movement towards a location other than the correct partition and / or a saccade not elicited within a time limit.

[0018] Another object of the present invention is as described above, wherein the cue corresponding to the first stimulus presented is excluded from the presented cue numbers.

[0019] Another object of the present invention is as described above, wherein the saccade is included in the step of calculating the WM effect and the saccade latency only when a saccade is induced for more than a minimum saccade latency after the step of presenting the cue digit.

[0020] Another object of the present invention is as described above, wherein the saccade is excluded from calculating WM if: a saccade is not made to one of the partitions within a time limit, fixation on the reference target cannot be maintained before starting a saccade to one of the diagonal partitions, and blinking results in an uncertain eye movement a. The object of the present invention is to provide a system for detecting one or more neurological disorders in a subject by measuring eye movements; the eye movements are measured while the subject is undergoing a visual test; the system comprises an eye tracker

[10] configured to monitor eye movements of the subject [5] while the subject [5] performs the vision test

[15] ; b. asking the subject to sequentially fixate on targets that are part of a group of targets (e.g., dots) presented together in the same picture (i.e., a maze or a maze)

[605] ; c. Ask the subject to fixate on only one target at a time until visualization of all targets throughout the picture is completed following the maze or maze directions (i.e., entering from the bottom of the maze or maze and exiting through the top)

[610] . d. a processor

[20] configured to receive data from the eye tracker

[10] while the subject [5] is performing the vision test

[15] ; and e. a display device

[40] configured to display a test report

[50] received from the processor

[20] ; Wherein the processor

[20] is further configured to analyze the eye tracking data for evidence of neurological disorders and attention disorders, and is configured to report the detection of one or more neurological disorders and attention disorders in the subject [5] in the test report

[50] .

[0021] Another object of the present invention is as described above, wherein the processor is further configured to, upon receiving the eye tracking data from the eye tracker when the subject visualizes, identifies, maintains, controls, inhibits, and sorts a target: a. counting the total number of eye fixations made by the subject while performing the visual test

[615] ; and b. If the subject's total number of eye fixations when visualizing the target is higher than the control group, then report in the test report that an impairment in attentional processing was detected.

[0022] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. measuring the average speed of saccades when the subject switches from one target to another

[620] ; and b. If the subject's average saccade velocity

[620] is lower than that of the control group; reporting in the test report

[50] that an impairment of executive function was detected.

[0023] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. counting the number of correct target identifications

[625] ; and b. If the subject's number of correct target identifications

[625] is lower than that of the control group, reporting in the test report that working memory impairment has been detected.

[0024] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. Calculate the average saccade amplitude

[630] ; and b. If the mean saccade amplitude

[630] is lower than the control group, reporting in the test report that an impairment in executive processing was detected.

[0025] Another object of the present invention is as stated above, wherein the processor is further configured to, after receiving the eye tracking data from the eye tracker: a. the total time spent performing the visual test

[635] ; and b. If the total time

[635] spent performing the visual test is higher than the control group, then reporting in the test report that impairment of attentional processes was detected.

[0026] Another object of the present invention is as described above, further comprising a device for measuring the pupil diameter of the subject

[17] , wherein the processor is further configured to: a. Tracking the pupil diameter of the subject undergoing the vision test

[640] ; and b. If the subject's pupil diameter

[640] does not show an increase as the task progresses, then report in the test report that impairment of attentional processing was detected.

[0027] Another object of the present invention is as described above, further comprising a device for measuring the pupil diameter of the subject

[17] , wherein the processor is further configured to calculate the duration of a person's gaze on a target when performing the visual test, and if the subject's gaze duration

[645] when looking at the target is lower than that of the control group, then reporting in the test report that impairment of attention and executive processes is detected.

[0028] The object of the present invention is to provide a method for detecting the presence of one or more neurological disorders or measuring general cognitive performance in a subject by measuring eye movements of the subject

[400] ; the measurement of eye movements is performed while the subject is reading

[405] ; the method comprising the following steps: a. providing a system for detecting one or more neurological disorders as claimed in claim 1 or claim 18; b. receiving eye tracking data and / or pupil diameter data of the subject while the subject is reading text

[415] ; The method further comprises the steps of analyzing the eye tracking data and / or pupil diameter data for evidence of one or more neurological disorders

[417] , and displaying a report on the detection of the one or more neurological disorders

[499] .

[0029] Another object of the present invention is as mentioned above, further comprising the following steps: a. counting the total number of eye fixations made by the subject while the subject is reading the text

[420] ; and b. If the total number of eye fixations made by the subject while reading the text is higher than that of the control group, an impairment of attentional processing is reported

[460] .

[0030] Another object of the present invention is as mentioned above, further comprising the following steps: a. counting the total number of eye fixations made by the subject while the subject is reading the text

[420] ; b. counting the number of positive eye fixations made by the subject while the subject is reading the text

[430] ; and c. If the subject's number of positive eye fixations while reading the text

[430] is lower than the control group; and the subject's number of eye fixations while reading is higher than the control group, then it is reported that working memory impairment is detected

[470] .

[0031] Another object of the present invention is as mentioned above, further comprising the following steps: a. counting the number of times the subject's eyes fixate on each word in the text while the subject is reading the text

[440] ; b. counting the number of words that the subject fixated on only once while reading the text

[445] ; and c. If the number of words that the subject fixated on only once while reading the text

[445] is lower than that of the control group, then it is reported that an impairment in retrieval memory is detected

[480] .

[0032] Another object of the present invention is as mentioned above, further comprising the following steps: a. counting the number of eye fixations made by the subject while reading the text

[450] ; and b. If the number of multiple eye fixations made by the subject while the subject was reading the text

[450] is higher than that of the control group, then an impairment of executive processing is reported to be detected

[490] .

[0033] Another object of the present invention is as mentioned above, further comprising the following steps: a. calculating the average amplitude of the subject's saccades from one eye fixation to the next while reading the text

[454] ; and b. If the subject's mean saccade amplitude from one eye fixation to the next while reading the text

[454] is lower than that of the control group, then reporting in the test report that an impairment in executive processing was detected

[491] .

[0034] Another object of the present invention is as mentioned above, further comprising the following steps: a. Tracking the pupil diameter of the subject while reading the text

[456] ; b. If the pupil diameter

[456] of the subject reading the text does not show a decrease as reading of the text progresses, then reporting in the test report that an impairment of executive processes was detected

[492] .

[0035] The object of the present invention is to provide a system for detecting a disorder of a subject's memory binding function

[100] , the system comprising: a. Eye tracker

[10] ; b. A device for measuring pupil diameter; c. A processor

[20] configured to: i. receiving eye tracking data of the subject [5] from the eye tracker

[10] ; ii. receiving the pupil diameter data of the subject [5] from the device for measuring pupil diameter; and d. a display device

[40] configured to display a test report

[50] received from the processor

[20] ; Wherein the processor

[20] is further configured to analyze the eye tracking data and the pupil diameter data, and is configured to report the detection of one or more impairments in the memory binding function of the subject [5] in the test report

[50] .

[0036] Another object of the present invention is as described above, wherein the processor

[20] is further configured to, after receiving the eye tracking data from the eye tracker

[10] : a. measuring one or more gaze durations of the subject [5] on each of the one or more targets observed by the subject [5]; b. calculating the average gaze duration of the subject [5] on the target; and c. If the mean gaze duration of the subject [5] is longer than the mean gaze duration of the control group, then report in the test report

[50] that impaired encoding and recognition of the target was detected in the subject [5].

[0037] Another object of the present invention is as described above, wherein the processor

[20] is further configured to, after receiving the eye tracking data from the eye tracker

[10] : a. counting the number of eye fixations made by the subject [5] while observing one or more targets; and b. If the number of eye fixations performed by the subject [5] while observing the target is higher than that of the control group, it is reported in the test report

[50] that an impairment of attention processes is detected in the subject [5].

[0038] Another object of the present invention is as described above, wherein the processor

[20] is further configured to apply an intelligent algorithm and is configured to: a. receiving the pupil diameter of the subject [5] from the device for measuring pupil diameter while the subject [5] is performing an activity requiring low cognitive effort; b. receiving the pupil diameter of the subject [5] from the device for measuring pupil diameter while the subject [5] is performing an activity requiring high cognitive effort; and c. If the pupil diameter of the subject [5] does not show an increase when performing the activity requiring the greater cognitive effort relative to the pupil diameter of the subject [5] when performing the activity requiring reduced / minimal cognitive effort, then it is reported in the test report

[50] that impairment of cognitive resources was detected in the subject [5].

[0039] Another object of the present invention is as described above, wherein the processor

[20] further reports in the test report

[50] the result of the impairment of the memory binding function in the subject [5] that was not detected by the system

[100] .

[0040] The object of the present invention is to provide a method

[500] for detecting a disorder of memory binding function in a subject

[505] , the method comprising the following steps: a. Provide a system as claimed in claim 1 or claim 33; b. Presentation target

[510] ; c. Asking subjects to look at the targets and remember them (encoding)

[515] ; d. Present an empty screen

[520] ; e. Present a target and ask the subject to identify whether the target is identical to a previously observed target (recognition). If the target is identical, then the answer "same" must be given. If not identical, then the answer "different" must be given. A keyboard or similar support must be used to collect both answers

[525] . Repeat steps from [510-525] for multiple trials

[530] ; f. Repeat steps [510-525] to perform multiple tests

[530] ; g. Receive eye tracking data; h. The subject observes one or more targets

[540] ; i. measuring the subject's gaze duration on each of the targets

[545] ; j. calculating the average gaze duration of the subject on the target

[550] ; k. measuring the pupil diameter of the subject while performing an activity requiring low cognitive effort

[555] ; l. counting the number of eye fixations made by the subject while observing the target

[560] ; m. wherein the method further comprises the following steps: i. if the subject's mean gaze duration is longer than the control group's mean gaze duration, then reporting that an impairment in target encoding and recognition processes is detected in the subject

[565] ; ii. if the subject's pupil diameter does not show an increase when performing the activity requiring greater cognitive effort relative to the subject's pupil diameter when performing the activity requiring less cognitive effort, then reporting that impairment of cognitive resources has been detected in the subject

[570] ; and iii. If the subject makes a greater number of eye fixations while observing the target than a control group, then it is reported that an impairment of attentional processing is detected in the subject

[575] .

[0041] Another object of the present invention is as described above, wherein the intelligent algorithm is configured to read at least one input selected from the group consisting of: a. Total number of eye fixations made by the subjects during each binding task. b. Binding evaluation task, i.e. “binding color” of “unbound color”. c. Identification number of the binding experiment. d. The correct behavioral answer for the trial (i.e., “same” or “different”). e. Behavioral responses of the subjects. f. Part of the experiment, i.e. encoding or retrieval. g. The pupil diameter of the subject at the time of the binding assessment. h. Number of blinks from the left eye, right eye, or both eyes. i. Microsaccades; Form Factor (FF): i. HEWI: shows the height / width relationship of microsaccades. ii.AREA: shows the area of ​​the rectangle inscribed in the microsaccade. LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade. iv.ANG: is the sum of all angles in the horizontal-vertical plane of the microsaccade. v.AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. These last two FFs give an estimate of the regularity of the microsaccade trajectory. vi.MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation. vii.TIME: is the duration of the microsaccade in milliseconds. viii. VMIN and VMAX: are the minimum and maximum speeds of microsaccades in degrees per second. ix. Microsaccade rate: It is the instantaneous rate in each time period. x. Direction consistency: It is the consistency between microsaccade direction and stimulus position. j. Eye position (ie, abscissa and ordinate) from the left eye, right eye, or both eyes when the binding evaluation was made. k. The magnitude of the saccade when processing the target. 1. The sequence of fixations (ie, eye behavior) during processing of the target. The sequence will be available from an image, matrix, etc. m. The distance between the gaze points of the right eye and the left eye when performing the binding evaluation. n. Filia information of the subject (ie, age; years of education; gender; ethnic group; occupation; hours of physical activity per week). o. Fixation duration while processing the target. p. Gaze duration while processing the target. q. Number of fixations on each target. r. Number of fixations outside each target. s. Number of fixations on each target.

[0042] The object of the present invention is to provide a method for detecting a neurological disorder and an attention disorder in a subject

[600] , the method comprising the following steps: a. Provide an eye tracker

[10] ; b. A device for measuring pupil diameter; c. A processor

[20] configured to: i. receiving eye tracking data of the subject [5] from the eye tracker

[10] ; ii. receiving pupil diameter data of the subject [5] from the device for measuring pupil diameter; and iii. a display device

[40] configured to display a test report

[50] received from the processor

[20] ; Wherein the processor

[20] is further configured to analyze the eye tracking data and the pupil diameter data, and is configured to report the detection of one or more neurological disorders and attention disorders of the subject [5] in the test report

[50] .

[0043] Another object of the present invention is as described above, wherein the processor

[20] is further configured to, after receiving the eye tracking data from the eye tracker

[10] : a. measuring one or more fixation durations of the subject [5] on each of the one or more targets observed by the subject [5]; b. calculating the average saccade amplitude of the subject [5] from each target to another target; and c. If the mean saccade amplitude of the subject [5] is shorter than the mean saccade amplitude of the control group, then it is reported in the test report

[50] that impairment of visualization, recognition, maintenance, control, inhibition and sequencing of targets was detected in the subject [5].

[0044] Another object of the present invention is as described above, wherein the processor

[20] is further configured to, after receiving the eye tracking data from the eye tracker

[10] : a. counting the number of eye fixations made by the subject [5] while observing one or more targets; and b. If the number of eye fixations performed by the subject [5] while observing the target is higher than that of the control group, it is reported in the test report

[50] that an impairment of attention processes is detected in the subject [5].

[0045] Another object of the present invention is as described above, wherein the processor

[20] is further configured to: a. receiving the pupil diameter of the subject [5] from the device for measuring the pupil diameter when the subject [5] is performing an activity requiring primary attention resources; b. receiving the pupil diameter of the subject [5] from the device for measuring pupil diameter while the subject [5] is performing an activity requiring primary attention; and c. If the pupil diameter of the subject [5] does not show an increase when performing the activity requiring primary attention relative to the pupil diameter of the subject [5] when performing the activity requiring secondary attention, then it is reported in the test report

[50] that impairment of cognitive resources was detected in the subject [5].

[0046] The object of the present invention is to provide a method for detecting a neurological disorder and an executive disorder in a subject

[600] , the method comprising the following steps: a. Provide a system as described above; b. receiving eye tracking data; c. The subject observes one or more targets [605-610]; d. calculating the average saccade amplitude of the subject to the target

[630] ; e. measuring the pupil diameter of the subject while performing an activity requiring primary attention

[640] ; f. measuring the pupil diameter of the subject while performing an activity that requires primary attention greater than secondary attention; and g. counting the number of eye fixations made by the subject while observing the target

[615] ; h. wherein the method further comprises the following steps: i. if the subject's mean saccade amplitude is shorter than the control group's mean saccade amplitude, then it is reported that impairments in target visualization, recognition, maintenance, control, inhibition and sequencing processes are detected in the subject; ii. if the subject's pupil diameter does not show an increase when performing the primary attention activity relative to the subject's pupil diameter when performing the secondary attention activity, then it is reported that impairment of cognitive and functional resources is detected in said subject; and if the number of said eye fixations made by said subject when observing said target is higher than that of a control group, then it is reported that impairment of attention processes is detected in said subject. iii. if the subject's mean saccadic latency (speed) is shorter than the mean saccadic latency of the control group, then reporting that an impairment of executive processing is detected in the subject;

[0047] Another object of the present invention is as described above, wherein the method is configured to report that an impairment of executive process is detected in the subject if the average saccade duration of the subject is shorter than the average fixation duration of the control group.

[0048] Another object of the present invention is as described above, wherein the nervous system disorder is selected from the group consisting of Parkinson's disease or attention deficit hyperactivity disorder.

[0049] Another object of the present invention is to provide a system and method for evaluating a person's performance, motor skills, and cognitive abilities. The system includes a three-dimensional (3D) virtual reality device configured to establish a 3D virtual reality environment in which a plurality of virtual objects are presented to the person, the objects having at least one characteristic that is different from one another, the objects moving toward or away from the person at a defined speed, acceleration, and direction. The system also includes an eye tracker configured to measure the person's eye movements while the person observes the virtual objects and performs a required task, the required task including a plurality of requirements requiring the person to virtually touch a specified virtual object, each having one of the specified characteristics. One or more motion sensors are configured to measure the person's limb movements while the person performs the required task. The processor is configured to receive data from the 3D virtual reality device, the eye tracker, and the one or more motion sensors while the person performs the required task, and is further configured to (i) identify selected eye movements and limb movements from the measured eye movements and limb movements that are related to the person's performance, motor skills, and cognitive abilities; (ii) determine the person's expected eye movements and limb movements while the person observes the virtual object and performs the required task, and compare the expected eye movements and limb movements with the selected eye movements and limb movements from the measured eye movements and limb movements to determine deviations therebetween; and (iii) evaluate the person's performance, motor skills, and cognitive abilities based on the deviations. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 and Figure 2 A system for detecting one or more neurological disorders in a subject according to some embodiments of the present invention is shown.

[0051] Figure 3A and Figure 3B A method for assessing impairment of nervous system function associated with MS according to some embodiments of the present invention is shown.

[0052] Figure 4A and Figure 4B A method for detecting one or more neurological disorders in a reading subject according to some embodiments of the present invention is shown.

[0053] Figure 5 A method for detecting impairment of a memory binding function according to some embodiments of the present invention is shown.

[0054] Figure 5B Shown are test results according to the evaluation method of 5A: Corrected recognition of controls and AD patients during two experimental conditions (error bars = standard error of the mean). Figure 5C Results of tests of the evaluation method according to 5A are shown: Effect of the binding task on gaze duration during encoding and recognition moments in controls and Alzheimer's disease (AD) patients. The inset shows the partial effect of the LMM (i.e., after removing other fixed effects and variance components). The shaded area indicates the 95% confidence interval. Gaze durations are plotted on a logarithmic scale to correspond to the LMM.

[0055] Fig. 6A and Figure 6B Methods for detecting Parkinson's disease and attention deficit hyperactivity disorder are shown.

[0056] Figure 7 Shown is the effect of dimethyl fumarate on saccade amplitude in multiple sclerosis patients who have been taking the drug for 4 years.

[0057] Figure 8 is a flow chart illustrating a method for identifying specific alterations in subjects with a defined disease, analyzing eye movement patterns when using specific visual stimuli, where a specific drug or treatment will enhance visual processing, cognitive performance and related brain activity.

[0058] Fig. 9 A conceptual illustration of a system for evaluating a person's performance by applying a 3-dimensional virtual reality (3DVR) environment in conjunction with embedded eye tracking technology (ET) and motion sensors that track the movement of limbs (such as hands and feet) while the person performs well-defined activities is shown.

[0059] Fig.10 Examples are shown of how objects can be positioned within the overall virtual environment and how objects and human hands can appear.

[0060] Fig.11 is a graphical representation of an example of the density of eye movements recorded from the right and left eyes when a person performs a task of touching a required object, presented as a heat map.

[0061] Fig.12 is a graphical representation of the recorded movement density of the right and left hand movements when a person performs a task of touching a required object, presented as a heat map.

[0062] Fig.13 is a flow chart describing one example of a method described herein for assessing a person's performance, motor skills, and cognitive abilities. DETAILED DESCRIPTION

[0063] The term "cognitive effort" reflects the total amount of mental effort required by a subject to perform a task. In the present application, the term "lower cognitive effort" refers to a reduction in working memory demands when performing a task.

[0064] In this application, the term "microsaccades" (also called "flicks") are small saccades made during a fixation period. They are the largest and fastest of the fixational eye movements. In this application, the term "saccades" refers to rapid, simultaneous movements of both eyes between two or more phases of fixation.

[0065] In this application, the term "ocular drift" is a fixational eye movement characterized by a smoother, slower wandering motion when the eyes are fixated on an object.

[0066] In this application, the term "ocular microtremors" (OMT) are small, rapid and synchronized oscillations of the eye that occur at frequencies ranging from 40 to 100 Hz, although they typically occur at about 90 Hz in normal healthy individuals. They are characterized by their high frequency and extremely small amplitude of only a few arc seconds.

[0067] In this application, the term "stimulus image" refers to a specific visual pattern or target presented to a subject in a display. The term "visual task" or "visual test" refers to the activity performed by the subject when processing each stimulus image.

[0068] Non-limiting embodiments of the present invention will now be described in detail.

[0069] Reference now Figure 1 , which shows a system

[100] for detecting a neurological disorder or neurological function in a subject [5] according to some embodiments of the present invention.

[0070] The system

[100] includes an eye tracker

[10] , a device for measuring pupil diameter

[17] , a processor

[20] , and a display device

[40] .

[0071] The eye tracker

[10] can be of any type known in the art; for example, an eye-attached tracker, an optical eye tracker, or an electro-oculographic eye tracker.

[0072] The apparatus for measuring pupil diameter

[17] may include, for example, a camera configured to capture an image of the eye and a processing unit for measuring pupil diameter from the image. As an alternative to the processing unit, the apparatus for measuring pupil diameter

[17] may include an image display for manual measurement while viewing the display.

[0073] The eye tracker

[10] and the device for measuring pupil diameter

[17] are in communication connection with the processor

[20] . The communication connection can be any one or more forms known in the art and can be wired (e.g., USB, parallel port or similar) or wireless (e.g., WiFi, Bluetooth or similar).

[0074] The processor

[20] receives and executes instructions stored in one or more storage media

[60] (such as RAM, CD / DVD, HDD, flash memory and / or any suitable medium). The instructions instruct the processor

[20] to: 1) receive eye tracking data from the eye tracker

[10] ; 2) receive pupil diameter data from the device for measuring pupil diameter

[17] ; 3) analyze the eye tracking data and pupil diameter data (further explained herein); 4) report the detection or non-detection of one or more impairments of the memory binding function of the subject [5] in a test report 50 for display on a display device

[40] . The display device

[40] can be a monitor, a screen of a mobile device (such as a smartphone), a printout, or any suitable device for displaying the test report

[50] . The processor

[20] can store any of the received eye tracking data, any one or more intermediate results of the analysis stage, and / or the test report

[50] in the storage medium

[60] .

[0075] Neurological disorders detected by the system

[100] may include reading functions, such as impaired encoding and recognition of objects, impaired attention processes, impaired cognitive resources, or any combination thereof. In other embodiments, the disorders detected may include multiple sclerosis (MS), attention deficit hyperactivity disorder (ADHD), Parkinson's disease (PD), Alzheimer's disease (AD), etc.

[0076] In some embodiments, the processor

[20] receives eye tracking data from the eye tracker

[10] while the subject [5] observes each of the one or more targets

[30] . The processor

[20] measures the gaze duration of the subject [5] on each target

[30] observed by the subject [5]. The processor

[20] calculates an average gaze duration of the subject [5] on each target

[30] . If the average of the gaze durations of the subject [5] on the target

[30] is longer than the average gaze duration of the control group, the processor

[20] reports in the test report

[50] that an impairment in the target encoding and recognition process was detected in the subject [5].

[0077] In some embodiments, the processor

[20] additionally or alternatively counts the number of eye fixations made by the subject [5] while observing each target

[30] . If the number of eye fixations made by the subject [5] while observing the target

[30] is higher than the control group, the processor

[20] reports in the test report

[50] that an impairment of attentional processes was detected in the subject [5].

[0078] In some embodiments, the processor

[20] receives pupil diameter data from the device for measuring pupil diameter

[17] while the subject [5] is performing an activity that requires lower cognitive effort. The processor

[20] further receives pupil diameter data from the device for measuring pupil diameter

[17] while the subject [5] is performing an activity that requires greater cognitive effort than the activity that requires lower cognitive effort. If the average pupil diameter of the subject 5 while performing the activity that requires greater cognitive effort does not show an increase relative to the average pupil diameter of the subject [5] while performing the activity that requires lower cognitive effort, the processor

[20] reports in the test report

[50] that impairment of cognitive resources was detected in the subject [5].

[0079] The control group may include a statistically representative sample of the same demographic subgroup as the subject [5] (e.g., the same gender, race, ethnic culture, age group, and / or other demographic characteristics as the subject [5]). Eye tracking data for the control group may be obtained by the system

[100] or otherwise collected from a previous research study and / or clinical study. The system

[100] may treat the average gaze duration or number of eye fixations of the subject [5] as equal to the average corresponding value of the control group if the average gaze duration or number of eye fixations of the subject [5] is within a selected limit of the average value of the control group (approximately one standard deviation of the distribution of the corresponding values ​​of the control group).

[0080] It will be appreciated that the eye tracking data received by the processor

[20] may be a series of eye positions measured by the eye tracker

[10] , which the processor

[20] analyses to find gaze duration and eye fixations of the subject [5]. Alternatively, the processor

[20] may receive a series of pre-processed signals from the eye tracker

[10] , each signalling a gaze duration or that an eye fixation has occurred. The signals may optionally be accompanied by metadata (e.g., eye position, time and / or length of an eye fixation). Multiple sclerosis

[0081] Reference now Figure 3A and Figure 3B , which shows a method

[300] for evaluating impaired nervous system function associated with multiple sclerosis [MS] according to some embodiments of the present invention. The method

[300] comprises the following steps: a. Provide a system for evaluating the impairment of neurological function associated with MS

[305] ; b. Asking the subject to look at a reference target on the chart

[310] ; c. presenting the stimulus images to the subject in one of the multiple partitions of the chart for multiple repetitions

[315] ; requiring the subject to remember in which partition each stimulus image appeared and in what order; d. presenting the subject with a cue corresponding to one of the presented stimulus images

[320] ; e. measuring the subject's saccades in response to the step of presenting the cue

[325] ; requiring the subject to look at the partition where the stimulus image corresponding to the cue is presented; f. Repeat the steps of presenting the cue and measuring saccades

[330] ; g. Repeat steps bf for multiple tests

[335] ; h. Calculate one or more of the following: i. WM effect

[340] (i.e., the WM effect is a measure that increases when WM demands increase. For each cue digit, the WM effect is represented by the ratio between the number of errors reported by the subject across all trials and the number of trials); and ii. mean saccadic latency

[345] , where saccadic latency is defined as the amount of time it takes a subject to elicit a saccade to a partition; and i. Report one or more of the following: i. the extent of working memory impairment as WM effects increase

[350] ; and ii. the extent to which executive processing is impaired as saccadic latency increases

[355] ; wherein the method further comprises an additional step performed during the step of presenting the stimulus image

[315] ; during which the subject is further required to look at the stimulus image; j. The additional steps include measuring one or more of the following: i. the amplitude of the subject's pupil dilation

[360] ; ii. the number of fixations made by the subject to the stimulus image

[365] ; and iii. The subject’s gaze duration on the stimulus image

[370] . k. The additional steps further include calculating and reporting one or more of the following: i. the extent to which subcortical processing is impaired while the magnitude of pupil dilation remains unchanged

[375] ; ii. the extent to which executive processing is impaired as the number of fixations increases

[380] ; and iii. The extent to which executive processes and working memory are impaired as gaze duration increases

[385] .

[0082] The method uses an intelligent algorithm to analyze subjects using the following variables: a. The total number of eye fixations made by the subjects during the n-Back task. The identification number of the bn-Back task trial (ie, if there are 20 n-Back task trials, the 5th trial is identified with the number 5. The 20th trial is identified with the number 20, etc.). c. Experimental parts, namely 1, 2 and 3. d. Part of the experiment, namely encoding; retrieval. e. Pupil diameter of subjects during the n-Back task. f. Number of blinks from the left eye, right eye, or both eyes. g. Microsaccades; Form Factor (FF): i. HEWI: shows the height / width relationship of a microsaccade. ii. AREA: shows the area of ​​a rectangle inscribed in a microsaccade. iii.LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade. iv.ANG: is the sum of all angles in the horizontal-vertical plane of microsaccades. v.AANG: is the horizontal plane of microsaccades - vertical plane The sum of all absolute values ​​of the angles in radians. These last two FF terms give an estimate of the regularity of the microsaccade trajectory. vi.MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the centerline of fixation. vii.TIME: is the duration of the microsaccade in milliseconds. viii. VMIN and VMAX: are the minimum and maximum speeds of microsaccades in degrees per second. ix. Microsaccade rate: It is the instantaneous rate in each time period. x. Direction consistency: It is the consistency between microsaccade direction and stimulus position. h. Eye position (ie, abscissa and ordinate) from the left eye, right eye, or both eyes while performing the n-Back task. i. The magnitude of the saccade when processing the target. j. Saccadic latency. k. The order of fixations (ie, eye behavior) when processing the target. The order will be available from an image, matrix, etc. l. The distance between the gaze points of the right eye and the left eye when processing the target. m. Filia information of the subject (ie, age; years of education; gender; ethnicity; occupation; hours of physical activity per week). n.Duration of fixation while processing a target. o. Gaze duration while processing the target. p. Number of fixations on each target. q. Number of fixations outside each target.

[0083] Measurements taken when the stimulus image is presented (feature j in method

[300] ) provide information during encoding, which occurs when the subject first identifies the location of the visual stimulus. In a pilot study conducted by the inventors, subjects with MS were found to be impaired when encoding visual information (e.g., the subject made many fixations to the display). Measurements during encoding are complementary to measurements taken during recognition, when the visual stimulus is presented with a cue, as in the study by Fielding et al. (step ai in method

[300] ). In summary, a subject's performance during encoding and recognition can help identify additional deficits (i.e., the extent to which subcortical processes, executive processes, and / or executive processes are impaired) and provide a deeper understanding of the subject's condition than performance during recognition alone. read

[0084] Reference now Figure 4A and Figure 4B , which shows a method for measuring a subject's general cognitive performance and detecting one or more neurological disorders by measuring the subject's eye movements and / or pupil diameter while the subject is reading, according to some embodiments of the present invention.

[0085] The method

[400] includes the following steps: providing a system for measuring general cognitive performance and detecting the presence of one or more neurological disorders by measuring eye movements and / or pupil diameter; receiving eye tracking data and / or pupil diameter data of a subject reading text; analyzing the eye tracking data for evidence of one or more neurological disorders; and displaying a report of the detection of one or more neurological disorders.

[0086] In some embodiments, the method

[400] includes the steps of: counting the total number of eye fixations of the subject while the subject is reading a text

[405] ; and reporting detection of an impairment of attentional processing if the total number of eye fixations of the subject while reading the text is higher than a control group

[460] .

[0087] In some embodiments, the method

[400] further includes the steps of counting the total number of eye fixations of the subject while reading the text

[405] ; counting the number of positive eye fixations of the subject while reading the text

[430] ; and reporting detection of working memory impairment if the number of positive eye fixations of the subject is higher than that of the control group and the total number of eye fixations of the subject while reading is higher than that of the control group

[470] .

[0088] Physiologically, impaired working memory is associated with frontal lobe degeneration. In some embodiments, reporting impaired working memory

[470] can be used in conjunction with additional treatment. For example, if neurosurgery is indicated, brain imaging of the subject's frontal lobe can be studied following method

[400] .

[0089] In some embodiments, the method

[400] includes the steps of counting the number of times the subject fixates on each word in a text as the subject reads the text

[440] ; counting the number of words that the subject fixates on only once

[445] ; and reporting detection of impaired retrieval memory if the number of words that the subject fixates on only once is lower than a control group

[480] .

[0090] Physiologically, impaired retrieval memory is associated with temporal lobe degeneration. In some embodiments, reporting impaired retrieval memory

[480] can be used in conjunction with additional treatment. For example, if neurosurgery is indicated, brain imaging of the subject's frontal lobe can be studied following method

[400] .

[0091] In some embodiments, the method

[400] includes the steps of: counting the number of multiple eye fixations of the subject while reading the text

[450] ; and reporting the detection of an impairment in executive processing if the number of multiple eye fixations is higher than a control group

[490] .

[0092] In some embodiments, the method

[400] includes the steps of calculating the average saccade amplitude of the subject from one eye fixation to the next while reading the text

[454] ; and reporting the detection of an impairment in executive processing if the average saccade amplitude is lower than a control group

[491] .

[0093] In some embodiments, the method

[400] includes the steps of: tracking the subject's pupil diameter while reading a text

[456] ; and if the subject's pupil diameter does not appear to decrease as reading the text progresses, reporting that an impairment in executive processing has been detected

[492] .

[0094] Physiologically, impaired executive processes are associated with degeneration of the frontal, temporal, and / or parietal lobes. In some embodiments, reporting impaired executive processes [490-491-492] can be used in additional treatments. For example, if neurosurgery is indicated, brain imaging of the subject's frontal, temporal, and / or parietal lobes can be studied following method

[400] .

[0095] The system and method were tested on 50 healthy controls and 50 mild AD patients

[400] . Both groups read 40 regular sentences. test Control group AD Group Attention Process 520(21) 882(317) Execution process 14(8) 37(6) Working Memory 85(14) 61(9) Retrieval memory 30(6) 12(11) Table 1 Bibliography

[0096] The above rules are based in part on the findings of the following studies: 1. Fernández G, Mandolesi P, Rotstein NP, Colombo O, Agamennoni O, Politi LE. (2013) Eye movement alterations during reading in patients with early Alzheimer disease. Invest Ophthalmol Vis Sci. pii:iovs.13-12877 v1. doi:10.1167 / iovs.13-12877. 2. Fernández G., Manes F., Politi L., Orozco D., Schumacher M., Castro L., Agamennoni O., Rotstein N. (2016). Patients with Mild Alzheimer Disease Fail When Using Their Working Memory: Evidence from the Eye Tracking Technique. Journal of Alzheimer Disease; 50, 827-828. 3. Fernández, G., Laubrock, J., Mandolesi P., Colombo O., Agamennoni O. (2014) Registering eye movements during reading in Alzheimer disease: difficulties in predicting upcoming words. Journal of Clinical and Experimental Neuropsychology; 36, 302-16. 4. Fernández G., Sapognikoff M., Guinjoan S., Orozco D., Agamennoni O. (2016). Word processing during reading sentences in patients with schizophrenia: evidence from the eyetracking technique. COMPREHENSIVE PSYCHIATRY; 68, 193-200. 5. Fernández G, Manes F, Rotstein N, Colombo O, Mandolesi P, Politi L, Agamennoni O. (2014) Lack of contextual-word predictability during reading in patients with mild Alzheimer disease. Neuropsychologia; 62, 143-51. 6. Fernández G., Schumacher M., Castro L., Orozco D., Agamennoni O., (2015). Patients with Alzheimer disease produced shorter outgoing saccades when reading sentences. Psychiatry Research, 229, 470-478. 7. Fernández G., Biondi J., Castro S., Agamennoni O. (2017). Pupil size behavior during online processing of sentences. Journal of Integrative Neurosciences, 15(4): 485-496 Memory Binding Non-limiting embodiments of the present invention will now be described in detail.

[0097] Reference now Figure 5 , which illustrates a method for detecting impairment of a subject's memory binding function according to some embodiments of the present invention

[500] .

[0098] The method includes the step of providing a system for detecting an impairment of a subject's memory binding function

[505] .

[0099] In some embodiments, method

[500] includes the steps of a subject observing one or more targets [510-535]; measuring the subject's gaze duration on each of the targets

[545] ; calculating the subject's average gaze duration on the targets

[550] ; and if the subject's average gaze duration is longer than the average gaze duration of a control group, reporting that an impairment in target encoding and recognition processes has been detected in the subject

[565] .

[0100] In some embodiments, method

[500] includes a step of measuring one or more pupil diameters of a subject while performing an activity requiring lower cognitive effort (e.g., identifying three objects or distinguishing objects)

[555] ; and a step of reporting detection of impairment of cognitive resources in subject [5] if the average pupil diameter of subject [5] while performing an activity requiring higher cognitive effort does not show an increase relative to the average pupil diameter of subject [5] while performing an activity requiring lower cognitive effort

[570] .

[0101] In some embodiments, method

[500] includes a step of counting the number of eye fixations made by subject [5] while observing target

[30]

[560] ; and a step of reporting that an impairment of attentional processing has been detected in subject [5] if the number of eye fixations made by subject [5] while observing target

[30] is higher than that of a control group

[575] . Bibliography

[0102] The above rules are based in part on the findings of the following studies: 1. Fernández G, Mandolesi P, Rotstein NP, Colombo O, Agamennoni O, Politi LE. (2013) Eye movement alterations during reading in patients with early Alzheimer disease. Invest Ophthalmol Vis Sci. pii:iovs.13-12877 v1. doi:10.1167 / iovs.13-12877. 2. Fernández G., Manes F., Politi L., Orozco D., Schumacher M., Castro L., Agamennoni O., Rotstein N. (2016). Patients with Mild Alzheimer Disease Fail When Using Their Working Memory: Evidence from the Eye Tracking Technique. Journal of Alzheimer Disease; 50, 827-828. 3. Fernández, G., Laubrock, J., Mandolesi P., Colombo O., Agamennoni O. (2014) Registering eye movements during reading in Alzheimer disease: difficulties in predicting upcoming words. Journal of Clinical and Experimental Neuropsychology; 36, 302-16. 4. Fernández G., Sapognikoff M., Guinjoan S., Orozco D., Agamennoni O. (2016). Word processing during reading sentences in patients with schizophrenia: evidence from the eyetracking technique. COMPREHENSIVE PSYCHIATRY; 68, 193-200. 5. Fernández G, Manes F, Rotstein N, Colombo O, Mandolesi P, Politi L, Agamennoni O. (2014) Lack of contextual-word predictability during reading in patients with mild Alzheimer disease. Neuropsychologia; 62, 143-51. 6. Fernández G., Schumacher M., Castro L., Orozco D., Agamennoni O., (2015). Patients with Alzheimer disease produced shorter outgoing saccades when reading sentences. Psychiatry Research, 229, 470-478. 7. Fernández G., Biondi J., Castro S., Agamennoni O. (2017). Pupil size behavior during online processing of sentences. Journal of Integrative Neurosciences, 15(4): 485-496. 8. Biondi J., Fernandez G., Castro S., Agamennoni O. (2018). Eye-movement behavior identification for Alzheimer Disease diagnosis. Journal of Integrative Neurosciences (in press). 9.Fernández,Orozco,Agamennoni,Schumacher, Biondi, Parra. (2018). Visual Processing during Short-Term Memory Binding in Mild Alzheimer's Disease. J Alzheimers Dis.; 63(1): 185-194. doi: 10.3233 / JAD-170728. Parkinson's disease (PD) and attention deficit hyperactivity disorder (ADHD)

[0103] Reference now Fig. 6A and Figure 6B , which shows a method according to some embodiments of the present invention for detecting one or more cognitive impairments, neurological impairments, and behavioral impairments in a person by measuring the person's eye movements and / or pupil diameter when the person takes a vision test.

[0104] The method

[600] includes the following steps: providing a system for detecting the presence of one or more cognitive impairment and neurological disorders by measuring eye movements while a person visualizes, identifies, maintains, controls, inhibits, and sorts an object; receiving eye tracking data of a person who visualizes, identifies, maintains, controls, inhibits, and sorts an object; analyzing the eye tracking data for evidence of one or more cognitive impairment and neurological disorders; and displaying a report of the detection of cognitive impairment and one or more neurological disorders.

[0105] In some embodiments, the method

[600] includes the steps of counting the total number of eye fixations made by the person while the person is taking a visual test

[615] ; and if the number of eye fixations made by the person is higher than a control group, reporting that impairment of attentional, executive, and inhibitory processes has been detected.

[0100] In some embodiments, the method

[600] includes the steps of calculating the average speed of saccades of the subject [5] from one target to another while the subject [5] is performing a visual test

[620] ; if the average saccade speed of the subject is lower than that of a control group, reporting that an impairment in executive function is detected. Physiologically, slower saccade speeds are associated with degeneration of the frontal eye fields, basal ganglia, and superior colliculus. In some embodiments, reporting an impairment in saccade speed can be used in additional treatments.

[0106] In some embodiments, the method

[600] includes the steps of: counting the number of correct target identifications made by a person while performing a visual test

[625] ; and reporting detection of working memory impairment if the number of correct target identifications is lower than a control group.

[0107] Physiologically, working memory impairment is associated with degeneration of the prefrontal cortex and posterior parietal cortex. In some embodiments, reporting impairments in working memory, inhibitory processes, and mental flexibility may be used in additional treatments.

[0108] In some embodiments, the method

[600] includes the steps of calculating an average saccade amplitude from one eye fixation to the next eye fixation

[630] ; and reporting that an impairment in executive processing was detected if the average saccade amplitude is lower than a control group.

[0109] In some embodiments, the method

[600] includes the steps of: tracking the diameter of a person's pupil while performing a visual test

[640] ; and reporting detection of an impairment in attentional processing if the subject's pupil diameter does not show an increase as the visual test progresses.

[0110] Physiologically, impaired attentional processing is associated with degeneration of the locus coeruleus, the noradrenergic system, and the epithalamus. In some embodiments, reporting impaired executive processing can be used in additional treatments.

[0111] In some embodiments, the method

[600] includes the steps of calculating the total time the person spends performing the visual trial

[635] ; and reporting that an impairment in attentional processing has been detected if the total time required to perform the trial is longer than the time reported by a control group.

[0112] Physiologically, impairments in attention and inhibition processes and mental flexibility are associated with degeneration of the prefrontal cortex, posterior parietal cortex, prefrontal-striatal-cerebellar and prefrontal-striatal-thalamic circuits. In some embodiments, reporting impairments in executive processes can be used in additional treatments.

[0113] In some embodiments, the method

[600] includes the steps of calculating the duration of a person's fixation on a target while performing a visual test

[645] ; and reporting that working memory impairment has been detected if the duration of fixation on the target is lower than that of a control group.

[0114] Physiologically, impairments in attention and inhibition processes and mental flexibility are associated with degeneration of the prefrontal cortex, frontal eye fields, and dorsal parietal cortex. In some embodiments, reporting impairments in executive processes may be used in additional treatments.

[0115] The method uses an intelligent algorithm to analyze subjects using the following variables: a. The total number of eye fixations made by the subject during the visual test. b. An identification number for each target depending on its location in the maze or maze. c. The pupil diameter of the subject during the visual test. d. Number of blinks from the left eye, right eye, or both eyes. e. Microsaccades; Form Factor (FF): i. HEWI: shows the height / width relationship of microsaccades. ii. AREA: shows the area of ​​the rectangle inscribed within the microsaccade. iii.LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade. iv.ANG: is the sum of all angles in the horizontal-vertical plane of microsaccades. v.AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. These last two FFs give an estimate of the regularity of the microsaccade trajectory. vi.MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation. vii.TIME: is the duration of the microsaccade in milliseconds. viii. VMIN and VMAX: are the minimum and maximum speeds of microsaccades in degrees per second. ix. Microsaccade rate: It is the instantaneous rate in each time period. x. Direction consistency: It is the consistency between microsaccade direction and stimulus position. f. Eye position (ie, abscissa and ordinate) from the left eye, right eye, or both eyes while performing the visual task. g. The magnitude of the saccade when processing the target. h. Saccadic latency. i. The order of fixations (ie eye behavior) when processing the target. The order will be available from an image, matrix, etc. j. The distance between the fixation points of the right eye and the left eye when processing the target. k. Filia information of the subject (ie, age; years of education; gender; ethnicity; occupation; hours of physical activity per week). l. Fixation duration while processing the target. m. Number of fixations on each target. n. Number of fixations outside each target. o. Total visual task time (ie, how long did the subject take to perform the entire trial).

[0116] This approach was tested in subjects with PD and ADHD

[600] and compared with healthy controls: ADHD Comparison ADHD Average gaze (MS) 283(±42.4) 370.3(±33.1) Correct gaze % 95％(±3) 73％(±7) Table 2 Evaluation of treatment options

[0117] This article describes a method for using the above-mentioned system and technology to evaluate the treatment regimen (e.g., medicaments, such as drugs, medicines, etc.) followed by a patient according to the guidance of a medical practitioner. In this way, medical practitioners and / or drug manufacturers can better track the effectiveness of the treatment regimen for the patient and change or supplement the regimen as needed based on the evaluation throughout the course of the disease. For example, the drugs or other agents that can be evaluated include neurological drugs and / or psychotropic drugs that have neurological effects and / or psychiatric effects.

[0118] For purposes of illustration only and not as a limitation of the methods described herein, examples are presented below in which eye movements are modeled in MS patients receiving different drugs (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab) or treatments that (a) reduce inflammation and prevent nerve damage that may cause symptoms of multiple sclerosis); (b) test sphingosine-1-phosphate receptor modulators, which sequester lymphocytes in lymphocyte ganglia, preventing them from causing autoimmune reactions); (c) examine immunosuppressants that act on lymphocyte pathways); and (d) analyze the effects of monoclonal antibodies that inhibit B lymphocyte activation. In addition, we explain how practitioners, drug manufacturers, etc. can evaluate the effects of these agents and any other treatments on cognitive performance and high-level motor ability.

[0119] Understanding the effects of various agents (e.g., drugs, etc.) on the central nervous system (CNS) and peripheral nervous system (PNS) by analyzing eye movements when performing well-defined activities such as those reported by us (e.g., go no-go test and n-back test) will allow practitioners to test the extent and efficacy of agents or treatments to produce the desired effects on the course of a patient's illness. In this sense, practitioners will have access to novel tools for testing the effects of agents on cognitive changes and fine motor changes in patients. In addition, pharmaceutical companies will also have objective and quantifiable measurements of the effects of agents on well-defined areas, opening up a new avenue to analyze, among other things, who should repeat new drug administrations (including dosages) and which are the patients who assimilate the agent better.

[0120] Some embodiments of the methods described herein can perform one or more of the following: calculate, model, and report one or more effects of a drug (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or treatment to test whether there is (a) reduction in inflammation and neural damage that may cause symptoms of multiple sclerosis; (b) impairment of sphingosine-l-phosphate receptor modulators that sequester lymphocytes in lymphocyte nodes, preventing them from causing autoimmune reactions; (c) impairment of immunosuppressants that act on lymphocyte pathways; and / or (d) therapeutic effects of monoclonal antibodies that inhibit lymphocyte B activation on some well-defined neurological processes and related cognitive activities.

[0121] We apply such mathematical models, wherein the dependent variables considered can be, for example, saccade amplitude, fixation duration, pupil behavior; and predictors can be, in particular, motor scales, cognitive scales, disease diagnosis and treatment (i.e., medication) years. We obtain regression coefficients, standard errors and t-values ​​from each model to understand what the effects of treatment are on specific eye movements (e.g., saccade amplitude), eye movement combination sets, related cognitive functions and related areas of the brain. We perform the first measurement (baseline) and repeat the exercise (when necessary) to check whether the treatment works properly.

[0122] The following example explains how to use saccade amplitude as a dependent variable: The saccade amplitude depends on the strategy developed by the person being evaluated to scan the numerical value when taking a specific test. If the test is an n-back task, people who perform better will make longer saccades due to the nature of the test. Longer saccades indicate good working memory performance, while short saccades imply poor performance (as previously shown in this patent). In order for the saccade amplitude to be longer, the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus must be preserved in this case. The reason behind this statement is that the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus are critical in defining where different fixations will occur (thus, affecting the saccade amplitude) (Fielding et al., 2015). In addition, when performance in cognitive scales (e.g., the Symbol Digit Modalities Test) and motor scales (e.g., the Extended Disability Status Scale) shows better output, the saccade should be longer. The reason behind this is that better cognitive scale outcomes are positively correlated with more preserved working memory, while better motor skills are positively correlated with more preserved high-level motor function. For this reason, the Symbol Digit Modalities Test and the Extended Disability Status Scale can be used as predictors.

[0123] If a person was treated with dimethyl fumarate (which can (a) produce a reduction in inflammation and neural damage that may cause symptoms of multiple sclerosis) and had longer saccades on the N-Back task, then one could infer that the treatment had a positive effect on working memory as well as the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus. Figure 7 In the study, a positive effect could be observed in patients taking dimethyl fumarate, showing longer saccade amplitudes as treatment progressed (4 years of treatment).

[0124] The following example explains how to use pupil behavior as a dependent variable: The patient's pupil behavior changes depending on the cognitive effort the patient is performing at a particular moment. When the task is more demanding, the size of the pupil increases (as previously explained in this patent). When performing the N-Back task, the pupil size must increase given the complexity of the test. This specific behavior shows that the noradrenergic system and the locus coeruleus respond correctly as the cognitive load increases. This statement is pupil size and cognitive load (Fernández et al., 2021). If a person is being treated with interferon-β (which can (b) reduce damage to immunosuppressants acting on lymphocyte pathways), and pupil size increases with increasing cognitive load, it can be inferred that the treatment has an effect on the amount of working memory resources used (Sweller et al., 2011) and has a positive effect on the noradrenergic system and the locus coeruleus.

[0125] Figure 8is a flow chart illustrating a method for identifying specific alterations in subjects with a defined disease, analyzing eye movement patterns when using specific visual stimuli, where a specific drug or treatment will enhance visual processing, cognitive performance and related brain activity. Additional details and examples of methods and systems for evaluating treatment regimens N-back task

[0126] In one example, in order to examine the effects of a particular agent (e.g., drug, pharmaceutical, etc.) or treatment, a method for evaluating neurological impairment, fine motor skills, executive processes, decision making, processing speed, and cognitive abilities associated with multiple sclerosis [MS] is provided, the method comprising a. Provide systems for evaluating neurological impairments, fine motor skills, executive processes, decision-making, fine motor skills, and cognitive abilities associated with MS; b. asking the subject to fixate on a reference target of a diagram, wherein the diagram includes a plurality of regions (eg, rectangles) placed in different partitions; c. Perform multiple repetitions, presenting stimulus images to the subject in one of the partitions, and requiring the subject to remember in which partition each stimulus image appeared and in what order; d. presenting a chart to the subject without including the stimulus image presented in step c, wherein the subject is required to look at the partition where the stimulus image of step c appears; e. measuring the subject's saccade in response to the presentation step d, requiring the subject to look at the partition presented in step c of the stimulus image; f. Repeat steps d and e for presenting the graph and measuring saccades; g. Repeat steps bf to modify the time of showing the stimulus image for multiple trials; h. Calculate one or more of the following: i. WM effect, where WM effect is a measure of the increase when WM demand increases. For each stimulus image looked at, the WM effect is represented by the ratio between the number of errors reported by the subject across all trials and the number of trials); and ii. Mean saccade latency, where saccade latency is defined as the amount of time it takes a subject to elicit a saccade to a partition; and report one or more of the following: iii. the extent of working memory impairment as WM effects increase; and i. The extent to which executive processing is impaired as saccadic latency increases; j. Wherein the method further comprises an additional step, wherein the additional step comprises a measurement performed during the step of presenting the stimulus image, during which the subject is further required to look at the stimulus image; the measurement comprises measuring one or more of the following: a. The extent of the subject's pupil dilation; b. The number of fixations made by the subject on the stimulus image; c. The subject's gaze duration on the stimulus image; d. Binocular disparity obtained through simultaneous visual exploration and target visualization; e. The subject fixated on the target where the visual stimulus had previously been; f. The number of consecutive target hits by the subject when considering the trial; g. Number of blinks from the left eye, right eye, or both eyes; h. Intelligent algorithms, where eye movement behavior is used as a benefit for classifying a person’s performance; i. Form Factor (FF) of Microsaccades: i) HEWI: shows the height / width relationship of a microsaccade. ii) AREA: shows the area of ​​a rectangle inscribed within a microsaccade; ii) LONG: is the longitude of the microsaccade trajectory in the horizontal-vertical plane. iii) ANG: is the sum of all angles in the horizontal-vertical plane of microsaccades; iv) AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. These last two FFs give an estimate of the regularity of the microsaccade trajectory; v) MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation; vi)TIME: is the duration of the microsaccade in milliseconds. vii) VMIN and VMAX: minimum and maximum speed of microsaccades in degrees / second; viii) Microsaccade rate: is the instantaneous rate in each time segment; ix) Direction consistency: It is the consistency between microsaccade direction and stimulus position; k. Obtain eye position information (ie, abscissa and ordinate) from the left eye, right eye, or both eyes when performing visual exploration.

[0127] It should be noted that any one or more (or all) of the items calculated in step h may be omitted. Likewise, any one or more (or all) of the additional steps shown in step i may be omitted.

[0128] In addition, in some embodiments, additional steps may further include calculating, modeling, and reporting one or more effects of a drug (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or treatment that (a) reduces inflammation that may cause symptoms of multiple sclerosis and prevents nerve damage that may cause symptoms of multiple sclerosis); (b) tests sphingosine-l-phosphate receptor modulators, which sequester lymphocytes in lymphocyte nodes, preventing them from causing autoimmune reactions); (c) examines immunosuppressants that act on lymphocyte pathways); and / or (d) analyzes the effects of monoclonal antibodies that inhibit lymphocyte B activation. GO NO-GO mission

[0129] In another example, methods and systems for evaluating neurological impairments, fine motor skills, processing speed, decision making, and cognitive processing impairments associated with multiple sclerosis (Go No-Go) are provided.

[0130] In one specific example, systems and methods are provided for detecting one or more neurological disorders in a subject and / or measuring fine motor skills, processing speed, decision-making, and cognitive processes by measuring eye movements, eye movement characteristics, or pupil behavior, the eye movements being measured as the subject visualizes (i.e., forms a picture of something in the mind in order to imagine or store it), recognizes (i.e., identifies something that has been previously encountered), maintains (i.e., stores in an already existing memory), controls (i.e., exercises restraint or guidance), inhibits (i.e., prevents or stops doing something), fixates (i.e., focuses the eyes on something), and analyzes an object. The system may include: a. An eye tracker configured to monitor the subject's eye movements as the subject visualizes, identifies, maintains, controls, fixates, and analyzes a target; b. a processor configured to receive data from the eye tracker as the subject visualizes, identifies, maintains, controls, fixates, and analyzes a target; and c. a display configured to display a test report received from a processor, wherein the processor is further configured to analyze the eye tracking data for evidence of one or more neurological disorders or general cognitive performance, and to report the detection of one or more neurological disorders or measurement of cognitive performance in the subject in the test report.

[0131] In a specific embodiment, the processor is further configured to, upon receiving the eye tracking data from the eye tracker, perform one or more (or all) of the following: a. Count the total number of eye fixations made by the subject while visualizing, identifying, maintaining, controlling, fixating, and analyzing the target; and b. If the total number of eye fixations during visualization, identification, maintenance, control, fixation, and analysis of the target is higher in the subject than in the control group, then impairment of attentional processing is reported in the test report; c. Count the number of correct placements of the subject during visualization, identification, maintenance, control, fixation, and analysis of the target; and d. If the subject's number of correct placements is lower than the control group's, report in the test report that an impairment in executive function was detected; e. Count the number of correctly cue-guided outward saccades during attempts to visualize, identify, maintain, control, fixate, track, and analyze the target; and f. If the subject makes a lower percentage of outward saccades guided by the correct cue (e.g., direction of an arrow) than the control group, report in the test report that an impairment in executive processing was detected; g. Count the number of outward saccades made by the subject in the opposite cue direction while attempting to visualize, identify, maintain, control, fixate, track, and analyze the target; h. If the percentage of outward saccades guided by the opposite cue is higher than that in the control group, report in the test report that an impairment of inhibitory processing was detected; i. Calculate the average saccade amplitude from one eye fixation to the next when visualizing, identifying, maintaining, controlling, fixating, tracking, and analyzing a target; j. If the mean saccade amplitude is lower than that of the control group, report in the test report that an impairment in executive processing was detected; k. Count the length of the subject's saccadic latency when directing the eyes to visualize, identify, maintain, control, inhibit, fixate, track, and analyze the target; l. If the saccade latency length (time) is higher than that of the control group, report in the test report that impaired speed processing was detected. m. Tracking of the subject's pupil diameter when visualizing, identifying, maintaining, controlling, inhibiting, fixating, tracking, and analyzing targets; and n. If the subject's pupil diameter does not show accommodation as the target is visualized, identified, maintained, controlled, inhibited, fixed, tracked, and analyzed, then report in the test report that noradrenergic impairment was detected. o. Consider the length of the subject's fixation duration when attempting to visualize, identify, maintain, control, fixate, track, and analyze the target; and p. If the length of fixation duration is longer than in the control group, report the detection of impaired online processing in the test report; q. Consider the subject's gaze duration when attempting to visualize, identify, maintain, control, fixate, track, and analyze a target; r. If the length of the fixation duration is longer than the control group, report the detection of impaired online processing in the test report; s. Count the number of correct targets identified while visualizing, identifying, maintaining, controlling, inhibiting, fixating, tracking, and analyzing targets; and t. If the number of correct targets identified is lower than that of the control group, report in the test report that impairment of executive and working memory processes was detected; u. Count blinks from the left eye, right eye, or both eyes when visualizing, identifying, maintaining, controlling, inhibiting, fixating, tracking, and analyzing a target; v. Applying intelligent algorithms, where eye movement behavior is used as a benefit for classifying a person’s performance; v. Form Factor (FF) for measuring microsaccades: i.HEWI: shows the height / width relationship of microsaccades; ii.AREA: shows the area of ​​the rectangle inscribed by the microsaccade; iii.LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade; iv.ANG: is the sum of all angles in the horizontal-vertical plane of microsaccades; v.AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades; vi.FF gives an estimate of the regularity of microsaccade trajectories; vii.MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation; viii.TIME: is the duration of the microsaccade in milliseconds; ix. VMIN and VMAX: the minimum and maximum speeds of microsaccades in degrees per second; x. Microsaccade rate: is the instantaneous rate in each time period; xi. Direction consistency: the consistency between microsaccade direction and stimulus position; w. measure eye position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes during visualization, recognition, maintenance, control, sequencing, and analysis of objects; x. Measuring total visualization, recognition, maintenance, control, fixation, tracking, and analysis target time (i.e., the time the subject spent visualizing the target across trials); y. Count the number of correct targets identified while visualizing, identifying, maintaining, controlling, inhibiting, fixating, tracking, and analyzing the targets.

[0132] The processor can be further configured to perform additional steps, comprising calculating, modeling, and reporting one or more effects of a drug (e.g., dimethyl fumarate, fingolimod, cladribine, ofatumumab, interferon-β) or treatment that (a) reduces inflammation that may cause symptoms of multiple sclerosis and prevents nerve damage that may cause symptoms of multiple sclerosis); (b) tests sphingosine-l-phosphate receptor modulators, which sequester lymphocytes in lymphocyte nodes, preventing them from causing autoimmune reactions); (c) examines immunosuppressants that act on lymphocyte pathways); and / or (d) analyzes the effects of monoclonal antibodies in inhibiting lymphocyte B activation. Using virtual reality environments to assess performance, motor skills, and cognitive abilities

[0133] This article describes such a system and method for combining virtual reality (VR), eye tracking (ET) and motion sensors on limbs (such as hands and feet) to evaluate changes in cognitive and motor abilities of healthy and unhealthy people using well-defined exercises. The application of VR and ET with motion sensors in cognitive exercises can improve the efficacy of interventions and the ability to quantify cognitive and motor abilities, enhancing the effectiveness of training for people. For example, a combination of VR, visual scanning, and arm and leg movements can provide new information about a person's decision-making process and brain circuit integrity (e.g., what a person does when visualizing a shape and deciding to move a hand to touch something in a VR environment). This method will enhance the ability of healthcare professionals to analyze, quantify, and train cognitive abilities and fine motor skills.

[0134] The eye tracking system described herein can be incorporated into a conventional head-mounted display (HMD) in which a VR world is provided and observed by the user. Each manufacturer of commercially available HMD / VR devices performs eye tracking in a slightly different way. However, the interface provided to developers allows them to access vectors (three numerical values) indicating the direction in which the eyes are looking in the 3D virtual space created by the VR application. In this way, the integration between the eye tracking system and the VR application is seamless. This interface is generally available in multiple languages / game development environments. Examples of some types of commercial VR devices that can be used include: HTC Vive Eye Pro; HP G2 Reverb Omnicept Edition; VarjoAero and Fove 0. The controllers provided with the VR device will generally work to track hand movements. Some of them (like valve controllers) are compatible with bases independent of the VR headset used. Virtual reality (VR) controllers play a key role in opening a fascinating immersive experience within a virtual environment. These tools enable users to interact and manipulate the digital world, ultimately bringing an extremely fascinating and immersive experience. Fundamentally, VR input devices act as a conduit for transferring hand motion data to a computer system. This information is then processed and utilized to control objects present within the simulated world. Currently, there are two main categories of input devices in use: motion controllers and game controllers. Motion controllers employ accelerometers and gyroscopes to detect motion and orientation changes. They may also incorporate buttons, analog sticks, and various input mechanisms, depending on the specific device. Motion controllers excel in scenarios where direct interaction with the virtual surroundings is most important, such as first-person exploration experiences. Game controllers, on the other hand, are more commonly associated with traditional gaming experiences. These controllers typically offer a range of input options, including dual analog sticks and multiple buttons. In the latest generation of input devices, an additional layer of immersion is added by including haptic feedback. This enhancement enhances the virtual experience by providing a perceptible reaction when interacting with elements within the virtual realm.

[0135] We apply mathematical models in which the dependent variables considered can be, for example, saccade amplitude, fixation duration, pupil behavior, hand reaction time, tracking accuracy; and the independent variables can be, among others, motor measures, cognitive measures, years of disease diagnosis, treatment. We obtain regression coefficients, standard errors and t-values ​​from each model in order to understand what the effect of the disease or treatment or training is on a specific eye movement (e.g., saccade amplitude) or limb movement (e.g., hand reaction time), a combination of eye movements, related cognitive functions, fine motor pathways and related areas of the brain.

[0136] The following illustrative example explains how to use saccade amplitude as the dependent variable:

[0137] Saccadic amplitude depends on the strategy developed by the person who casts eyes to a specific shape when being evaluated for a specific test. If the test is Go No-Go 3D, due to the nature of the test, the person who performs better will perform longer saccades. Longer saccades indicate that working memory performs well, while shorter saccades suggest poor performance (as previously discussed in this article). In order to make the saccade amplitude longer, in this case, the dorsolateral prefrontal cortex, basal ganglia and superior colliculus must be retained. The reason behind this statement is that the dorsolateral prefrontal cortex, basal ganglia and superior colliculus are key in defining where different fixations will occur (therefore, affecting saccade amplitude) (see Fielding, J. et al. Nat. Rev. Neurol. [Natural Review Neurology] 11, 637-645 (2015); doi: 10.1038 / nrneurol.2015.174). For example, if a person diagnosed with multiple sclerosis were studied, his / her saccades would be shorter and less accurate as disability increased, and it could be inferred that the disease has a significant effect on working memory as well as the dorsolateral prefrontal cortex, basal ganglia, and superior colliculus.

[0138] The following example explains how to use a person's hand response as a dependent variable.

[0139] A person's hand reaction time assesses the average time it takes for a person to elicit a manual response after visually perceiving a target. It can reflect a person's motor reaction time and coordination. This measurement can provide an understanding of the speed at which a person can convert visual information into motor actions. Hand reaction time can evaluate the potential efficiency of sensorimotor processing and a person's ability to quickly elicit a manual response. When performing Go No-Go 3D, the hand reaction time will decrease given the complexity of the test. This specific behavior suggests that the primary motor cortex and cerebellum respond correctly as the difficulty of the test increases. For example, if a person diagnosed with Parkinson's disease is studied, his / her hand reaction time will be slower and less flexible as the disability increases, and it can be inferred that the disease has a significant impact on hand speed fine motor flexibility as well as the primary motor cortex and cerebellum. We can also apply artificial intelligence algorithms and conduct a stepwise approach in which we combine eye movement and limb variables. We start by converting the input information into a tabular data set using a bio-perceptual feature extraction method and a variety of normalization and aggregation methods. When choosing an AI algorithm, we focus on robustness, choosing a model that inherently addresses overfitting and class imbalance. In the data processing pipeline or in the model itself, we try to make the process as white-box and interpretable as possible, allowing us to detect unexpected patterns and behaviors and analyze them. We also perform a set of statistical analyses (as described above) to ensure cross-device compatibility not only of the input data but also of the model results. Depending on the dataset size, the performance of the model is evaluated using either out-of-bag cross-validation or a random sample test set approach. Various evaluation metrics can be used that take into account general model performance, but also inherent class imbalances and the difference between type I and type II error costs found in the health domain.

[0140] Fig. 9 A conceptual illustration of a system for evaluating the performance of a person [5] by applying a 3-dimensional virtual reality (3DVR) environment in combination with embedded eye tracking technology (ET)

[10] and motion sensors

[110] that track the movement of limbs (such as hands and feet) while the person performs well-defined activities

[15] . Fig. 9 The system shown can be used in the methods described below. 3D Virtual Reality and Eye Tracking in the Go No-GO Mission

[0141] In order to quantify specific motor skills and cognitive processes, methods are provided for evaluating the performance of healthy or unhealthy persons [5]. The methods employ techniques such as those described above and Fig. 9 Depicted is a 3-dimensional virtual reality (3DVR) environment that can be used to assess human performance, fine motor skills, and cognitive abilities by applying 3DVR, ET, and limb motion tracking.

[0142] According to the method, while recording eye movements, the person is asked to visualize objects on the VR screen. Each object will have defined characteristics (such as color), and it will move toward the person at a defined speed, acceleration, and direction. Multiple repetitions are performed, objects are presented to the person in different partitions, and the person is asked to visually observe the objects on the screen. The presented objects appear to move toward the subject in the 3D virtual environment, and when the person effectively virtually touches the correct object by moving his limbs toward the shape, the apparent speed of the object will increase (or decrease). Similarly, when the person virtually touches an incorrect object by moving his limbs, the speed of the object will decrease (or increase).

[0143] Fig.10 An example of how objects can be positioned throughout a virtual environment and how objects and a person's hands can appear is shown. In this example, objects can be presented in two different colors and appear to come from the background of the screen. The person is asked to touch an object of one specific color. The person can use his or her right or left hand or right or left leg to touch the appropriate object, depending on whether the object is presented at the level of the hand or leg.

[0144] The method continues by measuring the person's saccades in response to the presentation of the objects. In particular, the person is asked to look at the objects in the virtual environment and touch them (or not touch them), and then the saccades are measured. These steps of asking the person to visually observe the objects and measuring the saccades can be repeated multiple times. Any of a variety of different requests may be made to the person to visualize and virtually touch (or not touch) objects on the VR screen while eye movements are recorded. For example, the person may be asked to virtually touch objects having a certain characteristic (e.g., red) but not to virtually touch objects having another characteristic (e.g., green). The above steps may be repeated for any number of instances of the object presented, which may be presented at different speeds.

[0144] Fig.11 is a graphical representation of an example of the density of eye movements recorded from the right and left eyes when a person performs a task of touching a required object, presented as a heat map.

[0145] Fig.12 is a graphical representation of the recorded movement density of the right and left hand movements when a person performs a task of touching a required object, presented as a heat map.

[0146] Based on the measurements of eye and limb movements in response to the presented demands, any one or more of the following metrics may be calculated: i. Suppress process errors (i.e., how many times a person touches an incorrect shape); and ii. Mean saccade latency, where saccade latency is defined as the amount of time it takes for the person to elicit a saccade to a new shape; and reporting one or more of the following: iii. the extent to which processing speed is impaired as the variation in the speed at which the successive shapes are presented to the subjects increases; and iv. The extent to which executive processing is impaired as inhibition errors increase. In some cases, the method may include additional steps including the step of taking additional measurements during the steps of presenting the object to the person and asking the person to look at and touch (or not touch) the object. Illustrative examples of such additional measurements may include one or more of the following: i. The pupil of a person is dilated greatly; ii. the number of times a person fixates on the stimulus image; and iii. The duration of a person's gaze on the stimulus image; iv. Human binocular parallax during visual exploration and object visualization v. The point of fixation where the person was before the object they touched and the visual stimulus vi. the number of consecutive objects that the person touches when conducting the test; vii. Number of blinks from the left eye, right eye, or both eyes. viii. The time it takes to visually detect the object (see Figure 3 for a heat map of eye movements). ix. The time from the visualization of the object until the moment the person starts moving the hands and / or feet. x. The time from when the person starts moving the hand and / or foot until the person touches or attempts to touch the object. xi. The number of times the subject touched or did not touch the virtual object (see Figure 4 for a heat map of hand movements). xii. The best locations for target visualization and the locations where it is less efficient to visualize the target. xii. Maintain tracking accuracy of visual focus on moving objects. xiii. The reach of the hand towards the object during the touching action. xiv. Maximum hand velocity achieved by the hand during movement towards the touched green object. xv. Ratio of dominant hand to hand use during manual interaction. xvi. Prediction time, which measures the average time it takes a person to anticipate and elicit a response following a visual cue. xvii. Microsaccades; Form Factor (FF): 1) HEWI: shows the height / width relationship of microsaccades. 2) AREA: shows the area of ​​the rectangle inscribed within the microsaccade. 3) LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade. 4) ANG: is the sum of all angles in the horizontal-vertical plane of microsaccades. 5) AANG: is the sum of all absolute values ​​of the angles in radians in the horizontal-vertical plane of the microsaccades. These last two FFs give an estimate of the regularity of the microsaccade trajectory. 6) MOD and THETA: are the modulus and angle of the polar coordinates of the sum of the Cartesian coordinates. They give the spatial orientation of the microsaccade relative to the midline of fixation. 7)TIME: is the duration of the microsaccade in milliseconds. 8) VMIN and VMAX: are the minimum and maximum speeds of microsaccades in degrees / second. 9) Microsaccade rate: It is the instantaneous rate in each time period. 10) Direction consistency: It is the consistency between microsaccade direction and stimulus position. Other additional steps that may be taken when presenting an object to a person and asking the person to look at and touch (or not touch) the object may include measuring the person's eye position (i.e., abscissa and ordinate) from the left eye, right eye, or both eyes when performing a visual exploration. Measurements may take into account: l. Eye movements when visualizing and touching objects; ll. The total time when the entire test is completed; and m. The number of seconds it takes a person to visualize the correct and incorrect objects.

[0147] Yet another additional step that may be performed includes quantifying neurological processes and related cognitive activities when considering pupil size behavior and / or binocular disparity and / or microsaccade characteristics and / or saccadic behavior and / or target touch rate and / or hand and foot movements and / or gaze and / or fixation duration and / or number of fixations and / or executive function performance in 3DVR.

[0148] Fig.13 is a flow chart describing one example of a method described herein for assessing a person's performance, motor skills, and cognitive abilities.

Claims

1. A method for evaluating a person's performance, motor skills and cognitive abilities, the method comprising: requiring a person to perform a task requiring the person to virtually touch designated virtual objects each having different designated characteristics, the designated virtual objects being presented in a three-dimensional (3D) virtual reality environment, the virtual objects moving toward or away from the subject at a defined speed, acceleration, and direction; repeating the requiring by requiring the person to perform the task a plurality of times with different ones of the virtual objects having different specified characteristics; measuring eye movements and limb movements of the person while the subject views the virtual object and performs the task; identifying selected ones of the measured eye movements and limb movements that are associated with the person's performance, motor skills, and cognitive abilities; determining expected eye and limb movements of the person while the person views the virtual object and performs the task, and comparing the expected eye and limb movements to the selected ones of the measured eye and limb movements to determine deviations therebetween; as well as The person's performance, motor skills, and cognitive abilities are evaluated based on the deviations.

2. The method of claim 1, wherein the eye movement measured comprises at least one of saccade amplitude, fixation duration, and pupil behavior.

3. The method of claim 1, wherein the limb movement measured includes limb reaction time required to perform a required task.

4. The method of claim 1, wherein the different specified features of the virtual object are colors (but not limited thereto).

5. The method of claim 1, wherein the evaluating comprises determining a metric, the metric comprising: i. Suppression process errors (i.e., how many times the person touches the incorrect object); ii. Mean saccadic latency, which represents the amount of time required for the person to initiate a saccade to view a subsequently viewed object.

6. The method of claim 1, wherein the evaluating further comprises determining (i) the extent to which processing speed is impaired as the speed at which sequential objects are presented to the person increases; and (ii) the extent to which executive processing is impaired as inhibition errors increase.

7. The method of claim 1, further comprising obtaining one or more additional measurements while the person is viewing the virtual object and performing the task, the one or more additional measurements selected from the group consisting of: i. the magnitude of pupil dilation of the person; ii. the number of times the person fixates on the stimulus image; iii. the duration of the person's gaze on the stimulus image; iv. binocular disparity of the person during visual exploration and object visualization; v. The fixation point of the target touched by the person and the location of the visual stimulus before; vi. the number of consecutive objects that the person touches when conducting the test; vii. Number of blinks from the left eye, right eye, or both eyes; viii. The time it takes to visually detect the object; x. the time from when the person starts moving the hand and / or foot until when the person touches or attempts to touch the object; xi. the number of times the subject touches or does not touch the virtual object; xii. The best position for object visualization and the position where the object visualization is less efficient; xii. Maintaining tracking accuracy of visual focus on moving objects; xiii. The reachable depth of the hand towards the object during the touching action; xiv. The maximum hand velocity achieved by the hand during movement towards the touched green object; xv.Ratio of dominant hand used to dominant hand during manual interaction; xvi. prediction time, which measures the average time it takes the person to anticipate and elicit a response following a visual cue; and xii. Microsaccades.

8. A system for evaluating human performance, motor skills and cognitive abilities, the system comprising: a three-dimensional (3D) virtual reality device configured to establish a 3D virtual reality environment in which a plurality of virtual objects are presented to the person, the objects having at least one characteristic that differs from one another, the objects moving toward or away from the person at defined speeds, accelerations, and directions; an eye tracker configured to measure eye movements of the person while the person views the virtual object and performs a required task, the required task comprising a plurality of requirements requiring the person to virtually touch a specified virtual object each having one of specified characteristics; one or more motion sensors configured to measure body movements of the person while the person performs the required task; A processor configured to receive data from the 3D virtual reality device, the eye tracker, and the one or more motion sensors while the person performs the required task, and further configured to (i) identify selected ones of the measured eye movements and limb movements that are associated with the person's performance, motor skills, and cognitive abilities; (ii) determine the person's expected eye movements and limb movements while the person observes the virtual object and performs the required task, and compare the expected eye movements and limb movements with the selected ones of the measured eye movements and limb movements to determine deviations therebetween; and (iii) evaluate the person's performance, motor skills, and cognitive abilities based on the deviations.

Images