Validating eye tracking data

Abstract

A computer-implemented method for validating eye tracking data for a system administering an eye-tracking-based test or task to a subject includes receiving (S8) eye tracking data for one or both eyes of the subject for each of a series of visual stimuli (200), the eye tracking data indicating one or more characteristics of the subject's gaze or position when each of the stimuli (200) is displayed. The method also includes receiving (S9) one or more subject metrics including at least one of the subject's age, one or more pathological or physiological conditions of the subject, and data representative of the calibration quality and / or attention level of one or more previous eye-tracking-based tests or tasks performed by the subject. The method also includes one or more of: a) quantifying the calibration quality of the eye tracking data and determining whether the calibration quality is lower than a predetermined minimum expected calibration quality associated with the subject metrics based on at least the eye tracking data, the location of each visual stimulus, and the subject metrics; and b) quantifying the subject's attention level based on the eye tracking data and determining whether the subject's attention level is lower than a predetermined minimum expected attention level associated with the subject metrics based on at least the eye tracking data and the subject metrics.

Description

[Technical Field]

[0001] The present disclosure relates to a computer-implemented method for validating eye tracking data for a system that administers eye tracking-based tests or tasks to subjects, and also to an apparatus and a computer program for performing the method. [Background technology]

[0002] Advances in eye tracking technology have enabled eye tracking devices to provide a more accurate means of analyzing a subject's ability to perform specific visual or cognitive tasks. Visual and cognitive tests and other tasks that were previously performed manually by medical professionals and relied on the medical professional's subjective analysis can now be performed objectively with eye tracking-based devices.

[0003] To be able to effectively analyze eye tracking data, the subject must focus on the provided visual stimuli, the system must be accurately calibrated, and the calibration must be valid for the entire period during which the eye tracking data is acquired. This is not always the case, and healthcare professionals may not realize that the subject has lost focus on the task or moved to a new position where the calibration is no longer valid. Ensuring that the subject pays attention to the task is particularly difficult for subjects who are uncooperative and have poor communication skills, increasing the likelihood that the subject will not pay much attention to the task.

[0004] Therefore, when acquiring eye tracking data, there is a need to objectively and reliably acquire the best possible quality data for each subject and to reliably determine when poor quality data has been acquired. Summary of the Invention

[0005] According to a first aspect of the present invention, there is provided a computer-implemented method of validating eye-tracking data for a system that administers an eye-tracking based test or task to a subject, the method comprising: receiving eye tracking data for one or both eyes of a subject for each of a series of visual stimuli, the eye tracking data indicative of one or more characteristics of the subject's gaze or position when each of the stimuli is displayed; receiving one or more subject indications, the one or more subject indications including at least one of data representative of the subject's age, one or more pathological or physiological conditions of the subject, and a calibration quality and / or attention level of one or more previous eye tracking-based tests or tasks performed by the subject; Including, The method comprises: a) quantifying a calibration quality of the eye tracking data and determining, based at least on the eye tracking data, the position of each visual stimulus, and the subject metric, whether the calibration quality is below a predetermined expected minimum calibration quality associated with the subject metric; and / or b) quantifying the subject's attention level based on the eye tracking data and determining, based at least on the eye tracking data and the subject metric, whether the subject's attention level is below a predetermined expected minimum attention level associated with the subject metric; Contains one or more of the following:

[0006] According to a second aspect of the present invention there is provided an apparatus arranged to perform a method according to the first aspect, the apparatus comprising: a display device configured to display the visual stimuli to the subject; and an eye tracking device configured to track the subject's gaze when the visual stimulus is presented to the subject and generate eye tracking data indicative of one or more characteristics of the subject's gaze or position; one or more processors configured to control the display device to display the visual stimuli, receive eye tracking data from the eye tracking device, receive the one or more subject metrics, and determine: a) based at least on the eye tracking data for each visual stimulus and one or more subject metrics of the subject, whether a calibration quality of the eye tracking data indicates a calibration quality lower than the predetermined minimum expected calibration quality; and / or b) based at least on the eye tracking data and the subject metrics, whether an attention level of the subject is lower than a predetermined minimum expected attention level associated with the subject metrics; Includes.

[0007] According to a third aspect of the present invention there is provided a computer program comprising computer readable instructions which, when executed by a processor, cause the processor to carry out the method according to the first aspect. [Brief explanation of the drawings]

[0008] For a better understanding of the present disclosure and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings, in which: [Figure 1] FIG. 1 illustrates a method for validating eye tracking data according to one or more embodiments. [Figure 2A] FIG. 2A shows a schematic diagram of an apparatus according to one or more embodiments. [Figure 2B] FIG. 2B shows a schematic front view of the device according to one or more embodiments. [Figure 3A] FIG. 3A illustrates a sequence of multiple visual stimuli presented during a method according to one or more embodiments. [Figure 3B] FIG. 3B illustrates a sequence of multiple visual stimuli presented during a method according to one or more embodiments. [Figure 3C] FIG. 3C illustrates a sequence of multiple visual stimuli presented during a method according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] As used herein, the term "eye-tracking-based test" may be understood to mean any test that displays a series of visual stimuli and utilizes qualitative or quantitative eye-tracking data of a subject viewing the stimuli to identify the subject's visual function or cognitive ability. The visual stimuli used in a given test may be configured as needed to identify a particular aspect of the subject's visual function or cognitive ability (e.g., the color, size, duration, shape, etc. of each visual stimulus or any other visual characteristic of the stimulus or display background may be varied depending on the type of test being performed, such that the subject's gaze serves as a marker of the subject's performance in the test). Depending on the test being performed, the visual stimuli may each be static (displayed at a fixed position) or dynamic (moving over time or varying in the visual characteristics of the stimulus over time (e.g., pulsating, changing color, and other changes in the appearance of the visual stimulus)).

[0010] As used herein, the term "eye-tracking based task" may be understood to mean any task that involves the presentation of a series of visual stimuli to be viewed by a subject, where the subject's gaze is tracked in order to identify, measure, evaluate or analyze the subject's performance on the task. The task may be part of a training session, a therapy, a game or any other task. As noted above, the visual stimuli used may vary depending on the type of task.

[0011] As used herein, the term "visual stimulus" may be understood to mean any physical or virtual object displayed to a subject. Visual stimuli may include physical objects that may be displayed on any of the electronic display devices disclosed herein or displayed to the patient at predetermined locations set by a medical professional.

[0012] As used herein, the term "subject" may be understood to mean any person performing a task involving the appearance of a visual stimulus. The subject may be a patient in a clinical setting performing one or more tasks related to calibrating an eye tracker in preparation for one or more tests of the subject's visual function or the patient's cognitive ability. While this disclosure is primarily concerned with determining whether the quality of the obtained eye tracking data is acceptable in a clinical setting (e.g., during one or more tests of the patient's visual function or cognitive ability), the subject may also be a user of a display device performing a visual task in other contexts.

[0013] As used herein, the term "static visual characteristics" may be understood to mean any characteristic of a visual stimulus that describes any aspect of the appearance of the visual stimulus at a point in time. Static visual characteristics include, for example, the shape, size, color, brightness, or contrast of the stimulus.

[0014] As used herein, the term "dynamic visual characteristic" may be understood to mean any characteristic that describes how the position or static visual characteristics of a visual stimulus change over time. For example, a stimulus may move or pulsate around the screen (changing size over a time cycle), or any of the static visual characteristics may change over time (e.g., shape, size, color, brightness, contrast).

[0015] Examples of eye-tracking based tests include: Eye exam Contrast Sensitivity Test Color perception test Eye movement control test Visual field test Visual Recognition Test Strabismus detection and / or measurement tests ·Pupillary response test Object Recognition Test Priority paradigm testing Visual Perception Test Visual-motor testing

[0016] As used herein, the term "eye tracking data" may be understood to mean any data indicative of one or more characteristics of a subject's gaze or position that can determine one or more of the precision or accuracy of the eye tracking device generating the data and whether the subject is paying attention to an eye tracking-based task. Eye tracking data may include direct data indicative of one or more characteristics of a subject's gaze or may include data from which one or more characteristics can be derived or calculated. Eye tracking data may be, for example, gaze data characterizing the position or type (e.g., fixation, saccade, pursuit, blink, etc.) of gaze of one or both eyes of a subject. Eye tracking data may be qualitative or quantitative.

[0017] As used herein, the term "characteristics" relating to a subject's gaze or position may be understood to mean any characteristic that describes a physical property of the subject's gaze or position. Gaze characteristics may include gaze direction relative to any coordinate system, gaze position relative to any coordinate system (e.g., relative to an eye tracker, display, or stimulus), and patient eye position. Position characteristics may include the position or orientation of the patient's head, eyes, or pupils.

[0018] Examples of quantitative eye tracking data include: The position or direction of the subject's gaze relative to the stimulus or relative to an absolute coordinate system. The position of the subject's pupils relative to the eye-tracking device, or relative to an imaginary tracking box projected onto the subject's face, or relative to another coordinate system. Pupil size. Image data of the subject's eyes (gaze position can be determined from known facial image analysis algorithms). A quantitative measure of how well the eye tracking data fit the calibration model of the eye tracking device. For example, the eye tracking device may include an automatic calibration algorithm that fits the subject's eye tracking data obtained during the calibration process to the eye tracking model. How well the data is calibrated may be indicated, for example, by a normalized numerical value. The number or percentage of visual stimuli that failed the eye tracker's calibration algorithm (i.e., those that the eye tracker determined were impossible to calibrate). The number or percentage of stimuli that the eye tracker had to repeat due to a failure to calibrate the stimulus data (or the number of times the eye tracker issued a request to repeat the stimuli for a calibration point). The duration for which each visual stimulus had to be displayed during the calibration process before the eye tracker's calibration algorithm successfully calibrated the calibration point. Vector formed by the pupil and the reflection of light in the eye.

[0019] Examples of qualitative eye tracking data include: For each data point, whether the subject looked at the visual stimulus or not. For each data point, whether the subject looked within a certain distance of the visual stimulus. For each data point, whether the subject's gaze was on the display. For each data point, whether the pupil size is smaller than a given threshold.

[0020] As used herein, the term "pathological or physiological condition" may be understood to mean any kind of anatomical, mental, or other condition that affects a subject's ability to perform an eye-tracking-based task, or that affects the subject's visual function (e.g., eye movement control), or that affects other aspects of the subject's vision or ability to concentrate on or view visual stimuli. Hereinafter, the term "pathological condition" may also be used, but the same meaning is intended.

[0021] As used herein, the term "non-communicative" in the context of a subject can refer to any subject who is limited in their ability to communicate effectively with a healthcare professional, either verbally or non-verbally. For example, the subject may be too young to communicate directly with a healthcare professional or may have a pathological condition that limits their ability to communicate with a healthcare professional.

[0022] As used herein, the term "uncooperative" in the context of a subject can refer to any subject who has limited ability to understand or follow the instructions of a healthcare professional. For example, the subject may be too young to understand instructions given by a healthcare professional or may have a pathological condition that limits the subject's ability to understand or follow the instructions of a healthcare professional.

[0023] As used herein, the term "calibration quality" in the context of eye tracking data may be understood to mean a measure of the reliability of the calibration of an eye tracking device used to generate eye tracking data for evaluating a subject's performance on an eye tracking-based test or task. Calibration quality may be affected by a poor initial calibration, which reduces the accuracy and / or precision of the eye tracking device and therefore reduces the correlation between the gaze position measured by the eye tracking device and the actual position of the gaze. In such cases, the system may require recalibration. Calibration quality may also be affected by the subject moving away from their original position when the calibration was performed. If the subject moves too far away from their original position, the calibration may no longer be considered valid because errors caused by the changed position reduce the accuracy and / or precision of the eye tracking device. In such cases, calibration quality may be improved without the need for recalibration. Poor calibration quality means that the system cannot reliably identify the subject's task performance, whereas high calibration quality means that the system can accurately identify the subject's task performance. Calibration quality can be quantified in a number of ways, some of which are described in more detail below.

[0024] As used herein, the term "attention level" may be understood to mean a measure of the attention a subject pays to the presentation of visual stimuli. Measuring a subject's attention level is preferable because it helps distinguish, for example, between poor performance on a task or test due to the subject's inability to see the visual stimuli due to a pathological condition and poor performance due to the subject's failure to pay attention to the visual stimuli. Attention level can be quantified in a number of ways, some of which are described in more detail below.

[0025] For simplicity, the term "reliability of data" may be used to mean either calibration quality, level of care, or a combination of the two.

[0026] It should be noted that for all of the embodiments disclosed herein, the disclosure may apply to one or both eyes of a subject. If both eyes of a subject are tracked, the data may include two data sets (one for each eye) or a single combined data set of combined gaze values ​​(e.g., the average value of the two eyes). Thus, when referring to one eye, it may alternatively apply to both eyes. Conversely, when referring to both eyes or multiple "eyes," it may alternatively apply to one eye. Those skilled in the art will understand that any of the tests, tasks, or calibrations disclosed herein may be applied to one or both eyes, and any other disclosure should not be construed as limiting the scope of the present invention.

[0027] FIG. 1 illustrates a computer-implemented method according to one or more embodiments of the present invention. In the illustrated method, dashed boxes are optional. In some embodiments, the method begins with receiving eye-tracking data for one or both eyes of a subject for each of a series of visual stimuli (i.e., multiple stimuli displayed sequentially, one at a time) (step S8). The visual stimuli may be displayed at multiple spatial locations. The eye-tracking data indicates one or more characteristics of the subject's gaze or position when each of the stimuli is displayed. Some or all of the multiple spatial locations may be at the same spatial location depending on the task to be performed. Note that the method does not require an actual eye-tracking step to be performed. For example, the eye-tracking data may be data previously generated by a separate device and received by the computer. Subsequent to step S9, the method includes receiving one or more subject metrics, including at least one of the subject's age, one or more pathological or physiological conditions of the subject, and data representative of the calibration quality and / or attention level of one or more previous eye-tracking-based tests or tasks performed by the subject. Next, the method (step S10) determines, based on the eye tracking data for at least each visual stimulus and subject index, whether the eye tracking data exhibits reliability lower than a predetermined minimum quality associated with the subject index. Step S10 includes quantifying the calibration quality of the eye tracking data based on at least the eye tracking data, the position of each visual stimulus, and the subject index and determining whether the calibration quality is lower than a predetermined minimum calibration quality associated with the subject index, and / or quantifying the subject's attention level based on the eye tracking data and determining whether the subject's attention level is lower than a predetermined minimum attention level associated with the subject index based on at least the eye tracking data and the subject index. As previously described, both the calibration quality and the attention level affect the reliability of the eye tracking data in determining the subject's performance on a test or task. Optionally, if the characteristics of the eye tracking data are calculated relative to the position of the stimuli, step S10 may be based on the position of each visual stimulus.The method advantageously takes into account subject metrics to determine whether improving the calibration quality or improving the subject's attention level will enable greater reliability of the tracking data. For example, it has been observed that the average calibration quality and attention level of standard eye tracking data varies among subjects of different ages and / or pathological or physiological conditions. Specifically, for example, younger subjects tend to have lower attention levels, and the maximum attention level expected for eye tracking data from younger subjects is generally lower than that of older subjects. In other examples, subjects may have pathologies that generally reduce calibration quality. To determine whether the calibration quality or attention level for each subject can be improved for a test or task, the data can be compared (by a rule-based algorithm or machine learning algorithm) with reference data given the provided subject metrics. Thus, the method can distinguish between subjects to indicate whether the calibration quality and / or attention level is lower than expected. For example, for the same eye tracking data with a given calibration quality and / or attention level, the calibration quality and / or attention level may be determined to be acceptable for a certain age and / or pathological condition, while the same eye tracking data for another age and / or pathological condition may be deemed to have a low calibration quality and / or attention level. This is because the subject's ability to maintain gaze and / or attention to visual stimuli depends on these factors. It is particularly important to automatically determine whether the reliability of the data is sufficiently high for uncommunicative or uncooperative subjects, as medical professionals may not be able to effectively communicate instructions to the subject, and the subject may not be able to correctly follow the instructions of the visual test. Therefore, validating the data is an important tool for medical professionals to determine whether the best possible eye tracking data has been correctly obtained for each subject; if not, further tests, tasks, or calibrations can be performed to improve the reliability of the data.

[0028] It should be noted that steps S8-S10 may be performed continuously in real time as new eye-tracking data is generated, for example, the eye-tracking data may be analyzed for quality over a moving time frame.

[0029] The age indication may be provided in years, months, and / or days, and if the subject's age is provided as an indication, the reference data to which the eye tracking data is compared may be reference data based on subjects in the same age group as the subject. For example, the method may include determining which of a predetermined age range the subject falls into (non-limiting examples include 0-6 months, 6-12 months, 12-18 months, 18-24 months, 2-3 years, 3-4 years, 4-5 years, 5-6 years, and over 6 years) and comparing the subject's eye tracking data to reference data of subjects in the same age range or using a reference curve. The pathological or physiological condition may include any of the conditions described herein or any other relevant pathological or physiological condition, and the data may be compared to reference data of subjects with the same pathological or physiological condition. If both age and pathological / physiological condition are provided, the eye tracking data may be compared to reference data of subjects in the same age range and with the same pathological / physiological condition. Alternatively or additionally, if data representative of the data quality of one or more previous eye-tracking-based tests or tasks performed by the subject is provided, this may be any previous eye-tracking data generated before the received eye-tracking data representative of the calibration quality and / or attention level of the one or more previous eye-tracking-based tests or tasks, and in some examples may include one or more statistical parameters characterizing the one or more previous eye-tracking data or quantitative quality parameters (e.g., secondary parameters disclosed herein) derived from the previous eye-tracking data representative of the calibration quality and / or attention level of the one or more previous eye-tracking-based tests or tasks. When age and / or pathological and / or physiological condition are provided, this method advantageously allows for determining whether the eye-tracking data is of acceptable quality for the subject even if no previous eye-tracking-based data is available for the subject, unlike embodiments in which only the subject's previous test or task performance is used and requires that previous eye-tracking tests or tasks have been administered to the same subject.

[0030] It should be noted that determining step S10 can be performed by a rule-based algorithm or a machine learning algorithm. In embodiments where a rule-based algorithm is used, the steps may include a first step of receiving or calculating, for one or both eyes, one or more statistical parameters describing a plurality of data points of the eye tracking data, or one or more secondary parameters derived from the one or more statistical parameters. The statistical parameters may be, for example, the mean, median, mode, standard deviation, or variance of any of the quantitative eye tracking data for each stimulus if quantitative eye tracking data is generated, a parameter describing the best fit of the quantitative data (e.g., describing a normal distribution of the data points), a bivariate contour ellipse area (BCEA) known in the art, and, in the case of qualitative data, the proportion of data points within one or more qualitative data categories. (e.g., the percentage of data points where the subject was looking at the stimulus or the screen). The data may include one or more sets of data with associated categories of gaze type (e.g., saccades, fixations, smooth pursuit movements) that characterize the set of data, such that a series of eye tracking data is characterized by gaze type over time.

[0031] The one or more secondary parameters may be one or more parameters generated according to a predetermined equation that is a function of one or more statistical parameters (e.g., a single secondary parameter that is a function of all of the statistical parameters, such that the statistical parameter is characterized by a single numerical value). In a second step, the one or more statistical or secondary parameters are compared to one or more predetermined thresholds associated with the subject index, i.e., one or more thresholds that indicate acceptable calibration quality and / or attention level for the given subject index. For example, each of the one or more statistical and / or secondary parameters is compared to that parameter's threshold value, which is a threshold that defines the boundaries indicating acceptable or unacceptable calibration quality and / or attention level for the subject index. For example, for any given age and / or pathological and / or physiological condition, normative data may be obtained for subjects of the same age group and / or subjects with the same pathological and / or physiological condition, and an average statistical / secondary parameter may be determined from the normative data. A predetermined threshold can then be defined based on a reference statistical / secondary parameter (e.g., one standard deviation from the mean reference value or a percentage difference from the mean reference value), above which the parameter is considered to have low calibration quality and / or attention level for the age and / or pathological or physiological condition. Note that, depending on the statistical / secondary parameter, a higher numerical value may indicate a higher or lower calibration quality or attention level, as will be apparent to those skilled in the art for a given statistical or secondary parameter. Thus, values ​​above or below the threshold may indicate acceptable or unacceptable data reliability (and vice versa). If the subject index includes data representing calibration quality or attention level for one or more previous eye-tracking-based tests or tasks performed by the subject, the same statistical or secondary parameter can be calculated for the previous data, and the calibration quality and / or attention level threshold can be set accordingly (again, e.g., one standard deviation or percentage difference from the statistical parameter, such that a substantial deviation from the previous calibration quality level or attention level indicates that more reliable data may be obtained).In some embodiments where data representative of the data quality of multiple previous eye-tracking-based tests performed by a subject over time is used, a calibration quality or attention level threshold may be set by analyzing trends in the data over time. For example, if the data indicates that the reliability of the data has improved over time (e.g., because the subject has received treatment or therapy for a pathology), a higher calibration quality and / or attention level threshold may be set for the eye-tracking data to account for this trend. Conversely, if the reliability of the previous data has improved over time, a higher calibration quality and / or attention level threshold may be set for the eye-tracking data to account for this trend. If the eye tracking data deteriorates over time (e.g., due to a degenerative disease), a lower calibration quality and / or attention level threshold can be set for the eye tracking data to account for this trend. In a third step, if a predetermined number of one or more statistical or secondary parameters do not meet one or more associated predetermined thresholds, the method determines that the eye tracking data has lower reliability than a predetermined minimum reliability (calibration quality and / or attention level); otherwise, determines that the eye tracking data has acceptable reliability. The predetermined number can be at least one of the calculated statistical and / or secondary parameters (i.e., one or more of the calculated parameters do not meet the predetermined threshold), or the predetermined number can be, for example, two or more, three or more, or all of the calculated parameters (i.e., all of the calculated parameters do not meet the predetermined threshold). In some embodiments, the method includes comparing the data to at least one calibration quality threshold and at least one attention level threshold.

[0032] In an alternative embodiment, step S10 is performed by an artificial intelligence (AI) algorithm, such as a machine learning algorithm. In such an embodiment, step S10 includes inputting the eye tracking data or statistical or secondary parameters of the eye tracking data, optionally the location of the visual stimulus, and a subject metric into an AI algorithm, which is trained to determine whether the reliability of the eye tracking data is acceptable or lower (calibration quality and / or attention level) than a predetermined minimum quality associated with the subject metric, compared to reference subject data having the same subject metric. The AI ​​algorithm may be any suitable algorithm known in the art, including any supervised, unsupervised, semi-supervised, and reinforcement learning algorithms. If the subject metric includes the subject's age and / or one or more pathological / physiological conditions, the training data for the AI ​​algorithm may include multiple data sets for subjects of each age group or pathological / physiological condition (or combinations thereof, as described above). Each dataset may include a plurality of data points indicating the position of gaze relative to the stimulus and / or one or more statistical parameters describing the plurality of data points and / or one or more secondary parameters derived from the one or more statistical parameters and / or qualitative data regarding the aforementioned plurality of data points and / or statistical parameters describing the aforementioned qualitative data. For each dataset, the training data may further include an indicator of the reliability (calibration quality and / or attention level) of the dataset with respect to the subject's age and / or pathological and / or physiological condition. The indicator may be, for example, a binary, continuous, or discrete quantitative number, one of several labels (e.g., "low," "medium," "high"), or any other suitable indicator. A machine learning algorithm can then be trained to determine whether the reliability of the eye tracking data is above or below a minimum quality assigned to the age and / or pathological and / or physiological condition for new input data given the subject indicators (e.g., age, pathological / physiological condition) or a combination thereof and the eye tracking data and / or the statistical parameters and / or secondary parameters of the eye tracking data.

[0033] If the subject metric includes data representing data reliability (calibration quality and / or attention level) for one or more previous eye-tracking-based tests or tasks performed by the subject, the AI ​​can be trained using training data including, for multiple subjects, eye-tracking data or a statistical or secondary parameter of the eye-tracking data, data representing data reliability for one or more previous eye-tracking-based tests or tasks performed by the subject, and an indicator of whether the reliability of the eye-tracking data or the statistical or secondary parameter of the eye-tracking data is acceptable compared to the data reliability of one or more previous eye-tracking-based tests or tasks performed by the subject. The indicator can be, for example, a quantitative number in binary, continuous, or discrete format, one of multiple labels (e.g., “low,” “medium,” “high”), or any other suitable indicator.

[0034] In some embodiments, the method ends after determining step S10 is performed regardless of the outcome of step S10. In some embodiments, a notification is communicated to the user regarding the outcome of step S10 (e.g., an audio or visual notification that the calibration quality and / or attention level is higher or lower than a predetermined minimum calibration quality and / or attention level). Optionally, in some embodiments, the method ends if it is determined (S11) that the eye tracking data has a calibration quality and / or attention level higher than a predetermined minimum calibration quality and / or attention level, and if it is determined (S11) that the eye tracking data has a calibration quality and / or attention level lower than the predetermined minimum calibration quality and / or attention level, the method may further include acquiring new eye tracking data by displaying (S12) a second plurality of visual stimuli to the subject, the visual stimuli being displayed individually in sequence, and the visual stimuli having a plurality of spatial locations. Some or all of the plurality of spatial locations may be the same spatial location depending on the task to be performed. If the calibration quality is determined to be unacceptable (e.g., to obtain data of higher calibration quality without recalibrating the eye-tracking device by, for example, changing the subject's position), new eye-tracking data can be acquired to recalibrate the eye-tracking device. If the attention level is determined to be unacceptable, new eye-tracking data can be acquired to obtain improved eye-tracking data for a given test or task. The method may further include tracking (S13) the subject's gaze with the eye-tracking device for each of the visual stimuli. The eye-tracking device may be an eye-tracking device capable of tracking the position of the subject's eye or eyes and placed at a known position relative to the visual stimuli (e.g., any commercially available eye tracker or any computing device including an imaging device such as a camera and appropriate software for tracking eye position based on captured images), or any other eye-tracking device capable of generating one or more characteristics of the subject's gaze or position.The method may further include generating eye tracking data (S14) indicative of one or more characteristics of the subject's gaze or position based on the gaze tracked by the eye tracking device. In some embodiments, step S10 may be performed on new eye tracking data. Steps S11-S14 may be repeated multiple times (e.g., two or three times, or until acceptable reliability of the data is achieved). Note that steps S11-S14 may be performed with or without preceding steps S1-S7, as described in further detail below. In embodiments in which steps S11-S14 are performed without preceding steps S5-S7, the data received in step S8 may include characteristics of the visual properties of stimuli presented to the subject, such that similar visual stimuli can be presented in steps S12-S14.

[0035] In some embodiments in which steps S12-S14 are performed, the second plurality of visual stimuli have different static or dynamic visual characteristics than those used to generate the eye tracking data determined in step S10 to have a calibration quality and / or attention level below a predetermined minimum calibration quality and / or attention level. Alternatively or additionally, if each of the plurality of visual stimuli has different static or dynamic visual characteristics among themselves, the plurality of stimuli may have a smaller overall range of different static or dynamic visual characteristics compared to the visual stimuli used to generate the eye tracking data from step S10 with an unacceptable calibration quality and / or attention level. Alternatively or additionally, the second plurality of visual stimuli are displayed over a smaller field of view than the visual stimuli used to generate the eye tracking data with an unacceptable calibration quality and / or attention level in step S10. Alternatively or additionally, one or more audio stimuli are provided to the subject between sequences of the second plurality of visual stimuli. This advantageously allows the visual stimuli to be tailored to the subject in order to more effectively capture the subject's attention. For example, a subject may not see a visual stimulus because it was too small for the subject to notice or because the subject has a pathological condition that makes the subject less likely to notice the stimulus (e.g., the subject is color blind). If the subject is unable to understand that a test or task is being performed (e.g., if they are very young), the subject may only sporadically pay attention to the visual stimulus, which may result in a low level of attention in the data. Therefore, by changing the visual characteristics or providing other stimuli in combination with the visual stimulus, the subject's attention may be captured to obtain the necessary eye tracking data with an acceptable level of attention. In some embodiments in which steps S12-S14 are repeated several times as described above, the visual characteristics may be changed each time steps S12-S14 are repeated. Particularly for subjects who are unable to communicate or are uncooperative, repeating the test or task with different visual characteristics increases the likelihood of obtaining eye tracking data of sufficient quality. Specific examples of changes to the second plurality of stimuli include: · Presenting stimuli of larger size. Displaying slower moving stimuli. Presenting the second set of stimuli in a narrower field of view, i.e., so that the stimuli appear less distant than before (increasing the likelihood that the subject will be able to notice the second stimulus while gazing at the location of the previous stimulus). - Playing sounds or music to attract the subject's attention. Presenting stimuli with additional dynamic visual characteristics, such as vibration, pulsation, or other movement. · Presenting stimuli that change color more frequently than previous multiple stimuli. Increasing the brightness and / or contrast of the stimulus. Reducing the range of visual properties between stimuli, e.g., restricting all stimuli to within the range of visual properties for which good quality data were obtained for the subject, or reducing the range of visual properties to a range that is generally visible to the subject. · Reducing the time for a test or task.

[0036] It should be noted that in any of the embodiments disclosed herein, the eye-tracking data may be obtained during calibration of the eye-tracking device. In such embodiments, a plurality of visual stimuli may be displayed to calibrate the subject's eye position using measurements made by the eye-tracking device, and the determining step determines whether the calibration quality of the calibration is lower than a predetermined minimum calibration quality for a subsequently administered eye-tracking-based task or test based on the subject's indicator. In some embodiments, the eye-tracking data may be obtained during an eye-tracking-based test or task, cognitive test or task, or vision therapy administered to the subject. In such embodiments, the determining step determines whether the calibration quality and / or attention level of the eye-tracking data is lower than a predetermined minimum calibration quality and / or attention level for the visual test or task, cognitive test or task, or vision therapy for the subject's indicator.

[0037] In some embodiments, the method may include generating actual eye tracking data. Specifically, the method may include displaying (S5) multiple visual stimuli to the subject. The visual stimuli are displayed sequentially and individually, and optionally have multiple spatial locations. As previously described, the multiple visual stimuli may be physical objects arranged sequentially at predetermined locations relative to the subject by a healthcare professional, or may be displayed on an electronic display device as described herein. In addition to the visual stimuli, one or more audio stimuli may be provided (e.g., upon display of each stimulus or in response to the subject's gaze, as described in more detail below). The method may further include tracking (S6) the gaze of one or both of the subject's eyes for each of the visual stimuli with an eye tracking device. The eye tracking device may be any suitable device for eye tracking disclosed herein. The method may further include generating (S7) eye tracking data characterizing the position of the gaze of one or both eyes relative to the displayed stimuli during the display of each visual stimulus, based on the gaze tracked by the eye tracking device.

[0038] In some embodiments, the generated eye tracking data may exclude data obtained within a predetermined period after the first appearance of each stimulus. For example, eye tracking data obtained within one second after the first appearance of each stimulus may be excluded. Alternatively or additionally, in some embodiments, the generated eye tracking data may exclude eye tracking data obtained within a predetermined period before the disappearance of each stimulus. For example, eye tracking data obtained within one second before the disappearance of the stimulus may be excluded. This reduces the possibility that the eye tracking data may artificially reduce the reliability of the eye tracking data if the subject has not yet noticed or tracked the new visual stimulus or has lost attention.

[0039] In some embodiments, once the system is calibrated, the device may be configured to detect whether the subject is looking at the visual stimuli or whether the subject has looked at each visual stimuli at any time. This may be calculated based on a preset algorithm performed on the eye tracking data, as known in the art. In some embodiments, if the eye tracking device determines that the subject has not looked at the visual stimuli for a predetermined period of time, the subject may be determined to be not gazing at or not looking at the visual stimuli.

[0040] Instead of or in addition to determining whether the subject is looking at the visual stimulus, some eye-tracking devices may provide a real-time indication of whether valid gaze data has been acquired. The real-time indication may be continuously monitored. In some embodiments, if no valid eye-tracking data has been acquired by the eye-tracking device for a predetermined period of time, it may be determined that the subject is not paying attention. Alternatively, the eye-tracking data may include any other data indicative of whether the patient is paying attention, as disclosed herein, and this data may be continuously monitored in real time to determine whether the subject is paying attention.

[0041] In embodiments in which the system includes an algorithm for detecting whether the subject is viewing or was able to view the visual stimulus, one or more display characteristics (static and / or dynamic) of the visual stimulus may be altered when it is determined that the subject is viewing the visual stimulus in order to maintain the subject's attention. The one or more display characteristics may be altered in real time (i.e., while the subject is viewing the stimulus) or may be altered after the stimulus disappears or after the task is completed (i.e., if the task is performed successfully or at the end of the displayed stimulus, to provide positive feedback to the subject at the end of the task). Alternatively or additionally, an audio stimulus may be provided to maintain the subject's attention when it is determined that the subject is viewing or was able to view the visual stimulus. Alternatively or additionally, one or more display characteristics (static and / or dynamic) of the visual stimulus may be altered to maintain the subject's attention when it is determined that the subject is looking away from the visual stimulus. Alternatively or additionally, an audio stimulus may be provided to maintain the subject's attention when it is determined that the subject is looking away from the visual stimulus. Specific examples include: - displaying a static or moving (e.g., rotating) star or other visual prompt symbol and / or emitting a sound when or after the subject is determined to be looking at the visual stimulus; and / or Increasing the brightness of the visual stimulus when the subject is looking at the stimulus, and / or Emitting a sound when the subject begins to look at and / or look away from the stimulus, and / or Gradually increasing the size of the visual stimulus when the subject is looking away from it, and optionally returning the visual stimulus to its original size when the subject is looking at it (to maintain the difficulty of the test or task), and / or If the visual stimulus is configured to move during the test or task, maintaining the visual stimulus in a static position until the subject is determined to be looking at the stimulus, and initiating the movement of the visual stimulus when the subject finds the stimulus; Examples include:

[0042] In some embodiments where calibration quality and / or attention level are continuously monitored over a moving time frame, the task or test is paused if the eye-tracking data is determined to indicate a calibration quality and / or attention level below a predetermined minimum calibration quality and / or attention level. Optionally, the task or test is resumed if the eye-tracking data is determined to have returned to an acceptable calibration quality and / or attention level for the subject. If the process of generating eye-tracking data is resumed when calibration quality and / or attention level return to an acceptable level, the process may, for example, By displaying the same stimulus that was displayed when the subject's gaze was judged to be away from the visual stimulus, or Discarding the data of the stimulus that was displayed when the subject's gaze was determined to be away from the visual stimulus and presenting an additional visual stimulus at the end of the visual stimulus sequence; By resetting the process to a predetermined time or a predetermined number of visual stimuli before the process is paused, or by continuing the sequence of stimuli from the point at which the process was paused, and optionally discarding eye-tracking data acquired while the process was paused, and optionally further discarding eye-tracking data acquired before the process was paused and for a predetermined period of time after the process was resumed; It can be resumed.

[0043] In some embodiments, prior to steps S5-S7, the method may include a step (S1) of tracking the subject's position via an eye-tracking device or a device for tracking the subject. The method may further include a step (S2) of indicating the subject's position on a display device. The subject's position on the display device may be indicated by displaying an image of the subject or a drawing or virtual representation of the subject (e.g., a rendered animated face) representing the subject's position in front of the display device. The subject's position is calculated by the eye-tracking device or a computing device connected to the eye-tracking device using eye-tracking data generated by the eye-tracking device and the known positions and orientations of the display device and the eye-tracking device relative to one another to calculate the subject's position in front of the display device (i.e., the subject's reflected position if the display device is a mirror). Such calculations are known in the art. The method may further include a step (S3) of overlaying a subject positioning guide on the display device to indicate the subject's correct position for obtaining eye-tracking data. The positioning guide may be placed on the display device to ensure that the subject is in the correct position when the subject's image or representation is aligned with the guide. The guides may be of any suitable form or shape, for example, a boundary line formed as the outline of a face or two circles configured to align with the eyes of the subject or representation. The guides are positioned until it is determined that the subject is correctly positioned (S4), at which point the method may proceed to step S5.

[0044] The systems and methods disclosed herein are suitable for obtaining the highest expected calibration quality and / or attention level of eye tracking data for a given set of metrics for a subject. It should be noted that in some cases, a parent or caregiver may hold the subject in a predetermined position so that the eye tracking device can detect multiple pairs of eyes within its field of view. For example, the subject may be an infant sitting on an adult's lap. In such cases, the method may include indicating that multiple eyes have been detected. Alternatively or additionally, the method may include generating eye tracking data for the subject's lowest pair of eyes. This reduces the amount of redundant data generated and reduces errors introduced by tracking the wrong pair of eyes.

[0045] FIG. 2A illustrates an apparatus for performing a method of the present invention according to one or more embodiments. The apparatus includes an eye tracking device 120. The eye tracking device may be any suitable device disclosed herein. The eye tracking device includes an imaging device, such as a camera 122, a memory 124, a processor 126, and a communication interface 128. The imaging device 122 may be any suitable imaging device for capturing images so that the subject's eyes can be tracked. The memory 124 may contain instructions for detecting and calculating the position of the subject's eyes relative to a displayed visual stimulus or any other coordinate system using image data generated by the imaging device 122. The memory may contain a list of predetermined locations of the visual stimuli relative to the eye tracking device (e.g., locations on a display device or locations of objects displayed by a medical professional), or the device may receive manual indication of these locations via a user interface (not shown). The processor 126 may be configured to calculate the position of the subject's eyes as a function of time from the image data collected by the imaging device and the locations of the visual stimuli. The eye tracking device 120 may further include a communication interface 128 for communicating imaging data generated by the imaging device 122 or any data calculated by the processor 126. The eye tracking device 120 may be configured to perform steps S1, S6, S7, S13, and S14.

[0046] It should be noted that although the memory 124 and processor 126 are provided as part of the eye tracking device, these components may be provided remotely from each other and from the imaging device 122, with data being transmitted between the components by any suitable wired or wireless connection (e.g., USB, internet, Bluetooth).

[0047] The device may further include an electronic display device 100 for displaying the plurality of visual stimuli. The display device 100 may include any suitable electronic display 102 known in the art. In addition to displaying the plurality of visual stimuli on the display 102, the display device 100 may include a memory 104 containing instructions for controlling the visual characteristics of the visual stimuli and providing any other stimuli (e.g., in embodiments where the display device 100 includes or is connected to a speaker and auditory stimuli are provided along with the visual stimuli). The display device 100 may further include a processor 106 for processing or calculating any of the data, such as the subject's eye position, based on image data generated by the imaging device 122, or for rendering images or representations of the guide and the subject and determining when the subject is aligned with the guide in steps S2 through S4. The display device 100 may further include a user interface 108 (e.g., a GUI) for controlling the device, such as for selectively initializing calibration or eye-tracking-based tests or tasks. The user interface 108 may further be used to select a test, task, or calibration, e.g., a method for modifying the visual characteristics or other stimuli provided in step S12. The user interface 108 may further be used to input, or the memory 104 may include, subject indicators. The display device 102 may include a communication interface 110 for communicating with an external device, e.g., to communicate results of a calibration or eye-tracking-based test or task to an external device, to indicate that the calibration quality and / or attention level is below a predetermined calibration quality and / or attention level, or to receive imaging data or other data from the eye-tracking device 120 or any other data from the eye-tracking device 120, such as data points related to the subject's gaze. Note that in some embodiments, the display device 100 may have an integrated eye-tracking device, as shown in FIG. 2B . In such a case, the memory 124 and the processor 126 may be the memory 104 and the processor 106, and the communication interface 128 may be the interface 110.The display device may be used to display any of the visual stimuli described in the methods disclosed herein or to calculate whether the calibration quality and / or attention level exceeds a predetermined minimum calibration quality and / or attention level.

[0048] It should be noted that in embodiments in which a display device 100 is provided, the functions of the memory 124 and the processor 126 may alternatively be performed by the memory 104 and the processor 106, or the eye tracking device 120 may not include the memory 124 or the processor 126.

[0049] The memories disclosed herein may include any suitable storage medium, such as optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / warehousing and / or any other type of device suitable for storing instructions and / or data.

[0050] The processors disclosed herein may include any suitable processor, including a CPU, GPU, microprocessor, distributed processing unit, or any other suitable processing unit.

[0051] It should be noted that the above-described embodiments of eye tracking device 120 and display device 100 are merely exemplary, and methods for tracking a user's gaze using eye tracking devices are known in the art. Such eye tracking devices and / or display devices may be used in place of certain embodiments disclosed herein. For example, any or all of the components of display device 100 and eye tracking device 120 may be provided within a single device.

[0052] 3A-3C illustrate an exemplary sequence of visual stimuli that may be displayed on the display 102 to obtain eye tracking data. A first visual stimulus 200a may be displayed in a first position on the display 102. Subsequently, a second visual stimulus 200b may be displayed in a second position on the display 102. Subsequently, a third visual stimulus 200c may be displayed in a third position on the display 102. It will be understood that the visual stimuli 200 may be the same or different in shape, color, size, duration, etc., in which they are displayed. It will be understood that the examples shown in FIGS. 3A-3C are illustrative only, and that the sequence of visual stimuli may be any suitable sequence suitable for obtaining eye tracking data for a subject. The sequence of visual stimuli may be for an eye tracking-based test or task, for example, to test a particular aspect of the subject's visual function, to test the subject's cognitive ability, or to provide vision therapy to the subject. Because the location of each stimulus is known and the position of the eye tracking device relative to the display 102 is known, the subject's gaze when each stimulus is displayed can be easily calculated once the eye tracking device 120 is calibrated, as is known in the art.

[0053] In some embodiments, a computer program is provided that includes computer-readable instructions that, when executed by a processor, cause the processor to perform any of the methods disclosed herein. The computer program may be included in the memory of any of the devices disclosed herein or in an external memory communicatively connected to any of the devices disclosed herein. The computer program may be implemented on hardware and / or software.

[0054] It should be noted that in any of the embodiments disclosed herein, the sequence of visual stimuli displayed to the subject (e.g., in step S5 or S10) may be adapted based on the received subject indicators. For example, the size or any other characteristics of the visual stimuli may be adapted based on the subject's age or any pathological or physiological condition of the subject. Similarly, any other aspect of the displayed visual stimuli (sequence, duration of each displayed stimulus, sound associated with each visual stimulus, static or dynamic visual characteristics) may be adapted based on the received subject indicators. The different characteristics of the visual stimuli and any associated sounds for each combination of patient indicators may be stored as computer-readable instructions in the memory of any of the devices disclosed herein.

[0055] Examples of using eye tracking data Different types of eye tracking data are described below in relation to determining whether eye tracking exhibits a quality below a predetermined minimum quality in accordance with the present invention.

[0056] In some embodiments, eye tracking data may include multiple data points representing the position or direction of the patient's gaze relative to each stimulus or relative to an absolute coordinate system. In such embodiments, data points corresponding to being within the range of a displayed stimulus or on the display may be associated with a good level of attention, as they at least indicate that the subject was paying attention to the display when the stimulus was displayed. The level of attention can be quantified, for example, by calculating the number or percentage of data points where the subject's gaze falls within a region considered to correspond to the subject paying attention (e.g., gaze falling on the display). In other examples, eye tracking data may be recorded when the subject is looking toward the display and not when the subject's gaze is away, so that it can be assumed that the subject is paying attention when data is available. This data can be compared to expected data for the subject's metrics (e.g., the maximum expected level of attention may vary depending on the subject's metrics, so that a better level of attention can be determined for the subject). In some embodiments, the gaze position can be compared to the position of the stimuli to determine whether the eye tracking device is properly calibrated. For example, the gaze position may be spread over a small angular range, meaning that the subject is focusing on a point on the display (and thus observing the stimulus), but that position is, on average, far from the stimulus position. There may also be a consistent difference between the gaze position and the position of each stimulus. This may indicate poor calibration quality of the data. Calibration quality may be quantified, for example, based on a normalized function that depends on the difference between the gaze position and the stimulus position, or any other suitable method for quantifying calibration quality known in the art. Calibration quality may also depend on the patient's pathology. For example, in a patient with strabismus, the gaze position of one eye may be far from the stimulus, whereas in a patient without strabismus, this may indicate poor calibration quality of the data. Therefore, for any given data, the assessment of whether the data is reliable or not may depend on the subject's indicators.

[0057] In some embodiments, the eye tracking data may include multiple data points representing the position of the subject's pupil relative to the eye tracking device, relative to an imaginary eye tracking box projected onto the subject's face, or relative to another coordinate system, such as the position of the subject's eye when the eye tracking device was calibrated. This data indicates good calibration quality if the pupil is located within a predetermined region of the coordinate system related to the working range of the eye tracking device. The predetermined region may be defined by an area surrounding the position of the subject's eye when the eye tracking device was calibrated. If the eye position deviates significantly from this position, the accuracy of the calibration decreases. The predetermined region may be selected based on a minimum acceptable calibration accuracy. If the eye is within the predetermined region, the eye tracking device can obtain accurate eye tracking data based on the calibration, whereas outside the predetermined region, the eye tracking device may not accurately track the eye, invalidating the calibration and reducing the accuracy of the data. The calibration quality of the data may be quantified by taking any other function of the data points, such as the number or percentage of data points within the predetermined region or a parametric calculation of the average pupil position. Furthermore, if the pupil is located within the predetermined region, it may indicate that the subject is paying attention to the stimulus when looking toward the display. The subject's attention level can then be determined from this data (e.g., quantified by taking the number or percentage of data points where the pupil position was within a predetermined region), and it should be noted that the predetermined region where the calibration is considered valid may vary in size relative to the predetermined region where the attention level is considered valid.

[0058] In some embodiments, the eye tracking data may include data indicating the subject's pupil size in absolute terms or relative to a reference pupil size (e.g., the pupil size when the eye tracking device was calibrated for the subject). The data may include an indication for each data point of whether the pupil size is smaller or larger than a predetermined threshold. The subject's pupil size is an indication of how close or far the subject is from the display and / or eye tracking device. This data indicates good calibration quality if the pupil size is within a predetermined range (e.g., within a predetermined range relative to the pupil size when the calibration was performed), from which it can be inferred that the subject's position has not substantially changed between when the eye tracking device was calibrated and when the eye tracking data was acquired. Calibration quality can be quantified based on this data, for example, by calculating the number or percentage of data points where the pupil size was within a predetermined range for which the calibration is valid. Similarly, a pupil size within a predetermined range indicates that the subject is facing the display at an appropriate distance from it and is therefore paying attention to the stimuli; attention level can be similarly quantified. The predetermined ranges of calibration quality and attention level may be the same or different.

[0059] In some embodiments, the eye tracking data may include image data of the subject's eyes. An image analysis algorithm may analyze the image data to identify multiple data points for one or more of face position, pupil position, gaze position, or gaze direction relative to any coordinate system. Pupil position or size, gaze position, and gaze direction may be evaluated for calibration quality and / or attention level, as disclosed herein. Similarly, the face position may be compared to predetermined areas where calibration quality and / or attention level are assumed to be acceptable, and calibration quality and / or attention level may be similarly quantified.

[0060] In some embodiments, the eye tracking data may include one or more quantitative indicators of how well the eye tracking data fits the calibration model of the eye tracking device. This data is a direct indicator of how well the system is calibrated for the subject, and may also be an indicator of the subject's attention level during calibration. Because the best calibration quality expected for a given subject may vary from subject to subject (e.g., subjects' attention may differ, or calibration quality may vary from subject to subject), it can be determined whether a better quality calibration or attention level is achievable based on previous data for subjects with the same or similar subject indicators.

[0061] In some embodiments, the eye tracking data may include data indicating one or more of the number or percentage of visual stimuli displayed during the calibration process for which the eye tracking device's calibration algorithm failed, the number or percentage of visual stimuli for which eye tracking had to be repeated due to failure of the eye tracking device's calibration algorithm, and the amount of time each visual stimulus had to be displayed during the calibration process before the eye tracking device's calibration algorithm successfully calibrated the calibration point. The calibration algorithm may fail for a number of reasons, including the subject's lack of attention or a subject pathology that causes the algorithm to fail more frequently. Thus, the number of calibration failures may be patient-specific, such that recalibration may or may not be expected to improve calibration quality and / or attention level. The length of time each stimulus is displayed during calibration and the number of repetitions also indicate the quality of the calibration (the greater the number of repetitions and / or the longer the display time, the more likely the calibration is of poor quality).

[0062] In some embodiments, the eye tracking data may include data indicative of a vector formed by the pupil and light reflection in the eye. The presence of the vector and its location within a predetermined tolerance, as well as its variation within the predetermined tolerance, are related to the subject's attention and good eye tracking quality. From this data, the gaze position can also be derived, and from the derived gaze position, it can be determined whether the subject was paying attention and whether the calibration quality is good, as described above.

[0063] In some embodiments, the eye tracking data may include data indicating, for each data point, whether the patient is looking at the visual stimulus or within a predetermined distance of the visual stimulus. For stimuli where the patient is expected to look, an indication that the patient is looking at the stimulus is a clear indication that the quality of the eye tracking calibration is good and that the subject is paying attention to the stimulus. The data points may be quantified by determining the number or percentage of data points where the subject was looking at the stimulus, and from the quantified data points, it can be determined whether the quality of the calibration and / or the level of attention is as good as a predetermined acceptable level of calibration quality and / or level of attention given the subject indicator.

[0064] In some embodiments, the eye tracking data may include data indicating whether the patient's gaze was on the display for each data point. This relates to whether the subject was paying attention to the display and therefore likely to notice the stimulus if it were visible to the subject. This data is therefore a good indicator of whether the subject was paying attention when the stimulus was presented. The data points can be quantified by determining the number or percentage of data points during which the subject was looking at the stimulus, and from the quantified data points, it can be determined whether the level of attention is as good as a predetermined acceptable level of attention given the subject's indicators.

[0065] In some embodiments, the eye tracking data includes two or more or all of the above types of data.

[0066] It should be noted that in some embodiments where the subject is assumed to have a sufficient level of attention during the entire process (e.g., a patient who is able to communicate and is cooperative), the methods disclosed herein may include determining whether the calibration quality is higher than a predetermined minimum calibration quality, rather than the attention level. In other embodiments, the calibration quality is assumed to be sufficiently high without checking against the calibration quality, and the methods may include determining whether the attention level is higher than a predetermined attention level, rather than the calibration quality.

[0067] It should be noted that the calibration quality check may include both determining whether the calibration quality itself is poor (i.e., whether all data is inaccurate) and whether the subject's position is making the eye tracking data less accurate (i.e., whether the data can be improved by repositioning the subject without requiring recalibration of the eye tracking device).

[0068] In some embodiments, the methods disclosed herein may include determining both whether the calibration quality is higher than a predetermined minimum calibration quality and whether the attention level is higher than a predetermined attention level. Note that the check for calibration quality may be performed on the same or different eye tracking data for the same or different series of visual stimuli presented to the subject. For example, a first subset of eye tracking data may be checked for calibration quality (e.g., eye tracking data obtained during a post-calibration sequence of visual stimuli), and a second subset of eye tracking data may be checked for attention level (e.g., eye tracking data obtained during an eye tracking-based test or task). Both calibration quality and attention level may be checked continuously by quantifying data quality and / or attention level over a time frame during the entire eye tracking-based process with the subject. For example, both may be continuously monitored during a series of eye tracking-based tests or tasks to determine whether calibration quality and / or attention level have degraded to an unacceptable level at any time, and to provide notification or pause the process if either has degraded to an unacceptable level. Advantageously, the method distinguishes between unreliable data due to poor calibration or poor positioning of the subject and unreliable data due to the subject's level of attention. Because these causes are corrected for in different ways, the overall reliability of the data can be improved by verifying that both are above acceptable levels given the subject's metrics or past performance on eye-tracking-based tasks or tests.

[0069] Although the present disclosure is primarily concerned with determining data quality for eye-tracking based tests and tasks related to visual function or treatment, it will be appreciated that the methods and apparatus disclosed herein may also be applied to other purposes and technical fields.

[0070] While several exemplary embodiments have been described, it should be understood that the foregoing is illustrative and not restrictive, and is presented by way of example only. In particular, while many of the examples presented herein involve particular combinations of device or software elements, these elements may be combined in other ways to achieve the same purpose. Acts, elements, and features described only in connection with one embodiment are not intended to be excluded from a similar role in other embodiments or other embodiments.

[0071] The apparatus described herein may be embodied in other specific forms without departing from its characteristics. The foregoing embodiments are illustrative rather than limiting of the described systems and methods. Accordingly, the scope of the apparatus described herein is indicated by the appended claims, rather than the foregoing description, and all changes that come within the meaning and range of equivalency of the claims are intended to be embraced therein.

Claims

1. 1. A computer-implemented method for validating eye-tracking data for a system that administers an eye-tracking-based test or task to a subject, the method comprising: receiving eye tracking data for one or both eyes of a subject for each of a series of visual stimuli, the eye tracking data indicative of one or more characteristics of the subject's gaze or position when each of the visual stimuli is displayed; receiving one or more subject metrics, the one or more subject metrics including at least one of data representative of the subject's age, one or more pathological or physiological conditions of the subject, and a calibration quality and / or attention level of one or more previous eye-tracking based tests or tasks performed by the subject; Including, The method comprises: a) quantifying a calibration quality of the eye tracking data and determining, based at least on the eye tracking data, the respective positions of the visual stimuli, and the subject metric, whether the calibration quality is lower than a predetermined expected minimum calibration quality associated with the subject metric; b) quantifying the subject's attention level based on the eye tracking data and determining, based at least on the eye tracking data and the subject metric, whether the subject's attention level is below a predetermined expected minimum attention level associated with the subject metric; A method comprising one or both of the following:

2. The step of determining whether the calibration quality is lower than the minimum calibration quality and / or the step of determining whether the attention level is lower than the predetermined expected minimum attention level comprises: receiving or calculating, for one or both eyes, one or more statistical parameters describing a plurality of data points of the eye tracking data over time when each of the visual stimuli is presented and / or one or more secondary parameters derived from the one or more statistical parameters; comparing the one or more statistical parameters or the secondary parameters to one or more predetermined thresholds associated with the subject metric, the one or more predetermined thresholds indicating an acceptable calibration quality and / or level of attention for the subject metric; if a predetermined number of said one or more statistical parameters or said secondary parameters do not satisfy the associated predetermined thresholds, determining that the calibration quality is below said predetermined minimum quality and / or that the attention level is below said predetermined minimum expected attention level, and otherwise determining that the eye tracking data is acceptable; The method of claim 1 , comprising:

3. 2. The method of claim 1 , wherein the step of determining whether the calibration quality is lower than the minimum calibration quality and / or the step of determining whether the attention level is lower than the predetermined expected minimum attention level comprises inputting the eye tracking data and / or one or more statistical parameters describing the eye tracking data and / or one or more secondary parameters derived from the one or more statistical parameters and the subject indicator into an AI algorithm, the AI ​​algorithm being trained to determine whether the calibration quality and / or the attention level is higher or lower than a predetermined expected minimum calibration quality and / or attention level associated with the subject indicator.

4. 4. The method of claim 1, wherein the eye tracking data is acquired by an eye tracking device, and the method comprises, if it is determined that the eye tracking data is of a calibration quality lower than the predetermined minimum expected calibration quality, recommending or performing a recalibration of the eye tracking device and / or indicating that the subject should be repositioned to improve the calibration quality of the data.

5. The method includes step b), wherein if the eye tracking data is determined to indicate an attention level below the predetermined minimum expected attention level and / or the calibration quality is determined to be below the predetermined minimum expected calibration quality, presenting a second plurality of visual stimuli to the patient, the visual stimuli being presented individually and sequentially, the visual stimuli having a plurality of spatial locations; and tracking the gaze of one or both of the patient's eyes with respect to each of the visual stimuli using an eye tracking device; generating eye tracking data indicative of one or more characteristics of the subject's gaze or position during presentation of each visual stimulus based on the tracked gaze; 5. The method of claim 1, further comprising acquiring new eye tracking data by:

6. 6. The method of claim 5, wherein the second plurality of visual stimuli have different static or dynamic visual characteristics or a smaller range of different static or dynamic visual characteristics compared to the visual stimuli used to generate the eye tracking data associated with an unacceptable level of attention and / or calibration quality, and / or the second plurality of visual stimuli are displayed with a smaller field of view than the visual stimuli used to generate the eye tracking data associated with an unacceptable level of attention and / or calibration quality, and / or one or more audio stimuli are provided to the subject between sequences of the second plurality of visual stimuli, and / or the visual stimuli are shown for a different duration compared to the visual stimuli used to generate the eye tracking data associated with an unacceptable level of attention and / or calibration quality.

7. 7. The method of claim 1, wherein the eye tracking data is eye tracking data obtained during calibration of an eye tracking device or the eye tracking data is eye tracking data obtained during an eye tracking based test or task.

8. The method comprises: presenting a plurality of visual stimuli to the subject, the visual stimuli being presented individually and sequentially; tracking the gaze of one or both eyes of the subject with respect to each of the visual stimuli using an eye tracking device; generating eye tracking data indicative of one or more characteristics of the subject's gaze or position during presentation of each visual stimulus based on the tracked gaze; The method of claim 1 , further comprising generating the eye tracking data by:

9. 9. The method of claim 8, wherein the eye tracking data excludes eye tracking data obtained within a predetermined period of time after the first appearance of one or more of the stimuli and / or excludes eye tracking data obtained within a predetermined period of time before the disappearance of one or more of the stimuli.

10. 10. The method of claim 8 or 9, wherein the eye tracking data includes data characterizing a gaze position of one or both eyes, the method including determining the gaze position relative to the stimulus, and wherein one or more display characteristics of the visual stimulus are changed and / or an audio stimulus is provided to the subject when it is determined that the subject is looking at the visual stimulus and / or when it is determined that the subject is not looking at the visual stimulus.

11. 11. The method of any one of claims 8 to 10, wherein the eye tracking data is continuously acquired and comprises data indicative of the subject's attention level and / or calibration quality of the data, the method comprising quantifying the subject's attention level and / or calibration quality of the data, wherein if it is determined that the subject is not paying attention and / or the calibration quality is below a predetermined minimum expected calibration quality, the series of visual stimuli is paused, and optionally, if it is determined from the continuously acquired data that the patient is paying attention and / or the calibration quality is above a predetermined minimum expected calibration quality, the series of visual stimuli is resumed.

12. The sequence of visual stimuli is resumed when it is determined that the subject is paying attention and / or the calibration quality is above the predetermined minimum expected calibration quality, and the sequence of visual stimuli is resumed when it is determined that the subject is paying attention and / or the calibration quality is above the predetermined minimum expected calibration quality. displaying the same stimuli that were displayed when the subject was determined to be not paying attention; discarding data of the stimulus that was displayed when it was determined that the subject was not paying attention, and displaying an additional visual stimulus at the end of the series of visual stimuli; resetting the series of visual stimuli to a predetermined period or a predetermined number of visual stimuli before the series of visual stimuli was paused; continuing the sequence of visual stimuli from the point at which the process was paused, and optionally discarding eye tracking data obtained while the process was paused, and optionally further discarding eye tracking data obtained for a predetermined period before the process was paused and after the process is resumed; The method of claim 11 , wherein the restart is initiated by one of:

13. Prior to generating the eye tracking data, the method further comprises: tracking the position of the subject via the eye tracking device or a device for tracking the subject; indicating the subject's position on a display device; overlaying a subject positioning guide on the display device to indicate the correct position of the subject for obtaining the eye tracking data; 13. The method of any one of claims 8 to 12, comprising:

14. 14. The method of any one of claims 8 to 13, wherein the eye tracking device is configured to detect whether there are multiple pairs of eyes within the field of view of the eye tracking device, and optionally, if multiple pairs of eyes are detected by the eye tracking device, the eye tracking data is generated only for the lowest pair of eyes that is the eye of the subject.

15. 15. An apparatus configured to perform a method according to any one of claims 1 to 14, the apparatus comprising: a display device configured to display the visual stimuli to the subject; and an eye tracking device configured to track the subject's gaze when the visual stimuli are presented to the subject and generate eye tracking data indicative of one or more characteristics of the subject's gaze or position; one or more processors configured to control the display device to display the visual stimuli, receive eye tracking data from the eye tracking device, receive the one or more subject indicators, and perform one or both of the steps of: a) determining, based at least on the eye tracking data for each visual stimulus and one or more subject indicators of the subject, whether a calibration quality of the eye tracking data indicates a calibration quality that is lower than the predetermined minimum expected calibration quality; and b) determining, based at least on the eye tracking data and the subject indicators, whether an attention level of the subject is lower than a predetermined minimum expected attention level associated with the subject indicator; 1. An apparatus comprising:

16. A computer program comprising computer readable instructions which, when executed by a processor, cause the processor to carry out a method according to any one of claims 1 to 14.

Images