Verification of eye tracking data

Abstract

A computer-implemented method of validating eye tracking data for a system for performing eye tracking-based tests or tasks on a subject. The method comprises receiving (S8) eye movement tracking data of one or both eyes of the subject for each of a series of visual stimuli (200), the eye movement tracking data indicating one or more characteristics of a gaze or position of the subject when each stimulus (200) is displayed. The method comprises receiving (S9) one or more subject indications, including at least one of: an age of the subject; one or more pathological or physiological conditions of the subject; and data representing a calibrated quality and / or attention level of one or more previous eye movement tracking-based tests or tasks performed by the subject. The method further comprises one or more of the following steps: a) quantifying a calibration quality of the eye movement tracking data and determining (S10) whether the calibration quality is below a predetermined minimum expected calibration quality associated with the subject indication based at least on the eye movement tracking data, the location of each visual stimulus and the subject indication; b) quantifying an attention level of the subject based on the eye movement tracking data, and determining (S10) whether the attention level of the subject is below a predetermined minimum expected attention level associated with the subject indication based at least on the eye movement tracking data and the subject indication.

Description

Technical Field

[0001] The present disclosure relates to a computer-implemented method for validating eye tracking data for a system for performing an eye tracking-based test or task on a subject. The present disclosure also relates to an apparatus and a computer program for performing the method. Background Art

[0002] Advances in eye tracking technology have enabled eye tracking devices to provide a more accurate means of analyzing a subject's ability to perform certain visual or cognitive tasks. Visual and cognitive tests and other tasks that were previously performed manually by medical practitioners and depended on subjective analysis by medical practitioners can now be performed objectively with eye tracking-based devices.

[0003] In order to be able to effectively analyze eye tracking data, the subject must be focused on the visual stimuli provided, the system must be properly calibrated, and the calibration must remain valid throughout the period during which the eye tracking data is acquired. This may not always be the case, and the healthcare practitioner may not notice that the subject is not focusing on the task, or has moved to a new position where the calibration is no longer valid. In particular, in the case of uncooperative and uncommunicative subjects, it is more difficult to ensure that the subject is focusing on the task, and there is a higher likelihood that less attention is given to the task.

[0004] Thus, there is a need to objectively and reliably obtain the highest possible quality data for each subject when obtaining eye tracking data, and to reliably determine when low quality data has been obtained. Summary of the invention

[0005] According to a first aspect of the present invention, there is provided a computer-implemented method for validating eye tracking data for a system for conducting an eye tracking-based test or task on a subject, comprising:

[0006] receiving eye tracking data for one or both eyes of a subject for each of a series of visual stimuli, the eye tracking data indicating one or more characteristics of the subject's gaze or position when each stimulus is displayed;

[0007] receiving one or more subject indications including at least one of: the subject's age; one or more pathological or physiological conditions of the subject; and data indicating calibration quality and / or attention level of one or more previous eye-tracking-based tests or tasks performed by the subject; and

[0008] One or more of the following:

[0009] a) quantifying the calibration quality of the eye tracking data and determining, based at least on the eye tracking data, the position of each visual stimulus and the subject's instructions, whether the calibration quality is below a predetermined expected minimum calibration quality associated with the subject's instructions; and / or

[0010] b) quantifying the subject's attention level based on the eye tracking data, and determining, based at least on the eye tracking data and the subject's indication, whether the subject's attention level is below a predetermined expected minimum attention level associated with the subject's indication.

[0011] According to a second aspect of the present invention, there is provided an apparatus configured to perform the method according to the first aspect, comprising:

[0012] - a display device configured to display visual stimuli to the subject;

[0013] - an eye tracking device configured to track the subject's gaze when visual stimuli are displayed to the subject and to generate eye tracking data indicative of one or more characteristics of the subject's gaze or position; and

[0014] - one or more processors configured to control the display device to display visual stimuli, receive eye tracking data from the eye tracking device, receive one or more subject indications, and determine one or more of the following: a) based at least on the eye tracking data for each visual stimulus and the one or more subject indications of the subject, determine whether a calibration quality of the eye tracking data indicates a calibration quality that is lower than a predetermined expected minimum calibration quality; and / or b) based at least on the eye tracking data and the subject indications, determine whether the subject's attention level is lower than a predetermined expected minimum attention level associated with the subject indications.

[0015] According to a third aspect of the invention there is provided a computer program comprising computer readable instructions which, when executed by a processor, cause the processor to perform a method according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order that the present disclosure may be better understood and to show how it may be put into practice, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0017] Figure 1 Methods for validating eye tracking data in accordance with one or more embodiments are shown.

[0018] Figure 2A Schematic diagrams showing apparatus in accordance with one or more embodiments.

[0019] Figure 2BA schematic front view of an apparatus in accordance with one or more embodiments is shown.

[0020] Figure 3A to Figure 3C A series of multiple visual stimuli are shown during a method in accordance with one or more embodiments. DETAILED DESCRIPTION

[0021] The term "eye-tracking-based test" as used herein may be understood to mean any test that displays a series of visual stimuli and utilizes qualitative or quantitative eye-tracking data from a subject viewing the stimuli to determine the subject's visual function or cognitive ability. The visual stimuli used for a given test may be configured as needed to determine specific aspects of the subject's visual function or cognitive ability (e.g., the color, size, duration, shape, etc. of each visual stimulus. Alternatively, any other visual characteristics of the stimulus or the background of the display may vary depending on the type of test being performed, so that the subject's gaze provides a marker for the subject's performance in the test). Depending on the test being performed, the visual stimuli may be both static (displayed in a fixed position) or dynamic (moving over time, or the visual characteristics of the stimuli changing over time, such as pulsation, color changes, and any other changes in the appearance of the visual stimuli).

[0022] The term "eye-tracking-based task" as used herein may be understood to mean any task involving the display of a series of visual stimuli to be viewed by a subject, for which the eyes of the subject are tracked to determine, measure, score or otherwise analyze the subject's task performance. The task may be part of a training course, a therapy, a game or any other task. As described above, the visual stimuli used may vary depending on the type of task.

[0023] The term "visual stimulus" as used herein may mean any physical or virtual object displayed to a subject. Visual stimuli may be displayed on any electronic display device disclosed herein, or may include a physical object displayed to a patient at a predetermined location set by a medical practitioner.

[0024] The term "subject" as used herein may be understood to mean any person who is performing a task involving the presentation of visual stimuli. The subject may be a patient in a clinical setting who is performing one or more tasks associated with calibration of an eye tracker in preparation for one or more tests of the subject's visual function or cognitive abilities. While the present disclosure is primarily directed to determining whether the quality of acquired eye tracking data is acceptable in a clinical setting (e.g., during one or more tests of a patient's visual function or cognitive abilities), the subject may be a user of a display device performing visual tasks in other contexts.

[0025] The term "static visual characteristics" used herein may be understood to mean any characteristics of a visual stimulus that describe any aspect of the appearance of the visual stimulus at a certain moment. Static visual characteristics include, for example, the shape, size, one or more colors, brightness or contrast of the stimulus.

[0026] As used herein, the term "dynamic visual property" may be understood to mean any property that describes how the position of a visual stimulus or a static visual property changes over time. For example, a stimulus may move across the screen or may pulsate (change in size over a period of time), or any static visual property may change over time (e.g., shape, size, color, brightness, contrast).

[0027] Examples of eye-tracking based tests include:

[0028] -Vision test.

[0029] -Contrast sensitivity test.

[0030] -Color perception test.

[0031] - Eye movement control test.

[0032] - Field of view test.

[0033] -Visual recognition test.

[0034] - Strabismus detection and / or measurement tests.

[0035] -Pupillary response test.

[0036] -Object recognition test.

[0037] -Preferential gaze paradigm test.

[0038] -Visual perception test.

[0039] -Visuomotor test

[0040] The term "eye tracking data" as used herein may be understood to mean any data indicating one or more characteristics of a subject's gaze or position, and from which one or more of the following may be determined: the precision or accuracy of the eye tracking device generating the data, and whether the subject is paying attention to an eye tracking-based task. Eye tracking data may include direct data indicating one or more characteristics of the subject's gaze, or may include data from which the one or more characteristics may be derived or calculated. Eye tracking data may, for example, be gaze data characterizing the position or gaze type (e.g., fixation, eye saccade, pursuit, blink, etc.) of one or both eyes of the subject. Eye tracking data may be qualitative, or may be quantitative.

[0041] The term "characteristic" used herein in relation to a subject's gaze or position may be understood to mean any characteristic that describes a physical feature of the subject's gaze or position. Gaze characteristics may include gaze direction relative to any coordinate system, gaze position relative to any coordinate system (e.g., relative to an eye tracker, display, or stimulus), and the position of the patient's eyes. Position characteristics may include the position or orientation of the patient's head, eyes, or pupils.

[0042] Examples of quantitative eye tracking data include:

[0043] - The subject's gaze position or direction relative to the stimulus or relative to an absolute coordinate system.

[0044] - The position of the subject's pupil relative to the eye tracking device, or relative to an imaginary tracking box projected onto the subject's face, or relative to another coordinate system.

[0045] - Pupil size.

[0046] - Image data of the subject's eyes (gaze position can be determined based on known facial image analysis algorithms).

[0047] - A quantitative indicator of how well the eye tracking data fits the eye tracker's calibration model. For example, the eye tracker may include an automatic calibration algorithm that fits the subject's eye tracking data obtained during a calibration process to the eye tracking model. How well the data is calibrated may be indicated by, for example, a normalized value.

[0048] - The number or proportion of visual stimuli for which the eye tracker's calibration algorithm failed (ie, visual stimuli for which the eye tracker determined that calibration was not possible).

[0049] - The number or proportion of these visual stimuli that the eye tracker needs to repeat due to failure to calibrate the data for these visual stimuli (or the number of times the eye tracker issues a request to repeat a visual stimulus for a calibration point).

[0050] - How long each visual stimulus needs to be shown during the calibration process before the eye tracker's calibration algorithm successfully calibrates the calibration points.

[0051] -The vector formed by the pupil and the reflection of light in the eye.

[0052] Examples of qualitative eye tracking data include:

[0053] - For each data point, whether the subject was looking at the visual stimulus.

[0054] - For each data point, whether the subject was looking within a certain distance of the visual stimulus.

[0055] - For each data point, whether the subject's gaze fell on the display.

[0056] -For each data point, whether the pupil size is below a predefined threshold.

[0057] As used herein, the term "pathological or physiological condition" may be understood to mean any anatomical, psychiatric, or any other type of condition that affects a subject's ability to perform eye-tracking-based tasks, or affects a subject's visual function (e.g., eye control), or affects any other aspect of a subject's vision or ability to focus on or view visual stimuli. This term may hereinafter be referred to as a "pathological condition," but the meaning is the same.

[0058] As used herein, the term "non-communicative" in the context of a subject may mean any subject that is limited in communicating effectively verbally or non-verbally with a healthcare practitioner. For example, the subject may be too young to communicate directly with a healthcare practitioner, or may have a pathological condition that limits the ability to communicate with a healthcare practitioner.

[0059] As used herein, the term "uncooperative" in the context of a subject may mean any subject who is limited in understanding or following the instructions of a medical practitioner. For example, the subject may be too young to understand the instructions given by the medical practitioner, or may have a pathological condition that limits the ability to understand or follow the instructions of the medical practitioner.

[0060] The term "calibration quality" used herein in the context of eye tracking data may be understood to mean a measure of the calibration reliability of an eye tracking device used to generate eye tracking data, which is used to evaluate the performance of a subject in a test or task based on eye tracking. The calibration quality may be affected by poor initial calibration, which results in reduced precision and / or accuracy of the eye tracking device, thereby resulting in a reduced correlation between the position of gaze measured by the eye tracking device and the actual position of gaze. In such cases, the system may need to be recalibrated. The calibration quality may also be affected by the movement of the subject away from the original position of the subject when the calibration is performed. If the subject moves too far away from the original position, the calibration may be considered no longer valid because the error caused by the changed position reduces the precision and / or accuracy of the eye tracking device. In such cases, the calibration quality can be improved without recalibration. Poor calibration quality means that the subject's task performance cannot be reliably determined, while high calibration quality means that the system is able to accurately determine the subject's task performance. The calibration quality can be quantified in a variety of ways, some of which will be described in further detail below.

[0061] The term "attention level" as used herein may be understood to mean a measure of the attention a subject pays to a display of visual stimuli. It is desirable to measure a subject's attention level because this helps to distinguish between poor performance on a task or test due to, for example, the subject's inability to see the visual stimuli due to a pathological condition and poor performance due to the subject's failure to pay attention to the visual stimuli. Attention level may be quantified in a variety of ways, some of which will be described in further detail below.

[0062] For brevity, the term “data reliability” may be used to refer to the quality of calibration, the level of attention, or a combination of both.

[0063] It should be noted that for all embodiments disclosed herein, the present disclosure may be applicable to one or both eyes of a subject. In the case of tracking both eyes of a subject, the data may include two data sets (one for each eye), or may include a single combined data set of combined gaze values ​​(e.g., an average of the two eyes). Thus, when one eye is referred to, the same may alternatively be applied to both eyes. Conversely, when two eyes or plural "eyes" are referred to, the same may alternatively be applied to only one eye. Those skilled in the art will appreciate that any test, task, or calibration disclosed herein may be applicable to one eye or both eyes, and any disclosure should not be construed as limiting the scope of the present invention.

[0064] Figure 1A computer-implemented method according to one or more embodiments of the present invention is shown. In the method illustrated, the boxes shown with dashed lines are optional. In some embodiments, the method begins (step S8) receiving eye tracking data for one or both eyes of a subject for each of a series of visual stimuli (i.e., multiple stimuli displayed one at a time in sequence). The visual stimuli can be displayed at multiple spatial locations. The eye tracking data indicates one or more characteristics of the subject's gaze or position when each stimulus is displayed. Depending on the task to be performed, some or all of the multiple spatial locations may be the same spatial location. It should be noted that the method does not require the actual eye tracking step, for example, the eye tracking data may be data previously generated by a separate device, which is received by a computer. In a subsequent step S9, the method includes receiving one or more subject indications, the subject indications including at least one of the following: the age of the subject; one or more pathological or physiological conditions of the subject; and data representing the calibration quality and / or attention level of one or more previous eye tracking-based tests or tasks performed by the subject. The method then (step S10) determines, based at least on the eye tracking data and the subject indication for each visual stimulus, whether the eye tracking data indicates a reliability lower than a predetermined minimum quality associated with the subject indication. Step S10 includes one or both of the following: quantifying the calibration quality of the eye tracking data, and determining, based at least on the eye tracking data, the position of each visual stimulus and the subject indication, whether the calibration quality is lower than a predetermined minimum calibration quality associated with the subject indication; and / or quantifying the subject's attention level based on the eye tracking data, and determining, based at least on the eye tracking data and the subject indication, whether the subject's attention level is lower than a predetermined minimum attention level associated with the subject indication. As previously described, both the calibration quality and the attention level affect the reliability of the eye tracking data in determining the subject's test or task performance. Optionally, if the characteristics of the eye tracking data are to be calculated relative to the position of the stimulus, step S10 may also be based on the position of each visual stimulus. Advantageously, the method takes into account the subject indication to determine whether a higher reliability of the tracking data is possible by improving the calibration quality or by improving the subject's attention level. For example, it has been observed that the average calibration quality and attention level of canonical eye tracking data varies for subjects of different ages and / or pathological or physiological conditions. In particular, for example, younger subjects tend to have lower attention levels, and the maximum expected attention level of eye tracking data for younger subjects is generally lower than that for older subjects. In other examples, subjects may have pathologies that generally reduce calibration quality.For each subject, in order to determine whether an improvement in the calibration quality or attention level of a test or task is possible, the data can be compared with the reference data (by a rule-based algorithm or a machine learning algorithm) in view of the subject's subject instructions provided. Thus, the method is able to distinguish between subjects to indicate whether the calibration quality and / or attention level is lower than expected. For example, for the same eye tracking data of a given calibration quality and / or attention level, for some ages and / or pathological indications, the calibration quality and / or attention level can be determined to be acceptable, while for other ages and / or pathological conditions, the same eye tracking data may be considered to have low calibration quality and / or low attention level. This is because the subject's ability to maintain its gaze and / or attention to visual stimuli depends on these factors. For uncommunicative or uncooperative subjects, it is particularly important to automatically determine whether the data reliability is high enough, because the practitioner cannot effectively convey instructions to the subject, and the subject may not be able to correctly follow the instructions of the visual test. Thus, verifying the data is an important tool for practitioners to determine whether the best possible eye tracking data for each subject has been correctly obtained, and if not, further tests, tasks or calibrations can be performed to improve data reliability.

[0065] It should be noted that steps S8 to S10 may be performed continuously in real time as new eye tracking data is generated. For example, the eye tracking data may be analyzed for quality within a moving time window.

[0066] The age indication may be provided in years, months and / or days, and when the subject's age is provided as an indication, the reference data to which the eye tracking data is compared may be reference data based on subjects in the same age group as the subject. For example, the method may include determining which age group the subject belongs to in a predetermined set of age groups (non-limiting examples are 0-6 months, 6-12 months, 12-18 months, 18-24 months, 2-3 years, 3-4 years, 4-5 years, 5-6 years, and over 6 years), and comparing the subject's eye tracking data with reference data for subjects in the same age group, or using a reference curve. The pathological or physiological condition may include any of those pathological or physiological states listed herein, or any other relevant pathological or physiological condition, and the data may be compared with reference data for subjects having the same pathological or physiological condition. When both age and pathological / physiological condition are provided, the eye tracking data may be compared with reference data for subjects in the same age group and having the same pathological / physiological condition. Alternatively or additionally, when data representing the quality of data of one or more previous eye-tracking-based tests or tasks performed by the subject is provided, this may be any previous eye-tracking data generated prior to the received eye-tracking data, said previous eye-tracking data representing the calibration quality and / or attention level of one or more previous eye-tracking-based tests and tasks, and in some examples may include one or more statistical parameters characterizing said previous eye-tracking data, or quantitative quality parameters derived from said previous eye-tracking data (e.g., secondary parameters as disclosed herein), representing the calibration quality and / or attention level of one or more previous eye-tracking-based tests or tasks. When age and / or pathological and / or physiological condition is provided, advantageously, unlike embodiments in which only the subject's previous test or task performance is used (which requires that a previous eye-tracking test or task has been performed on the same subject), the method can determine whether the eye-tracking data is of acceptable quality for the subject even if no previous eye-tracking-based data has been obtained for the subject.

[0067] It should be noted that the determination step S10 can be performed by a rule-based algorithm or by a machine learning algorithm. In an embodiment using a rule-based algorithm, the step may include a first step of receiving or calculating one or more statistical parameters describing a plurality of data points of eye tracking data for one or both eyes, or one or more secondary parameters derived from the one or more statistical parameters. The statistical parameters may, for example, include one or more of the following: the mean, median, mode, standard deviation, or variance of any quantitative eye tracking data for each stimulus when generating quantitative eye tracking data; a parameter describing the best fit of quantitative data (e.g., describing the normal distribution of data points); a bivariate contour ellipse area (BCEA) known in the art, and in the case of qualitative data, the proportion of data points within one or more qualitative data categories (e.g., the proportion of data points where the subject is looking at the stimulus or the screen). The data may also include one or more data sets having associated categories that characterize the gaze type of the data set (e.g., eye saccades, fixations, smooth pursuit movements), such that a sequence of eye tracking data over time is characterized by the type of gaze.

[0068] The one or more secondary parameters may be one or more parameters generated according to a predetermined equation that is a function of one or more statistical parameters (e.g., a single secondary parameter that is a function of all statistical parameters such that the statistical parameter is characterized by a single numerical value). In a second step, for one or more statistical parameters or secondary parameters, a comparison is made with one or more predetermined thresholds associated with the subject indication, the one or more thresholds indicating acceptable calibration quality and / or attention levels for a given subject indication. For example, each of the one or more statistical parameters and / or secondary parameters is compared with a threshold for the parameter, the threshold defining a boundary between acceptable or unacceptable calibration quality and / or attention levels for the subject indication. For example, for any given age and / or pathological and / or physiological condition, reference data may be obtained for subjects in the same age group and / or having the same pathological and / or physiological condition, and an average statistical / secondary parameter may be determined based on the reference data. Based on the reference statistical / secondary parameter, a predetermined threshold value may then be defined (e.g., one standard deviation from an average reference value, or a percentage difference from an average reference value), above which the parameter is considered to have a low calibration quality and / or a low attention level for the age and / or pathological or physiological condition. It should be noted that, depending on the statistical / secondary parameter, higher values ​​may indicate a higher or lower calibration quality or attention level, and that this will be apparent to a person skilled in the art for a given statistical parameter or secondary parameter. Thus, values ​​above or below the threshold value may indicate acceptable or unacceptable data reliability, and vice versa. If the subject indication includes data representing the calibration quality or attention level of one or more previous eye tracking-based tests or tasks performed by the subject, the same statistical parameter or secondary parameter may be calculated for the previous data and the calibration quality and / or attention level threshold value set accordingly (again, for example, one standard deviation from the statistical parameter, or a percentage difference, such that a substantial deviation from a previous level of calibration quality or attention level indicates that more reliable data may be obtained). In some embodiments using data representing the quality of data from multiple prior eye-tracking-based tests performed by a subject over time, a calibration quality or attention level threshold can be set by analyzing trends in the data over time. For example, if the data indicates that the reliability of the data has improved over time (e.g., because the subject is receiving treatment or therapy for a pathology), a higher calibration quality and / or attention level threshold can be set for the eye-tracking data in light of that trend. Conversely, if the reliability of prior data has decreased over time (e.g., due to a degenerative disease), a lower calibration quality and / or attention level threshold can be set for the eye-tracking data in light of that trend.In a third step, the method includes determining that the eye tracking data has a reliability lower than a predetermined minimum reliability (calibration quality and / or attention level) if a predetermined number of one or more statistical parameters or secondary parameters do not satisfy one or more associated predetermined thresholds, otherwise determining that the eye tracking data has an acceptable reliability. The predetermined number may be at least one of the calculated statistical parameters and / or secondary parameters (i.e., one or more calculated parameters do not satisfy the predetermined threshold), or the predetermined number may be two or more of the parameters, or three or more, and so on, or all of the calculated parameters (i.e., all calculated parameters do not satisfy the predetermined threshold). In some embodiments, the method includes comparing the data to at least one calibration quality threshold and at least one attention level threshold.

[0069] In an alternative embodiment, step S10 is performed by an artificial intelligence (AI) algorithm such as a machine learning algorithm. In such an embodiment, step S10 includes inputting eye tracking data or statistical parameters or secondary parameters of the eye tracking data, the location of the optional visual stimulus, and the subject indication into the AI ​​algorithm, wherein the AI ​​algorithm is trained to determine whether the eye tracking data has acceptable reliability or reliability (calibration quality and / or attention level) lower than a predetermined minimum quality associated with the subject indication compared to reference subject data with the same subject indication. The AI ​​algorithm can be any suitable algorithm known in the art, and can include any supervised, unsupervised, semi-supervised, and reinforced learning algorithms. When the subject indication includes the age of the subject and / or one or more pathological / physiological conditions, the training data for the AI ​​algorithm includes multiple data sets for subjects of each age group or pathological / physiological condition (or each combination thereof, as described above). Each data set may include a plurality of data points indicating the position of gaze relative to the stimulus, and / or one or more statistical parameters describing the plurality of data points, and / or one or more secondary parameters derived from the one or more statistical parameters, and / or qualitative data of the plurality of data points as described above, and / or statistical parameters describing the qualitative data as described above. For each data set, the training data also includes an indication of the reliability (calibration quality and / or attention level) of the data set for the subject's age and / or pathological and / or physiological condition. The indication may, for example, be a binary value, a quantitative number in continuous or discrete form, one of a plurality of labels (e.g., "low", "medium", "high"), or any other suitable indication. A machine learning algorithm may then be trained to determine, for new input data, whether the reliability of the eye tracking data is above or below the minimum quality for a given age and / or pathological and / or physiological condition, given the subject indication (e.g., age, pathological / physiological condition) or a combination thereof and the eye tracking data and / or the statistical parameters and / or secondary parameters of the eye tracking data.

[0070] When the subject indication includes data indicating the data reliability (calibration quality and / or attention level) of one or more previous eye-tracking-based tests or tasks performed by the subject, for multiple subjects, the AI ​​can be trained using training data, the training data including: eye-tracking data or statistical parameters or secondary parameters of the eye-tracking data, data indicating the data reliability of one or more previous eye-tracking-based tests or tasks performed by the subject; and an indication of whether the eye-tracking data or statistical parameters or secondary parameters of the eye-tracking data have acceptable reliability compared to the data reliability of one or more previous eye-tracking-based tests or tasks performed by the subject. The indication can be, for example, a binary value, a quantitative number in continuous or discrete form, one of a plurality of labels (e.g., "low", "medium", "high"), or any other suitable indication.

[0071] In some embodiments, regardless of the result of step S10, the method terminates after performing the determination step S10. In some embodiments, a notification about the result of step S10 is conveyed to the user (e.g., an audio or visual notification that the calibration quality and / or attention level is above or below a predetermined minimum calibration quality and / or attention level). Optionally, in some embodiments, the method may further include, if it is determined that the eye tracking data has a calibration quality and / or attention level higher than a predetermined minimum calibration quality and / or attention level (S11), then terminating, and if it is determined that the eye tracking data has a calibration quality or / attention level lower than a predetermined minimum calibration quality and / or attention level (S11), then obtaining new eye tracking data by displaying (S12) a second batch of multiple visual stimuli to the subject, the visual stimuli being displayed separately in sequence, and the visual stimuli having multiple spatial locations. Depending on the task to be performed, some or all of the multiple spatial locations may be the same spatial location. New eye tracking data may be obtained so as to recalibrate the eye tracking device when it is determined that the calibration quality is unacceptable (or to obtain data of higher calibration quality without recalibrating the eye tracking device, for example by repositioning the subject). When it is determined that the attention level is unacceptable, new eye tracking data can be obtained to obtain improved eye tracking data for a given test or task. The method may also include tracking (S13) the gaze of one or both eyes of the subject to each visual stimulus by an eye tracking device. The eye tracking device can be any eye tracking device that is capable of tracking the position of one or both eyes of the subject and is placed at a known position relative to the visual stimulus (e.g., any commercially available eye tracker, or any computing device including an imaging device such as a camera and suitable software for tracking eye positions based on captured images), or any other eye tracking data capable of generating one or more characteristics of the subject's gaze or position. The method may also include generating (S14) eye tracking data indicating one or more characteristics of the subject's gaze or position based on the gaze tracked by the eye tracking device. In some embodiments, step S10 may be performed on the new eye tracking data. Steps S11 to S14 may be repeated multiple times (e.g., twice or three times, or until acceptable data reliability is obtained). It should be noted that steps S11-S14 may be performed with or without the prior steps S1-S7, as described in further detail below. In embodiments where steps S11-S14 are performed without the prior steps S5-S7, the received data in step S8 may include an indication of the visual characteristics of the stimulus displayed to the subject, so that similar visual stimuli may be displayed in steps S12-S14.

[0072] In some embodiments of performing steps S12 to S14, the second plurality of visual stimuli have different static or dynamic visual characteristics for generating eye tracking data that has been determined in step S10 to have a calibration quality and / or attention level below a predetermined minimum calibration quality and / or attention level. Alternatively or additionally, if the plurality of visual stimuli have different static or dynamic visual characteristics between them, the plurality of stimuli may have a smaller overall range of different static or dynamic visual characteristics than the visual stimuli used to generate eye tracking data of unacceptable calibration quality and / or attention level from step S10. Alternatively or additionally, the second plurality of visual stimuli are displayed over a smaller field of view than the visual stimuli used to generate eye tracking data of unacceptable calibration quality and / or attention level in step S10. Alternatively or additionally, one or more audio stimuli are provided to the subject during the sequence of the second plurality of visual stimuli. Advantageously, this allows the visual stimuli to be adapted to the subject so as to more effectively capture the subject's attention. For example, it may be that the subject did not see the visual stimulus because the stimulus was too small for the subject to notice, or because the subject had a pathological condition that caused the stimulus to be less noticeable (e.g., the subject was color blind). In some cases where the subject is unable to understand that a test or task is being performed (e.g., if the subject is very young), the subject may only occasionally notice the visual stimulus, meaning that the attention level of the data is low. Thus, changing the visual characteristics or providing other stimuli in conjunction with the visual stimulus can attract the subject's attention to obtain the desired eye tracking data at an acceptable attention level. In some embodiments where steps S12 to S14 are repeated several times as described above, the visual characteristics may change each time steps S12 to S14 are repeated. Particularly in the case of uncommunicative or uncooperative subjects, repetition of tests or tasks with different visual characteristics increases the likelihood of obtaining eye tracking data of sufficient quality. Specific examples of changes in the second batch of multiple stimuli include:

[0073] - Display stimuli of larger size;

[0074] - Show stimuli moving at a slower speed;

[0075] - presenting the second batch of multiple stimuli over a smaller visual field, i.e. making the stimuli appear less far away from each other than before (increasing the probability that the subject will be able to notice the second stimulus when his gaze is at the location of the previous one);

[0076] - Play prompt audio or music to attract the subject's attention;

[0077] - Display stimuli with additional dynamic visual properties such as vibration, pulsation or other movement;

[0078] - Displaying a stimulus in which the stimulus changes color more frequently than in previous stimuli.

[0079] - Increase the brightness and / or contrast of the stimulus.

[0080] - reducing the range of visual properties between stimuli, such as limiting all stimuli to a range of visual properties for which good quality data is obtained for the subject, or reducing the range of visual properties to a range that is generally easier for the subject to see.

[0081] - Reduce the duration of a test or task.

[0082] It should be noted that in any embodiments disclosed herein, eye tracking data may be obtained during calibration of an eye tracking device. In such embodiments, a plurality of visual stimuli may be displayed in order to calibrate the position of the subject's eyes using measurements taken by the eye tracking device, and a determining step determines, given an indication by the subject, whether the calibration quality of the calibration is below a predetermined minimum calibration quality for any eye tracking-based task or test to be subsequently performed. In some embodiments, eye tracking data may be obtained during an eye tracking-based test or task, a cognitive test or task, or a vision therapy performed on the subject. In such embodiments, the determining step determines, based on the subject's indication, whether the calibration quality and / or attention level of the eye tracking data is a calibration quality and / or attention level that is below a predetermined minimum calibration quality and / or attention level for the visual test or task, the cognitive test or task, or the vision therapy.

[0083] In some embodiments, the method may include a step of actually generating eye tracking data. In particular, the method may include a step of displaying (S5) a plurality of visual stimuli to the subject, the visual stimuli being displayed separately in sequence, the visual stimuli optionally having a plurality of spatial positions. As previously described, the plurality of visual stimuli may be physical objects that are sequentially placed by a practitioner at predetermined positions relative to the subject, or may be displayed on an electronic display device such as described herein. In addition to the visual stimuli, one or more audio stimuli may also be provided (e.g., when each stimulus is displayed, or in accordance with the subject's gaze, as described in further detail below). The method may also include tracking (S6) the gaze of one or both eyes of the subject to each visual stimulus by an eye tracking device. The eye tracking device may be any suitable device for eye tracking as disclosed herein. The method may also include generating (S7) eye tracking data based on the tracked gaze of the eye tracking device, the eye tracking data characterizing the position of the gaze of one or both eyes relative to the displayed stimulus during the display of each visual stimulus.

[0084] In some embodiments, the eye tracking data generated may exclude data obtained within a predetermined time after each stimulus initially appears. For example, eye tracking data obtained within 1 second after each stimulus initially appears may be excluded. Alternatively or additionally, in some embodiments, the eye tracking data generated may exclude eye tracking information obtained within a predetermined time before each stimulus disappears. For example, eye tracking data obtained within 1 second before the stimulus disappears may be excluded. This reduces the possibility that eye tracking data where the subject has not yet noticed or tracked the new visual stimulus or has lost attention will artificially reduce the reliability of the eye tracking data.

[0085] In some embodiments, once the system is calibrated, the device can be configured to detect whether the subject is viewing the visual stimulus, or whether the subject has seen each visual stimulus at any time. This can be calculated based on a preset algorithm known in the art for eye tracking data. In some embodiments, if the eye tracking device determines that the subject has not viewed the visual stimulus for a predetermined time, it can be determined that the subject is not viewing or has not seen the visual stimulus.

[0086] Instead of determining whether the subject is looking at the visual stimulus or in addition to determining whether the subject is looking at the visual stimulus, some eye tracking devices can provide a real-time indication of whether valid gaze data has been obtained. The real-time indication can be monitored continuously. In some embodiments, if there is no valid eye tracking data obtained by the eye tracking device for a predetermined period of time, it can be determined that the subject is not paying attention. Alternatively, the eye tracking data can include any other data indicating whether the patient is paying attention as disclosed herein, which can be continuously monitored in real time to determine whether the subject is paying attention.

[0087] In an embodiment where the system includes an algorithm for detecting whether the subject is watching or can see the visual stimulus, when it is determined that the subject is looking at the visual stimulus, one or more display characteristics (static and / or dynamic) of the visual stimulus can be changed in order to maintain the subject's attention. One or more display characteristics can be changed in real time (i.e., when the subject is watching the stimulus), or can be changed after the stimulus disappears or after the entire task is performed (i.e., if the task is successfully performed, at the end of the task, or at the end of the displayed stimulus, positive feedback is provided to the subject). Alternatively or in addition, when it is determined that the subject is watching or can see the visual stimulus, audio stimulation can be provided in order to maintain the subject's attention. Alternatively or in addition, when it is determined that the subject is moving his eyes away from the visual stimulus, one or more display characteristics (static and / or dynamic) of the visual stimulus can be changed in order to maintain the subject's attention. Alternatively or in addition, when it is determined that the subject is moving his eyes away from the visual stimulus, audio stimulation can be provided in order to maintain the subject's attention. Specific examples include:

[0088] - displaying a static or moving (e.g., rotating) star or other encouraging symbol near the visual stimulus and / or emitting a sound when the subject is determined to be looking at the visual stimulus, or after looking at the visual stimulus; and / or

[0089] - increasing the brightness of the visual stimulus while the subject is looking at the stimulus; and / or

[0090] - a sound is emitted when the subject starts looking at the stimulus and / or when the subject starts looking away from the stimulus; and / or

[0091] - gradually increasing the size of the visual stimulus as the subject looks away from it, and optionally returning the visual stimulus to its original size while the subject is looking at it (in order to maintain the difficulty of the test or task); and / or

[0092] - Where the visual stimulus is configured to move during a test or task, maintaining the visual stimulus in a static position until it is determined that the subject is looking at the stimulus, and then initiating movement of the visual stimulus once the subject is looking at the stimulus.

[0093] In some embodiments where the calibration quality and / or attention level is continuously monitored over the moving time window, the task or test is paused when it is determined that the eye tracking data indicates a calibration quality and / or attention level that is lower than a predetermined minimum calibration quality and / or attention level. Optionally, the task or test is resumed when it is determined that the eye tracking data has returned to an acceptable calibration quality and / or attention level for the subject. When the process of generating eye tracking data is resumed when the calibration quality and / or attention level returns to an acceptable level, the process can be resumed by, for example:

[0094] - displaying the same stimulus that was being displayed when it was determined that the subject's gaze was away from the visual stimulus; or

[0095] - discarding data for the stimulus that was being displayed when it was determined that the subject's gaze was away from the visual stimulus, and displaying an additional visual stimulus at the end of the visual stimulus sequence;

[0096] - resetting the process to a predetermined amount of time or a predetermined number of visual stimuli before the process was paused; or

[0097] - resuming the stimulation sequence from the point at which the process was paused, and optionally discarding eye tracking data acquired while the process was paused, and optionally also discarding eye tracking data acquired within a predetermined amount of time before the process was paused and after the process was resumed.

[0098] In some embodiments, before steps S5 to S7, the method may include tracking (S1) the position of the subject via an eye tracking device or a device for tracking the subject. The method may also include indicating (S2) the position of the subject on the display device. The position of the subject on the display device may be indicated by displaying an image of the subject representing the position of the subject in front of the display device, or a drawing or virtual representation of the subject (e.g., a rendered animated face). The position of the subject may be calculated by an eye tracking device, or by a computing device connected to the eye tracking device using eye tracking data generated by the eye tracking device, and the known position and orientation of the display device and the eye tracking device relative to each other, thereby calculating the position of the subject in front of the display device (i.e., if the display device is a mirror, the reflected position of the subject). Such calculations are known in the art. The method may also include overlaying (S3) a subject positioning guide on the display device to indicate the correct position of the subject in order to obtain eye tracking data. The positioning guide may be positioned on the display device so that once the image of the subject or the representation of the subject is aligned with the guide, the subject is in the correct position. The guide may be of any suitable form or shape, such as a border in the shape of a facial outline or two circles configured to align with the eyes of the subject or representation. The guide may be positioned until (S4) it is determined that the subject is correctly positioned, at which point the method may proceed to step S5.

[0099] The systems and methods disclosed herein are well suited for indicating, for a given set of subjects, the highest expected calibration quality and / or attention level for obtaining eye tracking data. It should be noted that in some cases, the subject may be held in place by a guardian or caregiver such that the eye tracker may detect multiple pairs of eyes within its field of view. For example, the subject may be an infant held in an adult's lap. In such cases, the method may include indicating that multiple eyes are detected. Alternatively or in addition, the method may include generating eye tracking data for the bottom pair of eyes that are the subject's eyes. This reduces the amount of redundant data generated and reduces errors caused by tracking an incorrect pair of eyes.

[0100] Figure 2A An apparatus for performing the method of the present invention according to one or more embodiments is shown. The apparatus includes an eye tracking device 120. The eye tracking device may be any suitable device as disclosed herein. The eye tracking device includes an imaging device, such as a camera 122, a memory 124, a processor 126, and a communication interface 128. The imaging device 122 may be any suitable imaging device for imaging the subject's eyes so that the subject's eyes can be tracked. The memory 124 may include instructions for detecting and calculating the position of the subject's eyes relative to the displayed visual stimulus or relative to any other coordinate system using the image data generated by the imaging device 122. The memory may include an ordered list of predetermined positions for the visual stimulus relative to the eye tracking device (e.g., positions on a display device or positions of an object to be displayed by a practitioner), or the device may manually receive indications of these positions via a user interface (not shown). The processor 126 may be configured to calculate the position of the subject's eyes as a function of time based on the image data collected by the imaging device and the position of the visual stimulus. The eye tracking device 120 may also include a communication interface 128 for transmitting the imaging data generated by the imaging device 122 or any data calculated by the processor 126. The eye tracking device 120 may be configured to perform steps S1 , S6 , S7 , S13 , and S14 .

[0101] It should be noted that although the memory 124 and processor 126 are provided as part of the eye tracking device, these components may be provided remotely from each other and remotely from the imaging device 122, with data being transferred between the components via any suitable wired or wireless connection (e.g., USB, Internet, or Bluetooth).

[0102] The apparatus may also include an electronic display device 100 for displaying a plurality of visual stimuli. The display device 100 may include any suitable electronic display 102 known in the art. The display device 100 may include a memory 104 including instructions for displaying a plurality of visual stimuli on the display 102, and controlling the visual characteristics of the visual stimuli and providing any other stimuli (e.g., in an embodiment where the display device 100 includes a speaker or is connected to a speaker and provides auditory stimuli together with the visual stimuli). The display device 100 may also include a processor 106 for processing or calculating any data based on the image data generated by the imaging device 122, such as the position of the subject's eyes, or for rendering images or representations of the guide and the subject in steps S2 to S4, and determining when the subject is aligned with the guide. The display device 100 may also include a user interface 108 (e.g., a GUI) for controlling the device, such as selectively initializing a calibration or a test or task based on eye tracking. The user interface 108 may also be used to select how to change the test, task, or calibration, such as the visual characteristics or other stimuli provided in step S12. The user interface 108 may also be used to input subject indications, or the memory 104 may include subject indications. The display device 102 may include a communication interface 110 for communicating with an external device, such as transmitting to the external device the results of a calibration or eye-tracking-based test or task, or indicating that the calibration quality and / or attention level is below a predetermined calibration quality and / or attention level, or receiving imaging or other data from the eye-tracking device 120 or any other data from the eye-tracking device 120, such as data points about the subject's gaze. It should be noted that in some embodiments, the display device 100 may be integrated with an eye-tracking device, such as Figure 2B In such cases, the memory 124 and the processor 126 may be the memory 104 and the processor 106, and the communication interface 128 may be the interface 110. The display device may be used to display any visual stimulus described in the methods disclosed herein, or to calculate whether the calibration quality and / or the attention level is above a predetermined minimum calibration quality and / or attention level.

[0103] It should be noted that in an embodiment of setting display device 100, the functions of memory 124 and processor 126 may be performed by memory 104 and processor 106 instead, and eye tracking device 120 may not include memory 124 or processor 126.

[0104] The memory disclosed herein may include any suitable storage medium, such as optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / warehousing, and / or any other type of device suitable for storing instructions and / or data.

[0105] The processors disclosed herein may include any suitable processor, including a CPU, a GPU, a microprocessor, a distributed processing device, or any other suitable processing device.

[0106] It should be noted that the above-described embodiments of the eye tracking device 120 and the display device 100 are merely exemplary, and methods of tracking a user's eyes using an eye tracking device are known in the art. Instead of the specific embodiments disclosed herein, any such eye tracking device and / or display device may be used. For example, any and all components of the display device 100 and the eye tracking device 120 may be provided in a single device.

[0107] Figure 3A to Figure 3C An exemplary sequence of visual stimuli that may be displayed on display 102 to obtain eye tracking data is shown. A first visual stimulus 200a may be displayed at a first location on display 102. Subsequently, a second visual stimulus 200b may be displayed at a second location on display 102. Subsequently, a third visual stimulus 200c may be displayed at a third location on display 102. It should be understood that the visual stimuli 200 may be the same or different in shape, color, size, duration for which they are displayed, etc. Figure 3A to Figure 3C The examples shown in are illustrative only, and it will be appreciated that the sequence of visual stimuli may be any suitable sequence suitable for obtaining eye tracking data for a subject. The sequence of visual stimuli may be used in an eye tracking based test or task, such as testing some aspect of a subject's visual function, or testing a subject's cognitive abilities, or providing vision therapy to a subject. Since the position of each stimulus is known, and the positioning of the eye tracking device relative to display 102 is also known, once eye tracking device 120 is calibrated, the subject's gaze relative to each stimulus when each stimulus is displayed may be readily calculated, as is known in the art.

[0108] In some embodiments, a computer program comprising computer readable instructions is provided, which, when executed by a processor, causes the processor to perform any method disclosed herein. The computer program may be included in the memory of any device disclosed herein, or may be included in an external memory communicatively connected to any device disclosed herein. The computer program may be executed in hardware and / or software.

[0109] It should be noted that in any embodiment disclosed herein, the sequence of visual stimuli displayed to the subject can be adjusted based on the subject's instructions received (e.g., in step S5 or S10). For example, the size or any other characteristics of the visual stimuli can be adjusted based on the subject's age or any pathological or physiological condition of the subject. Similarly, any other aspects of the displayed visual stimuli (sequence, duration of each stimulus displayed, sounds associated with each visual stimulus, any static or dynamic visual characteristics) can be adjusted based on the subject's instructions received. The different characteristics of each combination of visual stimuli and any associated sounds for patient instructions can be stored as computer-readable instructions in the memory of any device disclosed herein.

[0110] Examples of Uses of Eye Tracking Data

[0111] Different types of eye tracking data are now discussed with respect to determining whether the eye tracking data indicates a quality lower than a predetermined minimum quality in accordance with the present invention.

[0112] In some embodiments, the eye tracking data may include a plurality of data points representing the patient's gaze position or direction relative to each stimulus or relative to an absolute coordinate system. In such embodiments, data points corresponding to the range of the displayed stimulus or on the display may be associated with a good level of attention because it at least indicates that the subject is paying attention to the display when the stimulus is displayed. The level of attention can be determined by, for example, calculating the number or proportion of data points in which the subject's gaze falls within an area that is considered to correspond to the subject's attention (e.g., the gaze falling on the display). In other examples, it can be assumed that the subject is paying attention when the data is available because the eye tracking data may be recorded when the subject looks at the display and not recorded when the subject looks away. The data can be compared with the expected data indicated for the subject (e.g., the maximum expected attention level may vary depending on the subject's instructions, so it can be determined whether a better level of attention can be obtained for the subject). In some embodiments, the position of the gaze can be compared with the position of the stimulus to determine whether the calibration of the eye tracker is correct. For example, the position of the gaze may be distributed within a small angle range, meaning that the subject is focused on a certain point on the display (so the stimulus has been observed), but the position is usually far away from the position of the stimulus. There may also be consistent differences between the position of gaze and the position of each stimulus. This may indicate that the calibration of the data has a low calibration quality. The calibration quality can be quantified based on, for example, a normalized function that depends on the difference between the position of gaze and the position of the stimulus, or any other suitable quantification method of calibration quality known in the art. The calibration quality may depend on the patient's pathological condition. For example, for a strabismic patient, the gaze position of one eye may be far away from the stimulus, while for a non-strabismic patient, this may indicate that the data has a low calibration quality. Therefore, for any given data, the assessment of whether the data reliability is high or low may depend on the subject's indication.

[0113] In some embodiments, the eye tracking data may include a plurality of data points representing the position of the subject's pupil relative to the eye tracking device, relative to an imaginary eye tracking box projected toward the subject's face, or relative to any other coordinate system, such as the position of the subject's eyes when the eye tracking device is calibrated. The data indicates good calibration quality when the pupil is located within a predetermined region of the coordinate system associated with the working range of the eye tracking device. The predetermined region may be defined by a region surrounding the position of the subject's eyes when the eye tracking device is calibrated. If the position of the eyes deviates significantly from this position, the accuracy of the calibration may be reduced. The predetermined region may be selected based on a minimum acceptable calibration accuracy. When the eyes are within the predetermined region, the eye tracking device is able to obtain accurate eye tracking data based on the calibration, while outside the predetermined region, the eye tracking device cannot accurately track the eyes and the calibration may be invalid, thereby reducing the accuracy of the data. The calibration quality of the data may be quantified, for example, by obtaining the number or proportion of data points within the predetermined region, or any other function of the data points, such as a parameter calculation of the average position of the pupil. In addition, when the pupil is within the predetermined region, this may indicate that the subject noticed the stimulus when looking at the display. The subject's attention level can then be determined from this data (and can be quantified, for example, by taking the number or proportion of data points where the pupil position is within a predetermined region). It should be noted that the predetermined region over which the calibration is considered valid may differ in size from the predetermined region over which the attention level is considered valid.

[0114] In some embodiments, the eye tracking data may include data indicating the size of the subject's pupil in absolute terms or relative to a reference size of the pupil (e.g., the size of the pupil when the eye tracking device was calibrated for the subject). For each data point, the data may include an indication of whether the pupil size is less than or greater than a predetermined threshold. The size of the subject's pupil is an indicator of how close or far the subject is from the display and / or the eye tracking device. When the size of the pupil is within a predetermined range (e.g., within a predetermined range relative to the size of the pupil when calibration was performed), the data indicates good calibration quality, such that it can be inferred that the subject's position did not substantially change between when the eye tracking device was calibrated and when the eye tracking data was obtained. The calibration quality may be quantified based on the data, for example by counting the number or proportion of data points for which the size of the pupil is within a predetermined range for which the calibration is valid. Similarly, the size of the pupil within a predetermined range may indicate that the subject noticed the stimulus when looking at the display at an appropriate distance from the display, and the attention level may be quantified in the same manner. The predetermined ranges for calibration quality and attention level may be the same or different.

[0115] In some embodiments, the eye tracking data may include image data of the subject's eyes. An image analysis algorithm may analyze the image data to determine multiple data points of one or more of the following: the position of the face relative to any coordinate system, the position of the pupil, the position of the gaze, or the direction of the gaze. As disclosed herein, the position or size of the pupil, the position of the gaze, and the direction of the gaze may be evaluated for calibration quality and / or attention level. Similarly, the position of the face may be compared to predetermined areas where calibration quality is considered acceptable and / or attention level is acceptable, and the calibration quality and / or attention level may be similarly quantified.

[0116] In some embodiments, the eye tracking data may include one or more quantitative indicators of how well the eye tracking data fits a calibration model for the eye tracking device. This data is a direct indicator of how well the system is calibrated for the subject, and may also be an indicator of the subject's attention level during calibration. The best expected calibration quality for a given subject may vary from subject to subject (e.g., the subject's attention may vary, or the quality of the calibration may vary from subject to subject), so a determination may be made whether a better quality calibration or attention level can be achieved based on previous data for subjects with the same or similar subject indications.

[0117] In some embodiments, the eye tracking data may include data indicating one or more of the following: the number or proportion of visual stimuli displayed during the calibration process for which the calibration algorithm of the eye tracker failed; the number or proportion of visual stimuli for which eye tracking needed to be repeated due to failure of the calibration algorithm of the eye tracking device for the visual stimuli; how long each visual stimulus needed to be displayed during the calibration process before the calibration algorithm of the eye tracking device successfully calibrated the calibration points. The calibration algorithm may fail for a variety of reasons, including a lack of attention by the subject, or a pathology of the subject causing the algorithm to fail more frequently. Thus, the number of calibration failures may depend on each patient, such that recalibration may or may not provide improved calibration quality and / or attention level. The number of repetitions and how long each stimulus is displayed during calibration also indicate the calibration quality (more repetitions and / or longer display times increase the likelihood of poor quality calibration).

[0118] In some embodiments, the eye tracking data may include data indicating a vector formed by the pupil and the reflection of light in the eye. The presence of this vector and its positioning within a predetermined acceptable range and its variation within a predetermined acceptable range are associated with the subject's attention and good eye tracking quality. In addition, the location of the gaze can be derived from this data, from which it can be determined whether the subject is paying attention and whether the calibration quality is good, as previously disclosed.

[0119] In some embodiments, the eye tracking data may include data indicating whether the patient is looking at the visual stimulus or within a predetermined distance of the visual stimulus for each data point. For a stimulus that the patient is expected to see, an indication that the patient is looking at the stimulus is a clear indication that the eye tracking calibration quality is good and the subject is paying attention to the stimulus. The data points may be quantified by determining the number or proportion of data points at which the subject is looking at the stimulus, from which it may be determined whether the calibration quality and / or attention level is as good as a predetermined acceptable calibration quality level and / or attention level indicated by a given subject.

[0120] In some embodiments, the eye tracking data may include data indicating whether the patient's gaze fell on the display for each data point. This is related to whether the subject is paying attention to the display, and thus may notice the stimuli (if these stimuli are visible to the subject). Thus, this data is a good indicator of whether the subject is paying attention when the stimuli are displayed. The data points can be quantified by determining the number or proportion of data points at which the subject is looking at the stimuli, from which it can be determined whether the attention level is as good as a predetermined acceptable attention level indicated by a given subject.

[0121] In some embodiments, the eye tracking data includes two or more or all of the above types of data.

[0122] It should be noted that in some embodiments where it can be assumed that the subject will have a sufficient level of attention throughout the procedure (e.g., a communicative and cooperative patient), the methods disclosed herein may include determining whether the calibration quality is above a predetermined minimum calibration quality, rather than the level of attention. In other embodiments, it may be assumed that the calibration quality is sufficiently high without any checks on the calibration quality, and the method may include determining whether the level of attention is above a predetermined level of attention, rather than the calibration quality.

[0123] It should be noted that the calibration quality check can include determining whether the calibration quality itself is low (i.e., causing all data to have low accuracy), as well as determining whether the position of the subject causes the eye tracking data to have low accuracy (i.e., causing the data to be improved by repositioning the subject without recalibrating the eye tracking device).

[0124] In some embodiments, the methods disclosed herein may include determining whether the calibration quality is above a predetermined minimum calibration quality and whether the attention level is above a predetermined attention level. It should be noted that the calibration quality may be checked for the same or different eye tracking data of the same or different visual stimulus sequences displayed to the subject. For example, a first subset of eye tracking data (e.g., eye tracking data obtained during a post-calibration sequence of visual stimuli) may be checked for calibration quality, and a second subset of eye tracking data (e.g., eye tracking data obtained during an eye tracking-based test or task) may be checked for attention level. By quantifying the data quality and / or attention level within a time window during the entire process of the eye tracking-based process for the subject, both the calibration quality and the attention level may be checked continuously. For example, both may be continuously monitored in a series of eye tracking-based tests or tasks to determine whether the calibration quality and / or attention level drops to an unacceptable level at any time, and if any of them drops to an unacceptable level, a notification is provided or the process is suspended. Advantageously, the method distinguishes between low reliability data caused by poor calibration or poor positioning of the subject and low reliability data caused by the subject's attention level. Since these causes are corrected in different ways, the overall reliability of the data can be improved by verifying that both are above acceptable levels, taking into account the subject's instructions or past performance on eye-tracking-based tasks or tests.

[0125] Although the present disclosure is primarily directed to determining data quality for eye-tracking based tests and tasks related to visual function or therapy, it will be appreciated that the methods and apparatus disclosed herein may also be applicable to other purposes and technical fields.

[0126] Now that some illustrative embodiments have been described, it is apparent that the foregoing is illustrative, not restrictive, and is presented by way of example. In particular, although many of the examples presented herein relate to specific combinations of devices or software elements, these elements may be combined in other ways to achieve the same goals. Actions, elements, and features discussed only in conjunction with one embodiment are not intended to be excluded from similar effects in other one or more embodiments.

[0127] The devices described herein may be embodied in other specific forms without departing from the characteristics thereof. The above embodiments are illustrative rather than limiting of the described systems and methods. The scope of the devices described herein is therefore indicated by the appended claims rather than the preceding description, and changes within the meaning and range of equivalents of the claims are also embraced therein.

Claims

1. A computer-implemented method for validating eye tracking data for a system for administering an eye tracking-based test or task to a subject, comprising: receiving eye tracking data for one or both eyes of a subject for each of a series of visual stimuli, the eye tracking data indicating one or more characteristics of the subject's gaze or position when each stimulus is displayed; receiving one or more subject indications including at least one of: an age of the subject; one or more pathological or physiological conditions of the subject; and data indicating calibration quality and / or attention level of one or more previous eye-tracking-based tests or tasks performed by the subject; as well as One or more of the following steps: a) quantifying the calibration quality of the eye tracking data and determining, based at least on the eye tracking data, the position of each visual stimulus and the subject's instructions, whether the calibration quality is below a predetermined expected minimum calibration quality associated with the subject's instructions; and / or b) quantifying the subject's attention level based on the eye tracking data, and determining, based at least on the eye tracking data and the subject's indication, whether the subject's attention level is below a predetermined expected minimum attention level associated with the subject's indication.

2. The method according to claim 1, wherein the step of determining whether the calibration quality is below a minimum calibration quality and / or the step of determining whether the attention level is below a predetermined expected minimum attention level comprises the following steps: - receiving or calculating one or more statistical parameters describing a plurality of data points of eye tracking data over time when each stimulus is displayed for one or both eyes, and / or one or more secondary parameters derived from the one or more statistical parameters; - performing a comparison of the one or more statistical parameters or secondary parameters with one or more predetermined threshold values ​​associated with the subject indication, the one or more threshold values ​​being indicative of an acceptable calibration quality and / or attention level for the subject indication; as well as - If a predetermined number of one or more statistical parameters or secondary parameters do not meet associated predetermined thresholds, determining that the calibration quality is below a predetermined expected minimum quality and / or the attention level is below a predetermined expected minimum attention level, and otherwise determining that the eye tracking data is acceptable.

3. The method according to claim 1, wherein the step of determining whether the calibration quality is lower than a minimum calibration quality and / or the step of determining whether the attention level is lower than a predetermined expected minimum attention level comprises the step of inputting eye tracking data and / or one or more statistical parameters describing the eye tracking data and / or one or more secondary parameters derived from one or more of the statistical parameters, and subject instructions into an AI algorithm, wherein the AI ​​algorithm is trained to determine whether the calibration quality and / or the attention level is higher or lower than a predetermined expected minimum calibration quality and / or attention level associated with the subject instructions.

4. A method according to any preceding claim, wherein the method includes step a) and eye tracking data is obtained by an eye tracking device and if it is determined that the eye tracking data has a calibration quality lower than a predetermined expected minimum calibration quality, recalibration of the eye tracking device is recommended or performed and / or an indication is given that the subject should be repositioned to improve the calibration quality of the data.

5. A method according to any preceding claim, wherein the method comprises step b) and if it is determined that the eye tracking data indicates a lower level of attention than a predetermined expected minimum level of attention and / or if it is determined that the calibration quality is below a predetermined expected minimum calibration quality, obtaining new eye tracking data by: displaying a second batch of multiple visual stimuli to the patient, wherein the visual stimuli are displayed individually in sequence, and the visual stimuli have multiple spatial locations; Tracking the gaze of one or both of the patient's eyes to each visual stimulus using an eye-tracking device; and Eye tracking data is generated based on the tracked gaze that indicates one or more characteristics of the subject's gaze or position during the presentation of each visual stimulus.

6. A method according to claim 5, wherein the second plurality of visual stimuli have different static or dynamic visual characteristics or a smaller range of different static or dynamic video characteristics than the visual stimuli used to generate eye tracking data associated with unacceptable attention level and / or calibration quality; and / or wherein the second plurality of visual stimuli are displayed over a smaller field of view than the visual stimuli used to generate eye tracking data associated with unacceptable attention level and / or calibration quality; and / or wherein one or more audio stimuli are provided to the subject during the sequence of the second plurality of visual stimuli; and / or the visual stimuli are displayed for a different time period than the visual stimuli used to generate eye tracking data associated with unacceptable attention level and / or calibration quality.

7. A method according to any preceding claim, wherein the eye tracking data is eye tracking data obtained during calibration of an eye tracking device, or wherein the eye tracking data is eye tracking data obtained during a test or task based on eye tracking.

8. A method according to any preceding claim, wherein the method comprises generating eye tracking data by the following preceding steps: presenting a plurality of visual stimuli to a subject, the visual stimuli being presented individually in sequence; Tracking the gaze of one or both eyes of the subject to each visual stimulus using an eye-tracking device; and Eye tracking data is generated based on the tracked gaze, the eye tracking data indicating one or more characteristics of the subject's gaze or position during presentation of each visual stimulus.

9. The method of claim 8, wherein the eye tracking data does not include eye tracking data obtained within a predetermined time after the initial appearance of one or more stimuli, and / or does not include eye tracking data obtained within a predetermined time before the disappearance of one or more stimuli.

10. A method according to claim 8 or 9, wherein the eye tracking data includes data characterizing the gaze position of one or both eyes, the method includes determining the position of the gaze relative to the stimulus, and wherein when it is determined that the subject is viewing the visual stimulus and / or when it is determined that the subject is not viewing the visual stimulus, one or more display characteristics of the visual stimulus are changed and / or audio stimulation is provided to the subject.

11. A method according to any one of claims 8 to 10, wherein the eye tracking data is obtained continuously and includes data indicating the subject's attention level and / or the calibration quality of the data, the method includes quantifying the subject's attention level and / or the calibration quality of the data, and wherein the visual stimulation sequence is paused when it is determined that the subject is not paying attention and / or the calibration quality is below a predetermined expected minimum calibration quality; optionally, wherein the visual stimulation sequence is resumed when it is determined from the continuously obtained data that the patient is paying attention and / or the calibration quality is above a predetermined expected minimum calibration quality.

12. The method of claim 11, wherein when it is determined that the subject is paying attention and / or the calibration quality is above a predetermined expected minimum calibration quality, the visual stimulus sequence is restored, and the sequence is restored in one of the following ways: - display the same stimulus that was being displayed when it was determined that the subject was not paying attention; or - discarding data for the stimulus being displayed when it was determined that the subject was not paying attention, and displaying an additional visual stimulus at the end of the visual stimulus sequence; - resetting the stimulation sequence to a predetermined amount of time or a predetermined number of visual stimuli before the stimulation sequence was paused; or - resuming the stimulation sequence from the point at which the process was paused and, optionally, discarding eye tracking data acquired while the process was paused and, optionally, also discarding eye tracking data acquired within a predetermined amount of time before the process was paused and after the process was resumed.

13. The method according to any one of claims 8 to 12, wherein before generating the eye tracking data, the method comprises the following steps: - tracking the subject's position via an eye tracking device or a device for tracking the subject; - indicating the position of the subject on a display device; as well as - Overlay a subject positioning guide on the display device to indicate the correct position of the subject in order to obtain eye tracking data.

14. A method according to any one of claims 8 to 13, wherein the eye tracking device is configured to detect whether there are multiple pairs of eyes within the field of view of the eye tracking device, and optionally, if the eye tracker detects multiple pairs of eyes, eye tracking data is only generated for the bottom pair of eyes that are the eyes of the subject.

15. An apparatus configured to perform the method according to any one of claims 1 to 14, comprising: - a display device configured to display visual stimuli to the subject; - an eye tracking device configured to track a subject's gaze while visual stimuli are displayed to the subject and to generate eye tracking data indicative of one or more characteristics of the subject's gaze or position; and - one or more processors configured to control the display device to display visual stimuli, receive eye tracking data from the eye tracking device, receive one or more subject indications, and: a) determine, based at least on the eye tracking data for each visual stimulus and the one or more subject indications of the subject, whether a calibration quality of the eye tracking data indicates a calibration quality that is lower than a predetermined expected minimum calibration quality; and / or b) determine, based at least on the eye tracking data and the subject indications, whether the subject's attention level is lower than a predetermined expected minimum attention level associated with the subject indications.

16. A computer program comprising computer readable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 14.

Images