Method for processing an image of a user's eye, associated system and computer program

Abstract

A computer-implemented method for selecting a sequence of images of a user's eye for an eye tracking application, wherein each image is captured while a stationary stimulus point is displayed to the user, the method comprising: for a plurality of different pairs of images: comparing the pairs of images to each other to determine an image score representing a degree of difference between the compared images; and calculating an image sequence score based on the image scores for the plurality of pairs of images.

Description

Technical Field

[0001] The present disclosure relates generally to the field of eye tracking. In particular, the present disclosure relates to systems and methods for processing and / or selecting an image sequence of a user's eye for use in an eye tracking system. Background Art

[0002] In an eye tracking application, a digital image of a user's eye is obtained and analyzed to estimate the user's gaze direction. The estimation of the gaze direction may be based on a computer-based image analysis of features of the imaged eye that may be identified in the image, such as pupil position or corneal reflection, etc. The image analysis may utilize calibration image data of the eye (binocular) of one or more users, which may be collated using a machine learning-based model or neural network. The calibration image data (which may be referred to as training data when used to train a machine learning algorithm or neural network) may use an image sequence of the eye, wherein each image in the image sequence is captured when the user's gaze is fixed on a stationary object or when a stationary stimulus point is displayed to the user to maintain their gaze. The quality of the calibration image data may be important for accurate gaze direction estimation. In particular, the quality of the training data may be important in terms of the resulting performance of the machine learning algorithm or neural network.

[0003] One known example method of eye tracking includes the use of infrared light and an image sensor. The infrared light is directed toward the user's pupil, and the reflection of the light is captured by the image sensor.

[0004] Portable or wearable eye tracking devices have also been described previously. One such eye tracking system is described in US Pat. No. 9,041,787. A wearable eye tracking device is described that uses an illuminator and an image sensor to determine gaze direction. Summary of the invention

[0005] According to a first aspect of the present disclosure, there is provided a computer-implemented method of processing or selecting a sequence of images of a user's eye for an eye tracking application (each image may be captured while a stationary stimulus point is displayed to the user), the method comprising:

[0006] For multiple different pairs of images:

[0007] comparing the pairs of images to each other to determine an image score representing a degree of difference between the compared images; and

[0008] An image sequence score is calculated based on the image scores of multiple pairs of images.

[0009] The method provides an image score and an image sequence score that can be used to determine the conformity of the image sequence with instructions provided to the user to keep the user's gaze on a stationary stimulus point. Determining the conformity of the image sequence can confirm that the image sequence is of sufficient quality to be used in eye tracking operations such as calibration of an eye tracking system or algorithm. This can improve the performance of the eye tracking system.

[0010] In one or more examples, each image in the image sequence may include a gaze position identifier, which indicates the position of a stimulus point presented to the user on the screen when the image was captured. The gaze position identifier may be the same for each image in the image sequence.

[0011] Including a gaze position marker in each image enables the method to ensure that processing or selection is performed on a sequence of images captured for the same displayed static stimulation point. The gaze position marker enables the method to verify that the sequence of images corresponds to the same stimulation point, or enables the method to select an image sequence from a longer sequence of images corresponding to a series of multiple static stimulation points.

[0012] In one or more examples, the method may further include controlling how the image sequence is passed to a downstream eye gaze tracking operation based on the image sequence score.

[0013] Controlling the delivery of image sequences to downstream eye gaze tracking operations based on image sequence scores can enable selective distribution of image sequences to downstream eye gaze tracking operations. In this way, by comparing the sequence scores to the requirements of the downstream operations, the method can advantageously deliver image sequences to one or more downstream operations.

[0014] In one or more examples, the method may further include setting a discard flag for at least a portion of the image sequence if the image sequence score is greater than a discard threshold.

[0015] Setting the discard flag may advantageously provide an indication to downstream operations that at least a portion of the image sequence may not represent an image sequence of sufficient quality. The downstream operations may then selectively analyze, use and / or discard the data as required by the downstream operations.

[0016] In one or more examples, the method may further include discarding at least a portion of the image sequence if the image sequence score is greater than a discard threshold.

[0017] Discarding at least a portion of the image sequence may prevent providing a bad image sequence to downstream operations.

[0018] In one or more examples, the method may further include:

[0019] setting a censorship flag for at least a portion of the image sequence if the image sequence score is greater than a censorship threshold, wherein the censorship threshold is lower than a discard threshold; and

[0020] If the check flag is set, at least a portion of the sequence of images is provided to a user to manually set the discard flag.

[0021] Setting a review flag based on a review threshold being below a reject threshold can advantageously provide a semi-autonomous process for image sequences having scores that are not clearly pass or clearly reject but are of potentially critical quality.

[0022] In one or more examples, the method may further include performing downstream eye gaze tracking operations on the image sequence for which the discard flag is not set, and not performing eye gaze tracking operations on the image sequence for which the discard flag is set.

[0023] By selectively performing downstream eye gaze tracking operations, such as calibration algorithms, on image sequences for which the discard flag is not set, the method may advantageously provide improved downstream operations, such as improved calibration methods.

[0024] In one or more examples, the method may further include:

[0025] performing downstream eye gaze tracking operations on the sequence of images regardless of whether an associated discard flag is set; and

[0026] Performance evaluation statistics are aggregated for image sequences for which the discard flag is not set, and performance evaluation statistics are not aggregated for image sequences for which the discard flag is set.

[0027] Performing downstream operations on the entire image sequence and selectively aggregating the output based on a discard flag can provide an efficient downstream method to provide high-quality output.

[0028] In one or more examples, the method may further include:

[0029] identifying an image subsequence as a plurality of consecutive images in the image sequence that satisfy a subsequence score threshold; and

[0030] Provides a subsequence of images to downstream eye gaze tracking operations.

[0031] Providing image subsequences can advantageously prune isolated image sequences having erroneously high image scores, so that the pruned image sequences have acceptable sequence scores.

[0032] In one or more examples, the method may further include:

[0033] identifying a plurality of spaced-apart image subsequences in the image sequence, wherein each image subsequence comprises a plurality of consecutive images in the image sequence that satisfy an image subsequence score threshold; and

[0034] concatenating a plurality of image subsequences to provide a concatenated image sequence; and

[0035] The concatenated image sequence is provided to downstream eye gaze tracking operations.

[0036] Identifying and concatenating multiple spaced-apart image subsequences that satisfy a subsequence score threshold can advantageously provide a method of filtering out falsely high image scores that are caused by isolated events such as a user blinking.

[0037] In one or more examples, eye gaze tracking operations may include:

[0038] a training algorithm for setting one or more trainable parameters of a machine learning model for performing eye gaze tracking on a received image;

[0039] Calibration algorithm;

[0040] Authentication algorithm; and / or

[0041] Performance evaluation algorithms.

[0042] In one or more examples, each image sequence may include:

[0043] a gaze position identifier, the gaze position identifier representing the position of a stimulus point presented to the user on the screen when the image sequence was captured; and

[0044] The eye gaze tracking operation may perform functions of the eye gaze tracking operation based on the image and the gaze location identification.

[0045] By performing the functionality of eye gaze tracking operations based on images and gaze position identification, the eye gaze tracking operations can advantageously perform their functionality based on known stationary stimulation points presented to the user, and optionally enable the eye gaze tracking operations to distinguish between multiple stationary stimulation points.

[0046] In one or more examples, calculating an image sequence score based on the image scores may include summing the image scores to provide a cumulative image sequence score.

[0047] In one or more examples, calculating an image sequence score based on the image scores may include setting a peak image sequence score to a maximum value of the image scores.

[0048] In one or more examples, the method may further include:

[0049] If the peak image sequence score is greater than the blink threshold, setting a blink flag for the image sequence; and

[0050] If the blink flag is set, pass the image sequence to a downstream eye gaze tracking operation that requires the presence of blinks in the image sequence; or

[0051] If the blink flag is not set, the image sequence is passed to downstream eye gaze tracking operations that require no blinks to be present in the image sequence.

[0052] In one or more examples, the method may further include:

[0053] If the peak image sequence score is greater than the motion threshold, setting a motion flag for the image sequence; and

[0054] If the motion flag is set, pass the image sequence to a downstream eye gaze tracking operation that requires motion to be present in the image sequence; or

[0055] If the motion flag is not set, the image sequence is passed to downstream eye gaze tracking operations that require no motion in the image sequence.

[0056] In one or more examples, comparing pairs of images may include comparing adjacent images in a sequence of images.

[0057] In one or more examples, the method may further include performing a comparison step for each of the second image and subsequent images in the sequence of images.

[0058] In one or more examples, the method may further include performing the comparing step for at least 90% of the images in the sequence of images.

[0059] In one or more examples, the method may further include:

[0060] causing the screen to display a static stimulus point to the user according to the gaze position identifier; and

[0061] A sequence of images of the user's eyes is captured while the screen is displaying a static stimulus point to the user.

[0062] In one or more examples, each image may include a plurality of pixel locations, and each pixel location may have a pixel value representing a brightness or hue of the pixel. Comparing the paired images may include:

[0063] comparing pixel values ​​at corresponding pixel locations in the pair of images to determine pixel value differences; and

[0064] The image score is determined based on the pixel value differences.

[0065] In one or more examples, determining the image score based on the pixel value differences may include summing the absolute values ​​of the pixel value differences.

[0066] In one or more examples, determining an image score based on pixel value differences may include:

[0067] comparing the absolute value of each pixel value difference to a pixel value threshold; and

[0068] The image score is determined as:

[0069] the number of pixel value differences that are greater than or equal to a pixel value threshold; or

[0070] The sum of the absolute pixel value differences that exceed the pixel value threshold.

[0071] In one or more examples, the sequence of images may represent a time period of at least 500 milliseconds.

[0072] In one or more examples, each image may include: a plurality of pixel locations representing a subset of pixels of the full sensor image, and a cropping mark representing the location of the subset of pixels in the full sensor image. Comparing the paired images may include: determining a cropping offset value as a difference between the cropping marks of the two images. The method may further include: setting a discard image sequence threshold based on the cropping offset value; and setting a discard flag for the image sequence if the image sequence score is greater than the discard image sequence threshold.

[0073] In one or more examples, each image can include:

[0074] a plurality of pixel locations representing a subset of pixels from the full sensor image, and

[0075] The crop flags that represent the location of the pixel subset in the full sensor image.

[0076] Comparing pairs of images in an image sequence may include:

[0077] determining a clipping offset value as a difference between clipping marks of the two images;

[0078] Setting a pixel value threshold based on the clipping offset value;

[0079] comparing pixel values ​​at corresponding pixel locations in the two images to determine a pixel value difference; and

[0080] The image score is determined as:

[0081] the number of pixel value differences that are greater than or equal to a pixel value threshold; or

[0082] The sum of the absolute pixel value differences that exceed the pixel value threshold.

[0083] According to another aspect of the present disclosure, a processor is provided, the processor being configured to:

[0084] receiving a sequence of images of a user's eyes for an eye tracking application;

[0085] For multiple different pairs of images:

[0086] comparing pairs of images to each other to determine image scores representing a degree of difference between the compared images; and

[0087] An image sequence score is calculated based on the image scores of multiple pairs of images.

[0088] According to another aspect of the present disclosure, a computer program is provided, the computer program being configured to execute any method disclosed herein.

[0089] According to another aspect of the present disclosure, a computer-readable medium is provided, which stores any computer program disclosed herein.

[0090] A computer program may be provided that, when run on a computer, causes the computer to configure any device (including a processor or apparatus disclosed herein), or to perform any method disclosed herein. The computer program may be a software implementation, and the computer may be considered to be any suitable hardware, including a digital signal processor, a microcontroller, and, as non-limiting examples, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) implementation. The software may be an assembler.

[0091] The computer program may be provided on a computer readable medium, which may be a physical computer readable medium such as a disk or a memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Embodiments of the present invention will now be described in detail, by way of example only, with reference to the following drawings, in which:

[0093] Figure 1 shows a schematic diagram of an eye tracking system that may be used to capture a sequence of images that may be used by example embodiments;

[0094] Figure 2 An example image of an eye is shown;

[0095] Figure 3 An example apparatus for processing and / or selecting a sequence of images of a user's eye is shown;

[0096] Figure 4An example image sequence is shown, along with example determinations of image scores and image sequence scores;

[0097] Figure 5 A sequence of eleven images and their associated image scores, and a concatenation of the image sequences are graphically shown;

[0098] Figure 6 graphically illustrates how sequence scores associated with multiple image sequences may be compared to various landmarks; and

[0099] Figure 7 is a flow chart of an example of a method.

[0100] All figures are schematic and generally only show parts which are necessary in order to elucidate the various embodiments, while other parts may be omitted or merely suggested. DETAILED DESCRIPTION

[0101] Figure 1 A simplified view of an eye tracking system 100 (which may also be referred to as a gaze tracking system) in a head-mounted device is shown, which appears in the form of a virtual or augmented reality (VR or AR) device, or VR or AR glasses. The system 100 includes an image sensor 120 (e.g., a camera) for capturing an image of a user's eye. The system may optionally include one or more illuminators 110 to 119 for illuminating the user's eye, which may be, for example, light-emitting diodes that emit light in an infrared band or a near-infrared band, and may be physically arranged in various configurations. The image sensor 120 may be, for example, any type of image sensor, such as a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor. The image sensor may be composed of an integrated circuit comprising an array of pixel sensors, each pixel comprising a photodetector and an active amplifier. The image sensor is capable of converting light into a digital signal. In one or more examples, the image sensor may be an infrared image sensor or an IR image sensor, an RGB sensor, an RGBW sensor, or an RGB or RGBW sensor with an IR filter.

[0102] The eye tracking system 100 may include a circuit system 125 for receiving and processing images captured by the image sensor 120, including, for example, a receiver 126 and a processing circuit system 127. The circuit system 125 may be connected to the image sensor 120 and optionally one or more illuminators 110 to 119, for example, via a wired connection or a wireless connection, and may be co-located with the image sensor 120 and one or more illuminators 110 to 119, or may be located at a distance from them (e.g., in a different device). In another example, the circuit system 125 may be disposed in one or more stacks below the photosensitive surface of the light sensor 120.

[0103] The eye tracking system 100 may include a display 128 for presenting information and / or visual stimulation points to the user. The display may include a VR display that presents imagery and substantially obscures the user's real-world view, or an AR display that presents imagery that is perceived as superimposed on the user's real-world view.

[0104] It should be noted that Figure 1 The positions of the image sensor 120 and the display 128 in are for illustration purposes only. In such a system 100, the position of the image sensor 120 for one eye is usually away from the user's line of sight so as not to block the display 128 for that eye. This configuration can be achieved, for example, with the help of a so-called hot mirror, which reflects a part of the light and allows the rest of the light to pass, for example infrared light is reflected and visible light is allowed to pass.

[0105] Although in the above example, the image of the user's eye is captured by the head-mounted image sensor 120, as it forms part of the eye tracking system 100, alternatively, the image may be captured by an image sensor separate from the eye tracking system 100, wherein the image sensor 120 may be head-mounted or non-head-mounted. It will be appreciated that a head-mounted image sensor directed toward the eye isolates any head movement from any eye movement in the captured image. However, if the image sensor is non-head-mounted, eye movement in the captured image may be due to movement of the user's eye or due to movement of the user's head. Therefore, for the example in which the image sensor is non-head-mounted, the system may be configured to receive information indicating movement of the user's head relative to the image sensor, so that the position and / or movement of the eye can be determined from the image captured by the image sensor. Alternatively, image processing may be performed on the full sensor image to identify an area in the image that contains the user's eye. Gaze tracking may then be performed on the identified image area. As will be discussed below, this may involve an algorithm that determines a clipping marker that represents the location of an area (which represents a subset of pixels) in the full sensor image that includes the user's eye.

[0106] Figure 2 Shown by Figure 1 1 is a simplified example of an image of eye 200 captured by image sensor 120. Circuit system 125 may, for example, employ image processing (such as digital image processing) to extract features in the image. Circuit system 125 may, for example, identify the location of pupil 210 in one or more images captured by image sensor 120.

[0107] In one or more examples, Figure 3 The illustrated apparatus 300 may provide for processing or selection of an image sequence. The image sequence may be used as calibration data, such as calibration data suitable for a single user or calibration data used as training data for a machine learning algorithm or neural network. In one or more examples, the apparatus 300 may form part of the circuit system 125. In one or more examples, the circuit system 125 may be configured to transmit the captured images for processing by the apparatus 300, which may include a computer or server remote from the eye tracking system 100. Thus, in general, the apparatus 300 may be considered to include a processor 301 that may access a memory 302 that receives images from the image sensor 120 and computer program code that defines the steps performed by the processor 301.

[0108] The eye tracking system 100 can perform eye tracking using calibration image data and / or a machine learning algorithm or neural network trained based on the calibration image data. As described above, the calibration image data (when used to train a machine learning algorithm or neural network, it can be referred to as training data) can use a sequence of images of the eye, wherein each image in the sequence of images is captured when the user's gaze is fixed on a stationary object or when a stationary stimulus point is displayed to the user to maintain their gaze. The stationary stimulus point may or may not be displayed on an electronic display. For example, the stationary stimulus point can be displayed on a whiteboard. Alternatively, the user can be commanded to direct their gaze to a fixed point on a wall or a stationary piece of furniture in a room. It may be important that the stimulus point and the camera are constant relative to each other in terms of distance and / or rotational position.

[0109] The quality of the calibration image data may be important for accurate gaze direction estimation. In particular, the quality of the training data may be important with respect to the resulting performance of a machine learning algorithm or neural network.

[0110] Reference Figure 4And according to the first embodiment, a method for processing an image sequence 400 of a user's eyes for use in an eye tracking application and a device 300 configured to perform the method are disclosed. In one or more examples, the processing may include selecting an image sequence or a portion thereof, such as a positive selection or a negative selection, for subsequent use as calibration data, performance monitoring data, or for other eye tracking purposes. Therefore, in one or more examples, the method may provide effective screening of one or more image sequences. In one or more examples, the method may provide effective editing of an image sequence based on a positive selection or a negative selection of a portion of the image sequence 400. For example, a positive selection may involve selecting an image sequence or a portion thereof for subsequent downstream processing, while a negative selection may involve selecting an image sequence or a portion thereof to exclude it from subsequent downstream processing. Therefore, a negative selection may also be viewed as discarding an image sequence from a downstream processing operation.

[0111] Figure 4 An image sequence 400 is shown, which can be received from the image sensor 120 and can have been transmitted to the memory 302 of the device 300 for performing the method. The image sequence 400 comprises a plurality of individual images that are sequential in time. Figure 4 In the example of , the first image 401 to the sixth image 406 are included. It should be understood that the image sequence may include any number of images more than three images. The image sequence 400 may include, for example, at least 20 images or at least 30 images or more images. The time period represented by the image sequence may include at least 0.5 seconds or at least 1 second. The rate of capturing the image sequence 400 may include at least 15 Hz or at least 30 Hz, and may typically include 30 Hz to 200 Hz. The resolution of images 401 to 406 of the image sequence 400 may include at least 2 pixels per mm of the eye being imaged. Therefore, the resolution of a typical image may include at least 50×50 pixels. In other examples, the resolution may include 40 pixels per mm of the eye being imaged, and therefore, the resolution of a typical image may include at least 1000×1000 pixels. It should be understood that the image sequence 400 processed by the method may be an image at the native resolution and / or native frame rate of the image sensor 120, or may be an image that is downsampled or upsampled from the native resolution or native frame rate of the image sensor 120 in terms of resolution and / or frame rate.

[0112] The method or the apparatus 300 performing the method may be configured to control the capturing of a sequence of images, or alternatively, the method or the apparatus may be fed with a sequence of images captured by the image sensor 120 of one or more systems 100 .

[0113] The image sequence 400 should be captured while the user's gaze remains in a stationary direction. In one or more examples, the image sequence 400 includes images captured when the user is commanded to maintain their gaze in the stationary direction, such as by visual or auditory instructions. In one or more examples, the user may be commanded to maintain their gaze on a point or object identified in their field of view. In other examples, the image sequence 400 includes images captured when a stationary stimulus point is displayed to the user, such as by an AR or VR display 128, and wherein the user is expected to maintain their gaze on the stationary stimulus point. The image sequence 400 may be captured while multiple individual stationary stimulus points are displayed to the user in sequence. As discussed further below, a gaze position identifier may be included with each image to indicate the location of the stimulus point presented to the user when the image was captured.

[0114] When capturing a sequence of images, the relative position of the stationary stimulus point to the image sensor should be substantially constant. In a head mounted device, the relative positions between the stationary stimulus point, the image sensor and the user's eyes should all remain substantially constant.

[0115] It will be appreciated that controlling the capture of a sequence of images and providing instructions for maintaining still eye gaze may or may not be part of an embodiment of the functionality of the method and apparatus 300. It will be appreciated that the method or apparatus 300 may be provided with one or more image sequences 400 that have been captured using any of the example methods above or other methods that encourage a user to maintain still eye position.

[0116] The method includes, for a plurality of different pairs of images 401 to 406 , comparing the pairs of images to each other to determine an image score 407 representing a degree of difference between the compared images; and calculating an image sequence score based on the image scores 407 for the plurality of pairs of images.

[0117] By calculating the image score 407 and the image sequence score, one or more disclosed methods may enable a conformity check of the image sequence 400 with the instruction to maintain fixation on a stationary stimulus point.

[0118] In one or more examples, the paired images include adjacent images in the image sequence, i.e., images captured consecutively in time, one after the other, in the image sequence 400. For example, a first paired image may include a first image 401 and a second image 402. A second paired image may include a second image 402 and a third image 403, and so on. Thus, the paired images may consist of overlapping selections of paired images that form the image sequence 400. Using overlapping selections of paired images can ensure that any sudden step changes in pixel differences are identified. For example, image differences associated with the start and / or end of a blinking motion may not be captured by non-overlapping consecutive paired images.

[0119] In other examples, the first paired image may include the first image 401 and the second image 402. The second paired image may include the third image 403 and the fourth image 404, and so on. Thus, the paired images may consist of non-overlapping selections of paired images that form the image sequence 400. In other examples, a combination of overlapping selections and non-overlapping selections may be used when determining the paired images. Comparing consecutive images may be advantageous because this may make it easy to identify the timing of specific events, such as blinks or eye movements.

[0120] In one or more examples, one or more reference images may be determined based on a predetermined criterion. The reference image may be one of the first image 401 to the sixth image 406. The paired image may include the reference image and another image other than the image immediately following in the image sequence 400. Accordingly, the first image 401 may be designated as the reference image, and thus the first paired image may include the first image 401 and the second image 402; the second paired image may include the first image 401 and the third image 403; and the third paired image may include the third image 401 and the fourth image 404; and so on.

[0121] Figure 4 A graph illustrating the image scores assigned to each pair of compared images is shown. Thus, point 408 represents an image score determined from a comparison of the first image 401 with the second image 402. In this example, the image score comprises zero because no differences were identified between the pair of images. Point 409 represents an image score determined from a comparison of the second image 402 with the third image 403, illustrated as zero. Point 410 represents an image score determined from a comparison of the third image 403 with the fourth image 404, and the score is six. In this example, the score of six represents the number of pixel positions in one of the paired images that have the following characteristics: these pixel positions differ relative to the equivalent pixel positions in the other image in the paired image. Thus, six pixel positions 413, 414, 415, 416, 417, and 418 in the fourth image 404 have brightness changes relative to the third image 403. Point 411 represents an image score determined from a comparison of the fourth image 404 with the fifth image 405, illustrated as zero. Point 412 represents the image score determined from a comparison of fifth image 405 and sixth image 406, illustrated as zero.

[0122] Whether there is a difference in the pixel values ​​at the corresponding pixel positions between the paired images can be determined based on the difference in pixel brightness. In other examples, the difference in pixel hue can be used. The pixel value difference can be determined by comparing the brightness or hue pixel values ​​at the corresponding pixel positions. The image score is determined based on the pixel value difference.

[0123] In some examples, the absolute difference between pixel values ​​may be calculated so it does not matter which pixel value is greater than the other; the absolute value of the difference will be used. This may be particularly useful when subsequently summing the image scores so that positive and negative pixel values ​​do not cancel each other out.

[0124] In one or more examples, the image score is determined as the sum of the absolute values ​​of the pixel value differences. For example, brightness or hue can be represented by an 8-bit digital value between 0 and 255. The absolute value of the pixel value difference can also be represented as an 8-bit digital value. Determining the image score can include summing the 8-bit pixel value differences for each pair of pixel values ​​at corresponding pixel locations.

[0125] In one or more examples, the image score can be determined by comparing the absolute pixel value difference to a pixel value threshold. The image score can include: an integer of absolute pixel value differences greater than or equal to the pixel value threshold; or a sum of absolute pixel value differences exceeding the pixel value threshold.

[0126] In summary, in this example, each image includes a plurality of pixel positions, and each pixel position has a pixel value representing the brightness or hue of the pixel. The method may include comparing the paired images by: comparing the pixel values ​​at corresponding pixel positions in the paired images to determine a pixel value difference; and determining an image score based on the pixel value difference.

[0127] Thus, in one or more examples, the image score represents the degree of difference between the compared images in terms of the number of sub-region positions (e.g., pixel positions) at which the difference between the images of the paired images exceeds a threshold amount. Thus, while pixels may be considered to segment an image into a grid of pixel positions, in one or more examples, a sub-region may be larger than a pixel position. Further, at different locations in the image, the sub-region may have different sizes, which may improve processing efficiency.

[0128] Although in this example the difference between corresponding pixel positions or sub-region positions is used to determine the image score, other examples may use different score assignment schemes.

[0129] It should be understood that a low score (such as zero) indicates little or no difference, while a higher positive number indicates a greater degree of difference. In other examples, a high score may indicate a low degree of difference, while a low score may indicate a high degree of difference.

[0130] The sequence score is based on the image scores of multiple paired images. In one or more examples, the sequence score may include a function of the image scores, such as a sum of the image scores, or a multiple of the image scores. In one or more examples, as the method steps through multiple paired images, the calculation of the image sequence score can be performed simultaneously with determining the image scores. Therefore, calculating the sequence score based on the image scores may include summing the image scores to provide a cumulative image sequence score. Once all expected image scores have been summed, the cumulative image sequence score may include the image sequence score. Alternatively, the image sequence score can be calculated once all image scores have been determined.

[0131] exist Figure 4 In the example of , the image sequence score comprises the sum of the image scores, and thus comprises 0+0+6+0+0=6. The image sequence score may be used (eg, by comparison with a threshold) to determine how to further use the image sequence 400.

[0132] An example computer routine for determining an image score and an image sequence score is provided below, wherein:

[0133] "previous_image" comprises the working image assignment used to step through the image sequence 400,

[0134] "recording.images" includes a data structure representing a sequence of images 400,

[0135] Stimulus_point_is_stationary(image) represents a check on the gaze position flag, which is described below, and

[0136] "score" and "scores" represent the image score and the cumulative image sequence score, respectively.

[0137]

[0138]

[0139] Furthermore, "total_abs_diff" calculates the sum of the absolute differences between a pixel in the first input image and the corresponding pixel in the second input image for all pixels in the images assumed to be of the same size.

[0140] It has been found that the image sequence score provides an efficient way of determining whether the image sequence 400 is suitable for downstream use in the field of eye tracking. Thus, even when a user attempts to keep their eye position still, this may not always be successful, and while a single or a few short deviations in time in a relatively large image sequence may not be problematic, many deviations or deviations representing a larger degree of difference may result in the image sequence 400 being suboptimal for downstream operations. The disclosed method of determining a sequence score represents a technically efficient way of processing an image sequence of a user's eye.

[0141] The plurality of paired images may be selected to include all images in the image sequence 400 in at least one of the plurality of paired images. In other examples, fewer images (e.g., at least 80%, 90%, or 95%) than all images 401-406 may be included in one of the plurality of paired images. For example, if a threshold value for an image sequence score (e.g., a cumulative image sequence score) has been reached before all image scores are considered, the method may end before all expected image scores are accumulated into the image sequence score.

[0142] As described above, when performing the method, it can be assumed that the eye position or related gaze direction is stationary for the image sequence (or at least the user attempts to maintain a stationary position). In one or more examples, each image in the image sequence 400 can include a gaze position identifier, which indicates the position of the stimulus point presented to the user on the display 128 when the image is captured. The gaze position identifier can be embodied as metadata or other indicators linked to each image in the image sequence 400. Therefore, the method can be performed on the image sequence 400, and the gaze position identifier is the same for each image 401 to 406 in the image sequence 400. In one or more examples, the method may include the following steps: before continuing the comparison step to determine the image score, verifying for each image 401 to 406 whether the gaze position identifier indicates the same stimulus point position. In other examples, the method may include selecting a subset of consecutive images from the image stream for forming the image sequence 400 based on the fact that the gaze position identifiers are the same for the images in the subset. The method can be configured to select a subset having a predetermined minimum number of constituent images. The method can be further configured to select a subset that does not include images near the transition point in the gaze position identifier. For example, the method may select a subset of images that is at least a predetermined number of images from the transition point of the gaze position identification. This may allow the user to have reaction time to respond to new stimulus points and / or gaze instructions.

[0143] The method may include controlling how the image sequence 400 is passed to a downstream eye gaze tracking operation based on the sequence score. The control may include whether to pass the image sequence to the downstream eye gaze tracking operation (or, for example, discard the image sequence). The control may include determining a portion of the image sequence passed to the downstream eye gaze tracking operation. For example, the control may include whether to retain the image sequence for a downstream operation that evaluates eye tracking performance, or to discard the image sequence for the evaluation of eye tracking performance. In some applications, subsequent eye gaze tracking operations are performed on the image sequence regardless of whether a discard flag is set, but the method only performs performance evaluation statistics aggregation for frames where the discard flag is not set. In this way, the method can aggregate performance evaluation statistics for image sequences where the discard flag is not set, and not aggregate performance evaluation statistics for image sequences where the discard flag is set. When the aggregation of performance evaluation statistics itself is regarded as a downstream eye gaze tracking operation, this can also be regarded as not performing an eye gaze tracking operation on the image sequence with the discard flag. That is, when evaluating eye gaze tracking performance, the results from frames (images) where the discard flag is set are ignored. The control may be based on a comparison of the image sequence score to one or more thresholds, such that sequence scores on one side of a threshold result in a first control being performed, while sequence scores on the other side of the threshold result in a second, different control being performed.

[0144] Eye gaze tracking operations may include: a training algorithm for setting one or more trainable parameters of a machine learning model to perform eye gaze tracking on received images; a calibration algorithm; a validation algorithm; and / or a performance evaluation algorithm. Various examples of the types of performance that may be evaluated by the performance evaluation algorithm may include:

[0145] - Accuracy of gaze estimation (difference between measured gaze and actual gaze assumed to be towards the stimulus point);

[0146] - the accuracy of the gaze estimate, i.e. the difference between gaze estimates made from different images (in the case where the user is looking at a fixed point); and / or

[0147] - Output rate, i.e., the percentage of images for which the eye gaze tracking operation was fully able to make any gaze estimation. If images in which the user is blinking are included in the evaluation, this value will typically not be 100%. Embodiments disclosed herein can remove any such suboptimal images from the evaluation, which can enable a higher output rate to be achieved.

[0148] It should be understood that if the user is not looking at the fixed point, the accuracy estimate will be wrong and cannot reflect the true performance of the performance evaluation algorithm. One or more examples disclosed herein can help solve this problem by only selecting frames where the user is looking at the fixed point for performance evaluation.

[0149] In one or more examples, for an image sequence score accumulated from the image scores (where a higher score indicates a greater degree of difference between paired images), if the image sequence score is greater than a threshold (referred to as a discard threshold), the image sequence may be marked for discard and, therefore, may not be passed to downstream eye gaze tracking operations. If the sequence score is less than or equal to the discard threshold, the image sequence may be passed to downstream eye gaze tracking operations. Thus, in one or more examples, the image sequence 400 or a portion thereof may be supplemented with a discard flag (e.g., metadata) to identify the image sequence 400 or a portion thereof that does not have an acceptable sequence score relative to the discard threshold.

[0150] In one or more examples, it may not be necessary to pass all images of the image sequence 400 to the downstream eye gaze tracking operation. For example, the downstream eye gaze tracking operation may be able to operate with fewer images than are present in the image sequence 400. In such an example, the method may include: segmenting the image sequence into a plurality of image subsequences, each image subsequence including a plurality (e.g., at least three) of consecutive images; and determining a plurality of image sequence scores based on image scores of pairs of images in each of the plurality of image subsequences (referred to as image subsequence scores to separate them from image sequence score partitions, but the image subsequence scores and the image sequence scores are calculated in the same manner). The plurality of image subsequence scores may be compared to an image subsequence score threshold to determine whether to pass each image in the image subsequence to the downstream eye gaze tracking operation. Image subsequences with unacceptable image subsequence scores may be discarded. Thus, the method may effectively prune the image sequence so that the pruned image sequence has an acceptable image sequence score.

[0151] In one or more examples, the downstream eye gaze tracking operation includes a process of evaluating the performance of eye gaze tracking to determine performance statistics, and wherein the performance statistics are calculated based on image sequences 400 without a discard flag set and not based on image sequences 400 with a discard flag set.

[0152] In one or more examples, the image sequence score can be used to identify the image sequence 400 or a portion thereof that should be forwarded for manual inspection by a user (not necessarily a user whose eyes have been imaged in the plurality of images). Thus, in one or more examples, the method may include setting an inspection flag for the image sequence 400 or at least a portion of the image sequence 400 if the image sequence score is greater than an inspection threshold. The inspection flag may be recorded as metadata associated with the entire image sequence or each of its images 401 to 406. If the inspection threshold is used in conjunction with a discard threshold, the inspection threshold is typically less than the discard threshold. The method may include the following steps: if the inspection flag is set (and optionally, if the discard flag is not set (if there is a discard flag)), providing at least a portion of the image sequence to a user to manually set the discard flag. In one or more examples, the step may include providing a display of at least one or more images of the plurality of images for which the inspection flag is set, and receiving user input as to whether to set the discard flag.

[0153] For either or both of the discard flag or the review flag, the method may include determining whether the image sequence score is above an associated threshold. If so, the method may be configured to identify which images score above the other threshold and only set the relevant flag for images with such image scores. Thus, the method may advantageously use the image sequence score to evaluate the entire image sequence 400, and then set (multiple) flags for specific images in the image sequence for discard or manual review. This may provide an efficient method.

[0154] Although the user attempts to maintain a stationary eye position, i.e., gaze direction, one or more of the images 401 to 406 may include a blink. Given the typical instantaneous nature of a blink, multiple images before and after the blink in the image sequence 400 may be satisfactory even if the entire image sequence is unsatisfactory because its sequence score exceeds a discarding threshold. Therefore, the method may be configured to automatically edit the image sequence 400 so that the edited sequence has an acceptable sequence score.

[0155] Figure 5 A sequence of images 500 is shown, designated for simplicity by image numbers one through eleven. Table 501 shows a series of image scores for pairs of images above each box in the table. Accordingly, the image score for pair one and two is zero. The image score for pair three and four is one. The image score for pair five and six is ​​ten, and the image score for pair six and seven is also ten. Image six may have a larger difference than images five and seven due to the presence of a blink. Identification of a blink may be based on which image scores exceed a blink threshold and which image scores do not exceed a blink threshold.

[0156] Thus, the method can be configured to identify a plurality of spaced-apart image subsequences 502, 503 in an image sequence that may be temporally before and after a blink, wherein each image subsequence includes a plurality of consecutive images in the image sequence 400 that satisfy an image subsequence score threshold. In this example, the image subsequence score is determined in the same manner as the image sequence score. Thus, the image scores for images one through five in the first image subsequence 502 include zero, zero, one, zero. Summing (or other functions in other examples) yields an image subsequence score of one. In this example, the image scores for images seven through eleven in the second image subsequence 503 include zero, zero, zero, zero. Summing (or other functions in other examples) yields an image subsequence score of zero.

[0157] In one or more examples, the image subsequences can be identified as being spaced apart in the image sequence 500 based on determining that one or more image scores of consecutive images exceed a blink threshold. In this example, the blink threshold can be nine, and thus an image score of ten for the pair of images five and six and the pair of images six and seven can indicate that a blink occurred in the image(s) common to these pairs of images (i.e., image six).

[0158] Additionally or alternatively, the method may include identifying the image subsequences 502, 503 by testing candidate image subsequences of the image sequence to determine whether an image subsequence score threshold is satisfied. Thus, the method may start with a candidate image subsequence having a predetermined number of images 500 and gradually reduce the candidate subsequences in terms of the number of images therein, and / or different consecutive groups of the image sequence 500 may be designated as candidate image subsequences. The method may determine whether each candidate image subsequence satisfies the image subsequence score threshold. The image candidate subsequence having the largest number of images may be determined as at least one of the image subsequences. Testing may include calculating an image subsequence score for each of the candidate image subsequences.

[0159] Therefore, consecutive images in which blinking occurs in the image sequence 500 may be excluded from the spaced-apart image subsequences.

[0160] The method may then include concatenating the plurality of image subsequences 502, 503 to provide a concatenated image sequence 504; and providing the concatenated image sequence 504 to downstream eye gaze tracking operations. The concatenated image sequence may be considered to include an image sequence in which a blink was "edited out". Thus, the concatenated image sequence 504 is based on the image sequence 500, but with image six deleted. Table 505 shows a plurality of image scores for the concatenated image sequence 504. In this example, the image sequence score for the concatenated image sequence 504 is therefore 0+0+1+0+0+0+0+0+0=1.

[0161] In one or more examples, certain downstream eye tracking operations may require an image sequence that includes eye blinks.

[0162] The method may include determining the highest or maximum value among the plurality of image scores determined from the image sequence 500 and designating the maximum value as the "peak image sequence score". Figure 5 In the example shown, the peak image sequence has a score of ten.

[0163] Then, if the peak image sequence score is greater than the blink threshold, the method may set a blink flag for the image sequence 500, indicating that the image sequence contains a blink.

[0164] Thus, if the blink flag is set, the method may include passing the image sequence to a downstream eye gaze tracking operation that requires the presence of a blink in the image sequence, or if the blink flag is not set, passing the image sequence to a downstream eye gaze tracking operation that requires the absence of a blink in the image sequence.

[0165] In one or more examples, certain downstream eye tracking operations may require image sequences that include eye movements (such as involuntary eye movements).

[0166] The method may include determining the highest or maximum value among the plurality of image scores determined from the image sequence 400 and designating the maximum value as the "peak image sequence score."

[0167] Then, if the peak sequence score is greater than the motion threshold, the method may set a motion flag for the image sequence 400, indicating that the image sequence contains motion. The motion threshold may be lower than the blink threshold.

[0168] Thus, if the motion flag is set, the method may include: passing the image sequence to a downstream eye gaze tracking operation that requires motion to be present in the image sequence; or if the motion flag is not set, passing the image sequence to a downstream eye gaze tracking operation that requires no motion to be present in the image sequence.

[0169] Therefore, the method can effectively classify an image sequence into any two of an image sequence comprising images with static eye positions, an image sequence comprising images containing blinks, and an image sequence comprising images containing motion.

[0170] Figure 6 Examples of how image sequence scores may be compared to thresholds to set various flags are shown diagrammatically.

[0171] Figure 6 A plurality of image sequences 620 are shown. Each image sequence is labeled with a letter A through K. Figure 6 Also shown are a cumulative image sequence score 622 and a peak image sequence score 628 associated with each image sequence 620. These scores 622, 628 may be calculated in the same manner as described above.

[0172] In this example, the cumulative sequence score 622 is compared to a discard threshold and a check threshold to determine whether to set the corresponding discard flag 626 and check flag 624 associated with each image sequence. Figure 6 , the value of the discard threshold is 50, and the value of the check threshold is 20. Therefore, the discard flag 626 is set for the image sequences B, F, and G, and the check flag 624 is set for the image sequences A, B, F, and G.

[0173] Also in this example, the peak image sequence score 628 is compared to the motion threshold and the blink threshold to determine whether to set the corresponding motion flag 630 and blink flag 632 associated with each image sequence. Figure 6 , the value of the motion threshold is 10, and the value of the blink threshold is 20. Therefore, the motion flag 630 is set for image sequences B, C, F, and G, and the blink flag 624 is set for image sequences F and G.

[0174] It should be understood that Figure 6 This is just one example of how a flag may be set, and either or both of the cumulative sequence score 622 and the peak sequence score 628 may be compared to an appropriate threshold to set any flag. It should also be understood that the values ​​used for the cumulative sequence score 622 and the peak sequence score 628, as well as these thresholds, are entirely optional to illustrate the principle of how flags may be set based on sequence scores.

[0175] The method or processor may be used in any manner described herein. Figure 6 Any of the marks 624, 626, 630, 632 shown in .

[0176] Figure 7A flow chart outlining an embodiment of the method is shown. The flow chart optionally shows receiving 701 a sequence of images of a user's eyes for an eye tracking application, wherein each image is captured when the user's eyes are expected to be stationary; and for a plurality of different pairs of images:

[0177] comparing pairs of images to each other 702 to determine an image score representing a degree of difference between the compared images; and

[0178] An image sequence score is calculated 703 based on the image scores of the plurality of pairs of images.

[0179] As described above, in one or more examples, the sequence of images can be captured by a non-head-mounted image sensor. The non-head-mounted image sensor can include a wider field of view than the head-mounted image sensor. Thus, the captured images can include images that cover significantly more than one or both eyes of the user, such as images of the user's entire head or upper torso.

[0180] In such an example, the movement of the user's eyes will be superimposed on any movement of the user's head. Therefore, in one or more examples, the disclosed method may further include determining the movement of the user's head before determining the relative movement of the user's eyes. In addition, the movement of the user's head may result in different ambient lighting conditions in the user's eye region. Therefore, the discard sequence threshold may be reduced accordingly.

[0181] Each image may include a plurality of pixel locations representing a subset of pixels from the full sensor image. For example, the subset of pixels may correspond to the eyes of a user. Determining the subset of pixels may be accomplished using an object recognition algorithm to separate out the portion of the image that includes the eyes of the user. The image may further include a cropping identifier representing the location of the subset of pixels in the full sensor image. One or more example methods may include determining the subset of pixels and the cropping identifier. Alternatively, the subset of pixels and the cropping identifier may be determined as part of a separate process, and the subset of pixels and the cropping identifier may be received with the captured sequence of images.

[0182] In some examples, where an image sequence includes a subset of pixels and a cropping identifier, a comparison of paired images may be performed only on the corresponding subset of pixels. Comparing the paired images may include determining a cropping offset value as the difference between the cropping identifiers of the two images. The cropping offset value may provide an indication of head movement. The method may also include setting a discard image sequence threshold based on the cropping offset value. In some applications, the discard image sequence threshold may be based on a statistical measure of the cropping offset value, such as an average of the cropping offset values. The discard image sequence threshold may be inversely proportional to the absolute value of the cropping offset value. In other words, the discard image sequence threshold decreases as the amount of head movement increases. Head movement may cause a change in the ambient lighting conditions of the user's eyes. Setting the discard image sequence threshold in this manner allows the threshold to be dynamically set based on the movement of the user's head. The method may then include setting a discard flag for the image sequence if the image sequence score is greater than the discard image sequence threshold.

[0183] In one or more examples, the method may further include adjusting a pixel value threshold based on a clipping offset identifier. For example, comparing paired images in an image sequence may include: comparing pixel values ​​at corresponding pixel positions in the paired images to determine a pixel value difference; and setting a pixel value threshold based on a clipping offset value. Then, determining an image score may include any of: an integer of absolute pixel value differences greater than or equal to the pixel value threshold; or a sum of absolute pixel value differences exceeding the pixel value threshold.

[0184] As mentioned above, the method described herein may be performed by the apparatus 300. Therefore, the apparatus 300 may be configured to perform one or more method steps described herein.

[0185] The apparatus 300 may have only one processor 301 and one memory 302, but it should be understood that other embodiments may utilize more than one processor and / or more than one memory (e.g., the same or different processor / memory types). Further, the apparatus 300 may be an application specific integrated circuit (ASIC). The processor 300 may be a general purpose processor dedicated to executing / processing information received from other components according to instructions stored in the memory 302 in the form of computer program code. The output signals generated by such operation of the processor are provided onwardly to other components such as providing downstream eye tracking operations.

[0186] The memory 302 (not necessarily a single memory unit) is a computer readable medium (solid state memory in this example, but may be other types of memory such as a hard drive, ROM, RAM, flash memory, etc.) that stores computer program code. The computer program code stores instructions that are executable by the processor when the program code runs on the processor.

Claims

1. A computer-implemented method of selecting a sequence of images of a user's eyes for an eye tracking application, wherein: Each image in the sequence of images is captured while a stationary stimulus point is displayed to the user and the user is provided with instructions to maintain their gaze on the stationary stimulus point, the method comprising for a plurality of different pairs of images: comparing the pairs of images to each other to determine an image score representing a degree of difference between the compared images; as well as An image sequence score is calculated based on the image scores of the plurality of pairs of images to determine compliance of the image sequence with the instructions provided to the user to maintain the user's gaze on the stationary stimulus point.

2. The method of claim 1, wherein: Each image in the sequence of images includes a gaze position identifier, the gaze position identifier indicating the position of the stimulus point presented to the user on the screen when the image was captured; and The gaze position identifier of each image in the image sequence is the same. 3 . The method of claim 1 , further comprising controlling how the image sequence is passed to a downstream eye gaze tracking operation based on the image sequence score.

4. The method of claim 1, further comprising: If the image sequence score is greater than a discard threshold, a discard flag is set for at least a portion of the image sequence.

5. The method of claim 1, further comprising: If the image sequence score is greater than a discarding threshold, at least a portion of the image sequence is discarded.

6. The method of claim 4, further comprising: If the image sequence score is greater than a censoring threshold, setting a censoring flag for at least a portion of the image sequence, wherein the censoring threshold is lower than the discarding threshold; and If the check flag is set, the at least a portion of the sequence of images is provided to a user to manually set the discard flag.

7. The method of claim 4, further comprising: Downstream eye gaze tracking operations are performed on image sequences for which the discard flag is not set, and the eye gaze tracking operations are not performed on image sequences for which the discard flag is set.

8. The method of claim 4, further comprising: performing a downstream eye gaze tracking operation on the sequence of images regardless of whether an associated discard flag is set; as well as The performance evaluation statistical data are summarized for the image sequence where the discard flag is not set, and the performance evaluation statistical data are not summarized for the image sequence where the discard flag is set.

9. The method of claim 1, further comprising: identifying a plurality of spaced-apart image subsequences in the image sequence, wherein each image subsequence comprises a plurality of consecutive images in the image sequence that satisfy an image subsequence score threshold; and concatenating the plurality of image subsequences to provide a concatenated image sequence; and The concatenated sequence of images is provided to downstream eye gaze tracking operations.

10. The method of claim 7, wherein: The eye gaze tracking operation includes: a training algorithm for setting one or more trainable parameters of a machine learning model for performing eye gaze tracking on a received image; Calibration algorithm; Authentication algorithm; and / or Performance evaluation algorithms.

11. The method of claim 10, wherein: Each image sequence includes a gaze position identifier, the gaze position identifier representing the position of a stimulus point presented to the user on the screen when the image sequence was captured; and The eye gaze tracking operation performs a function of the eye gaze tracking operation based on the image and the gaze location identification.

12. The method of claim 1, wherein: Calculating the image sequence score based on the image score comprises: The image scores are summed to provide a cumulative image sequence score.

13. The method of claim 1, wherein: Calculating the image sequence score based on the image score comprises: The peak image sequence score is set to the maximum of the image scores.

14. The method of claim 13, further comprising: If the peak image sequence score is greater than a blink threshold, setting a blink flag for the image sequence; as well as If the blink flag is set, passing the image sequence to a downstream eye gaze tracking operation that requires the presence of a blink in the image sequence; or If the blink flag is not set, the image sequence is passed to a downstream eye gaze tracking operation that requires no blinks to be present in the image sequence.

15. The method of claim 13, further comprising: If the peak image sequence score is greater than a motion threshold, setting a motion flag for the image sequence; as well as If the motion flag is set, passing the image sequence to a downstream eye gaze tracking operation that requires motion to be present in the image sequence; or If the motion flag is not set, the image sequence is passed to a downstream eye gaze tracking operation that requires the absence of motion in the image sequence.

16. The method of claim 1, wherein: The method further comprises: Displaying a static stimulation point to the user on a screen according to the gaze position identifier; and The sequence of images of the user's eyes is captured while the screen is displaying the stationary stimulus point to the user.

17. The method of claim 1, wherein: Each image includes a plurality of pixel locations, and each pixel location has a pixel value representing a brightness or hue of the pixel; and Comparing the pair of images comprises: comparing pixel values ​​at corresponding pixel positions in the paired images to determine a pixel value difference; as well as The image score is determined based on the pixel value differences.

18. The method of claim 1, wherein: Each image includes: a plurality of pixel locations representing a subset of pixels of the full sensor image, and a cropping marker indicating a location of the subset of pixels in the full sensor image; and Comparing the pair of images includes determining a crop offset value as a difference between the crop identifiers of the two images; The method further comprises: Setting a discarded image sequence threshold based on the clipping offset value; and If the image sequence score is higher than the discard image sequence threshold, a discard flag is set for the image sequence.

19. A processor configured to: receiving, for an eye tracking application, a sequence of images of a user's eyes, wherein each image is captured while a stationary stimulus point is displayed to the user and while the user is provided with instructions to maintain their gaze on the stationary stimulus point; For multiple different pairs of images: comparing the pairs of images to each other to determine an image score representing a degree of difference between the compared images; and An image sequence score is calculated based on the image scores of the plurality of pairs of images to determine compliance of the image sequence with the instructions provided to the user to maintain the user's gaze on the stationary stimulus point.

20. A non-transitory computer-readable storage medium for storing instructions that, when executed by a processor, cause a computer to perform the method of any one of claims 1 to 18.

Images