High-performance light pupil eye tracking

Abstract

A method (800) and system for controlling one or more illumination devices in an eye tracker system (100) is described herein to cause a measured pupil / iris contrast to exceed a predetermined minimum pupil / iris contrast. The method (100) includes: a. capturing an image of a subject (102) within a predetermined image capture period, the image including one or both of the subject's eyes; b. illuminating one or both of the subject's eyes from one or more illumination devices (108 and 110) within the predetermined image capture period, at least one of the illumination devices (108 and 110) positioned sufficiently close to a camera lens to create a bright pupil effect; and c. selectively varying the output of at least one of the illumination devices (108 and 110) to create a bright pupil reflex intensity such that the measured pupil / iris contrast in the captured image exceeds the predetermined minimum pupil / iris contrast.

Description

[Technical Field]

[0001] The present invention relates to lighting device control, and in particular to lighting devices for eye trackers.

[0002] Although embodiments of the present invention are specifically configured to illuminate a subject's face to extract facial features such as pupils under bright pupil conditions, it will be appreciated that the present invention is applicable in broader contexts and other applications. [Background technology]

[0003] Many eye tracking systems rely on detecting the contours of the pupil to accurately track a subject's gaze. To detect these contours, there must be enough contrast between the imaged pupil and the surrounding iris to identify the contour.

[0004] When the illuminator is placed far away from the optical axis of the imaging camera, the pupil appears dark and there is a large contrast between the pupil and the surrounding bright iris.

[0005] Conversely, when the illuminator is positioned close to the optical axis of the imaging camera (typically within about 3.25 degrees of the camera's optical axis), the pupil may appear brighter due to retroreflection from the eye. In conventional photography, this manifests as the well-known "red-eye" phenomenon, due to the optical properties of the interior region of the eye. In infrared imaging, where grayscale images are produced, this manifests as a "light pupil" effect, where the pupil appears white or nearly white and is typically brighter than the iris.

[0006] Eye tracking can be performed using cameras and lighting devices under both bright and dark pupil conditions. Under dark pupil conditions, pupil contrast is typically high, making eye tracking more accurate. However, in a typical dashboard-mounted driver monitoring system, achieving dark pupil conditions when operating at a wavelength of 950 nm requires that a single lighting device or multiple lighting devices be placed at least approximately 2 cm away from the camera. This directly translates into larger camera and lighting units. As vehicles become more electronic, available space on the vehicle dashboard becomes increasingly valuable, a trend expected to continue with the advent of semi-autonomous driving.

[0007] Thus, there is a commercial desire to minimize the spatial footprint of driver monitoring systems in vehicles. For this reason, in the future it may be advantageous to implement light pupil eye tracking systems.

[0008] To achieve a bright pupil image, the angular separation between the camera and illuminator must be very small. Such a small separation not only poses significant challenges for lens and illuminator design, but also creates a problematic total internal reflection situation. In particular, under this camera / illuminator geometry, the pupil and iris are imaged at approximately the same brightness, resulting in very low pupil contrast. These situations are called "gray pupil" situations. Any angular separation within the above range will, in some scenarios, cause a gray pupil situation. Without contrast between the pupil and iris, it is impossible to accurately identify the pupil's location.

[0009] Any discussion of background art throughout the specification should in no way be construed as an admission that such art is well known or forms part of the common general knowledge in the art. Summary of the Invention [Problem to be solved by the invention]

[0010] According to a first aspect of the present invention, there is provided a method for controlling one or more illumination devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the method comprising: capturing an image of a subject, including one or both of the subject's eyes, for a predetermined image capture period; illuminating one or both of the subject's eyes from one or more lighting devices during the predetermined image capture period; At least one illuminator is positioned sufficiently close to the camera lens to produce a bright pupil effect. To illuminate and Selectively varying the output of at least one of the illumination devices to produce a bright pupil reflex intensity such that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast.

[0011] In some embodiments, the output of at least one of the lighting devices is selectively varied based on a direct measure of pupil / iris contrast as determined by the intensity of pixels in a pupil region relative to an iris region of one or both of the subject's eyes. i. Ambient light measurements, ii. a measurement of the subject's pupil diameter; and / or iii. The subject's current or recent gaze direction The signal is selectively varied based on one or more of:

[0012] In some embodiments, the ambient light measurement is determined from an exposure setting of the camera and / or a light setting of the one or more lighting devices.

[0013] In some embodiments, the output of at least one of the lighting devices is selectively varied based on a measurement of contrast in the subject's eye pupil from a previous image capture period. In some embodiments, the output of at least one of the lighting devices is selectively varied based on a physiological parameter of the subject. In some embodiments, the output of at least one of the lighting devices is selectively varied between at least two different power levels within an image capture period.

[0014] In some embodiments, the camera captures at least two images within an image capture period, the two images being captured using different lighting or image capture settings, where a first image is captured while one or more lighting devices have a first output level and a second image is captured while one or more lighting devices have a second output level that is different from the first output level.

[0015] In one embodiment, the method includes performing image subtraction on the two images to produce a resultant image with increased pupil contrast.

[0016] In some embodiments, the eye tracker includes a single illuminator. In some embodiments, a controller is configured to modulate the illumination power of the illuminator during an image capture period. In some embodiments, the eye tracker includes two illuminators positioned at different distances from the camera. Preferably, each illuminator is positioned close enough to the camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics.

[0017] In some embodiments, selectively varying the output of at least one of the lighting devices includes deactivating one of the two lighting devices within an image capture period.

[0018] According to a second aspect of the present invention, there is provided a method of controlling two or more illuminators in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the method comprising: capturing an image of the subject within a predetermined image capture period, the image including one or both of the subject's eyes; illuminating one or both of the subject's eyes during the predetermined image capture period from a system of two or more illuminators, each illuminator being positioned close enough to the camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics; and selectively varying the output of the two or more illumination devices to produce a characteristic of a photopic pupil reflex such that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast.

[0019] According to a third aspect of the present invention, there is provided a system for controlling one or more illumination devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the system comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; one or more illumination devices configured to selectively illuminate one or both of the subject's eyes within the predetermined image capture period, at least one of the illumination devices being positioned sufficiently close to the camera lens to produce a bright pupil effect; and a controller configured to selectively vary the output of at least one of the illumination devices to produce a light pupil reflex intensity such that a measured pupil / iris contrast in a captured image exceeds a predetermined minimum pupil / iris contrast.

[0020] In one embodiment, the system includes one illuminator. Preferably, the illuminator is positioned within a distance of 7 mm-15 mm from the camera. More preferably, the illuminator is positioned within a distance of 8 mm-14 mm from the camera. In another embodiment, the system includes two illuminators. Preferably, each illuminator is positioned close enough to the camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics.

[0021] According to a fourth aspect of the present invention, there is provided a system for controlling two or more illumination devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the system comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; two or more illumination devices configured to illuminate one or both of the subject's eyes within the predetermined image capture period, each of the illumination devices positioned sufficiently close to the camera lens to produce a bright pupil effect, but positioned at different distances from the lens to produce different bright pupil reflex characteristics; and a controller configured to selectively vary the output of the two or more illuminators to produce a characteristic of a photopic pupil reflex such that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast.

[0022] In one embodiment, the system includes two illuminators. Preferably, a first illuminator is positioned 3 mm-15 mm from the camera and a second illuminator is positioned 7 mm-50 mm from the camera. More preferably, the first illuminator is positioned 8 mm-13 mm from the camera and a second illuminator is positioned 20 mm-30 mm from the camera.

[0023] In one embodiment, the controller is configured to deactivate one of the lighting devices during an image capture period.

[0024] In some embodiments, the characteristic of the light pupillary response comprises a measure of the retroreflection effect by one or both of the subject's eyes, hi some embodiments, the characteristic of the light pupillary response comprises a direct measure of pupil / iris contrast as determined by the intensity of pixels in the pupil region relative to the iris region of one or both of the subject's eyes.

[0025] According to a fifth aspect of the present invention there is provided an illumination system for an eye tracker, the system comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; one or more illumination devices configured to selectively illuminate one or both of the subject's eyes within the predetermined image capture period, at least one of the illumination devices being positioned sufficiently close to the camera lens to produce a bright pupil effect; A controller, the controller comprising: processing the captured images to measure contrast of the subject's eye pupils in at least a subset of the images; controlling the output of one or more illumination devices based on a controlled signal to produce a light pupil reflex intensity such that the measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast, the control signal being derived based on measurements of pupil contrast from a previous image capture period; a controller configured to Includes:

[0026] According to a fifth aspect of the present invention, there is provided an eye tracking system, the system comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; one or more illumination devices configured to selectively illuminate one or both of the subject's eyes within the predetermined image capture period, at least one of the illumination devices being positioned sufficiently close to the camera lens to produce a bright pupil effect; A controller, the controller comprising: processing the captured image to perform an eye-tracking routine to track the subject's eyes, the eye-tracking routine including determining one or more control parameters; controlling the output of the one or more illuminators based on the one or more control parameters to generate a light pupil reflex intensity such that the measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast; a controller configured to Includes.

[0027] In one embodiment, the control parameters are: i. Ambient light measurements, ii. a measurement of the subject's pupil diameter; and / or iii. The subject's current or recent gaze direction Includes.

[0028] In some embodiments, the control parameter includes a physiological parameter of the subject.

[0029] In some embodiments, the processor is configured to process the captured images to determine a measure of pupil contrast of the subject's eye, the measure of pupil contrast from the previous image capture period being used as a control parameter for controlling the illumination power of one or more illumination devices. Thus, in any of the described embodiments, the measure of pupil contrast from the previous image capture period can be a parameter or factor based on which the illumination power of at least one of the illumination devices is selectively varied. [Brief explanation of the drawings]

[0030] Exemplary embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings. [Figure 1] FIG. 1 is a perspective view of the interior of a vehicle having a driver monitoring system including a camera and two LED light sources mounted inside the vehicle. [Figure 2] FIG. 2 is a perspective view of a driver's view of a vehicle dashboard having the driver monitoring system of FIG. 1 mounted thereon. [Figure 3] FIG. 3 is a schematic functional diagram of the driver monitoring system according to FIGS. [Figure 4] FIG. 4 is a schematic diagram of an eye illuminated by an infrared illuminator to illustrate the bright pupil effect. [Figure 5] FIG. 5 is a schematic diagram of exemplary bright and dark pupil effect situations and associated images. [Figure 6] FIG. 6 is a plan view of the driver monitoring system of FIGS. 1-3 showing the camera field of view and the LED illumination field of view relative to the subject. [Figure 7] FIG. 7 is a plan view of the driver monitoring system of FIGS. 1-3 showing the illumination and imaging geometry relative to the subject being imaged. [Figure 8] FIG. 8 is a process flow diagram illustrating the major steps in an illumination method for an eye tracker. [Figure 9] FIG. 9 is an exemplary setup of a first embodiment of a camera and a pair of LEDs. [Figure 10] FIG. 10 is a perspective view of a driver's view of a vehicle dashboard having a driver monitoring system with a single LED mounted on the vehicle dashboard. [Figure 11] FIG. 11 is a schematic functional diagram of the driver monitoring system of FIGS. 1-3 showing the data flow between the different system elements. [Figure 12] FIG. 12 illustrates an example situation where illumination from the far-side LED is preferred. [Figure 13] FIG. 13 illustrates an example situation where illumination from the near-side LED is preferred. [Figure 14] FIG. 14 shows a graph of the detected contrast of the iris and pupil as a function of the infrared light source and the visual angle of the camera for four different pupil sizes. DETAILED DESCRIPTION OF THE INVENTION

[0031] The lighting systems and methods described herein can be applied and used in numerous eye tracking environments. One example is monitoring drivers or passengers of automobiles or other vehicles, such as buses, trains, or airplanes. In addition, the described systems can be applied to operators using or operating any other equipment, such as machinery and flight simulators. For ease of understanding, embodiments of the present invention are described herein in the context of a driver monitoring system for a vehicle. Furthermore, while the lighting devices are described as being light-emitting diodes (LEDs), it will be understood that the present invention can be applied to other types of light sources, such as vertical-cavity surface-emitting lasers (VCSELs). [System Overview]

[0032] 1-3, a driver monitoring system 100 is illustrated for capturing images of a vehicle driver 102 operating a vehicle 104. The system 100 is further configured to perform various image processing algorithms on the captured images, such as face detection, facial feature detection, facial recognition, facial feature recognition, face tracking, or facial feature tracking, such as tracking a person's eyes. Exemplary image processing parameters are described in U.S. Patent No. 7,043,056, entitled "Facial Image Processing System," by Edwards et al. and assigned to Seeing Machines Pty Ltd (hereinafter "Edwards et al."), the contents of which are incorporated herein by cross-reference.

[0033] As best illustrated in FIG. 2, the system 100 includes an imaging camera 106 positioned on or within an instrument display of a vehicle dashboard 107 and oriented to capture images of the driver's face in the infrared wavelength range for identifying, locating, and tracking one or more human facial features.

[0034] Camera 106 may be a conventional CCD or CMOS-based digital camera having a two-dimensional array of light-sensitive pixels (or photodetectors) and, optionally, distance or depth-determining capabilities (such as through one or more phase-detection elements). The light-sensitive pixels are capable of sensing electromagnetic radiation in at least the infrared range. Camera 106 may also be a three-dimensional camera, such as a time-of-flight (TOF) camera, or other scanning or range-based camera capable of imaging a scene in three dimensions. In other embodiments, camera 106 may be replaced by a pair of similar cameras operating in a stereo configuration and calibrated to extract depth information of objects in images captured by camera 106. Camera 106 is preferably configured to image in the infrared wavelength range, although it will be understood that in alternative embodiments, camera 106 may image in the visible range. Although not shown, camera 106 also includes imaging optics, including a first imaging lens, for focusing light onto the array of light-sensitive pixels.

[0035] 2, in the first embodiment, system 100 also includes a pair of infrared illuminators in the form of light emitting diode arrays (LED arrays) 108 and 110, which are positioned at horizontally spaced locations proximate to the camera on the vehicle's dashboard 107. As described below, LEDs 108 and 110 are positioned at different distances from camera 106. In other embodiments described below, only a single LED is used. Also, in some embodiments, more than two light sources may be employed in the system.

[0036] To improve imaging of the driver's face so as to obtain high-quality images of the driver's face or facial features, LEDs 108 and 110 are configured to illuminate driver 102 with infrared radiation during predefined image capture periods when camera 106 is capturing images. Operation of camera 106 and LEDs 108 and 110 in the infrared range reduces visual distraction for the driver. Operation of camera 106 and LEDs 108 and 110 is controlled by an associated controller 112 that includes a computer processor or microprocessor and memory for storing and buffering captured images from camera 106. In other embodiments, different types of light sources, such as VCSELs, may be used instead of LEDs.

[0037] As best illustrated in FIG. 2 , the camera 106 and the LEDs 108 and 110 may be manufactured or assembled as a single unit 111 having a common housing. This unit 111 is shown mounted to the vehicle's dashboard 107, but may be installed during vehicle manufacture or later installed as an aftermarket product. In other embodiments, the driver monitoring system 100 may include one or more cameras and light sources positioned in any location suitable for capturing images of the head or facial features of the driver, subject, and / or passenger within the vehicle. By way of example, the cameras and LEDs may be positioned on the vehicle's steering column, rearview mirror, center console, or driver's side front pillar. In the illustrated embodiment, the first and second light sources each include a single LED. In other embodiments, each light source may include multiple individual LEDs.

[0038] Turning now to FIG. 3 , functional elements of system 100 are illustrated schematically. System controller 112 serves as the central processing unit for system 100 and is configured to perform several functions described below. Controller 112 may be located within dashboard 107 of vehicle 104 and may be coupled to or integrated with the vehicle's on-board computer. In another embodiment, controller 112 may be located within a housing or module along with camera 106 and LEDs 108 and 110. This housing or module may be sold as an aftermarket product, installed in the vehicle's dashboard, and subsequently calibrated for use in that vehicle. In further embodiments, such as a flight simulator, controller 112 may be an external computer or unit, such as a personal computer.

[0039] The controller 112 may be implemented in any form of computer processing device or portion of such a device that processes electronic data, for example from registers and / or memory, and converts the electronic data into other electronic data that may be stored, for example, in registers and / or memory. As shown in Figure 3, the controller 112 includes a microprocessor 114 that executes code stored in memory 116, such as random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), and other equivalent memory or storage systems that will be readily apparent to those skilled in the art.

[0040] The microprocessor 114 of the controller 112 includes a vision processor 118 and a device controller 120. The vision processor 118 and the device controller 120 both represent functional elements performed by the microprocessor 114. However, it will be appreciated that in alternative embodiments, the vision processor 118 and the device controller 120 may be implemented as separate hardware, such as a microprocessor used in conjunction with custom or specialized circuitry.

[0041] Vision processor 118 is configured to process the captured images to perform driver monitoring. For example, driver monitoring may involve determining the three-dimensional head pose and / or gaze position of driver 102 within a monitored environment. To accomplish this, vision processor 118 utilizes one or more gaze determination algorithms, which may include, by way of example, the method described in Edwards et al. Vision processor 118 may also perform various other functions, including determining attributes of driver 102, such as eye closure and blink rate, and tracking the driver's head movements, to detect driver attention, drowsiness, or other issues that may interfere with the driver's safe operation of the vehicle.

[0042] The raw image data, gaze position data, and other data acquired by the vision processor 118 are stored in the memory 116 .

[0043] Device controller 120 is configured to control various parameters of camera 106, such as shutter speed and / or image sensor exposure / integration time, and to selectively activate LEDs 108 and 110 in a manner described below, synchronized with the exposure time of camera 106, or more generally within a predefined image capture period. LEDs 108 and 110 are preferably electrically coupled to device controller 120, although they may also be controlled wirelessly by controller 120 through wireless communication, such as Bluetooth™ or WiFi™ communication.

[0044] Thus, during operation of the vehicle 104, the device controller 120 activates the camera 106 to capture images of the driver 102's face in the form of a video sequence. The LEDs 108 and 110 are activated and deactivated synchronously with the image frames captured by the camera 106 to illuminate the driver for a predefined image capture period. The device controller 120 and the vision processor 118 cooperate to capture and process images of the driver to obtain information about the driver's state, such as drowsiness, attention, and gaze position, during normal operation of the vehicle 104. The device controller 120 and the vision processor 118 also cooperate to perform dynamic lighting control, as described below.

[0045] Additional components of the system may also be included in a common housing of unit 111 or may be provided as separate components according to other additional embodiments. In one embodiment, the operation of controller 112 is performed by an on-board vehicle computer system that is coupled to camera 106 and LEDs 108 and 110.

[0046] Throughout this specification, specific functions performed by the vision processor 118 or the device controller 120 may be more broadly described as being performed by the controller 112. [Bright pupil condition]

[0047] 4, a schematic diagram of the bright pupil condition is shown. With reference to this figure, the concept of the bright pupil condition will be described.

[0048] When a point light source, such as an infrared LED 400, is used to illuminate the eye 402, the lens 404 of the eye 402 focuses the point light source into an image 406 on the surface of the retina 408. Because the lens 404 is an imperfect lens, the image 406 becomes a blurred disk on the retina 408. Some light is diffusely reflected from the retina 408, and a subset of this light is incident on the lens 404. A subset of this reflected light is focused by the lens 404 back toward the light source. Again, because the lens 404 is imperfect, the reflected light is imaged as a disk of finite diameter around the LED 400, rather than as an ideal point. The disk represents a region 410 where the captured image will exhibit a bright pupil effect.

[0049] Because only light that passes through the eye's pupil 412 can enter the interior of the eye 402, the bright pupil region defined by region 410 decreases as the pupil diameter decreases and brightness decreases. As the eye rotates to view different regions in space (different gaze angles), the position, shape, and size of image 406 on retina 408 change. As a result, the size and brightness of region 410 change with gaze angle.

[0050] The bright pupil effect is Pupil dilation (size), - Viewing angle relative to the camera, the age and ethnicity of the subject, and Wavelength of light is influenced by many factors, including

[0051] Under the bright pupil effect, the pupil appears brighter than the surrounding iris in a grayscale image. When imaged by a camera, the iris appears as a dark area surrounding the very bright pupil. This is illustrated in the left portion of Figure 5, which also shows example angles at which the eye is illuminated. The right portion of Figure 5 illustrates a more common dark pupil condition, where the pupil is darker than the surrounding iris.

[0052] Given the physiology of the human eye, the bright pupil effect typically appears when the imaging camera is positioned at an angle from the light source relative to the eye that is less than 3.25 degrees, although it will be appreciated that the particular angle at which the bright pupil effect appears will vary with the above and other factors.

[0053] 6 and 7, plan views of the driver monitoring system 100 are shown. FIG. 6 illustrates the illumination areas of the LEDs 108 and 110 and the field of view of the camera 106. FIG. 7 illustrates the geometric angle between the camera 106 and the LEDs 108 and 110, including the camera optical axis. Under normal driving conditions, the camera 106 and the LEDs 108 and 110 are typically positioned between 30 cm and 80 cm from the face of the driver 102. To achieve a bright pupil condition (the LEDs 108 and 110 illuminate the driver 102 at an angle less than 3.25 degrees (θ1 and θ2 in FIG. 7) from the optical axis of the camera 106), the LEDs 108 and 110 should be positioned within approximately 30 mm of the camera 106. However, the outer LEDs 110 may be positioned up to approximately 50 mm from the lens of the camera 106.

[0054] For a given angular separation between the camera and illuminator, the larger the pupil size, the brighter the pupil. Similarly, for a given camera / illuminator angle, a subject looking directly at the camera will have a darker pupil than when the subject looks off-axis.

[0055] When there is significant ambient illumination, the contrast between the bright pupil and the iris is determined by the pupil luminance and the iris luminance. The pupil luminance is almost entirely due to the bright pupil effect resulting from the controlled illumination from the LED alone. Ambient light does not significantly affect the bright pupil intensity. The iris luminance is due to a combination of the controlled illumination and the ambient illumination.

[0056] It will therefore be appreciated that in other monitoring environments and under different conditions, the LEDs 108 and 110 may be positioned at a greater distance from the camera 106 while still achieving a bright pupil condition.

[0057] As mentioned above, there exists a situation called the "gray pupil" situation in which the iris and pupil are imaged at approximately the same intensity, resulting in very low pupil contrast. Because gray pupils typically occur only when the pupil size is small (when pupil luminance is relatively low), gray pupils typically occur only under bright visible ambient conditions (when iris luminance is generally high).

[0058] Gray pupil situation: Changing the intensity of the bright pupil effect (e.g., by moving the illumination source closer to or further away from the imaging camera lens), Adjusting the intensity of controlled lighting, and Adjusting the intensity of ambient light can be reduced by

[0059] It is also possible to vary the pupil / iris contrast by changing the exposure period of the imaging camera, such as by varying the sensor's integration time (shutter period). However, the inventors have found that dynamically controlling illumination power is advantageous because it does not increase complexity or require changes in the exposure control loop of the device controller 120's control algorithm. Dynamically controlling illumination power is also advantageous for improving the signal-to-noise ratio in object tracking under poor light conditions, as described in PCT Patent Application Publication No. WO 2019 / 084595, entitled "System and Method for Improving Signal to Noise Ratio in Object Tracking Under Poor Light Conditions" by Noble and assigned to Seeing Machines Pty Ltd (hereinafter "Noble"). Dynamic illumination control may also be performed in conjunction with dynamic exposure control to improve system performance under certain conditions, at the expense of increased complexity of the control algorithm. [Lighting Control]

[0060] Referring now to FIG. 8 , a process flow diagram illustrates major steps in an illumination method 800 for ensuring that a minimum pupil / iris contrast between the pupil and iris of an eye is maintained in an eye tracker system such as system 100. Method 800 includes, in step 801, configuring camera 106 to capture an image of subject 102 for a predefined image capture period. The predefined image capture period may represent a normal shutter period for camera 106 operating at a normal frame rate. However, camera 106 may be configured to operate in a higher frame rate mode in which multiple integrations of the image sensor occur during a single shutter period. Similarly, some cameras are configured to record multiple images during a single shutter period using multiple image sensors while simultaneously storing image data. Here, the predefined image capture period may represent multiple periods of sensor integration. Thus, multiple images may be captured within a single image capture period. The term “image capture period” is intended to cover all variations of image capture, whether at the normal frame rate or a higher frame rate of the camera 106’s sensor integration mode.

[0061] The captured image includes one or both of the subject's eyes. In some eye tracker systems, eye tracking for each eye is performed independently, while in other eye tracker systems, both eyes are tracked simultaneously. When both eyes cannot be tracked in an image frame, only one eye may be tracked. When neither eye can be tracked (e.g., when the subject's eyes are closed), that image frame may be discarded for eye tracking purposes, and a subsequent image may be loaded.

[0062] In step 802, one or both of the subject's eyes are illuminated during a predefined image capture period by one or both of LEDs 108 and 110. In practice, steps 801 and 802 are performed synchronously with one another so that illumination occurs during the predefined image capture period.

[0063] In step 803, controller 112 selectively varies the output of at least one of LEDs 108 and 110 to produce a light pupil reflex intensity such that the measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast. As part of this step, vision processor 118 may calculate pupil / iris contrast from a current or past image. This contrast calculation is described in detail below.

[0064] As described in more detail below, the present invention includes different embodiments having different combinations of LED clusters (or other lighting devices). In a first embodiment, illustrated in Figures 1-3, 6, and 7, two LEDs (108 and 110) are provided, each positioned close enough to the lens of camera 106 to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics. By way of example, the configuration of this first embodiment is illustrated schematically in Figure 9, where LED 108 is positioned 9 mm from the lens of camera 106 and LED 110 is positioned 50.3 mm from the lens of camera 106.

[0065] It will be appreciated that other placements of the LEDs 108 and 110 are possible to create a bright pupil effect and produce different bright pupil reflex characteristics. By way of example, the near LED 108 may be positioned within a range of approximately 3 mm-15 mm from the lens of the camera 106, and the far LED 110 may be positioned within a range of approximately 7 mm-50 mm from the lens of the camera 106. In some embodiments, the near LED 108 is preferably positioned within a range of approximately 8 mm-13 mm from the lens of the camera 106, and the far LED 110 is preferably positioned within a range of approximately 20 mm-30 mm from the lens of the camera 106.

[0066] It will also be appreciated that the optimal separation distance between camera 106 and LEDs 108 and 112 will depend on factors such as the physical limitations of the camera lens and LED / secondary optics, and the presence and design of a cover glass (to avoid excessive internal reflections).

[0067] In a second embodiment of the system 200, shown in FIG. 10 , only a single LED 108 is used to illuminate one or both eyes of a subject. The single LED 108 is positioned close enough to the lens of the camera 106 to create a bright pupil effect. In both embodiments, the LEDs are dynamically driven by the controller 112 to selectively vary their output to create a bright pupil reflex intensity such that the measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast. Achieving a minimum pupil / iris contrast improves the performance of the eye tracking system, allowing it to properly detect and track the gaze of the subject being imaged.

[0068] Both of these embodiments are described below. It will be understood that each embodiment is simply concerned with controlling an LED group (or more generally, an illumination device) that is placed close enough to the camera lens to induce a bright pupil effect. This differs from other illumination systems, where the light source may be placed farther away from the camera lens and therefore operate in dark pupil conditions well off the camera's optical axis. [Embodiment 1 - Lighting with two LEDs]

[0069] Referring again to FIG. 9 , each illuminator is positioned close enough to the lens of the camera 106 to produce a bright pupil effect (as described above), but at different distances from the lens to produce different bright pupil reflex characteristics. The bright pupil characteristic includes a measurement of the magnitude of the retroreflection effect experienced by the illuminated eye. The magnitude of the retroreflection effect can be quantified by measuring pixel values ​​of the pupil and iris at different illumination positions for a given LED power. In some embodiments, this can be measured as a measurement of the luminance or pixel intensity of one or more pixels in the iris region and one or more pixels in the pupil region in the captured image. These luminance measurements are performed by the vision processor 118 during image processing of raw image data from the camera 106.

[0070] The luminance measurements are expressed as grayscale luminance measurements, typically in the format of the captured image (e.g., 16-bit). The luminance measurements may include average luminance values ​​of multiple pixels determined to be located in the iris and pupil of the image, respectively. For example, in a dark pupil situation, the raw 16-bit pupil luminance may be 3,540 and the iris luminance may be 18,050. In a light pupil situation, the raw 16-bit pupil luminance may be 32,501 and the iris luminance may be 3,460. The determination of the different regions of the eye is performed by the vision processor 118 using known image processing techniques such as edge detection, shape recognition, and contour detection.

[0071] Determination of the iris and pupil regions may be performed as part of a more general eye tracking algorithm, such as the algorithm described in Edwards et al., but typically these regions may be identified using image processing techniques such as edge detection, shape recognition, and the Hough transform. Thus, within an image, the iris and pupil regions may be defined as two-dimensional regions or groups of regions made up of individual pixels.

[0072] A measurement of pupil / iris contrast may be performed by comparing the luminance values ​​of one or more pixels within a defined pupil region with the luminance values ​​of one or more pixels within a defined iris region. The pixel samples may be a single pixel from each region, a group of pixels distributed around each region, or the average pixel value of some or all of the pixels within each region.

[0073] 11, a variation of FIG. 3 is illustrated showing various data communicated between different elements of the system 100. Here, the vision processor 118 is shown receiving raw image data from the camera 106 and transmitting various data to the device controller 120. For simplicity, the memory 116 is not shown in FIG. 11, however, data communication between the camera 106, the vision processor 118, and the device controller 120 may include storing and retrieving data from the memory 116.

[0074] During the illumination control step 803 of method 800, controller 112 controls the output of one or both of LEDs 108 and 110, respectively, based on the generated control signals 130 and 132. Control signals 130 and 132 are determined by device controller 120 and include current and / or voltage signals that drive the corresponding LEDs to generate infrared radiation. Typically, the LEDs are controlled using a pulsed output having a defined peak power and pulse width. In system 100, the pulse width is set by device controller 120 to substantially coincide with the exposure time of camera 106.

[0075] Modification of the output by device controller 120 covers modifications such as changing the pulse peak value, pulse waveform, or pulse width. In some embodiments, the LEDs are controlled such that the energy content of the pulse is varied by device controller 120. In other embodiments, the energy content of the pulse remains constant, but the pulse peak value and / or pulse waveform is varied. By way of example, the LEDs may be dynamically controlled based on a pulse processing curve, as described in the above-referenced Noble publication.

[0076] The control signals required by the device controller 120 may be based on one or more of the following inputs: Ambient light measurements Ambient light measurements may be obtained from an external ambient light sensor 150 and provided to the device controller 120. Alternatively, ambient light measurements may be estimated by the vision processor 118 from captured images through comparison of background features across many images. The vision processor 118 may implement an algorithm that extracts ambient light measurements from captured images by considering object reflectivity, object distance, and the amount of controlled lighting. If the object being measured is fixed relative to the camera (such as an object in a vehicle interior), the distance factor remains constant. In some embodiments, a proxy measurement of ambient light is derived from the exposure settings and / or LED bank illumination settings of the camera 106. Because cameras have built-in hardware and software configured to control image exposure times based on lighting conditions, using these exposure settings may be used as a proxy measurement of ambient light. For example, in scenes with low ambient light conditions, the camera 106 automatically detects light levels via the image sensor and sets a longer image exposure time. Subject's pupil diameter measurement This is measured by vision processor 118 from a previous image where pupil diameter is discernible. A larger diameter pupil indicates less ambient light, while a smaller pupil indicates more ambient light. The pupil diameter measurement may also be used as a surrogate measurement of the ambient light present. Current or recent gaze direction of the subject The complex geometry of the human eye means that different areas of the eye receiving light have different reflective properties. When the eye is looking directly toward the light source, a strong retroreflection occurs, resulting in a bright pupil effect. However, at different gaze angles, such as away from the light source, different reflective properties are present, and the bright pupil effect may or may not be present. Furthermore, because the geometry of the eye varies from person to person, these properties also vary from person to person. Thus, the current or recent gaze direction measured by driver monitoring system 100 can be used as an input to control LEDs 108 and 110. Subject's physiological parameters Human individuals of different ages and ethnicities have different eye geometries, resulting in variations in eye size. In particular, the size of a human lens and the amount of pupil dilation can vary from person to person. This results in different light and dark pupil responses for a given optical system. Exemplary physiological parameters that vary from person to person include the size, shape, and reflectivity of the fundus or lens, and the shape and response of the pupil. In some embodiments, these parameters can be input to device controller 120 and considered in determining appropriate control signals 130 and 132. In some embodiments, these physiological parameters are measured directly. In other embodiments, the physiological parameters are fitted by optimizing the parameters based on training data and the like. Direct measurement of pupil contrast In one embodiment, the vision processor 118 is configured to process the captured image to determine a measure of pupil contrast of the subject's eye. This measure of pupil contrast, which may be obtained from a current or past captured image, may form an input to the device controller 120 to control the output of the LEDs 108 and 110. In one embodiment, as described above, the measure of pupil contrast is determined by comparing the luminance values ​​of one or more pixels in a defined pupil region with the luminance values ​​of one or more pixels in a defined iris region. In other embodiments, pupil contrast may be derived by other techniques, such as determining the slope of pixel values ​​from the iris region to the pupil region.

[0077] The device controller 120 includes one or more algorithms that generate desired control signals 130 and 132 for the LEDs 108 and 110 based on one or more of the above inputs. The specific voltage or current values ​​of the control signals may be based on a combination of the above measured inputs determined by the control algorithm. In some embodiments, the controller 120 operates specific control algorithms that set the desired voltage and / or current for the LEDs 108 and 110 based on the measured inputs. In some embodiments, each input is weighted based on its importance.

[0078] Typically, in dark environments (low ambient light), the near-side LED 108 may be activated and the far-side LED 110 may be deactivated. For enlarged pupils, the upper limit of the angular separation between the camera and the LEDs may be larger, which mitigates the problem of internal reflections. In situations where the pupil size is small but the subject is looking off-axis, or where the pupil size is large enough, it is preferable to activate the near-side LED 108 to enhance pupil / iris contrast.

[0079] Typically, in bright environments (high ambient light), the far-side LED 110 may be activated and the near-side LED 108 may be deactivated. For constricted pupils, the lower limit of the angular separation between the camera 106 and the LEDs 108 and 110 may be smaller, meaning a smaller size camera assembly. In situations where the subject is looking at the camera 106 or the pupil size is small enough, the far-side LED 110 may also be activated.

[0080] Figure 12 illustrates an example situation in which illumination from the far-side LED 110 is preferred, and Figure 13 illustrates an example situation in which illumination from the near-side LED 108 is preferred. Note that there is some overlap between Figures 12 and 13. This means that in some situations, either the near-side LED 108 or the far-side LED 110 may be activated. For a given LED position and pupil size, if the subject looks directly at the camera, the pupil will be darker than when the subject looks in other directions.

[0081] Thus, in step 803, the device controller may implement the control signals 130 and 132 such that the following exemplary lighting conditions are provided: LED 108 is activated and LED 110 is deactivated. LED 110 is activated and LED 108 is deactivated. The power of LED 108 is increased and LED 110 is deactivated. The power of LED 110 is increased and LED 108 is deactivated. The power of LED 108 is increased and LED 110 is decreased. The output of LED 110 is increased and LED 108 is decreased. The power of LED 108 is reduced and LED 110 is deactivated. The power of LED 110 is reduced and LED 108 is deactivated.

[0082] It will be understood that the above lighting conditions are merely exemplary and not an exhaustive list of possible conditions. The amount by which each LED is increased or decreased may be based on a combination of the above inputs. In some embodiments, each LED can be driven at one of several predefined voltage or current levels based on the specific values ​​or ranges of the detected inputs. In some embodiments, device controller 120 varies the illumination power of one or more lighting devices to at least two different power levels within an image capture period.

[0083] It will be appreciated that in other embodiments, the system 100 includes more than two groups of LEDs arranged to create a bright pupil effect. [Embodiment 2 - Single LED]

[0084] 10, a system 200 represents a second embodiment in which only the near-side LED 108 is implemented. In this embodiment, the LED 108 is preferably, but not necessarily, positioned 3 mm-15 mm from the lens of the camera 106.

[0085] This single LED embodiment relies on the fact that the presence of the gray pupil effect can be mitigated without changing the angle of illumination by simply dynamically adjusting the intensity of the controlled illumination of the LED until the ratio of ambient light to controlled light restores pupil / iris contrast to a minimal level, where controlled light represents the amount of light generated in a controlled manner from the LED 108 and ambient light represents all other light imaged by the photosensor array of the camera 106.

[0086] Dynamic control of LED 108 (and also LED 110 in the first embodiment) may be performed by device controller 120 based on the following understanding. i. In low ambient light conditions, the pupil is large but decreases in size as the ambient light increases. ii. The iris brightens with increasing ambient light, but the pupil does not. iii. The controlled illumination by LED 108 (and / or LED 110) is reduced by the camera's automatic exposure control algorithm executed by device controller 120, which reduces the luminance of the pupil and the iris by the same proportion. That is, the luminance of the pupil is reduced by a larger absolute amount because it starts at a higher value. This reduces the absolute contrast between the pupil and the iris.

[0087] The combination of the above three effects is that under bright pupil conditions, pupil size decreases, which reduces the bright pupil effect (however, in very low ambient light environments, a usable bright pupil effect still exists down to small pupil diameters). At the same time, the combination of increased ambient light and adjustment of the LED output control algorithm reduces pupil luminance and increases iris luminance. All these effects combined act to significantly reduce the bright pupil effect and cause the gray pupil effect to appear at smaller illumination angles.

[0088] Thus, LED output control algorithms should consider variations in pupil size and iris luminance as well as ambient conditions. Furthermore, there is likely to be no LED configuration that can successfully produce a light pupil across all ambient conditions and resulting pupil sizes and controlled illumination levels.

[0089] 14, a graph of detected pupil / iris contrast as a function of the angle between the LED and the camera lens is shown for four different pupil sizes. The graph illustrates the relationship between pupil size, pupil / iris contrast, and LED / camera separation for a given distance. These relationships allow for estimation of ambient light levels and a set of rules to be constructed to control the LED(s) to achieve the desired iris / pupil contrast.

[0090] In some embodiments, the camera 106 is configured to capture multiple images per image capture period. In these embodiments, the device controller 120 may control the LEDs 108 to capture at least two images within an image capture period under different lighting and / or image capture conditions, e.g., by adjusting the output power of the LEDs 108. For example, one image may be captured while the LEDs 108 are driven at a first output level, and a second image may be captured while the LEDs 108 are driven at a second output level different from the first output level. This results in two simultaneous or closely spaced images with different controlled light levels but a common ambient light level. In other embodiments, image capture settings, such as exposure time or sensor gain, may be changed between images.

[0091] Vision processor 118 may then perform image subtraction on the two images to produce a resultant image with increased pupil contrast. In this image subtraction process, pixel values ​​of corresponding pixels in the two images are subtracted to remove ambient light components and enhance the bright pupil effect.

[0092] A similar image subtraction process can be performed with one or both of the LEDs 108 and 110 modulated at different power levels as for the first embodiment.

[0093] The invention described above can provide efficient eye tracking using pupil / iris contrast with performance comparable to standard eye tracking systems operating in dark pupil mode, but with a 50 percent reduction in package size (for the first embodiment). This miniaturization is advantageous in modern vehicles where space on the dashboard instrument panel is at a premium. For the second embodiment (single LED), the package size can be further reduced at the expense of reduced performance in situations with small pupil sizes. [interpretation]

[0094] The term "infrared" is used throughout this description and specification. Within the scope of this specification, infrared refers to the general infrared region of the electromagnetic spectrum, including near-infrared, infrared, and far-infrared frequencies or light waves.

[0095] Unless specifically stated otherwise, as will be apparent from the discussion that follows, it will be understood that throughout this specification, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "analyzing," or the like, refer to the operations and / or processing of a computer or computing system or similar electronic computing device that manipulates and / or transforms data expressed as physical quantities, such as electronic quantities, into other data similarly expressed as physical quantities.

[0096] Similarly, the terms "controller" or "processor" may refer to any device or part thereof that processes electronic data from, for example, registers and / or memory and converts the electronic data into other electronic data that may be stored, for example, in registers and / or memory. A "computer" or "computing machine" or "computing platform" may include one or more processors.

[0097] The references throughout this specification to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "one embodiment," "some embodiments," or "an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments, as would be apparent to one of ordinary skill in the art from this disclosure.

[0098] As used herein, unless specifically stated otherwise, the ordinal adjectives "first," "second," "third," etc., used to describe a common object merely indicate that different instances of a similar object are being referred to and are not intended to imply that the objects so described must be present in the given order in time, space, hierarchy, or in any other respect.

[0099] In the following claims and in the description herein, the words comprise, comprised, or comprising are all open terms, meaning the inclusion of at least the element(s) / feature(s) that follow, but not the exclusion thereof. Thus, when used in a claim, the word comprise should not be interpreted as being limited to the means or elements or steps listed thereafter. For example, the scope of the expression "device comprising A and B" should not be limited to a device consisting only of elements A and B. As used herein, the word comprise is also open terms, meaning the inclusion of at least the element(s) / feature(s) that follow, but not the exclusion thereof. Thus, comprise is synonymous with comprise, meaning comprising.

[0100] In the above description of exemplary embodiments of the present disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof to streamline the disclosure and facilitate understanding of one or more various inventive aspects. However, this method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present disclosure.

[0101] Furthermore, as will be understood by those skilled in the art, although some embodiments described herein may include some features but not other features included in other embodiments, combinations of features of different embodiments are also intended to be within the scope of the present disclosure and to form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

[0102] In the description provided herein, many specific details are set forth. However, it will be understood that embodiments of the present disclosure may be practiced without these specific details. In other words, well-known methods, structures, and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0103] Similarly, it should be noted that the term coupled, when used in the claims, should not be interpreted as being limited to direct connections. The terms "coupled" and "connected," along with their derivatives, may be used. These terms are not intended as synonyms for each other. Thus, the scope of the expression device A coupled to device B should not be limited to devices or systems in which the output of device A is directly connected to the input of device B. It means that a path exists between the output of A and the input of B, and this may be a path that includes other devices or means. "Coupled" can mean that two or more elements are in either direct physical, electrical, or optical contact, or that two or more elements are not in direct contact but still cooperate or interact with each other.

[0104] The embodiments described herein are intended to cover any adaptations or variations of the present invention. While the invention has been described and illustrated in terms of specific exemplary embodiments, those skilled in the art will recognize that further embodiments can be readily envisioned that are within the scope of this invention.

Claims

1. 1. A method of controlling one or more lighting devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the method comprising: capturing an image of a subject, including one or both of the subject's eyes, for a predetermined image capture period; illuminating one or both of the subject's eyes from one or more lighting devices during the predetermined image capture period; At least one illuminator is positioned sufficiently close to the camera lens to produce a bright pupil effect. To illuminate and selectively varying the output of at least one of the illuminators to produce a bright pupil reflex intensity and vary the ratio of ambient light to light from the illuminators such that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast; A method comprising:

2. The output of at least one of the lighting devices is selectively varied based on a direct measure of pupil / iris contrast as determined by the intensity of pixels in a pupil region relative to an iris region of one or both of the subject's eyes. The method of claim 1.

3. The output of at least one of the lighting devices is: i. the ambient light measurement; ii. A measurement of the subject's pupil diameter; and / or iii) The current or most recent gaze direction of the subject is selectively varied based on one or more of The method of claim 1.

4. The ambient light measurement is determined from the exposure setting of the camera and / or the light setting of the one or more lighting devices. The method of claim 3.

5. The output of at least one of the lighting devices is selectively varied based on a physiological parameter of the subject. The method according to any one of claims 1 to 4.

6. The output of at least one of the lighting devices is selectively varied between at least two different power levels within an image capture period. The method according to any one of claims 1 to 5.

7. The camera captures at least two images within an image capture period, the two images being captured using different lighting or image capture settings. The method of claim 6.

8. performing image subtraction on the two images to produce a resultant image having increased pupil contrast. The method of claim 7.

9. The eye tracker includes a single lighting device. The method according to any one of claims 1 to 8.

10. A controller is configured to modulate the illumination power of the illumination device during an image capture period. The method according to any one of claims 1 to 9.

11. The method of any one of claims 1 to 8, wherein the eye tracker comprises two lighting devices positioned at different distances from the camera.

12. Selectively varying the output of at least one of the lighting devices includes deactivating one of the two lighting devices within an image capture period. The method of claim 11.

13. Each illuminator is positioned close enough to the camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics.

13. The method of claim 11 or 12.

14. 1. A system for controlling one or more lighting devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the system comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; one or more illumination devices configured to selectively illuminate one or both of the subject's eyes within the predetermined image capture period, at least one of the illumination devices being positioned sufficiently close to the camera lens to produce a bright pupil effect; a controller configured to selectively vary an output of at least one of the illuminators to produce a bright pupil reflex intensity and vary a ratio of ambient light to light from the illuminators such that a measured pupil / iris contrast in a captured image exceeds a predetermined minimum pupil / iris contrast; A system including:

15. Includes one lighting device The system of claim 14.

16. The system includes two lighting devices; Each of the illuminators is positioned sufficiently close to the camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics. The system of claim 14.

17. The first illuminator is positioned at a distance of 3 mm to 15 mm from the camera, and the second illuminator is positioned at a distance of 7 mm to 50 mm from the camera.

17. The system of claim 16.

18. The controller is configured to deactivate one of the lighting devices during an image capture period.

18. A system according to claim 16 or 17.

19. The characteristics of the light pupillary response include a direct measure of pupil / iris contrast, determined by the intensity of pixels in the pupil region relative to the iris region of one or both of the subject's eyes. A system according to any one of claims 16 to 18.

20. 1. An eye tracking system, comprising: a camera configured to capture an image of a subject within a predetermined image capture period, the image including one or both of the subject's eyes; one or more illumination devices configured to illuminate one or both of the subject's eyes within the predetermined image capture period, at least one of the illumination devices being positioned sufficiently close to the camera lens to create a bright pupil effect; A controller, the controller comprising: executing an eye tracking routine for tracking the eyes of the subject by processing the captured image, the eye tracking routine including determining one or more control parameters; controlling the output of the one or more illumination devices based on the one or more control parameters to generate a bright pupil reflex intensity and to vary the ratio of ambient light to light from the illumination devices so that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast; a controller configured to An eye tracking system comprising:

21. 1. A method of controlling two or more lighting devices in an eye tracker so that a measured pupil / iris contrast exceeds a predetermined minimum pupil / iris contrast, the method comprising: capturing an image of the subject within a predetermined image capture period, the image including one or both of the subject's eyes; illuminating one or both of the subject's eyes during the predetermined image capture period from a system of two or more illuminators, each of which is positioned close enough to a camera lens to produce a bright pupil effect, but at different distances from the lens to produce different bright pupil reflex characteristics; selectively varying the output of the two or more illuminators to produce a bright pupil reflex intensity and vary the ratio of ambient light to light from the illuminators so that a measured pupil / iris contrast in the captured image exceeds a predetermined minimum pupil / iris contrast; A method comprising:

22. The output of at least one of the lighting devices is selectively varied based on a measurement of contrast in the subject's eye pupil from a previous image capture period. The method according to any one of claims 1 to 13.

Images