Device for detecting a viewing direction, system for detecting a viewing direction, display device and method for detecting a viewing direction

Abstract

A device (100) for detecting a viewing direction comprises a sensor apparatus (105), which comprises a laser diode (110) for emitting a light beam (120) and an integrated photodiode (115) for detecting interference between a portion of the light beam (120) and a reflection beam (145), and a scanning device (130), which can be adjusted using an angle signal, for scanning a scanning field (150) comprising a left eye and a right eye using the light beam (120). In addition, the device (100) comprises an evaluation unit (155) which is designed to detect a first pupil position of the right eye and a second pupil position of the left eye using an interference signal provided by the photodiode (115) and a current emission angle defined by the angle signal, and to detect a viewing direction using the first pupil position and the second pupil position.

Description

[0001] R. 416792

[0002] - 1 -

[0003] Description

[0004] title

[0005] Device for detecting a direction of gaze, system for detecting a direction of gaze, display device and method for detecting a direction of gaze

[0006] State of the art

[0007] The invention relates to a device or a method according to the preamble of the independent claims. The present invention also relates to a computer program.

[0008] Monitors are equipped with so-called eye-tracking sensors, devices that use a camera to detect a user's gaze direction.

[0009] Disclosure of the invention

[0010] Against this background, the approach presented here introduces a method, a device that uses this method, and finally a corresponding computer program according to the main claims. Advantageous further developments and improvements of the device specified in the independent claim are possible through the measures listed in the dependent claims.

[0011] The advantages achievable with the invention include, in particular, the possibility of retrofitting monitors and a power-saving design by eliminating the need for cameras. R. 416792

[0012] - 2 -

[0013] A device for detecting a gaze direction is presented, comprising a sensor assembly that includes a laser diode for emitting a light beam and an integrated photodiode for detecting interference between a portion of the light beam and a reflected beam. The device also includes a scanning unit configured to successively adjust different emission angles of the light beam from the scanning unit using an angular signal defining the different emission angles. This enables the scanning unit to scan a field encompassing a left eye and a right eye using the light beam, and to return the reflected beam to the sensor assembly when the light beam is reflected from the right eye or the left eye.Furthermore, the device includes an evaluation unit designed to determine, sequentially, a first pupil position of the right eye and a second pupil position of the left eye using an interference signal provided by the photodiode and a current beam angle defined by the angular signal. The direction of gaze is then determined using the first and second pupil positions.

[0014] The device can, for example, be built into a monitor.

[0015] Alternatively, the device can be designed as a separate unit or system that can be attached to monitors. The device can, for example, be in the form of a strip or be pluggable. The sensor assembly can comprise the laser diode and the photodiode. The sensor assembly can be arranged within a housing of the device, which may have an opening. The laser diode, which is part of the sensor assembly, can be a surface emitter or a VCSEL. The light beam emitted by the laser diode is, for example, infrared light. According to one embodiment, the photodiode can detect a difference, an interference, between the light beams emitted by the laser diode and those reflected back by the photodiode.The scanning device may, for example, include mirrors to move the emitted light beam horizontally and vertically in order to capture a pupil across a large scan field. The scan field may be large enough to capture a right pupil and a left pupil. R. 416792.

[0016] - 3 -

[0017] to capture the eyes. The scan field can only detect the pupil.

[0018] No personal data is contained in a captured 2D image, which offers advantageous data protection. The scanning device eliminates the need for multiple laser diodes, thus enabling the creation of a cost-effective device with this embodiment.

[0019] According to one embodiment, the sensor device can be designed to implement laser feedback interferometry as a measurement method.

[0020] The laser diode can also be designed as a surface emitter (VCSEL).

[0021] Furthermore, the sensor device can be designed as a surface emitter.

[0022] Furthermore, the evaluation unit can be designed to determine the direction of gaze by triangulation from the first and second pupil positions. The evaluation unit analyzes the reflected light beam, for example, using triangulation.

[0023] The laser diode can be configured to emit the light beam as infrared light. Infrared light results in comparatively low light scattering, making the entire system with the laser diode resistant to stray light. Furthermore, infrared light prevents the user from being blinded by the light beam.

[0024] According to an advantageous embodiment, the scanning device can be configured to deflect the light beam both vertically and horizontally. This allows the eye to be illuminated from different angles in order to better determine eye position and gaze direction.

[0025] The scanning device can, for example, be designed as a microelectromechanical system. This allows the scanning device to require less energy and to use already known, more cost-effective components. R. 416792

[0026] - 4 -

[0027] The device can include a collimating lens arranged between the sensor and the scanning unit. The collimating lens can be designed to direct light emitted by the output unit and / or to return and focus reflected light back to the output unit. The collimating lens saves space in the device, as no additional lenses are required.

[0028] Furthermore, a system for detecting gaze direction is presented, wherein the system comprises at least two devices arranged at a distance from each other. This system with the two devices can, for example, determine pupil position more precisely or even detect multiple pairs of eyes simultaneously.

[0029] Furthermore, a display device is presented, comprising at least one device. The display device can be a monitor or screen of an electronic device, in particular a tablet, laptop, television, PC monitor, or smartphone. The device integrated into the display device can be relatively small to save space and additional costs. The device can also be retrofitted to an existing display device. Multiple devices can be attached to a single display device to improve accuracy. To prevent detection conflicts, these devices can differ in polarization or wavelength.

[0030] Furthermore, a method for detecting a direction of gaze using an embodiment of a device mentioned herein is presented, wherein the method comprises a sending step, a setting step, a determining step and a detecting step.

[0031] In the emission step, a light beam can be emitted using the sensor device, which includes the laser diode and the integrated photodiode for detecting interference between a portion of the light beam and a reflected beam. R. 416792

[0032] - 5 -

[0033] In the setting step, different beam angles of the light beam can be set successively by the scanning device using an angular signal defining the different beam angles, in order to scan a scan field encompassing a left eye and a right eye using the light beam, and to return the reflected beam to the sensor device when the light beam is reflected at the right eye or the left eye.

[0034] In the determination step, using an interference signal provided by the photodiode and a current emission angle defined by the angle signal, a first pupil position of the right eye and a second pupil position of the left eye can be determined successively in time.

[0035] In the determination step, the direction of gaze can be determined using the first pupil position and the second pupil position.

[0036] This method can be implemented, for example, in software or hardware, or in a hybrid form of software and hardware, for example in a control unit or device.

[0037] Examples of the approach presented here are shown in the drawings and explained in more detail in the following description. It shows:

[0038] Fig. 1 shows a schematic representation of a device according to an exemplary embodiment;

[0039] Fig. 2 shows a schematic representation of a pupil position according to an exemplary embodiment;

[0040] Fig. 3 shows a further schematic representation of a device according to an exemplary embodiment; R. 416792

[0041] - 6 -

[0042] Fig. 4 shows a schematic representation of an eye movement according to an exemplary embodiment;

[0043] Fig. 5 shows a schematic representation of a device according to an exemplary embodiment with one captured viewing direction;

[0044] Fig. 6 shows a further schematic representation of a device according to an exemplary embodiment with a captured viewing direction;

[0045] Fig. 7 shows a schematic representation of a display device according to an exemplary embodiment; and

[0046] Fig. 8 shows a flowchart of a process according to an exemplary embodiment.

[0047] In the following description of favorable embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and acting similarly, without repeating these elements.

[0048] Fig. 1 shows a schematic representation of a device 100 according to an exemplary embodiment. The device 100 comprises a sensor unit 105 with a laser diode 110 and a photodiode 115. The laser diode 105 emits a light beam 120, which passes through a collimating lens 125 to a scanning unit 130. The collimating lens 125 is arranged between the sensor unit 105 and the scanning unit 130. The scanning unit 130 directs the light beam 120 at an angle to an eye 135. The light beam 120 then strikes a retina 140 and is reflected back as a ray 145 via the scanning unit 130 and the collimating lens 125 to the photodiode 115.

[0049] According to one embodiment, the photodiode 115 is shaped to detect interference between a portion of the light beam 120 and the reflection beam 145 in order to determine the pupil position of the eye 135. R. 416792

[0050] - 7 -

[0051] The light beam 120 is, for example, infrared light, more precisely an infrared laser beam. The laser diode 110 continuously emits the light beam 120, which is directed by the scanning device 130 at different beam angles onto the eye 135. The scanning device 130 adjusts the light beam 120 with respect to a horizontal coordinate system h and a vertical coordinate system v. Thus, the light beam 120 is guided over the area of ​​a scan field 150. An infrared light beam is used as the light beam 120 because infrared light is not perceptible to the user. The MEMS scan field 150 should be large enough to detect both the left and right eyes. The scan field 150 encompasses the left and right eyes, which are scanned using the light beam 120. Furthermore, the scanning device 130 is, for example, designed as a microelectromechanical system.

[0052] Furthermore, the device 100 includes an evaluation unit 155, which is designed to continuously detect a first pupil position of the right eye and a second pupil position of the left eye. The first and second pupil positions then make it possible to determine the user's gaze direction. The pupil positions are detected using an interference signal provided by the photodiode 115 and a current beam angle defined by the angular signal.

[0053] The sensor unit 105, implemented here as a VCSEL (vertical cavity surface emitting laser), emits the light beam 120 and includes the integrated photodiode 115, which scans the viewer's eye 135 using the scanning device 130 by means of a horizontal microelectromechanical system h and a vertical microelectromechanical system v (MEMS). The sensor unit 105 uses, for example, the ViP measurement method, which operates with laser feedback interferometry (LFI). The reflection beam 145, reflected back from the retina 140, generates an interference signal in an area of ​​overlap with the light beam 120. This interference signal then enables the device 100 to determine the pupil positions more accurately. R. 416792

[0054] - 8 -

[0055] Fig. 2 shows a schematic representation of a pupil position 200 according to an exemplary embodiment. The pupil position 200 is determined by the device described in Fig. 1.

[0056] The pupil position 200 shown here is determined at a first time point tO. The image is a scan of the eye, generated by the light emitted by the scanning device, i.e., light reflected from the retina by a horizontal and vertical microelectromechanical system. The photodiode is integrated into the scanning device. Light scattered or reflected back from the retina can be combined to form the 2D image described here. The pupil position 200 is shown as a circle in the 2D image.

[0057] Fig. 3 shows another schematic representation of a device 100 according to an exemplary embodiment. In this embodiment, the device 100 is shaped like the device described in Fig. 1. Here, as in Fig. 1, the light beam 120 is emitted by the laser diode 110 and reflected by the scanning device 130. The difference is that the pupil of the eye 135 is in a different position.

[0058] Due to the changed pupil position, the light beam 120 strikes the retina 140 of the eye 135 at a different angle than in Fig. 1, and accordingly, the reflected beam 145 is also reflected back at a different angle than in Fig. 1. The photodiode 115 detects that the interference signal has changed and that the pupil is in the altered pupil position.

[0059] Fig. 4 shows a schematic representation of an eye movement 400 according to an exemplary embodiment. The eye movement 400 is detected here by a second emission of the light beam using the device from Fig. 1 and / or Fig. 3, whereby an interference detected by the photodiode is different from the interference detected in Fig. 2.

[0060] The pupil position 200 shown here is determined at a first time point tO. The eye movement 400 is then determined at a second time point R. 416792

[0061] - 9 -

[0062] t1 determined In Fig. 4 it can be clearly seen that the circle representing the pupil has moved to the right.

[0063] Since the scan angle of the scanning device is known at all times, this signal can be displayed in a 2D image. With eye movement 400, the position of the pupil changes in the reconstructed or composited 2D image. A change in the direction of gaze also changes the pupil position 200 in the 2D image.

[0064] Fig. 5 shows a schematic representation of a device 100 according to an exemplary embodiment with a captured gaze direction. According to this exemplary embodiment, the device 100 is shaped like the device described in Fig. 1. Here, the device 100 is arranged on a display device 500. A right eye 510 and a left eye 520 look at the display device 500. A first pupil position 530 is visible below the right eye 510, and a second pupil position 540 is visible below the left eye 520.

[0065] The device 100 detects that the user is looking at the left part of the display device 500. This result is derived from the detected straight second pupil position 540 of the left eye 520 and the leftward-facing first pupil position 530 of the right eye 510. By tracking the right eye 510 and the left eye 520, the device 100 can then use triangulation to determine the direction of gaze. To make the eye tracking even more reliable, calibration software can be provided. For calibration, this software prompts the user to look at various positions on the monitor. If the user does not wear glasses permanently, they should perform the calibration once with and once without glasses. Wearing glasses would introduce additional reflections into the reconstructed image.If these are present, the program knows that the user is wearing glasses and automatically loads the corresponding calibration.

[0066] Fig. 6 shows a further schematic representation of a device 100 according to an exemplary embodiment with one captured viewing direction. The device 100R. 416792

[0067] - 10 -

[0068] According to one embodiment, the device is shaped like the one described in Fig. 1. The representation in Fig. 6 is similar to the representation in Fig. 5. The device 100 is arranged in the display device 500. The right eye 510 and the left eye 520 look at the display device 500.

[0069] Here it can be seen that the user's gaze is directed towards the right side of the display device 500. This result is derived from the detected straight first pupil position 530 of the right eye 510 and the rightward-directed second pupil position 540 of the left eye 520. By tracking the right eye 510 and the left eye 520, the gaze direction can then be determined by triangulation from the device 100.

[0070] Fig. 7 shows a schematic representation of a display device 500 according to an exemplary embodiment. The display device 500 is like the one in Fig.

[0071] The display device described in section 5 is designed as follows. Display device 500 comprises device 100, which enables eye tracking, or gaze detection of the user's eyes, by means of an eye-tracking sensor. Display device 500 is designed as a monitor, for example, intended for use with a PC. Device 100 is very small and is integrated, for example, into a housing 700 of display device 500. The housing 700 surrounds a screen 710. Device 100, or an eye-tracking sensor, can also be retrofitted to existing monitors.

[0072] The device 100 also serves for PC control via eye tracking, which is based on scanned laser feedback interferometry (LFI). Thus, a head or eye movement can adjust an image or video displayed on the display device 500. The display device 500 can be, for example, a tablet, a laptop, a television, or a smartphone, to name a few examples. R. 416792

[0073] - 11 -

[0074] Figure 7 also shows a system 720 for detecting a direction of gaze. The system 720 comprises at least two devices 100 arranged at a distance from each other.

[0075] Fig. 8 shows a flowchart of a method 800 according to an exemplary embodiment. The method 800 is designed to detect a direction of gaze using the device from one of the figures described above or a similar device. The method 800 comprises a step 801 of sending, a step 803 of setting, a step 805 of determining, and a step 807 of determining.

[0076] In step 801 of the emission process, a light beam is emitted using the sensor device which includes the laser diode and the integrated photodiode for detecting interference between a portion of the light beam and a reflection beam.

[0077] In step 803 of the setting process, different beam angles of the light beam are sequentially set by the scanning device using an angular signal defining the different beam angles, in order to scan a scan field encompassing a left eye and a right eye using the light beam, and to return the reflected beam to the sensor device when the light beam is reflected at the right eye or the left eye.

[0078] In step 805 of the determination, a first pupil position of the right eye and a second pupil position of the left eye are determined successively using an interference signal provided by the photodiode and a current emission angle defined by the angle signal.

[0079] In step 807 of the determination process, the gaze direction is determined using the first pupil position and the second pupil position.

Claims

R. 416792 - 12 - Claims 1. Device (100) for detecting a direction of gaze, wherein the device (100) has the following features: a sensor device (105) comprising a laser diode (110) for emitting a light beam (120) and an integrated photodiode (115) for detecting interference between a portion of the light beam (120) and a reflection beam (145); a scanning device (130) configured to successively adjust different emission angles of the light beam (120) from the scanning device (130) using an angular signal defining the different emission angles, in order to scan a scan field (150) encompassing a left eye (520) and a right eye (510) using the light beam (120), and to return the reflection beam (145) to the sensor device (105) when the light beam (120) is reflected at the right eye (510) or the left eye (520); An evaluation unit (155) is configured to determine, using an interference signal provided by the photodiode (115) and a current beam angle defined by the angular signal, a first pupil position (530) of the right eye (510) and a second pupil position (540) of the left eye (520) in successive temporal sequence, and to determine the gaze direction using the first pupil position (530) and the second pupil position (540). R. 416792 - 13 - 2. Device (100) according to claim 1, wherein the sensor device (105) is configured to implement laser feedback interferometry as the measurement method.

3. Device (100) according to one of the preceding claims, wherein the laser diode (110) is configured as a surface emitter (VCSEL).

4. Device (100) according to one of the preceding claims, wherein the sensor device (105) is configured as a surface emitter.

5. Device (100) according to one of the preceding claims, wherein the evaluation unit (155) is designed to determine the direction of gaze by means of triangulation from the first pupil position (530) and the second pupil position (540).

6. Device (100) according to one of the preceding claims, wherein the laser diode (110) is configured to emit the light beam (120) as infrared light.

7. Device (100) according to one of the preceding claims, wherein the scanning device (130) is configured to deflect the light beam (120) both vertically and horizontally.

8. Device (100) according to one of the preceding claims, wherein the scanning device (130) is designed as a microelectromechanical system.

9. Device (100) according to one of the preceding claims, comprising a collimation lens (125) which is arranged between the sensor device (105) and the scan device (130).

10. System for detecting a direction of gaze, wherein the system comprises at least two spaced-apart devices (100) according to claims 1 to 9. R. 416792 - 14 - 11. Display device (500), in particular tablet, laptop, television, monitor of a PC or smartphone, with at least one device (100) according to claims 1 to 9.

12. Method (800) for detecting a direction of gaze using a device (100) according to claims 1 to 9, wherein the method comprises the following steps: Emitting (801) a light beam (120) using the sensor device (105) which has the laser diode (110) and the integrated photodiode (115) for detecting interference between a portion of the light beam (120) and a reflection beam (145); Setting (803) different emission angles of the light beam (120) from the scanning device (130) successively using an angular signal defining the different emission angles, to scan a scan field (150) encompassing a left eye (520) and a right eye (510) using the light beam (120), and to return the reflection beam (145) to the sensor device (105) when the light beam (120) is reflected at the right eye (510) or the left eye (520); Determine (805) a first pupil position (530) of the right eye (510) and a second pupil position (540) of the left eye (520) successively using an interference signal provided by the photodiode (115) and a current emission angle defined by the angle signal; and Determine (807) the direction of gaze using the first pupil position (530) and the second pupil position (540).

Images