Method for detecting the viewing direction of an eye, control unit, and data glasses

Abstract

The invention relates to a method (1400) for detecting the viewing direction (150) of an eye (115), the method (1400) having a step of illuminating (1410) at least one part of the eye (115) using a light beam (110) provided by a laser feedback interferometry sensor (105). The method (1400) additionally has a step of receiving (1420) a sensor signal (140) of the laser feedback interferometry sensor (105), the sensor signal having an amplitude value which represents the reflection intensity of a light beam (110) reflected by at least one part of the eye (115). Lastly, the method (1400) has a step of determining (1430) the viewing direction (150) of the eye (115) using the amplitude value.

Description

[0001] R. 414942

[0002] - 1 -

[0003] Description

[0004] title

[0005] Method for detecting the gaze direction of an eye, control unit and data glasses

[0006] State of the art

[0007] The invention relates to a method for detecting the gaze direction of an eye, a control unit, and data glasses, or according to the preamble of the independent claims. The present invention also relates to a computer program.

[0008] When displaying images to the eye with smart glasses, it is often necessary to determine the precise direction of the eye's gaze in order to position the displayed images accurately. This is often achieved using camera-based image analysis to capture the position of the eye's pupil. However, reflections or image artifacts can sometimes make such analysis insufficiently precise in determining the direction of gaze.

[0009] Disclosure of the invention

[0010] Against this background, the approach presented here comprises a method for detecting the gaze direction of an eye, a control unit that uses this method, and finally a corresponding computer program according to the main claims. Advantageous further developments and improvements of the device specified in the independent claim are possible through the measures listed in the dependent claims. R. 414942

[0011] - 2 -

[0012] A method for detecting the gaze direction of an eye is presented here, the method comprising the following steps:

[0013] - Illuminating at least part of the eye with a light beam provided by a laser feedback interferometry sensor;

[0014] - Receiving a sensor signal from the laser feedback interferometry sensor, which has an amplitude value representing a back-reflectance intensity of at least part of the light beam reflected by the eye; and

[0015] - Determining the eye's gaze direction using the amplitude value.

[0016] A laser feedback interferometry sensor uses a laser beam emitted from the sensor onto an external reflective surface and reflected back into the sensor. Depending on the reflectivity of this surface, and its distance from the sensor, the intensity or amplitude of a specific wavelength varies, resulting in a corresponding sensor signal. Specifically, by analyzing this amplitude, represented by the sensor signal, the reflectivity of the surface onto which the light beam is emitted can be determined. Knowing that the retina of the eye, which is illuminated by the light beam through the pupil, has a different refractive index than the sclera (the outer part of the eye), the position of the retina can be used to determine the refractive index of the retina.The angle of emission of the light beam determines whether the light beam passes through the pupil and onto the retina of the eye. By evaluating this sensor signal, the position of the pupil, and thus the eye itself, can be determined. In this way, the eye's gaze direction can be determined. Even when the light beam is spread out and illuminates a larger area of ​​the eye, the amplitude value of the sensor signal, which represents the degree of retinal reflection, can be used to determine what proportion of the light beam passes through the pupil and is reflected by the retina. This also allows the position of the pupil to be determined, and therefore, the eye's gaze direction. R. 414942.

[0017] - 3 -

[0018] By evaluating the amplitude of the sensor signal, a technically very simple method for determining the eye's gaze direction can now be advantageously implemented. This approach leverages the fact that evaluating the amplitude value can be implemented much more simply and therefore more energy-efficiently, both in terms of circuitry and numerical computation. At the same time, errors in determining the eye's gaze direction caused by stray light can be largely reduced or completely compensated for. The already mature technology of laser feedback interferometry sensors also offers a very cost-effective implementation option for the approach presented here.

[0019] According to a favorable embodiment of the approach presented here, in the illumination step, at least a portion of the pupil and a portion of an area outside the pupil can be illuminated. Such an embodiment offers the advantage that the direction of emission of the light beam does not need to precisely follow the movement of the pupil, since, for example, it can be evaluated that, with eye movement, a larger or smaller proportion of the light beam passes through the pupil and is reflected at the retina with a correspondingly higher reflectance. This leads to a variation in the amplitude value of the sensor signal, from which the movement of the eye and thus the current gaze direction can then be calculated.

[0020] A further advantageous embodiment of the approach presented here is one in which, during the determination step, the gaze direction is recognized as being directed towards the light beam when an increasing amplitude value is received during the reception step. Such an embodiment offers the advantage of a particularly simple determination of the eye's gaze direction from the sensor signal.

[0021] Another conceivable embodiment of the approach proposed here involves illuminating at least part of the eye in the illumination step using an optical element of the laser feedback interferometry sensor, wherein the optical element is configured to direct the light beam from R. 414942

[0022] - 4 -

[0023] The laser feedback interferometry sensor is directed onto the part of the eye. Such an embodiment of the approach proposed here offers the advantage of being able to mount the laser feedback interferometry sensor in a convenient position within a suitable device or smart glasses and to redirect the light beam accordingly. Additionally or alternatively, the light beam can also be shaped, for example, widened, to illuminate a larger area of ​​the eye. This embodiment thus allows for a greater degree of flexibility in implementing the approach presented here using cost-effective means.

[0024] According to a further embodiment of the approach proposed here, in the illumination step, at least another part of the eye can be illuminated with a further light beam provided by another laser feedback interferometry sensor. In the receiving step, a further sensor signal from the further laser feedback interferometry sensor is received, which has a further amplitude value representing a back-reflectance intensity of at least a part of the further light beam reflected by the eye. In the determining step, the eye's gaze direction is determined using this further amplitude value. By taking this at least one further amplitude value into account, a very reliable and robust determination of the eye's gaze direction can thus be obtained, further reducing, for example, the influence of stray light.

[0025] According to another embodiment of the approach proposed here, the eye's gaze direction can also be determined in the determination step by combining the amplitude value and the subsequent amplitude value, in particular by comparing the amplitude value and the subsequent amplitude value. By combining the amplitude value and the subsequent amplitude value, for example, a rotation of the eye or the angle of rotation can be determined and / or verified very easily. R. 414942

[0026] - 5 -

[0027] A further advantageous embodiment of the approach presented here involves illuminating parts of the eye with light beams from a sensor field during the illumination step. In particular, the sensor field contains a plurality of laser feedback interferometry sensors arranged in rows and columns. During the reception step, each laser feedback interferometry sensor receives a sensor signal, each signal having an amplitude value representing the back-reflectance intensity of at least one part of the light beam reflected by the eye. During the determination step, the eye's gaze direction is determined using these amplitude values. Such an embodiment of the proposed approach offers the advantage of illuminating a large area of ​​the eye with appropriate light, while the use of the corresponding sensors in the sensor field can be implemented cost-effectively.By evaluating the sensor signals of the sensors arranged in the sensor field, a highly precise determination of the eye's gaze direction can be achieved.

[0028] A particularly advantageous embodiment of the approach proposed here includes a step in controlling an image output to the eye, responding to the eye's specific gaze direction. By outputting the image to the eye, for example, through appropriate control of a display or indicator unit to output corresponding image positions on that display or indicator unit, relevant information can be presented to the user very precisely and reliably recognized or identified by the user's eye.

[0029] An embodiment of the approach presented here can be implemented very efficiently, either numerically or in terms of circuit design, as a method with a single step of selecting one of several laser feedback interferometry sensors, which is to be used to illuminate at least part of the eye in a subsequent illumination step. This allows the evaluation of the sensor signals to be limited to a small number of sensors, for example, a sensor array, which is highly likely to contain a detection range R. 414942

[0030] - 6 - this sensor can also detect the pupil, so that the direction of gaze can be determined simply and reliably. The determination of the laser feedback interferometry sensor to be used for a subsequent step can be done, for example, based on the amplitude value, which is, for example, larger than an amplitude value that should not be used for illuminating at least part of the eye in a subsequent step.

[0031] This process can be implemented, for example, in software or hardware, or in a hybrid form of software and hardware, for example in a control unit.

[0032] The approach presented here further creates a control unit designed to execute, control, and implement the steps of a variant of the method presented here in appropriate devices. This embodiment of the invention, in the form of a control unit, also allows the underlying problem to be solved quickly and efficiently.

[0033] For this purpose, the control unit can have at least one processing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The processing unit can be, for example, a signal processor, a microcontroller, or the like, and the storage unit can be flash memory or a magnetic storage device.The communication interface can be configured to read or output data wirelessly and / or via wired connections, wherein a communication interface capable of reading or outputting wired data can, for example, read this data electrically or optically from or output it into a corresponding data transmission line. R. 414942.

[0034] - 7 -

[0035] In this context, a control unit can be understood as an electrical device that processes sensor signals and outputs control and / or data signals accordingly. The device may have an interface, which can be implemented in hardware and / or software. In the case of a hardware-based interface, the interfaces can, for example, be part of a so-called system ASIC, which incorporates various functions of the device. However, it is also possible that the interfaces are separate integrated circuits or at least partially comprised of discrete components. In the case of a software-based interface, the interfaces can be software modules, which, for example, are located on a microcontroller alongside other software modules.

[0036] Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out, implement and / or control the steps of the method according to one of the embodiments described above, in particular if the program product or program is executed on a computer or device.

[0037] A particularly advantageous embodiment of the approach presented here is that of eyeglasses, especially smart glasses, with at least one laser feedback interferometry sensor for outputting a light beam to illuminate at least part of the eye and a control unit according to a variant presented here. Such an embodiment allows the information about the eye's gaze direction to be used reliably and very advantageously to precisely display information to a user of the eyeglasses or smart glasses.

[0038] One embodiment of the approach proposed here requires particularly little installation space if at least part of the laser feedback interferometry sensor or the laser feedback interferometry sensor itself is integrated into a spectacle lens of eyeglasses or smart glasses. Such an embodiment is R. 414942.

[0039] - 8 -

[0040] The design enables the simple and interference-free output of the light beam to the eye of a user of the data glasses.

[0041] According to a further embodiment of the approach proposed here, the glasses can have a varifocal lens whose refractive power can be changed by the direction of gaze detected by a variant of a control unit presented here. Such an embodiment of the approach presented here advantageously allows, for example, the adjustment of the refractive power depending on the direction of gaze. In this way, very flexible optical glasses can also be realized in which the adjustment of the refractive behavior is adapted to the direction of gaze, so that, for example, static areas for distance vision or night vision in such lenses can be avoided, thus significantly increasing user comfort. At the same time, an adjustment to changing eye parameters can be easily achieved, for example, by reprogramming the corresponding control unit.

[0042] Examples of the approach presented here are shown in the drawings and explained in more detail in the following description. It shows:

[0043] Fig. 1 shows a schematic representation of an embodiment of glasses, which can be designed, for example, as data glasses;

[0044] Fig. 2 shows an exemplary amplitude spectrum for operation of the glasses in the first case, i.e., during movement without modulation;

[0045] Fig. 3 shows an exemplary amplitude spectrum for operation of the glasses in the second case, i.e., during movement with modulation;

[0046] Fig. 4 shows a diagram of two time histories of the parameters l(t) and the power Pt using a triangular modulation approach, which also allows for simultaneous measurement of the distance to the target; R. 414942

[0047] - 9 -

[0048] Fig. 5 shows a diagram of two time courses of the parameters l(t) and the power Pt using a triangular modulation approach, which also allows for simultaneous measurement of the distance to the target;

[0049] Fig. 6 shows a cross-sectional view of the eye, which is illuminated by two LFI illumination cones or two light beams emitted by LFI sensors;

[0050] Fig. 7 is a diagram to illustrate the procedure according to the approach presented here;

[0051] Fig. 8 shows another diagram to illustrate the procedure according to the approach presented here;

[0052] Fig. 9 shows a schematic representation of an eye illuminated by light rays from a sensor field of LFI sensors;

[0053] Fig. 10 shows a schematic representation of a determination unit for determining the direction of gaze of the eye;

[0054] Fig. 11 shows a diagram of the distribution of intensity values ​​plotted on the ordinate, representing the intensities of reflected light received by each of the LFI sensors plotted on the abscissa;

[0055] Fig. 12 shows another schematic representation of a part of a pair of glasses, which includes the components presented here.

[0056] Fig. 13 shows a schematic representation of a sensor field of LFI sensors or of the light rays emitted by these sensors as an example, which are emitted onto a part of the eye, here the pupil and the sclera or the iris; and

[0057] Fig. 14 shows a flowchart of an embodiment of a method for detecting the gaze direction of an eye. R. 414942

[0058] - 10 -

[0059] In the following description of favorable embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and acting similarly, without repeating these elements.

[0060] Figure 1 shows a schematic representation of an embodiment of a pair of glasses 100, which can, for example, be configured as data glasses. The glasses include a laser feedback interferometry sensor 105 (hereinafter also referred to as LFI sensor) configured to emit a light beam 110, which here is configured as a laser beam, onto a part of the eye 115 of a user of the glasses 100. The light beam 110 can strike at least partially the sclera 120, i.e., the sclera, the iris 125, or the pupil 130. If the light beam 110 strikes the pupil 130, it enters the interior of the eye 115 and is reflected by the retina 135. If the light ray 110 hits the sclera 120 or the iris 125, it will also be reflected, but due to the lower reflectivity of the sclera 120 or the iris 120, it will be reflected with significantly less intensity than if it were reflected by the retina 135.The reflected beam can then be coupled back into the laser feedback interferometry sensor 105, so that a sensor signal 140 can be provided from the intensity of the reflected light beam 110, the amplitude of which represents the intensity of the reflected light beam 110. From this sensor signal 140, it can then be determined, for example, in a corresponding determination unit 145, in which direction 150 the eye 115 is oriented.This allows, for example, the exploitation of the fact that at a high intensity of the reflected light beam 110, the gaze direction 150 of the eye 115 is directed towards an area very close to, or near, the laser feedback interferometry sensor 105, and at a low intensity of the reflected light beam 110, the gaze direction 150 of the eye 115 is not directed towards the laser feedback interferometry sensor 105, so that the light beam 110 falls on the sclera 120 or the iris 125 and is thus reflected with lower intensity than if it were to fall on the retina 135. In the determination unit 145, an alignment signal 155 can then be determined, which represents the current gaze direction 150 of the eye 115 and which R. 414942.

[0061] - 11 - for example, output to a display unit 160 or a display to output or display information in a corresponding area of ​​a spectacle lens 165 of the glasses 100, so that the output or display is directly and clearly visible to the user of the glasses 100 in their field of vision. It is also conceivable that the alignment signal 155 is output to a corresponding varifocal lens 170 to control a corresponding change in refractive power, so that, for example, the user is enabled to adjust the refractive power of the spectacle lens 165 very comfortably, so that, depending on the direction of gaze 150, the refractive power of the spectacle lens 165 is set for distance vision or night vision.It is also conceivable that, with appropriate adjustment of eye parameters, which can change with age, for example, a new adjustment of the refractive powers required for the different directions of gaze can be set, so that the glasses can continue to be used as such and only the lens power for different directions of gaze is readjusted.

[0062] An important element for realizing the approach proposed here is the use of a laser feedback interferometry sensor 105, which is already technically mature and available and which, for example, is embedded directly on or in the spectacle lens 165. However, it is also conceivable that the laser feedback interferometry sensor 105 is arranged in a lateral area of ​​the spectacle 100, for example next to the display unit 160, and that only a corresponding optical element for deflecting, widening, or reflecting the light beam 110 is provided on or in the spectacle lens 165. This would allow the sensor 105 to be positioned on or in the spectacle 100 according to the available installation space.Simultaneously or additionally, the light beam 110 could also be widened to illuminate a larger area of ​​the eye 115, so that, for example, when the eye 115 moves or rotates, an increasingly larger portion of the (widened) light beam 110 passes through the pupil 130, thus increasing the intensity of the reflected light beam 110 accordingly. Conversely, if the eye 115 rotates out of the (widened) light beam 110, the intensity of the reflected light beam 110 will decrease accordingly. This can be achieved by using meta-optics or diffractive optics for beam widening, or at least a corresponding element for R. 414942.

[0063] - 12 -

[0064] Implementing these functions allows for a very efficient beam expansion and an enlargement of the eyebox without requiring multiple light-emitting elements. This also enables a cost-effective implementation of the approach described here, along with its resulting advantages.

[0065] The fundamental operating principle of a laser feedback interferometry sensor is not the core of the approach presented here. Rather, this approach focuses on the advantageous application of this sensor component. However, the basic procedure is described below to clarify the functionality of the approach presented here.

[0066] The operating principle of a laser is based on an optical resonator. Inside the resonator, electrons are excited by an external energy input. The radiation, initially generated by spontaneous emission, is guided multiple times through the interior of the optical resonator, for example, by mirrors. The resulting oscillating electromagnetic wave excites the electrons to emit in phase, producing coherent radiation. A mirror element with low transmittance, for example, 1%, is located on one side to couple the generated laser radiation out of the resonator. In the case of a Vertical Cavity Surface Emitting Laser (VCSEL), layers designed as a Distributed Bragg Reflector (DBR) are used as the optical mirror.

[0067] The underlying physical principle of an oculography system is based on so-called "laser self-mixing." When the coherent radiation emitted by a laser is scattered by a surface, some of this radiation returns to the laser cavity, the optical resonator. If twice the distance to the scatterer corresponds to an integer multiple of the wavelength, the backscattered radiation is in phase with the radiation in the laser cavity. It thus adds constructively to the radiation already present there, reducing the lasing threshold and thereby increasing the laser's output power. If the distance to the scatterer, and thus R. 414942, is now increased, the backscattered radiation is in phase with the radiation in the laser cavity.

[0068] - 13 - If the optical path length changes, positive or negative interference occurs repeatedly within the laser cavity, depending on the distance. This causes the laser power to modulate in an oscillating pattern between a radiation maximum and a radiation minimum. Alternatively, the current driving the laser can be ramp-modulated, thereby modulating the laser wavelength. At a fixed distance, this also changes the number of wavelengths that "fit" into the optical path, resulting in the same oscillating temporal interference pattern. If the optical radiation power is measured by a photodiode (monitoring photodiode), the change in the amplitude of the radiation power can be used to infer the change in intensity of the backscattered laser power.

[0069] Alternatively, the voltage at the PN junction of the laser can be measured to detect the self-mixing effect.

[0070] By analyzing the number of oscillations (e.g., by counting zero crossings or maximum values, or by calculating a Fourier spectrum and analyzing the amplitude in the frequency domain), the number of oscillations (transmissions of constructive and destructive interference) can be determined, and thus, given a known laser wavelength, the distance between the laser cavity and the scatterer can be calculated. A similar effect occurs with a scatterer moving parallel to the laser beam. Here, according to the Doppler effect, the frequency of the backscattered laser light changes. At low speeds, this can be approximated as a phase shift of the backscattered laser light in the laser cavity, leading, analogously to the effect described above, to oscillating oscillations of positive and negative interference (formation of a beat frequency).This beat frequency fb is directly proportional to the velocity v of the scatterer, where the speed of light Co, the angle a between the laser beam and the motion vector, and the exciting laser frequency fo are known: ft = 2 v / co * fo cos(a) (Eq. 1) R. 414942.

[0071] - 14 -

[0072] The ViP system can be operated in two different modes. In the first case, the laser operates unmodulated, meaning the wavelength / frequency of the laser does not change over time. In the second case, the frequency of the laser is modulated over time, for example, in the form of a triangular ramp function.

[0073] Figure 2 shows an exemplary amplitude spectrum 200 for operation of the glasses 100 in the first case, i.e., during movement without modulation. The amplitude on the ordinate 210 is shown as a function of the frequency plotted on the abscissa 220. During movement, a signal 230 is displayed, whose center frequency is directly correlated with the velocity component in the beam direction.

[0074] Figure 3 shows an exemplary amplitude spectrum 300 for operation of the glasses 100 in the second case, i.e., during movement with modulation. Without movement, a signal comparable to the signal in Figure 2 is obtained, where the frequency represents the distance between the scatterer and the ViP sensor. If movement occurs in addition, the signal splits into two signal components 310 and 320, where the distance 330 between the signal components is correlated with the velocity. In this way, distance, velocity, and the direction vector of the velocity can be determined.

[0075] Figure 4 shows a diagram of two time courses of the parameters l(t) (partial figure 4a) and the power Pf (partial figure 4b) using a triangular modulation approach, which also allows for simultaneous measurement of the distance to the target.

[0076] The distance-dependent beat frequency (fb) can be determined by an FFT. If several targets are located in the area illuminated by the LFI sensor, they all scatter a portion of the scattered light, resulting in a superposition of several frequencies that leads to a spectral distribution in the FFT. This superposition is subsequently referred to as the distance spectrum. Similarly, there is a velocity spectrum containing the Doppler frequencies (fd). If different targets are moving at different speeds, a superposition can also be observed in the spectrum. R. 414942

[0077] - 15 -

[0078] The superposition of the Doppler frequencies in the spectrum is subsequently referred to as the velocity spectrum.

[0079] Furthermore, the laser sensor can be modulated by changing the reflectivity in terms of interference intensity. This measurement mode is used for this application because no electronics or signal processing are required to capture the spectral features.

[0080] The target, in this case the eye, is modulated as the external cavity of the laser sensor.

[0081] The interference signal, which can be measured as a current signal by the photodiode integrated into the cavity or via the voltage signal at the PN junction of the laser diode, is modulated in intensity and thus in amplitude by a change in the reflectivity of the external mirror.

[0082] The feedback power Pf in the cavity corresponds to the optical power of the emitted light P02, which is firstly amplitude-modulated by an amplitude component m and secondly frequency-modulated by the length of the external cavity. The amplitude component m is essentially described by the external component and the coupling factor kf. R. 414942

[0083] - 16 -

[0084] An array of at least one LFI sensor is equipped with an optical element (meta lens, DOE, HOE, classic glass / plastic lens) or operated lensless, so that the beam diverges and illuminates a part of the eye.

[0085] The approach presented here dispenses with light modulation and spectral analysis, representing an alternative to the state of the art. This eliminates the need for costly and complex evaluation units, especially spectral analysis of a sensor signal, allowing for fast and resource-efficient pupil position detection.

[0086] Figure 5 shows a schematic top view of an eye 115 illuminated by a light beam 110, showing its iris 125 and pupil 130. In this example, the eye 115 is illuminated by three LFI laser spots or corresponding light beams 110, 500, and 510, which are emitted, for example, by three LFI sensors arranged in an array or sensor field (these LFI sensors are not shown in Figure 5). It can be seen that light beam 110, the other light beam 500, and, for example, the additional light beam 510 illuminate different areas of the eye 115 and partially pass through the pupil 130.

[0087] Figure 6 shows a cross-sectional view of the eye 115, which is illuminated by two LFI illumination cones or two light beams 110 and 500 emitted by LFI sensors, respectively. The eye 115 is illuminated by the two cones, with part of the light from one cone striking the sclera 120, iris 125, or the skin of the eye 115 with low reflectivity R1, and a second part of the cone 110 or 500 striking the pupil 130 and lens R. 414942

[0088] - 17 -

[0089] 600 of the eye 115 and enters the eye 115. Here, the second part of the light strikes the retina 135 with a high reflectivity R2. The eye 115 rotates at an angle 0 about a rotation axis 610, so that the portion of the light from the light ray 110 and the further light ray 500 that enters the eye 115 through the pupil 130 varies.

[0090] Figure 7 shows a diagram illustrating the procedure according to the approach presented here, in which the absolute value of the amplitude is plotted on the ordinate against time on the abscissa. The diagram thus shows the measured amplitude of the interference signal or sensor signal over time t for the two light cones 110 and 500, respectively, as shown in Figure 6. For the two light cones, at an eye rotation angle 0, as depicted in the preceding Figure 6, there is an approximately equal ratio of light from the cone on the outer surface of eye 115 with reflectivity R1 and on the retina 135 of eye 115 with reflectivity R2. Therefore, the amplitudes of the sensor values ​​140 and 700 of the two sensors, or rather the LFI sensors emitting the respective light beams, are equal in this arrangement.

[0091] Figure 8 shows another diagram illustrating the procedure according to the approach presented here, in which the absolute value of the amplitude is plotted on the ordinate against the angle 0 on the abscissa. When eye 115 rotates to the right, the amplitude ratio changes. The additional light beam 500 provided by the further LFI sensor, which has an amplitude signal of 700, shows an increase in amplitude, while the LFI sensor 105, which provides light beam 110, shows a decrease in its amplitude signal of 700. From the correlation and the relationship between the amplitude signals 140 and 700, the viewing angle 0 of eye 115 can be estimated.

[0092] Figure 9 shows a schematic representation of an eye 115 illuminated by light rays 110, 500, 510, ... of a sensor field 900 of LFI sensors 105, 910, 920 ... . It can again be seen that part of the light ray passes through the pupil 130 and is reflected at the retina 135 with a higher reflectance than other light rays such as the R. 414942

[0093] - 18 - further light beam 500 or the additional light beam 510, which, for example, are partially directed onto the sclera 125 or the iris 125 of the eye 115.

[0094] Each of the LFI sensors 105, 910, 920 or 930 can then be configured to output a corresponding sensor signal, the amplitude of which also represents the intensity of the reflected light beam received by the respective LFI sensor.

[0095] Figure 10 shows a schematic representation of a determination unit 145 for determining the eye's gaze direction. The sensor signals 140 of the LFI sensor 105 and / or the other LFI sensors 910, 920, 930 of the sensor array 900 from Figure 9 are fed to the determination unit 145. Each of these signals has an amplitude value that depends on the intensity of the reflected light beam. The gaze direction 150, the rotation angle 0 of the eye 115, and the upward deflection p of the eye are then determined in this determination unit 145 based on one or more of these sensor signals.

[0096] The amplitude signals of the sensor array 900 are thus preprocessed, e.g., averaged over a moving window, and then processed in the determination unit 145, for example, in a regression algorithm, by converting the individual sensor signals to viewing angles (0, p) using polynomial interpolation. In an alternative, simpler implementation, only the probability that the pupil of the eye is within the corresponding light cone is output for the grid or sensor field.

[0097] Figure 11 shows a diagram of the distribution of intensity values ​​P plotted on the ordinate, representing the intensities of reflected light received by each of the LFI sensors 105, 910, 920, 930 plotted on the abscissa. Thus, for an exemplary arrangement with four LFI sensors arranged in an array or sensor field (for example, according to Figure 9), the eye is scanned one-dimensionally by the respective laser beams and two-dimensionally by the extent of the respective laser beams.

[0098] Because the light from sensor 105 falls primarily through the pupil / lens 130 of the eye 115 onto the retina and with a correspondingly high intensity R. 414942

[0099] - 19 - is reflected, this sensor 105 measures a particularly high amplitude, the highest compared to the other sensors, and accordingly indicates a high probability that the eye is looking in a direction that lies within this sensor 105.

[0100] Figure 11 shows another schematic representation of a part of a pair of glasses 100, which includes the components presented here. In this embodiment of the approach presented here, the LFI sensors 105, 910, 930 are integrated into the lens 165 so that they are directed straight at the eye 115. From the amplitude signals of the respective LFI sensors 105, 910, 930, it is then possible to determine how the eye looks through the lens or how the gaze vector 150 is oriented with respect to the lens 165. For example, a varifocal lens 170 is installed in the lens 165, so that the function of adaptive progressive lenses can be replicated by appropriately controlling the refractive power of the varifocal lens 170 depending on the specific direction of gaze. If one of the sensors measures, for example, at the lower part of the glasses 100, or...If the spectacle lens 165 emits a high amplitude signal, thus indicating a high probability that the eye is looking through the glasses in this area, a corresponding control signal can be sent to the varifocal lens 170, which should then also set a near accommodation in order to offer the user a high level of user comfort.

[0101] Since only a portion of the LFI laser sensors used in a sensor field deliver stable amplitude signals on the surface of the lens and pupil, a ROI-based activation of individual LFI sensors is proposed, specifically those sensors that exhibited the highest amplitude values ​​within a given time window. This approach assumes that the gaze direction within a specific area yields amplitude values ​​indicating that the pupil, and thus the gaze direction, is located within this area of ​​the sensor field and is detected there. It can also be assumed that, for anatomical reasons, the gaze direction will not change too rapidly, at least not so quickly that evaluating the amplitude values ​​of all sensors is necessary.Rather, for example, those sensors for which there is a high probability that the eye's gaze direction is in the R. 414942.

[0102] - 20 -

[0103] Within the detection range of these sensors, a higher sampling rate is used to quickly and promptly detect changes in gaze direction. Upon detection of a gaze change, the number or quantity of sensors whose amplitude value or sensor signal is to be evaluated can be adjusted. This approach can then lead to a reduction in the required processing effort and / or higher measurement resolution and signal-to-noise ratio (SNR) due to the higher sampling rate of the activated sensors. Furthermore, the laser power emitted towards the eye can be reduced, ensuring the system's eye safety in the presence of artificial IR light. Reducing laser power and processing effort also leads to a further reduction in the system's energy consumption. This increases battery life and thus user comfort.Furthermore, a less powerful processing unit (e.g., a microcontroller instead of an FPGA) can be used, reducing overall construction costs. The measured amplitude of the reflected light beam is then used, for example, to estimate the current ROI (ROI = Region of Interest, i.e., the area of ​​sensors in a sensor array where the line of sight to the pupil appears to lie). The future ROI could be estimated using a predictive algorithm (e.g., Kalman filter) to select the correct subset of LFI sensors for the next measurement, whose amplitude value will be evaluated.

[0104] Figure 13 shows a schematic representation of a sensor array 900 of LFI sensors and the light beams 110 and 500 emitted by these sensors, which are directed onto a part of the eye 115, specifically the pupil 130 and the sclera 120 or the iris 125. The dotted circles describe the light beams of the current ROI (Region of Interest) containing the active sensors, i.e., those LFI sensors whose sensor signal or amplitude values ​​are to be evaluated. A predictor, arranged, for example, in the determination unit 145 as shown in Figure 1, calculates the subset of sensors in the ROI for the subsequent measurement(s) based on amplitude measurements. R. 414942

[0105] - 21 - a new subset of sensors is then selected, for example, for the next measurement or determination of the viewing direction.

[0106] The approach presented here thus enables the realization of several advantages. For example, multiple MESAS (laser cavities or discrete individual LFI sensors) with their own meta-optics can be integrated directly at the chip level. LFI sensors can also be integrated into the spectacle frame or temple, in combination with an optical element (e.g., holographic) that deflects the beams. Integration of the approach presented here at the chip level and / or within a spectacle temple is also possible. For example, the integration of multiple MESAS (laser cavities), each with its own meta-optics, can be implemented directly at the chip level.For example, such a system, or individual elements of the approach presented here, could include integrating the LFI sensors into the spectacle frame or temples, possibly with holographic optical elements for beam deflection (e.g., to extend the eyebox). Modulation of the laser power output by the sensor(s), measurement of velocities and distances, and / or evaluation of the distance distributions can also be performed, correlating distance in the spectrum with amplitude in the time signal as a "weighted" distance measurement of the individual sensors. Detecting whether an eye is looking into a beam, based on the distance information, is also technically feasible, allowing for easy evaluation of whether the beam is hitting the eye or the retina. Furthermore, the implementation of multi-modal eye tracking is conceivable.This approach combines laser feedback interferometry with other eye-tracking methods, such as computer vision, electrooculography (EOG), or video-based tracking, to create a more robust and accurate system. An adaptive sampling rate can also be employed, allowing, for example, the implementation of an algorithm that dynamically adjusts the sampling rate of the laser feedback interferometry system to the user's gaze behavior, such as during saccades or fixations. Gaze-based user authentication is also possible. For this purpose, the LFI sensor system R.414942 is used.

[0107] - 22 - is used to authenticate users based on their unique gaze patterns or retinal amplitude, providing an additional layer of security for augmented reality applications. Eye-tracking analysis can also be implemented for health monitoring. The approach presented here is used to monitor eye movements for the early detection of neurological disorders such as Parkinson's disease, or to track the effects of fatigue or attention deficits. The implementation of a "virtual try-on" with gaze-based feedback is also conceivable. This involves integrating the eye-tracking system with virtual try-on applications to provide users with real-time feedback on their gaze behavior while trying on virtual products. A gaze-dependent display is also possible.This can be achieved by developing a display that adapts its content and layout based on the user's gaze, providing a personalized and efficient user experience. Eye tracking can also be implemented for gaming and simulation. The system presented here can be used to create immersive gaming experiences where the user's gaze can control game elements or interact with virtual objects.

[0108] Furthermore, improved foveal rendering can be implemented. This involves using high-resolution gaze data to optimize foveal rendering techniques, thereby reducing computing resources while maintaining visual quality. Gaze-based human-computer interaction can also be implemented, exploring new interaction methods based on gaze input, such as gaze-based scrolling, zooming, or selection. Eye tracking for advertising and marketing research is also theoretically feasible with the approach presented here. This system would be used to analyze users' attention and interest in advertisements, providing valuable insights for advertisers and market researchers. Finally, the implementation of gaze-based accessibility features is conceivable.For this purpose, assistive technologies are being developed that use eye tracking to support people with disabilities, such as mobility or speech impairments. Real-time gaze analysis is also possible (R. 414942).

[0109] - 23 - can be used to create a system that provides real-time analysis of users' gaze behavior, enabling applications such as attention tracking, sentiment analysis, or mental workload assessment. Gaze-based learning and training applications are also feasible. These involve developing learning and training applications based on gaze input to improve learning processes and increase the efficiency of training programs. Furthermore, an innovative eye-tracking system for driver assistance systems can be developed. This system is used to monitor drivers' gaze movements and enhance road safety.

[0110] The approach presented here offers the advantage that using static LFI sensors without a scanner allows for very high update rates (e.g., greater than 1 kHz), which is not possible with conventional camera-based solutions. Furthermore, the presented approaches exhibit low power consumption and avoid the disadvantage of static laser sensors, namely that the laser beam of a single LFI sensor must directly hit the eye to measure the eye's surface velocity and eye movement. The eyebox, i.e., the area in which the eye can be located and tracked, can also be enlarged, enabling a system consisting of static LFI sensors to be used in the mass market (with varying head shapes, interpupillary distances, and eye / lid shapes).An important aspect of the approach presented here is the use of at least one laser feedback interferometry sensor for eye tracking. Meta-optics or diffractive optics can also be employed to widen the beam of a single sensor. The use of a sensor array to enlarge the eyebox is also conceivable. Sensor modulation and spectral analysis are unnecessary. A key advantage of this approach is the requirement for significantly simpler electronics and thus lower power consumption, which can result, for example, in lower ASIC costs or even the complete elimination of an ASIC. Relative eye tracking is also possible, meaning that only relative, rather than absolute, accuracy of the measurement or sensor alignment is required (see R. 414942).

[0111] - 24 -

[0112] Stray light does not significantly impair robust detection of the viewing direction.

[0113] Figure 14 shows a flowchart of an embodiment of a method 1400 for detecting the gaze direction of an eye, wherein the method 1400 includes a step 1410 of illuminating at least a part of the eye with a light beam provided by a laser feedback interferometry sensor. A step 1420 of receiving a sensor signal from the laser feedback interferometry sensor, which has an amplitude value representing the back-reflectance intensity of at least a part of the light beam reflected by the eye, may also be included. Finally, the method 1400 includes a step 1430 of determining the gaze direction of the eye using the amplitude value.If an embodiment includes an “and / or” connection between a first feature and a second feature, this is to be read as meaning that the embodiment according to one embodiment has both the first feature and the second feature, and according to another embodiment either only the first feature or only the second feature.

Claims

R. 414942 - 25 - Claims 1. Method (1400) for detecting a gaze direction (150) of an eye (115), wherein the method (1400) comprises the following steps: Illuminate (1410) at least part of the eye (115) with a light beam (110) provided by a laser feedback interferometry sensor (105); Receiving (1420) a sensor signal (140) from the laser feedback interferometry sensor (105) which has an amplitude value representing a back-reflectance intensity of at least part of the light beam (110) reflected by the eye (115); and determining (1430) the gaze direction (150) of the eye (115) using the amplitude value.

2. Method (1400) according to claim 1, wherein in the step (1410) of illumination at least a part of a pupil (130) of the eye (115) and a part of an area (120, 125) outside the pupil (130) of the eye (115) is illuminated.

3. Method (1400) according to one of the preceding claims, wherein in the step (1430) of determining the viewing direction (150) is recognized as being directed towards the light beam (110) when in the step (1420) of receiving a sensor signal (10) is received which represents an increasing amplitude value.

4. Method (1400) according to one of the preceding claims, wherein in the step (1410) of illumination at least one part of the eye (115) is illuminated using an optical element of the laser feedback interferometry sensor (105), wherein the optical element is configured to direct the light beam (110) from the laser feedback interferometry sensor (105) onto the part of the eye (115). R. 414942 - 26 - 5. Method (1400) according to one of the preceding claims, wherein in step (1410) of illumination at least a further part of the eye (115) is illuminated with a further light beam (500) provided by a further laser feedback interferometry sensor (910), wherein in step (1420) of receiving a further sensor signal from the further laser feedback interferometry sensor (910) is received, which has a further amplitude value representing a back-reflectance intensity of at least a part of the further light beam (500) reflected by the eye (115), and wherein in step (1430) of determining the gaze direction (150) of the eye (115) is determined using the further amplitude value.

6. Method (1400) according to claim 5, wherein in step (1430) of determining the gaze direction (150) of the eye (115) is determined using a combination of the amplitude value and the further amplitude value, in particular wherein the gaze direction (150) of the eye (115) is determined using a comparison of the amplitude value and the further amplitude value.

7. Method (1400) according to one of the preceding claims, wherein in step (1410) of illumination parts of the eye (115) are illuminated with light beams (110, 500, 510) from laser feedback interferometry sensors (105, 910, 920) from a sensor field (900), in particular wherein a plurality of laser feedback interferometry sensors (105, 910, 920) are arranged in rows and columns in the sensor field (900), wherein in step (1420) of receiving a sensor signal (140) is received from each of the laser feedback interferometry sensors (105, 910, 920), each signal having an amplitude value representing a back-reflectance intensity of at least one part of the light beam (110) reflected by the eye (115), and wherein in step (1430) of determining the gaze direction (150) of the eye (115) is determined using the amplitude values. R. 414942 - 27 - 8. Method (1400) according to one of the preceding claims, comprising a step of controlling an image output to the eye (115), responding to the specific viewing direction (150) of the eye (115).

9. Method (1400) according to one of the preceding claims, comprising a step of selecting one of several laser feedback interferometry sensors (105, 910, 920) to be used for illuminating at least a part of the eye (115) in a subsequent step (1410) of illumination.

10. Control unit (105, 145) configured to execute and / or control the steps (1410, 1420, 1430) of the method (1400) according to any of the preceding claims in corresponding units (105, 145).

11. Computer program configured to execute and / or control the steps (1410, 1420, 1430) of the method (1400) according to any of the preceding claims.

12. Machine-readable storage medium on which the computer program according to claim 11 is stored.

13. Glasses (100), in particular data glasses, with at least one laser feedback interferometry sensor (105) for outputting a light beam (110) for illuminating at least one part of the eye (115) and a control unit (105, 145) according to claim 10.

14. Glasses (100) according to claim 13, wherein at least a part of the laser feedback interferometry sensor (105) or the laser feedback interferometry sensor (105) is integrated into a lens (165) of the glasses (100).

15. Glasses (100) according to one of claims 13 or 14, with a varifocal lens (170) whose refractive power can be changed by the direction of gaze (150) detected by means of the control unit (105, 145) according to claim 10.

Images