Method for three-dimensional eye tracking of an eye of a user of a virtual retinal display, virtual retinal display and smart glasses

Abstract

The invention relates to a method for three-dimensional eye tracking of an eye (10a-c) of a user of a virtual reality retinal display (12a-c) (retinal scan display), for example in smart glasses (14a-c), by means of stereoscopy which is based on an evaluation of perspectively different images (16a-c) of the same eye (10a-c), which images are recorded by means of laser feedback interferometry (LFI). It is proposed that from scanned infrared laser signals (18a-c) of an LFI sensor unit (20a-c), in particular infrared laser signals starting from a single starting point (26a-c), at least two different virtual camera perspectives (22a-c, 24a-c) are created for a recording of the perspectively different images (16a-c) used for the stereoscopy.

Description

[0001] Description

[0002] Method for three-dimensional eye tracking of an eye of a user of a virtual retinal display, virtual retinal display and data glasses

[0003] State of the art

[0004] A method for three-dimensional eye tracking of a user's eye of a virtual retinal display (retinal scan display) by means of stereoscopy has already been proposed, which is based on an evaluation of perspectively different images of the same eye taken by means of laser feedback interferometry (LFI).

[0005] Disclosure of the invention

[0006] The invention is based on a method for three-dimensional eye tracking of an eye of a user of a virtual retinal display (retinal scan display), for example in data glasses, by means of stereoscopy, which is based on an evaluation of perspectively different images of the same eye, each of which is recorded by means of laser feedback interferometry (LFI).

[0007] It is proposed that at least two different virtual camera perspectives be generated from scanned infrared laser signals of an LFI sensor unit, in particular from a single starting point, for recording the perspectively different images used for stereoscopy. This advantageously enables particularly stray light-robust eye tracking, especially since, when using an LFI sensor, only infrared light previously emitted by the LFI sensor unit itself can contribute to the signal from which the eye position is determined, in contrast to conventional photodiode sensors or CMOS / CCD camera sensors, which are susceptible to fluctuating and therefore difficult to remove by computation.Furthermore, the multiple superimposed virtual camera perspectives can advantageously achieve a particularly robust determination of the eye and / or pupil position and / or a gaze vector of the eye, preventing any slippage of the virtual retinal display. In particular, this can be achieved even when the pupil is partially covered by the eyelid when viewed from a certain perspective or when the signal from a perspective is weak for another reason. Furthermore, the proposed method, in particular through the use of LFI sensors, can advantageously extend the measurement of the eye and / or pupil position and / or a pupil diameter to include the third spatial dimension (in the case of data glasses, this can correspond, for example, to a distance to a lens of the data glasses).This advantageously enables highly accurate pupil measurement without parallax or scale errors, particularly through multi-camera calibration. Furthermore, the proposed method advantageously enables a particularly energy-efficient implementation of multiple virtual camera sensors. The required power consumption is also particularly low due to the low computational complexity of the algorithms for determining the eye and / or pupil position from the stereoscopic data. Furthermore, the use of LFI sensors advantageously enables high compactness and thus also high integration into an RGB laser scanner already present in the virtual retinal display, particularly since no additional separate detector is required. The LFI sensor unit is advantageously invisible to the user of the virtual retinal display / data glasses.Advantageously, a simple design can be achieved because an existing RGB projection system can be utilized for the virtual retinal display. A high responsiveness of eye and / or pupil tracking can be achieved, particularly because at least two pupil images can be captured simultaneously within a single scan of the virtual retinal display. Furthermore, the detectable measurement volume of a pupil scan area within which the pupil is searched is advantageously increased, particularly compared to measurements with only a single perspective.

[0008] “Data glasses” should be understood in particular as a wearable (head-mounted display) by means of which information can be added to a user’s field of vision. Data glasses preferably enable augmented reality, virtual reality and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses or VR glasses or AR glasses. In particular, the data glasses have the virtual retinal display (also called a retinal scan display or light field display), which is particularly familiar to those skilled in the art. The virtual retinal display is particularly designed to sequentially scan an image content by deflecting at least one visible laser beam from at least one temporally modulated light source, such as one or more (RGB) laser diodes of a laser projector, and to project it directly onto the retina of the user’s eye using optical elements.The method can be used in particular for eye tracking by an eye tracking system preferably integrated into the data glasses. An eye tracking system is particularly configured to track and / or record a movement of at least the eye and / or at least the pupil of the user. In particular, the LFI sensor unit with at least one LFI sensor forms a, in particular integral, component of the eye tracking system of the data glasses. The eye tracking system could be provided for detecting an eye and / or pupil position of the user's eye using the so-called "bright pupil effect," which is based in particular on the relatively high infrared reflectivity of the retina of the eye compared to neighboring / other components of the user's eye. When using the bright pupil effect, the pupil of the eye can advantageously be determined directly from the image captured by the LFI sensor unit.In particular, when a laser beam, especially a sufficiently coherent laser beam, is reflected, the retina produces a so-called laser speckle pattern. The laser speckle pattern (also called laser granulation or light granulation) is in particular a grainy-looking interference phenomenon that can be observed when optically rough object surfaces, such as the retina of the eye (cf. cones and rods of the retina), are illuminated with sufficiently coherent light. “Designed” or “intended” should be understood in particular to mean specially programmed, designed and / or equipped. The fact that an object is designed or intended for a specific function should be understood in particular to mean that the object fulfills and / or executes this specific function in at least one application and / or operating state.

[0009] The LFI sensor of the LFI sensor unit can be designed, for example, as a VCSEL, preferably a ViP-VCSEL (“vertical-cavity surface-emitting laser with integrated photo diode”). The LFI sensor, in particular the LFI sensor unit, is integrated in particular into the data glasses, e.g., into a frame, lens, or temple of the data glasses. The LFI sensor, in particular the LFI sensor unit, can be integrated into a common laser module / a common laser projector with the (RGB) laser diodes of the data glasses that generate the projected image content. Alternatively, the LFI sensor, in particular the LFI sensor unit, can also be designed separately from the laser projector with the (RGB) laser diodes that generate the projected image content. It is also conceivable for the data glasses to have more than one LFI sensor, e.g., two, three, or more than three LFI sensors. The LFI sensor of the LFI sensor unit is based on an interferometric measuring method.The LFI sensor of the LFI sensor unit can also be capable of detecting a distance to a target (e.g., the user's eye) as well as a surface velocity of the target. Specifically, the LFI sensor emits a laser beam in the infrared spectrum, which then strikes a surface with a reflectivity R at an angle. The light from the laser beam is then backscattered by this surface, re-entering a laser cavity of the LFI sensor. In the laser cavity of the LFI sensor, the backscattered light interferes with a locally oscillating field of the LFI sensor. This leads, in particular, to a modulation of the laser power of the laser source, which can be detected either by a photodiode integrated into a back reflector of the laser cavity or by a direct measurement of a voltage of the laser source.In particular, the photodiode of the LFI sensor, integrated into the laser module, is designed to detect the backscattered portion of the laser beam and convert it into an electrical signal. Furthermore, a current from the laser source can be modulated with a modulation signal, preferably a triangular one, to achieve a cyclic shift in the laser wavelength. Given known laser parameters, a beat frequency and a Doppler frequency can then be determined. From this, the surface velocity of the eye, the distance of the LFI sensor from the eye, and other eye parameters can be determined using known equations. Stereoscopy methods are known to those skilled in the art.

[0010] In particular, the infrared laser signal can be scanned by a MEMS micromirror system of a microscanner. The microscanner represents, in particular, a micro-opto-electro-mechanical system from the class of micromirror actuators for the dynamic modulation of light. A modulating movement of individual mirrors of the MEMS micromirror system can be rotational about at least one or two axes, which in particular results in a deflection of light beams incident on the individual mirrors. In particular, the scanned laser beam is scanned line by line. However, scanning the scanned laser beam column by column is also conceivable. Scanning frequencies of the MEMS micromirror system are in the range of several kilohertz, in particular in the range up to 50 kHz.

[0011] The relevant infrared laser beams of the scanned infrared laser signals that generate the images leave the LFI sensor unit / laser module / ViP together and always at the same location. The infrared laser beams of the scanned infrared laser signal can be part of a laser beam bundle emitted by the virtual retinal display, which also generates the visible images of the virtual retinal display. The "different virtual camera perspectives" and / or the differently positioned "virtual cameras" are preferably generated by different deflections of the infrared laser beams in an optical system of the virtual retinal display. Pupil detection from the images is preferably performed by evaluating a laser speckle pattern generated by the retina of the eye in the reflection signal. In particular, recording a glint is neither necessary nor intended.Advantageously, the use of a retinal reflection signal (bright pupil) for eye tracking significantly improves measurement accuracy compared to using a corneal reflection signal (glint). Furthermore, it is proposed that at least three different virtual camera perspectives be generated from the scanned infrared laser signals of the LFI sensor unit, particularly those emanating from the single starting point, for recording three perspectively different images used for stereoscopy. This can improve accuracy, particularly spatial accuracy. Furthermore, redundancy can advantageously be created.

[0012] It is also proposed that virtual cameras associated with the different virtual camera perspectives assume discrete spatial positions in the virtual retinal display. This advantageously makes it possible to achieve a simple and / or robust design. Advantageously, a particularly interference-resistant method for three-dimensional eye tracking can be achieved. For example, the discrete spatial positions of the virtual cameras associated with the different virtual camera perspectives can be achieved by using suitable optical elements, in particular suitable segmented optical elements, such as segment lenses, in the beam path of the scanned infrared laser signal, wherein the infrared laser beam of the infrared laser signal is scanned over an area comprising several segments, in particular duplication segments, of the segment lens in the method according to the invention.By using the segment lens, the beam paths of the scanned infrared laser signal are redirected segment by segment, creating spatially and perspectively offset scanning areas on the eye. Alternatively or in addition to a lens, a (segmented) prism, a (segmented) mirror, a (segmented) diffractive optical element (DOE), a (segmented) hologram, a (segmented) phase plate, or other (segmented) optical components can be used to divide the images. The discrete spatial position is, in particular, a fixed spatial position that is unchanging, at least during normal operation. In particular, the virtual camera can be viewed as a point camera that captures the reflection signal of the scanned infrared laser signal. Through a suitable design of the optical elements, in particular the segment lens, etc. (inclined surfaces, coatings, etc.)) can advantageously minimize the occurrence of back reflections within the optical system, in particular within the virtual retinal display. This can advantageously minimize the influence of internal interference with the signal reflected back from the eye, e.g., the laser speckle pattern. In particular, by using a segmented optical element, in particular the segment lens, with a full scan of the infrared laser signal via the segment lens, a number of virtual cameras with different perspectives can be generated, which corresponds to a number of segments, in particular duplication segments, of the segmented optical element.The resulting virtual cameras are then spatially offset from one another and / or positioned in different spatial alignments according to the optical functions of the segments, in particular duplication segments, of the segmented optical element, in particular the segment lens.

[0013] Alternatively, it is proposed that virtual cameras associated with the different virtual camera perspectives can be spatially positioned continuously or quasi-continuously in the virtual retinal display at least along one direction, in particular spatial direction or panning direction. This advantageously makes it possible to obtain a particularly high-resolution measurement range for eye and / or pupil tracking, in particular by creating a continuum of virtual camera positions within the virtual retinal display. For example, to achieve the spatial positionability of the virtual cameras, the scanned infrared laser beam of the scanned infrared laser signal can be guided over a pivotable, rotatable and / or translatable optical element, such as a mirror, a hologram, a lens, etc.In particular, to achieve the spatial positioning of the virtual cameras, the scanned infrared laser beam of the scanned infrared laser signal can be guided via a preferably controllable tilting mirror. At the same time, the RGB projection generating the visible image of the virtual retinal display can also be guided via the same beam path, e.g., via the same tilting mirror, as the scanned infrared laser signal. This allows the exit pupil of the RGB projection generating the visible image of the virtual retinal display to be displaced along with the infrared laser signal. This advantageously enables control of the tilting mirror, whereby the exit pupil of the RGB image of the virtual retinal display is kept in the center of the pupil of the eye by controlled centering of the infrared laser signal (e.g., in the center of the infrared image captured by the LFI sensor unit).

[0014] If the continuous or quasi-continuous adjustability of the spatial positioning of the virtual cameras is achieved by means of a tilting mirror deflecting the scanned infrared laser signals, which is in particular additional to and different from scanning mirrors of the microscanner scanning the infrared laser beam of the infrared laser signal across an eye plane of the virtual retinal display, a particularly high-resolution measuring range for eye and / or pupil tracking can advantageously be obtained.

[0015] Furthermore, it is proposed that, in order to generate at least two of the perspectively different images, at least two infrared laser signals with different wavelengths are emitted from the LFI sensor unit, which are refracted differently by an optical system of the virtual retinal display, which is passed by both infrared laser signals, in particular on a path to the eye and on a path from the eye. This advantageously allows a plurality of differently positioned virtual cameras to be created from a single scanned infrared laser signal in a structurally simple manner. The optical system preferably comprises at least one optical element, e.g., a lens, a mirror, a hologram, a DOE, a grating, etc., which generates wavelength-dependent refraction and / or diffraction. The infrared laser signals with the different wavelengths can be offset in time, e.g.,alternately one after the other, or simultaneously, e.g. in an infrared laser beam, preferably emitted by the LFI sensor unit.

[0016] If at least two of the perspectively different images are recorded simultaneously, at least in a constant operating mode of the virtual retinal display, a particularly high temporal resolution, e.g., for eye and / or pupil tracking, can advantageously be achieved. Simultaneous recording can be achieved, for example, by scanning the infrared laser signal across multiple segments, in particular multiplication segments, of a segmented optical element during a single scanning process. Simultaneous recording can be achieved, for example, by simultaneously emitting multiple infrared laser beams with different wavelengths onto a wavelength-dependent interacting optical element.

[0017] If, at least in the constant operating mode of the virtual retinal display, an infrared laser signal, in particular one having a constant wavelength, is scanned in a scan cycle of a microscanner of the virtual retinal display over at least two segments, in particular duplication segments, of a duplication optics of an optical system of the virtual retinal display, which duplication optics is arranged in particular in an optical path between the eye and the LFI sensor unit and comprises a plurality of duplication segments, in order to generate the at least two perspectively different images, a structurally simple, cost-effective and / or energy-saving construction can advantageously be achieved.

[0018] If, alternatively or additionally, at least in a multiplexing operating mode of the virtual retinal display, all perspectively different images are recorded at different times, a particularly high level of resource efficiency, in particular energy efficiency and / or computing power efficiency, can advantageously be achieved. The time-shifted recording can be achieved, for example, by a time-shifted scanning of the infrared laser signal, in particular comprising only a single wavelength, across individual segments, in particular duplication segments, of the segmented optical element. The time-shifted recording can be achieved, for example, by a time-shifted scanning of the infrared laser signal, in particular comprising only a single wavelength, across the tilting mirror at different tilt positions.The temporally offset recording can be achieved, for example, by temporally offsetting the infrared laser signals with different wavelengths onto the wavelength-dependent interacting optical element. The temporally offset images with different perspectives can be recorded completely without temporal overlap or can exhibit a partial temporal overlap (e.g., overlapping the end of one recording with the beginning of the next).If, at least in the multiplexing operating mode of the virtual retinal display, an infrared laser signal, in particular one having a constant wavelength, is scanned in temporally successive scan cycles of a microscanner of the virtual retinal display via at least one different duplication segment of a duplication optics of an optical system of the virtual retinal display, in particular arranged in an optical path between the eye and the LFI sensor unit and comprising a plurality of duplication segments, in order to generate at least two of the perspectively different images, a structurally simple, cost-effective and / or energy-saving construction can advantageously be achieved.Although the temporal offset can lead to a slowing down of the update, especially of a z-position of the eye, assuming that the data glasses with the virtual retinal display slide only slowly, a measurement error can still be avoided due to an unknown distance between the eyeball center of the eye and the lens of the data glasses.

[0019] Furthermore, it is proposed that the constant operating mode or the multiplexing operating mode of the virtual retinal display be activated depending on measurement data from an acceleration sensor assigned to the virtual retinal display. This advantageously allows for high operational flexibility, particularly combined with high overall energy efficiency. The acceleration sensor can be used, in particular, to check whether a more frequent measurement frequency is necessary due to strong accelerations acting on the virtual retinal display / data glasses. For example, the virtual retinal display / data glasses could slip more quickly during mountain biking than during walking, making more frequent stereoscopic pupil measurements (switching from the multiplexing operating mode to the constant operating mode) necessary or at least useful.The multiplexing operating mode forms, in particular, a standard mode or an energy-saving mode of the virtual retinal display / data glasses. The constant operating mode forms a high-speed mode or a precision mode of the virtual retinal display / data glasses. In particular, the constant operating mode is activated when the acceleration sensor detects high or strong accelerations, in particular accelerations above a definable acceleration limit. It is conceivable that activating the constant operating mode requires the acceleration limit to be exceeded multiple times within a definable period of time.In particular, the multiplexing operating state is activated upon detection of low or weak accelerations, in particular only accelerations below the definable acceleration limit within a definable period of time.

[0020] If a visible scanned laser signal is output from the virtual retinal display for projecting visible images onto the retina of the eye, which is guided by an optical system of the virtual retinal display via the same optical beam path, in particular via the same optical tilting mirror, as the infrared laser signal, a high degree of compactness, high degree of integration, and / or high degree of user-friendliness can advantageously be achieved. In particular, in this case, an element for adjusting, in particular pivoting or shifting, the virtual camera position, such as the tilting mirror, is adjusted only to the extent that the visible scanned laser signal still enters the pupil of the eye in all set virtual camera positions.In particular, the fact that the pupil, with a diameter of 2 mm or more, is significantly larger than the exit pupil of the RGB image can be exploited, so that different virtual camera positions can be approached without interrupting the RGB projection by the visible scanned laser signal, and in particular without the user of the virtual retinal display noticing this in their visual impression. In this case, a numerical interpolation of infrared camera calibration parameters for intermediate tilting mirror positions is preferably required, which includes the shifting of extrinsic virtual camera parameters (pose of the virtual camera in space relative to the glasses) from a finite number of known calibrated support points of virtual camera positions and the associated tilting mirror positions.

[0021] Furthermore, the virtual retinal display and / or the data glasses, in particular AR and / or VR headset, with the virtual retinal display, with the device for three-dimensional eye tracking, in particular by means of the method described above, with the laser feedback interferometry (LFI) sensor unit for recording the images of the eye, and with a computing unit for carrying out stereoscopy based on an evaluation of the images of the LFI sensor unit, are proposed, wherein the LFI sensor unit and / or an optical system of the virtual retinal display are configured to generate at least two different virtual camera perspectives for recording the images used for the stereoscopy from scanned infrared laser signals of the LFI sensor unit, which emanate from a single starting point, in particular leave the LFI sensor unit.This makes it possible, in particular, to achieve the advantages already described in connection with the proposed method. A “computing unit” should be understood in particular as a unit with an information input, an information processing unit, and an information output. The computing unit advantageously has at least one processor, a memory, input and output means, further electrical components, an operating program, control routines, control routines, and / or calculation routines. The components of the computing unit are preferably arranged on a common circuit board and / or advantageously arranged in a common housing, in particular the data glasses. Alternatively, the computing unit can also be arranged externally to the data glasses, for example in a mobile device connected to the data glasses, such as a smartphone or tablet, or in a cloud.

[0022] The method according to the invention, the virtual retinal display according to the invention, and the data glasses according to the invention are not intended to be limited to the application and embodiment described above. In particular, the method according to the invention, the virtual retinal display according to the invention, and the data glasses according to the invention may have a number of individual elements, components, units, and method steps that differs from the number stated herein to fulfill a functionality described herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0023] Drawing: Further advantages will become apparent from the following description of the drawing. The drawing illustrates three exemplary embodiments of the invention. The drawing, the description, and the claims contain numerous features in combination. Those skilled in the art will also expediently consider the features individually and combine them into useful further combinations.

[0024] They show:

[0025] Fig. 1 schematically shows part of a virtual retinal display of data glasses,

[0026] Fig. 2 shows a schematic cross-section through an eye illuminated by an infrared laser signal,

[0027] Fig. 3 shows an enlarged section of Fig. 2 in the area of ​​a retina of the eye,

[0028] Fig. 4 an inverted raw signal of an LFI sensor unit of the virtual retinal display for a two-dimensional scan over the eye,

[0029] Fig. 5 is a schematic representation of the virtual retinal display with an optical system and the LFI sensor unit,

[0030] Fig. 6 is a schematic representation of a duplication optics of the optical system with a schematically indicated scan path of the infrared laser signal,

[0031] Fig. 7 exemplary beam paths for the virtual retinal display with two virtual cameras,

[0032] Fig. 8a shows an exemplary scan area of ​​a full scan of a projector unit having the LFI sensor unit with two virtual cameras,

[0033] Fig. 8b shows a first exemplary scan area of ​​a full scan of the projector unit with four virtual cameras,

[0034] Fig. 8c shows a second exemplary scan area of ​​a full scan of the projector unit with four virtual cameras,

[0035] Fig. 9 shows an inverted measurement signal of the LFI sensor unit of an exemplary embodiment of the duplication optics with four duplication segments, Fig. 10 shows a schematic geometric illustration of a method for determining the gaze vector,

[0036] Fig. 11 is a schematic flow diagram of the method for three-dimensional eye tracking of the eye of the user of the virtual retinal display in the data glasses by means of stereoscopy,

[0037] Fig. 12 is a schematic representation of an alternative virtual retinal display with an alternative optical system and an alternative LFI sensor unit and

[0038] Fig. 13 is a schematic representation of another alternative virtual retinal display with another alternative optical system.

[0039] Description of the embodiments

[0040] Figure 1 schematically shows part of a virtual retinal display 12a of a data glasses 14a. The data glasses 14a can form an AR headset or a VR headset. The virtual retinal display 12a has a device for three-dimensional eye tracking using stereoscopy. The virtual retinal display 12a is provided for carrying out a method for three-dimensional eye tracking of an eye 10a of a user of the virtual retinal display 12a of the data glasses 14a using stereoscopy. In the method, an infrared laser signal 18a is scanned across the eye 10a from multiple directions / perspectives, and the backscattered light is observed using a laser feedback interferometry (LFI) sensor of an LFI sensor unit 20a in order to determine a current position of a pupil 54a of the eye 10a.

[0041] The device has a projector unit 56a. The projector unit 56a includes a laser module 58a. The laser module 58a can be integrated into a frame (not shown) of the data glasses 14a. The laser module 58a includes an RGB laser diode system 60a. The virtual retinal display 12a is provided for outputting a visible scanned laser signal 46a for projecting visible images onto a retina 48a of the eye 10a. The RGB laser diode system 60a is provided for generating the visible scanned laser signal 46a. The projector unit 56a, in particular the laser module 58a, includes the LFI sensor unit 20a. The LFI sensor unit 20a is provided for outputting an infrared laser signal 18a. The LFI sensor unit 20a comprises an infrared laser diode 62a for generating the infrared laser signal 18a. The LFI sensor unit 20a is designed to capture images 16a of the eye 10a.The LFI sensor unit 20a is provided for detecting reflection signals generated by the eye 10a from the self-emitted infrared laser signal 18a. The LFI sensor unit 20a includes an infrared photodiode 64a for detecting the reflection signals generated by the eye 10a. The photodiode 64a can be embodied as a silicon photodiode. The photodiode 64a can be integrated into a Bragg reflector of the infrared laser diode 62a. This advantageously allows the bright pupil effect to be utilized, in which, similar to a red-eye effect, the pupil 54a of the eye 10a can be directly detected from an image 16a (see Fig. 4) of the infrared photodiode 64a.

[0042] The projector unit 56a includes a microscanner 36a. The microscanner 36a includes at least one scanning mirror 34a. The microscanner 36a could include either two 1D micromirrors or one 2D micromirror. The microscanner 36a is configured to scan the visible laser signal 46a using the scanning mirror 34a. The microscanner 36a is configured to scan the infrared laser signal 18a using the scanning mirror 34a. The scanning of the laser signals 18a, 46a could alternatively be performed by a DMD micromirror array, a prism array, an optical phased array, an LCOS (Liquid Crystal on Silicon) array, a digital light processor, a liquid lens array, or an electrically controllable phase plate. The virtual retinal display 12a comprises an optical system 52a, which redirects the laser signals 18a, 46a output by the projector unit 56a toward the eye 10a.The optical system 52a can comprise one or more optical elements, e.g., mirrors, diffractive optical elements, holographic optical elements, prisms, partially transparent mirrors, lenses, etc. The optical system 52a preferably generates beams directed essentially parallel to the eye 10a. At least part of the optical system 52a, e.g., a reflection HOE, can be integrated into a spectacle lens 66a of the data glasses 14a. By means of the known deflection of the microscanner 36a, in particular its scanning mirror 34a, the image 16a of the eye 10a can be generated from the signal strengths of the reflection signals determined by the infrared photodiode 64a. The optical system 52a comprises a duplication optics 42a. The duplication optics 42a are arranged at the output of the projector unit 56a, in particular of the microscanner 36a.

[0043] The virtual retinal display 12a, in particular the data glasses 14a, has an acceleration sensor 44a. The acceleration sensor 44a is provided for detecting accelerations and / or vibrations of the virtual retinal display 12a, in particular the data glasses 14a.

[0044] If the infrared laser beam of the infrared laser signal 18a falls on the retina 48a, this leads to strong backscattering, since the reflectivity of the retina 48a is stronger than that of the other components of the eye 10a, such as the iris or the sclera. In addition, the infrared laser beam incident on the eye 10a can be collimated such that the eye 10a itself focuses the infrared laser beam onto the retina 48a (see Fig. 2), whereby the light backscattered by the retina 48a (quasi-Lambertian) is coherently directed backward onto the LFI sensor of the LFI sensor unit 20a, where it leads to a significantly increased amplitude modulation of the optical power of the laser of the infrared laser diode 62a for a scan area in which the infrared laser beam is imaged onto the retina 48a through the pupil 54a. The amplitude of the backscattered laser light is modulated by speckle.The speckle is generated by the optical roughness of the retina 48a (see Fig. 3). The roughness of the retina 48a is related to the anatomy of the eye 10a and behaves roughly similarly for all people. Fig. 2 schematically shows a cross-section through the eye 10a illuminated by the infrared laser signal 18a.

[0045] The infrared laser beam of the infrared laser signal 18a penetrates the cornea 68a of the eye 10a and is focused by the lens 70a of the eye 10a onto the retina 48a. There, the infrared laser beam of the infrared laser signal 18a strikes a rough surface 72a of the retina 48a with its coherent wavefront. A portion of the incoming wavefront is directly reflected by the surface 72a of the retina 48a, whereby the wavefront 74a of the reflected wave is distorted by the signal propagation times through the rough surface 72a of the retina 48a. A further portion of the incoming wavefront enters the upper tissue layers of the retina 48a and is reflected from there, which also leads to a distortion of the retroreflected wavefront 74a.These two effects result in constructive or destructive interference in the LFI sensor of the LFI sensor unit 20a, so that a laser speckle pattern with bright speckles and dark speckles is formed, normally distributed, in the imaged area of ​​the retina 48a and thus of the pupil 54a in the obtained image 16a. On the iris and surrounding structures of the eye 10a, however, the reflectivity is low and the infrared laser beam of the infrared laser signal 18a is not focused, so that a solid angle for the back reflection in the direction of the LFI sensor of the LFI sensor unit 20a is comparatively much smaller. In addition, the phase position of neighboring positions on the iris varies greatly due to roughness, so that only minimal interference occurs in the LFI sensor due to the reflection signal of the iris. The image areas not attributable to the retina 48a therefore appear dark in the recorded image 16a.Essentially, only coherent infrared light, which has passed through the beam path between infrared photodiode 64a and retina 48a in both directions, contributes to structures in image 16a through interference in the laser resonator of infrared photodiode 64a, so that non-coherent stray light or other wavelengths cannot interfere with the signal relevant to LFI sensor unit 20a. This highly selective effect is now exploited to determine a particularly efficient algorithm for determining the position of pupil 54a from the obtained perspective-different images 16a. To isolate the signal contribution of pupil 54a in the overall signal, a partial area of ​​the histogram must first be selected from image 16a. Fig. 4 shows the raw signal of the amplitude signal of LFI sensor unit 20a for a two-dimensional scan across eye 10a inverted. The dark area 76a in image 16a represents the area of ​​the pupil 54a.In the uninverted image 16a, the dark area 76a would appear brighter compared to the rest of the image 16a. Due to the higher reflectivity of the retina 48a, the measured amplitude of the optical power of the LFI sensor in this area 76a differs from the rest of the image 16a in amplitude (height of the pixel values). Thus, the corresponding signal contribution can be separated from the residual signal using a histogram transformation and thresholding. From an image of the pupil 54a extracted in this way, a three-dimensional pupil position in the eyeglass coordinate system (x, y, z) of the data glasses 14a, a pupil diameter of the pupil 54a, and other properties of a 3D eye model can be determined. The structure that results in an image 16a with a single pupil image results in a single virtual camera sensor. This allows the eye 10a to be scanned and the pupil 54a to be detected, so that pupil tracking can basically be realized.The disadvantage of only a single virtual camera sensor, however, is that the position of the pupil 54a can only be determined relative to an optical element of the optical system 52a, and a rotation of the eye 10a cannot be distinguished from a movement of the data glasses 14a relative to the eye 10a. This disadvantage can be remedied by a stereoscopic approach with multiple virtual cameras 28a, 30a (see Fig. 5). The device has a computing unit 50a (see Fig. 1), which is provided for performing the stereoscopy based on an evaluation of the perspectively different images 16a of the LFI sensor unit 20a.

[0046] Figure 5 shows a schematic representation of the virtual retinal display 12a with the optical system 52a and the LFI sensor unit 20a. The LFI sensor unit 20a and / or the optical system 52a is configured to generate at least two different virtual camera perspectives 22a, 24a for recording the images 16a used for stereoscopy from scanned infrared laser signals 18a of the LFI sensor unit 20a, which emanate from a single starting point 26a, in particular leave the LFI sensor unit 20a. Each of the different virtual camera perspectives 22a, 24a can be assigned a virtual camera 28a, 30a. The optical system 52a has the duplication optics 42a. In the embodiment of Fig. 5, the duplication optics 42a is designed as a segment lens.The multiplication optics 42a comprises a plurality of multiplication segments 38a, 40a. The multiplication optics 42a are intended to convert the scanned infrared laser signal 18a into a plurality of partial signals 78a, 80a, each of which illuminates the eye 10a from different perspectives. Each of the multiplication segments 38a, 40a deflects an incident light beam of the same wavelength in a different direction. This creates the different virtual camera perspectives 22a, 24a. In this embodiment, virtual cameras 28a, 30a associated with the different virtual camera perspectives 22a, 24a occupy discrete spatial positions in the virtual retinal display 12a.

[0047] The infrared laser beam of the scanned infrared laser signal 18a is scanned by the microscanner 36a across several (all) duplication segments 38a, 40a of the duplication optics 42a (see Fig. 6). This optically duplicates a beam path of the scanned laser signal 18a. If the duplication optics 42a is arranged between the microscanner 36a and the spectacle lens 66a, as shown in Fig. 1, each of the partial signals 78a, 80a resulting from the duplication is redirected in an offset manner toward the eye 10a via an optical element (e.g., a reflection HOE) of the optical system 52a integrated into the spectacle lens 66a. This creates offset scanning areas 82a, 84a, 86a, 88a, particularly in an eye plane of the virtual retinal display 12a (see Figures 8a to 8c).Each of the reproduction segments 38a, 40a generates a virtual camera 28a, 30a, which are arranged offset from one another according to the respective optical function of the respective reproduction segment 38a, 40a.

[0048] Figure 7 shows exemplary beam paths for a virtual retinal display 12a with two virtual cameras 28a, 30a. The two virtual cameras 28a, 30a are created from two spatially offset scans via the optical element of the optical system 52a integrated in the spectacle lens 66a. The two spatially offset scans via the optical element integrated in the spectacle lens 66a originate from a scan of the infrared laser beam emitted by the projector unit 56a via two duplication segments 38a, 40a of the duplication optics 42a. In the case shown, the projector unit 56a is integrated into a temple piece 94a of the data glasses 14a. The virtual cameras 28a, 30a each view the eye 10a from a different perspective through the spectacle lens 66a. The optical element in the spectacle lens 66a serves as the image plane 100a.Due to the offset of the centers of the two virtual cameras 28a, 30a, an image of the pupil 54a of the eye 10a appears as a circle in the images 16a of the virtual cameras 28a, 30a spanned by the respective scan areas 82a, 84a. The circles depicting the pupil 54a are spatially offset or arranged differently in the images 16a of the scan areas 82a, 84a (see also Figs. 8a to 8c). In Fig. 7, the beam paths generating the pupil circles 126a, 126'a in the images 16a, 16'a are provided with the reference numerals 90a and 90'a, respectively, and 92a and 92'a, respectively. If the eye 10a rotates, this leads to a translation 128a of the pupil 54a in the image plane 100a of the two virtual cameras 28a, 30a as well as to an elliptical distortion 130a of the pupil circles 126a, 126'a, which depends on the respective camera perspective 22a, 24a of the respective virtual camera 28a, 30a.The geometric properties of the virtual cameras 28a, 30a relative to each other are known and firmly defined through suitable calibration. Therefore, the rotation of the eye 10a and a translation of the data glasses 14a relative to the eye 10a can be clearly differentiated from the images 16a of the two scan areas 82a, 84a. Advantageously, this allows a robust detection of the position of the pupil 54a. From homographic images of the virtual cameras 28a, 30a, the position and orientation of the pupil 54a in the virtual retinal display 12a can thus be clearly determined, and from this, a pupil diameter and a gaze vector can also be clearly assigned. Other stereo image processing methods for determining the gaze vector are conceivable, such as the use of neural networks (DNNs, CNNs, LSTMs, etc.), photogrammetry, a model-based estimator, etc.

[0049] Figures 8a, 8b, and 8c each show exemplary scan areas 82a, 84a, 86a, 88a of a full scan of the projector unit 56a. The position of the eye 10a in the scan areas 82a, 84a, 86a, 88a is also sketched in Figures 8a to 8c. The images 16a of the respective virtual cameras 28a, 30a can thus also be read from Figures 8a to 8c. In the example of Figure 8a, the virtual retinal display 12a generates only two virtual cameras 28a, 30a. In the examples of Figures 8b and 8c, the virtual retinal display 12a generates four virtual cameras 28a, 30a each. The corresponding scan areas 82a, 84a, 86a, 88a overlap each other. Each scan area 82a, 84a, 86a, 88a hits the pupil 54a. Each scan area 82a, 84a, 86a, 88a is generated by a different multiplication segment 38a, 40a of the multiplication optics 42a. To enable stereoscopy, at least two virtual cameras 28a, 30a must be implemented.By adding additional virtual cameras 28a, 30a (e.g., by adding additional duplication segments 38a, 40a to the duplication optics 42a or by modifying the optical element integrated into the spectacle lens 66a, etc.), the robustness of eye tracking can be increased. With at least three virtual cameras 28a, 30a, a triangulation algorithm can be used to determine the gaze vector of the eye 10a. The virtual camera orientations shown in Figures 8a to 8c represent particularly advantageous implementations.

[0050] Figure 9 shows, by way of example, an inverted measurement signal from the LFI sensor unit 20a of an exemplary embodiment of the multiplication optics 42a with four multiplication segments 38a, 40a / with four virtual cameras 28a, 30a. The exemplary measurement signal comprises four images of the same pupil 54a, each of which was recorded from different virtual camera perspectives 22a, 24a.

[0051] Figure 10 illustrates an alternative to triangulation and advantageously particularly efficient method for determining the gaze vector. This method reduces complexity and is independent of the orientation of the individual virtual camera perspectives 22a, 24a, thus representing a generalized solution to the problem of determining the gaze vector. A simplified model of the eye 10a consisting of a sphere and the pupil 54a as a disk on the sphere is assumed. Subsequently, the overlapping camera images (cf. Figs. 8a to 8c and Fig. 9) of the virtual cameras 28a, 30a are transferred into a common coordinate system, and the (elliptically distorted) pupil contours 96a, 98a contained therein are displayed in a common image plane 100a. Three-dimensional cones 102a, 104a are then calculated from the origins of the virtual cameras 28a, 30a and the corresponding pupil contours 96a, 98a.From an intersection point 106a of the two cones 102a, 104a, the pupil diameter and the rotation of the pupil 54a relative to the virtual cameras 28a, 30a can now be determined. A normal on the constructed (disk-like) pupil 54a then corresponds to the gaze vector 108a (gaze vector) in three-dimensional space. By observing the gaze vector 108a over time and projecting a plurality of pupils 54a with different virtual cameras 28a, 30a, a center 110a of the eye 10a relative to the data glasses 14a can also be determined. This advantageously creates a fully described reference system. By using additional virtual cameras 28a, 30a, e.g., a third virtual camera, further cones can be generated, further improving the robustness of the gaze angle determination.Furthermore, the system is thus advantageously overdetermined and there is no need for a temporal determination of the center 110a of the eye 10a via a projection of the pupil 54a over time.

[0052] Since geometries of virtual nodes are hard-coded in holograms, when holograms are used as an optical element of the optical system 52a integrated into the spectacle lens 66a, the resulting virtual single-pixel cameras can be calibrated using a stereo calibration algorithm known from the prior art. Thus, a rotation and translation of the virtual cameras 28a, 30a relative to one another and in relation to the image plane 100a are determined (C. extrinsic camera calibration parameters"), and the imaging and camera properties are determined ("intrinsic camera calibration parameters").With known rotation and translation, the position of the pupil 54a in all three spatial dimensions, the gaze vector 108a, and the center 110a of the eye 10a can finally be reconstructed three-dimensionally using triangulation and / or stereo matching (see also the publication "Holography Single-Pixel Stereo Camera Sensor for Calibration-free Eye-Tracking in Retinal Projection Augmented Reality Glasses" by Meyer et al., 2021). In contrast to the use of only one virtual camera, systematic measurement errors (parallax errors and scale errors) are advantageously avoided when the data glasses 14a are moved away from the eye 10a or toward the eye 10a (= along z) out of the calibrated image plane 100a.

[0053] The camera calibration parameters of the virtual cameras 28a, 30a are determined fully automatically during the production of the data glasses 14a by presenting suitable calibration objects (e.g., plates or solid bodies with defined structures visible in the IR image) at one or more locations in the measurement volume and by calculating suitable camera model parameters. The calculated camera model parameters are permanently stored on the data glasses 14a and are used to calculate the pupil properties in the field. The calibration target is designed to enable low-noise infrared LFI image acquisition. The calibration target has a high infrared reflectivity of a background, which is additionally moved during calibration to avoid speckles. Commercially available image processing algorithms (thresholding, ellipse fitting, etc.) are applied to the calibration images thus obtained.) in order to be able to carry out the three-dimensional camera calibration for all virtual cameras 28a, 30a in the entire measurement volume of the virtual retinal display 12a.

[0054] Figure 11 shows a schematic flow diagram of the method for three-dimensional eye tracking of the eye 10a of the user of the virtual retinal display 12a in the data glasses 14a using stereoscopy. The method is based on an evaluation of the perspectively different images 16a of the same eye 10a, which are recorded using laser feedback interferometry (LFI). In the method, perspectively different images 16a are used to perform the stereoscopy. The perspectively different images 16a are recorded from at least two, preferably at least three different virtual camera perspectives 22a, 24a, wherein the infrared laser signals 18a of the LFI sensor unit 20a are used to record the images 16a, all of which originate from a single starting point 26a. In at least one method step 112a, the infrared laser signal 18a is generated and output by the LFI sensor unit 20a.The infrared laser signal 18a can comprise only light of a single infrared wavelength (see embodiments of Figures 1 to 10 and 13) or light of several different (discrete) infrared wavelengths (see embodiment of Figure 12). If the infrared laser signal 18a comprises several different infrared wavelengths, these can be output simultaneously or offset in time. Additionally, in method step 112a, the visible scanned laser signal 46a is output by the virtual retinal display 12a for projecting visible images onto the retina 48a of the eye 10a.

[0055] In at least one further method step 114a, the infrared laser signal 18a is directed onto the user's eye 10a via the optical system 52a of the virtual retinal display 12a. The different virtual camera perspectives 22a, 24a are generated by the optical system 52a of the virtual retinal display 12a. The virtual cameras 28a, 30a are generated / positioned in space by the optical system 52a of the virtual retinal display 12a. In the embodiments of Figures 1 to 10 and 12, the virtual cameras 28a, 30a associated with the different virtual camera perspectives 22a, 24a occupy discrete spatial positions in the virtual retinal display 12a.In the embodiment of Figures 1 to 10, the discrete spatial positions of the virtual cameras 28a, 30a are generated by multiplying the scanned infrared laser signal 18a by means of the multiplying optics 42a having multiple multiplying segments 38a, 40a. In the embodiment of Fig. 12, the discrete spatial positions of the virtual cameras 28a, 30a are generated by using multiple infrared laser signals 18a with different wavelengths. The multiple infrared laser signals 18a with different wavelengths are refracted differently by the optical system 52a of the virtual retinal display 12a, which is passed through by all infrared laser signals 18a on the way to the eye 10a and on the way from the eye 10a. In the embodiment of Fig.13, the virtual cameras 28a, 30a associated with the different virtual camera perspectives 22a, 24a can be spatially positioned continuously or quasi-continuously in the virtual retinal display 12a at least along one direction, in particular spatial direction or panning direction. The continuous or quasi-continuous adjustability of the spatial positioning of the virtual cameras 28a, 30a is achieved in this case by means of a tilting mirror 32c that deflects the scanned infrared laser signals 18a (see Fig. 13). Additionally, in method step 114a, the visible laser signal 46a is directed by the optical system 52a of the virtual retinal display 12a along the same optical beam path as the infrared laser signal 18a.

[0056] In at least one further method step 116a, a measurement signal is acquired by the LFI sensor unit 20a. The infrared laser signals 18a are reflected by the eye 10a and scattered back into the LFI sensor unit 20a by the optical system 52a. In the LFI sensor unit 20a, the backscattered infrared light of the infrared laser signals 18a generates the measurement signal. In method step 116a, each virtual camera 28a, 30a generates an image 16a from the acquired measurement signals. In at least one further method step 118a, the images 16a of the multiple virtual cameras 28a, 30a are (algorithmically) evaluated. By evaluating the images 16a of the plurality of virtual cameras 28a, 30a, the three-dimensional eye and / or pupil position of the eye 10a in the virtual retinal display 12a and / or the gaze vector 108a of the eye 10a in the virtual retinal display 12a are determined in method step 118a.In at least one further method step 120a, an eye tracking function of the virtual retinal display 12a / data glasses 14a is carried out based on the determined eye and / or pupil positions and / or based on the detected gaze vectors.

[0057] The eye tracking function of the virtual retinal display 12a / data glasses 14a includes a constant operating mode. In the constant operating mode, several perspectively different images 16a are recorded simultaneously. For example, in the constant operating mode of the virtual retinal display 12a, to generate the several perspectively different images 16a, the infrared laser signal 18a is scanned in one scan cycle of the microscanner 36a via at least two duplication segments 38a, 40a of the duplication optics 42a of the optical system 52a arranged in the optical path between the eye 10a and the LFI sensor unit 20a. The eye tracking function of the virtual retinal display 12a / data glasses 14a includes a multiplexing operating mode. In the multiplexing operating mode, several (all) perspectively different images 16a are recorded with a temporal offset.For example, in the multiplexing operating mode for generating the plurality of perspectively different images 16a, the infrared laser signal 18a is scanned in temporally successive scan cycles of the microscanner 36a via at least one different replication segment 38a, 40a of the replication optics 42a of the optical system 52a comprising a plurality of replication segments 38a, 40a.

[0058] In at least one further method step 122a, the constant operating mode or the multiplexing operating mode of the virtual retinal display 12a is activated depending on measurement data from the acceleration sensor 44a assigned to the virtual retinal display 12a. If high accelerations are detected, the constant operating mode is activated. For example, a high acceleration can be detected by exceeding an acceleration limit once or multiple times (close in time). If low accelerations are detected, the multiplexing operating mode is activated. For example, a low acceleration can be detected by not exceeding an acceleration limit within a period of time.

[0059] Figures 12 and 13 show two further embodiments of the invention. The following descriptions and the drawings are essentially limited to the differences between the embodiments, whereby with regard to components with the same designation, in particular with regard to components with the same reference numerals, reference can in principle also be made to the drawings and / or the description of the other embodiments, in particular Figures 1 to 11. To distinguish the embodiments, the letter a is placed after the reference numerals of the embodiment in Figures 1 to 11. In the embodiments in Figures 12 and 13, the letter a is replaced by the letters b and c.

[0060] Figure 12 shows a schematic representation of an alternative virtual retinal display 12b with an alternative optical system 52b and an alternative LFI sensor unit 20b. The alternative LFI sensor unit 20b generates infrared laser signals 18b, 18'b with at least two different wavelengths. The alternative optical system 52b comprises an optical element 124b. The optical element 124b generates wavelength-dependent refraction and / or diffraction. The two infrared laser signals 18b, 18'b with the different wavelengths are refracted differently by the optical system 52b, in particular by the optical element 124b. This generates the two virtual cameras 28b, 30b, which are spatially spaced apart (discretely positioned) from one another, in the alternative virtual retinal display 12b.The optical element 124b has a similar splitting effect on the infrared laser signal 18b, 18'b at different wavelengths as the duplication optics 42a of the first embodiment at the same wavelengths.

[0061] Figure 13 shows a schematic representation of a further alternative virtual retinal display 12c with a further alternative optical system 52c and a projector unit 56c with an LFI sensor unit 20c. The LFI sensor unit 20c generates an infrared laser signal 18c. The further alternative optical system 52c comprises a tilting mirror 32c. The tilting mirror 32c is in addition to and different from scanning mirrors 34c of a microscanner 36c of the projector unit 56c, which scans the infrared laser signal 18c across an eye plane of the virtual retinal display 12c. The further alternative optical system 52c generates at least one virtual camera 28c. By adjusting the tilting mirror 32c, a continuous or quasi-continuous adjustability of the spatial positioning of the virtual camera 28c is generated.This allows different virtual camera perspectives 22c, 24c to be set and perspectively different images 16c to be taken.

Claims

Claims 1. A method for three-dimensional eye tracking of an eye (10a-c) of a user of a virtual retinal display (12a-c) (retinal scan display), for example in data glasses (14a-c), by means of stereoscopy, which is based on an evaluation of perspectively different images (16a-c) of the same eye (10a-c) recorded by means of laser feedback interferometry (LFI), characterized in that at least two different virtual camera perspectives (22a-c, 24a-c) for recording the perspectively different images (16a-c) used for the stereoscopy are generated from scanned infrared laser signals (18a-c) of an LFI sensor unit (20a-c), in particular emanating from a single starting point (26a-c).

2. Method according to claim 1, characterized in that at least three different virtual camera perspectives (22a-c, 24a-c) for recording three perspectively different images (16a-c) used for stereoscopy are generated from the scanned infrared laser signals (18a-c) of the LFI sensor unit (20a-c), in particular emanating from the individual starting point (26a-c).

3. Method according to claim 1 or 2, characterized in that virtual cameras (28a-b, 30a-b) associated with the different virtual camera perspectives (22a-b, 24a-b) assume discrete spatial positions in the virtual retinal display (12a-b).

4. Method according to claim 1 or 2, characterized in that virtual cameras (28c, 30c) associated with the different virtual camera perspectives (22c, 24c) are spatially positioned continuously or quasi-continuously in the virtual retinal display (12c) at least along one direction, in particular spatial direction or panning direction. can be.

5. The method according to claim 4, characterized in that the continuous or quasi-continuous adjustability of the spatial positioning of the virtual cameras (28c) is achieved by means of a tilting mirror (32c) which deflects the scanned infrared laser signals (18c), which is in particular additional to and different from scanning mirrors (34c) of a microscanner (36c) which scans the infrared laser signal (18c) over an eye plane of the virtual retinal display (12c).

6. Method according to one of the preceding claims, characterized in that for generating at least two of the perspectively different images (16b) at least two infrared laser signals (18b, 18'b) are emitted with different wavelengths by the LFI sensor unit (20b), which are refracted differently by an optical system (52b) of the virtual retinal display (12b), which is passed by both infrared laser signals (18b, 18'b), in particular on a path to the eye (10b) and on a path from the eye (10b).

7. Method according to one of the preceding claims, characterized in that at least in a constant operating mode of the virtual retinal display (12a-c) at least two of the perspectively different images (16a-c) are recorded simultaneously.

8. The method according to claim 7, characterized in that at least in the constant operating mode of the virtual retinal display (12a) for generating the at least two perspectively different images (16a), an infrared laser signal (18a), in particular having a constant wavelength, is transmitted in a scan cycle of a microscanner (36a) of the virtual retinal display (12a) via at least two duplication segments (38a, 40a) of a duplication optics (42a) of an optical system (52a) of the virtual retinal display (12a), in particular arranged in an optical path between the eye (10a) and the LFI sensor unit (20a) and comprising a plurality of duplication segments (38a, 40a). is scanned.

9. Method according to one of the preceding claims, characterized in that at least in one multiplexing operating mode of the virtual retinal display (12a-c) all perspectively different images (16a-c) are recorded at different times.

10. The method according to claim 9, characterized in that at least in the multiplexing operating mode of the virtual retinal display (12a) for generating at least two of the perspectively different images (16a), an infrared laser signal (18a), in particular having a constant wavelength, is scanned in temporally successive scan cycles of a microscanner (36a) of the virtual retinal display (12a) via in each case at least one different duplication segment (38a, 40a) of a duplication optics (42a) of an optical system (52a) of the virtual retinal display (12a), in particular arranged in an optical path between the eye (10a) and the LFI sensor unit (20a) and comprising a plurality of duplication segments (38a, 40a).

11. Method at least according to claims 7 and 9, characterized in that the constant operating mode or the multiplexing operating mode of the virtual retinal display (12a-c) is activated as a function of measurement data from an acceleration sensor (44a-c) assigned to the virtual retinal display (12a-c).

12. Method according to one of the preceding claims, in particular at least according to claim 5, characterized in that a visible scanned laser signal (46a-c) for projecting visible images onto a retina (48a-c) of the eye (10a-c) is output from the virtual retinal display (12a-c), which is directed by an optical system (52a-c) of the virtual retinal display (12a-c) via the same optical beam path, in particular via the same optical tilting mirror (32a-c), as the infrared laser signal (18a-c).

13. Virtual retinal display (12a-c), in particular for data glasses (14a-c), with a device for three-dimensional eye tracking, in particular by means of a method according to one of the preceding claims, with a laser feedback interferometry (LFI) sensor unit (20a-c) for recording images (16a-c) of an eye (10a-c), and with a computing unit (50a-c) for carrying out stereoscopy based on an evaluation of the images (16a-c) of the LFI sensor unit (20a-c), characterized in that the LFI sensor unit (20a-c) and / or an optical system (52a-c) of the virtual retinal display (12a-c) are configured to generate, from scanned infrared laser signals (18a-c) of the LFI sensor unit (20a-c), which emanate from a single starting point (26a-c), in particular the LFI sensor unit (20a-c), at least two different virtual camera perspectives (22a-c,24a-c) for recording the images (16a-c) used for stereoscopy.

14. Data glasses (14a-c), in particular AR and / or VR headset, with a virtual retinal display (12a-c) according to claim 13.

Images