Method and device for spatially reconstructing an eye and determining a viewing vector of the eye

Abstract

The invention relates to a method (800), in particular for operating smart glasses (500), comprising: illuminating (806) an eye (2), which comprises an eyeball (4), a cornea (8), a sclera (6) and a pupil (10), with diffusely scattered light (107a, 107b) of a laser beam (106, 106a, 106b, 106c), for example an infrared laser beam, in particular a modulated infrared laser beam, by means of a plurality of scattering portions (202a, 202b; 602a … 606e '; 702a … 702f '), the scattering portions (202a, 202b; 602a … 606e '; 702a … 702f ') being irradiated by the laser beam (106, 106a, 106b, 106c) for specific emission angles (α), as a result of which the laser beam (106, 106a, 106b, 106c) is diffusely scattered; sensing (808), by means of an event camera (300), a position (304a, …, 304f ') of a plurality of glints (108a, …, 108f) which the diffusely scattered laser beam (106, 106a, 106b, 106c) produces on the eye (2), the number of glints (108a, …, 108f) corresponding to the number of scattering portions (202a, 202b; 602a … 606e '; 702a … 702f '); determining (810) a spatial position (32) of the cornea (8) of the eye (2) according to the position (304a, …, 304f ') of the sensed glints (108a, …, 108f), a geometric arrangement of the event camera (300) and of the plurality of scattering portions (202a, 202b; 602a … 606e '; 702a … 702f '), a specified refractive power of the cornea (8), a specified radius (50) of the cornea (8), and a specified distance between a center (36) of the cornea (8) and a specified center point (38) of the sclera (6), and / or determining (812) a spatial position (32) of the cornea (8) of the eye (2) according to the position (304a, …, 304f ') of the sensed glints (108a, …, 108f) and a geometric arrangement of the event camera (300) and of the plurality of scattering portions (202a, 202b; 602a … 606e '; 702a … 702f '), the plurality of scattering portions (202a, 202b; 602a … 606e '; 702a … 702f ') comprising at least four scattering portions (202a, 202b; 602a … 606e '; 702a … 702f ').

Description

[0001] Description

[0002] title

[0003] Method and device for spatial reconstruction of an eye and determination of a gaze vector of the eye

[0004] State of the art

[0005] The present invention relates to a method and a device for the spatial reconstruction of an eye and for determining a gaze vector of the eye.

[0006] To estimate the pupil ellipse of an eye, current technology uses scanning laser systems with a photodiode integrated into the laser system or externally. The pupil contour of the eye appears bright or dark. Using an image processing algorithm, the dark or bright pupil contour is detected in an image from the photodiode, and the pupil ellipse is estimated. The photodiode is evaluated at equidistant intervals with a fixed sampling rate.

[0007] Alternatively, a combination of eye illumination using infrared LEDs arranged in a spectacle frame is realized, with a photo camera sensor and an event camera capturing the eye to determine the eye's direction of gaze.

[0008] The aforementioned approaches require complex image generation by the photodiodes or camera sensors, as well as matching the image generation sampling rate to the laser system or illumination. This results in high computational complexity and energy consumption. Furthermore, photodiodes and camera sensors are highly sensitive to light interference. Therefore, a method and device for determining a gaze vector are desirable that reduces computational complexity and energy consumption and is more robust against interference.

[0009] Disclosure of the invention

[0010] This is achieved by an apparatus and a method according to the independent claims.

[0011] The method, in particular for operating data glasses, comprises: illuminating an eye, which comprises an eyeball, a cornea, a sclera, and a pupil, with diffusely scattered light from a laser beam, for example an infrared laser beam, in particular a modulated infrared laser beam, by means of a plurality of scattering sections, wherein the scattering sections are irradiated by the laser beam for specific radiation angles, whereby the laser beam is diffusely scattered; detecting a position of a plurality of glints generated by the diffusely scattered laser beam on the eye by means of an event camera, wherein a number of glints corresponds to a number of scattering sections;determining a spatial position of the cornea of ​​the eye depending on the position of the detected glints, a geometric arrangement of the event camera and the plurality of scattering sections, a predetermined refractive power of the cornea, a predetermined radius of the cornea and a predetermined distance between a center of the cornea and a predetermined center of the sclera, and / or determining a spatial position of the cornea of ​​the eye depending on the position of the detected glints and a geometric arrangement of the event camera and the plurality of scattering sections, wherein the plurality of scattering sections comprises at least four scattering sections;

[0012] Because the scattered sections are illuminated by the laser beam only at specific beam angles, the characteristic of a flashing light source is achieved. This eliminates the need for flashing LEDs (light-emitting diodes) that are positioned in or near the detection zone of the eye. Energy consumption is also reduced, as any number of scattered sections, and thus any number of glints, can be realized using just one laser beam.

[0013] Preferably, a holographic optical element is segmented in such a way that it forms the plurality of scattering sections, for example by means of diffusely scattering holograms. In particular, the holographic optical element is designed in such a way that it deflects rays emanating from the eye, in particular for a wavelength range of the laser beam, in such a way that the eye is captured by the event camera as if from a central perspective relative to the eye.

[0014] By using the holographic optical element, it is possible to arrange the scattering sections flexibly and space-savingly. By redirecting the rays emanating from the eye to the event camera, advantageous capture of the eye from the central perspective relative to the eye is possible without the event camera having to be positioned at a specific spatial location for this capture. This improves flexibility regarding the arrangement of the event camera while maintaining consistent capture quality. Because only outgoing rays from the eye in the wavelength range of the laser beam are redirected, robustness against stray light is improved.

[0015] Preferably, a waveguide comprises a coupling structure by means of which the laser beam is coupled into the waveguide, wherein the waveguide comprises a plurality of coupling-out structures by means of which the laser beam is coupled out for the respective determined radiation angle, and wherein the waveguide comprises a scattering element, wherein the plurality of scattering sections are formed by the plurality of coupling-out structures and the scattering element.

[0016] By using the waveguide, it is possible to arrange the scattering sections flexibly and space-savingly as desired. Furthermore, the use of the waveguide also improves the flexibility of the laser beam generating device. The waveguide is preferably designed such that rays emanating from the eye, in particular the glints, are redirected to the event camera.

[0017] By redirecting the rays emanating from the eye to the event camera, advantageous capture of the eye from the central perspective relative to the eye is possible without requiring the event camera to be positioned at a specific spatial location. This improves flexibility regarding the placement of the event camera while maintaining consistent capture quality.

[0018] Preferably, at least one scattering section of the plurality of scattering sections is arranged such that the eye is illuminated by it from the central perspective relative to the eye with diffusely scattered light of the laser beam.

[0019] Due to at least one of the plurality of scattering sections, the eye is illuminated from the same axis as it is captured by the event camera. Thus, the eye's pupil appears bright and can be captured by the event camera, making the method more robust. Furthermore, no camera sensor is required to determine the pupil of an eye, and energy consumption and space requirements are reduced.

[0020] The method according to claim 1 preferably comprises: detecting the pupil using the event camera; determining a position of a center of the pupil of the eye; determining a position of a center of the spatial position of the cornea; and determining a gaze vector of the eye depending on the position of the center of the pupil and the determined center of the spatial position of the cornea. This allows the gaze direction of the eye, in the form of the gaze vector, to be determined with low computational complexity.

[0021] The method preferably comprises: classifying the detected positions of the glints into at least two clusters as to whether a glint is to be assigned to the cornea or the sclera of the eye, for example depending on a shape of the detected position of the glint on the event camera; determining a spatial position of the sclera of the eye depending on the detected positions of the glints assigned to the sclera; determining a position of a center of the spatial position of the cornea; determining a position of a center of the spatial position of the sclera; determining a gaze vector of the eye depending on the position of the center of the spatial position of the cornea and the center of the spatial position of the sclera. As a result, a gaze direction of the eye, in the form of the gaze vector, is determined with low computational complexity.

[0022] The device comprises: an illumination device which is designed to generate a laser beam, for example an infrared laser beam, in particular a modulated infrared laser beam, and to emit this in at least one dimension at a radiation angle, in particular in two dimensions with a first radiation angle and a second radiation angle, in particular by means of a deflection device arranged in the illumination device, for example a micro-electro-mechanical system mirror; an illumination device which is designed to illuminate an eye, which comprises an eyeball, a cornea, a sclera and a pupil, with diffusely scattered light from the laser beam, wherein the illumination device comprises a plurality of scattering sections, wherein the scattering sections are designed to be irradiated by the laser beam for specific radiation angles and to diffusely scatter the laser beam;an event camera configured to detect a position of a plurality of glints on the eye; an evaluation device configured to determine a spatial position of the cornea of ​​the eye depending on the position of the detected glints, a geometric arrangement of the event camera and the plurality of scattering sections, a predetermined refractive power of the cornea, a predetermined radius of the cornea, and a predetermined distance between a center point of the cornea and a predetermined center point of the sclera, and / or to determine a spatial position of the cornea of ​​the eye depending on the position of the detected glints and a geometric arrangement of the event camera and the plurality of scattering sections, wherein the plurality of scattering sections comprises at least four scattering sections;

[0023] Thanks to the illumination device and its multiple scattering sections, the laser beam generated by the illumination device and emitted at different deflection angles is sufficient to create the effect of a multitude of diffusely scattering flashing light sources. This reduces energy consumption and space requirements.

[0024] Preferably, the illumination device is designed as a holographic optical element, and the holographic optical element is segmented such that it forms the plurality of scattering sections, for example, by means of diffusely scattering holograms. In particular, the holographic optical element is designed to redirect rays emanating from the eye, particularly for a wavelength range of the laser beam, such that the event camera captures the eye as if from a central perspective relative to the eye. This allows a multitude of optical functions to be integrated into the illumination device without increasing the space requirement, thereby increasing the quality and robustness of the glint capture by the event camera.

[0025] Preferably, the illumination device is designed as a waveguide, wherein the waveguide comprises an input coupling structure that couples the laser beam into the waveguide, a plurality of output coupling structures configured to output the laser beam for the specific beam angle, and a scattering element, wherein the plurality of output coupling structures and the scattering element form the plurality of scattering sections. The waveguide enables the illumination device to be arranged flexibly and in a more space-saving manner.

[0026] Preferably, the waveguide is designed to redirect rays emanating from the eye, in particular the glints, to the event camera. This allows for a more flexible arrangement of the event camera without compromising the quality and robustness of the event camera's detection of the eye.

[0027] Preferably, one of the plurality of scattering sections is configured to illuminate the eye from the central perspective relative to the eye with diffusely scattered light from the laser beam. Thus, the eye is illuminated from the same axis as it is captured by the event camera. Thus, the pupil of the eye appears bright and can be captured by the event camera, making the method more robust. Furthermore, no camera sensor is required to determine the pupil of an eye, and energy consumption and space requirements are reduced.

[0028] Preferably, the event camera is configured to capture the pupil, and the evaluation device is configured to determine a position of a center of the pupil of the eye, a position of a center of the spatial position of the cornea, and a gaze vector of the eye based on the spatial position of the center of the pupil and the determined spatial position of the center of the spatial position of the cornea. This allows the gaze direction of the eye, in the form of the gaze vector, to be determined with low computational complexity.

[0029] Preferably, the evaluation device is designed to classify the detected positions of the glints into at least two clusters as to whether a glint is to be assigned to the cornea or the sclera of the eye, in particular depending on a shape of the detected position of the glint on the event camera; to determine a spatial position of the sclera of the eye depending on the detected positions of the glints assigned to the sclera; to determine a position of a center of the spatial position of the cornea; to determine a position of a center of the spatial position of the sclera; to determine a gaze vector of the eye depending on the position of the center of the spatial position of the cornea and the position of the center of the spatial position of the sclera. As a result, a gaze direction of the eye, in the form of the gaze vector, is determined with low computational complexity.

[0030] Preferably, the illumination device and / or the event camera and / or the evaluation device are arranged on smart glasses, in particular in a frame and / or temple of the smart glasses, and / or the illumination device is arranged in a lens of the smart glasses. The device can be flexibly arranged on the smart glasses in a space-saving and unobtrusive manner. This makes them more comfortable to wear. Furthermore, the quality of the smart glasses is improved because, as described above, computational complexity, energy consumption, and robustness are improved.

[0031] Further advantageous embodiments will become apparent from the following description and the drawing. The drawing shows: Figure 1 shows a schematic representation of a device for determining a gaze vector of an eye;

[0032] Figure 2 is a schematic representation of an embodiment of the device arranged on data glasses;

[0033] Figure 3 is a schematic representation of an embodiment of the device;

[0034] Figure 4a is a schematic representation of an illumination device designed as a holographic optical element;

[0035] Figure 4b is a schematic representation of an embodiment of the illumination device designed as a waveguide;

[0036] Figure 4c is a schematic representation of an embodiment of the waveguide;

[0037] Figure 5 is a schematic representation of a detection range of an event camera and a temporal progression of detected glints by the event camera for the waveguide;

[0038] Figure 6 is a schematic representation of the detection range of the event camera and a temporal progression of detected glints by the event camera for the waveguide embodiment;

[0039] Figure 7 is a schematic representation of a reconstruction of the eye;

[0040] Figure 8 is a schematic representation of glints on a cornea and a sclera of the eye;

[0041] Figure 9 shows a schematic representation of a reconstruction of the eye; Figure 10 shows a flowchart of a method for determining the gaze vector of the eye;

[0042] Figure 11 is a flowchart of part of an embodiment of the method for determining the gaze vector of the eye;

[0043] Figure 12 is a flowchart of part of an embodiment of the method for determining the gaze vector of the eye.

[0044] Figure 1 shows a device 900 for determining a gaze vector of an eye 2 comprising a lighting device 100, an illumination device 200, an event camera 300 and an evaluation device 400.

[0045] The illumination device 100 is configured to generate a laser beam 106, for example, an infrared laser beam, in particular a modulated infrared laser beam, and to emit it in at least one dimension at an emission angle α. The laser beam 106 is generated, for example, by means of a light source 102 included in the illumination device 100. The emission angle α is variable, and the illumination device 100 changes it, for example, in predetermined steps and patterns. For this purpose, the illumination device 100 in the example comprises a deflection device 104, which is configured, in particular, as a prism, galvo mirror, or micro-electro-mechanical system mirror.

[0046] It can be provided that the illumination device 100 is configured to emit the laser beam 106 in two dimensions with a first emission angle α and a second emission angle β. For example, the deflection device 104 is configured to vary the first and second emission angles α and β.

[0047] The illumination device 200 is designed to illuminate the eye 2, which comprises an eyeball, a cornea 8, a sclera 6, and a pupil 10, with diffusely scattered light 107a, 107b of the laser beam 106. The illumination device 200 comprises a plurality of scattering sections 202a, 202b, which are designed to be irradiated by the laser beam 106 for specific radiation angles α and to diffusely scatter the laser beam 106. For a specific radiation angle α, at least one scattering section of the plurality of scattering sections 202a, 202b is irradiated. By changing the radiation angle α, the at least one scattering section of the plurality of scattering sections 202a, 202b is not continuously illuminated, but only for the specific radiation angle α or a specific range of the radiation angle α. This creates a radiation characteristic for the majority of scattering sections 202a, 202b of flashing diffusely scattering LEDs.

[0048] The diffusely scattered light 107a, 107b of the laser beam 106 generates a plurality of specular reflections (glints) 108a, 108b on the eye 2. These glints 108a, 108b flash depending on the radiation angle a. A number of the plurality of glints 108a, 108b corresponds to a number of the plurality of scattering sections 202a, 202b.

[0049] The event camera 300 is configured to detect a position 304a, 304b of the plurality of glints 108a, 108b on the eye 2. The event camera 300 includes, for example, an imaging optics 302 that images the detected plurality of glints 108a, 108b onto an event camera sensor.

[0050] The evaluation device 400 is configured to determine a spatial position of the cornea 8 of the eye 2 depending on the position 304a, 304b of the detected glints 108a, 108b, a geometric arrangement of the event camera 300 and the plurality of scattering sections 202a, 202b, a predetermined refractive power of the cornea 8, a predetermined radius 50 of the cornea 8, and a predetermined distance between a center 36 of the cornea 8 and a predetermined center point 38 of the sclera 6. The geometric arrangement of the event camera 300 and the plurality of scattering sections 202a, 202b is predetermined or can be predetermined and remains unchanged in the example. The predetermined distance between a center 36 of the cornea 8 and a predetermined center 38 of the sclera 6 and the predetermined radius 50 of the cornea 8 are known, for example, as average values ​​from human anatomy.

[0051] It can be provided that the evaluation device 400 is configured to determine a spatial position of the cornea 8 of the eye 2, depending on the position 304a, 304b of the detected glints 108a, 108b as well as the geometric arrangement of the event camera 300 and the plurality of scattering sections 202a, 202b, wherein the plurality of scattering sections 202a, 202b comprises at least four scattering sections 202a, 202b. In this case, the cornea 8, including a spatial position of the center point 36 and the radius 50 of the cornea 8, can be approximated using a spherical function.

[0052] Figure 2 shows an embodiment of the device 900 arranged on data glasses 500. The illumination device 100, the event camera 300, and the evaluation device 400 are arranged in a temple 502 of the data glasses 500, and the illumination device 200 is arranged in a lens 504 of the data glasses 500. Provision can be made for the illumination device 100 and / or the event camera 300 and / or the evaluation device 400 to be arranged on other elements of the data glasses 500, for example, in a frame of the data glasses 500 or a nose pad of the data glasses 500.

[0053] In the example, the illumination device 200 is embodied as a holographic optical element 600. The holographic optical element 600 is segmented such that it forms the plurality of scattering sections 202a, 202b, 602a, 602b, for example, by means of diffusely scattering holograms 602a, 602b. In the example, the plurality of scattering sections 202a, 202b, 602a, 602b are formed in an edge region of the holographic optical element 600. A laser beam 106a, 106b, each with a specific deflection angle a, strikes a scattering section of the plurality of scattering sections 202a, 202b, 602a, 602b and the laser beam 106a, 106b is diffusely scattered and the eye 2 is illuminated with the diffusely scattered light 107a, 107b of the laser beam 106a, 106b.

[0054] Laser beams 106c with further specific deflection angles a are guided unchanged through the holographic optical element 600 in the example, for example by means of empty holograms.

[0055] It can be provided that the holographic optical element 600 is designed to deflect rays emanating from the eye 2, in particular for a wavelength range of the laser beam 106, 106a, 106b, 106c, such that the event camera 300 captures the eye 2 as if from a central perspective relative to the eye 2. Figure 3 shows an embodiment of the device 900, wherein the illumination device 200 is designed as a waveguide 700, wherein the waveguide 700 comprises a coupling structure 706 which couples the laser beam 106, 106a, 106b, 106c into the waveguide 700. The waveguide 700 comprises a plurality of coupling-out structures 701a ... 701f, which are configured to couple out the laser beam 106, 106a, 106b, 106c for the specific emission angle α. Furthermore, the waveguide 700 comprises at least one scattering element 712, wherein the plurality of coupling-out structures 701a ... 701f and the scattering element 712 form the plurality of scattering sections 702a ... 702f.In the example, the coupling-out structures 701a...701f are located on a side of the waveguide 700 facing away from the eye 2, with the laser beam 106, 106a, 106b, 106c being coupled out and diffusely scattered in the direction of the eye 2 by the scattering element 712. This forms the plurality of scattering sections 702a...702f. It can be provided that the coupling-out structures 701a...701f couple the laser beam 106, 106a, 106b, 106c in a direction opposite to the eye, and the scattering element 712 is arranged on the side of the waveguide 700 facing away from the eye 2, with a further reflective layer radiating the diffusely scattered light in the direction of the eye 2.

[0056] In the example, the laser beam 106, 106a, 106b, 106c propagates within the waveguide 700 by total internal reflection until it encounters the outcoupling structures 701a...701f. These are designed, for example, as a diffractive optical element, such as a hologram. Furthermore, the waveguide 700 is segmented, so that for one hop length in the waveguide, the laser beam 106, 106a, 106b, 106c encounters another segment, or several hop lengths encounter the same segment. If the laser beam 106, 106a, 106b, 106c encounters the outcoupling structure 701a...701f at the specific emission angle α, a Bragg condition is met, and the laser beam 106, 106a, 106b, 106c is outcoupled.

[0057] It can be provided that the waveguide 700 is designed to redirect rays emanating from the eye 2, in particular the glints 108a...108f, onto the event camera 300. In this case, the position 304a...304f of the plurality of glints 108a...108f is detected. It can be provided that the device 900 comprises a coupling device 110, for example, a beam splitter or a prism. This is designed to image the outgoing rays of the eye 2 onto the event camera 300. Furthermore, the coupling device 110 is also designed, for example, to redirect the laser beam 106, 106a, 106b, 106c of the illumination device 100 onto the coupling structure 706 of the waveguide 700.

[0058] It is also conceivable that a bandpass filter, laser line filter or microlens array adapted to the wavelength of the laser beam 106... 106c is coupled into a beam path of the rays emanating from the eye.

[0059] For example, to determine a viewing direction of the eye 2, in addition to the spatial position of the cornea 8, a position of a center of the pupil 10 is required.

[0060] It can be provided that a scattering section of the plurality of scattering sections 202a, 202b; 602a, 602b; 702a ... 702f is arranged such that the scattering section is designed to illuminate the eye 2 from the central perspective relative to the eye 2 with diffusely scattered light from the laser beam 106 ... 106c (on-axis). Thus, the pupil 10 of the eye 2 appears bright, and a corresponding event cluster is created on the event camera 300. Alternatively, an image of the eye 2 illuminated by a scattering section of the plurality of scattering sections 202a, 202b; 602a, 602b; 702a ... 702f that is not in the central perspective relative to the eye 2 can be recorded by a photo camera (off-axis). In this case, the pupil 10 of the eye 2 appears dark. The photo camera can be integrated into the Event Camera 300.

[0061] Figure 4a shows an embodiment of the illumination device 200 as the holographic optical element 600, which is arranged in the spectacle lens 504. The holographic optical element 600 comprises the plurality of scattering sections 602a...602e' as well as the scattering section 608 of the plurality of scattering sections 602a...602e', which is designed to illuminate the eye 2 from the central perspective relative to the eye 2 with diffusely scattered light from the laser beam 106...106c. Furthermore, the holographic optical element 600 in the example comprises a region 604 that transmits the laser beam 106a...106c. Figure 4b shows an embodiment of the illumination device 200 as a waveguide 700, which is arranged in the spectacle lens 504. In the example, the waveguide 700 comprises the plurality of scattering sections 702a...702f, 702a '...702f '. The waveguide 700 also has an expander 710, which directs the laser beam 106...106c to the plurality of scattering sections 702a...702f, 702a '...702f '. In the example, the waveguide 700 is configured such that opposing scattering sections, for example, the scattering sections 702a and 702a ', decouple the laser beam 106... 106c for the same specific radiation angle a and thus simultaneously emit the diffusely scattered light 107a, 107b of the laser beam 106... 106c. Thus, two glints are created on the eye 2 simultaneously.

[0062] Figure 4c shows an embodiment of the illumination device 200 as a waveguide 700, which is arranged in the spectacle lens 504. In the example, the waveguide 700 comprises the plurality of scattering sections 702a...702f, 702a'...702f'. The waveguide 700 also has a first expander 710a and a second expander 710b. Furthermore, the coupling structure 706 comprises a first coupling region 706a and a second coupling region 702b. The first coupling region 706a is irradiated in a second dimension for a specific or specific range of the second radiation angle β of the laser beam 106...106c and is guided via the first expander 710a to a first group of the plurality of scattering sections 702a...702f'. The first radiation angle α varies while the second radiation angle β remains constant, wherein the laser beam 106... 106c is emitted from at least one scattering section of the first group of the plurality of scattering sections 702a...702f ' is coupled out and diffusely scattered depending on the specific radiation angle a.

[0063] The second coupling region 706b is irradiated for a further specific or specific range of the second emission angle β of the laser beam 106... 106c in the second dimension and is guided via the second expander 710a to a second group of the plurality of scattering sections 702a...702f'. The first emission angle α varies while the further second emission angle β remains constant, wherein the laser beam 106... 106c is coupled out and diffusely scattered from at least one scattering section of the second group of the plurality of scattering sections 702a...702f' depending on the specific emission angle α.

[0064] It can be provided that the plurality of scattering sections 202, 202b; 602a...602e'; 702a...702f ' are of different geometric configurations, for example elongated, angular, round or curved.

[0065] Figure 5 shows a time course of the detected positions 304a...304f' of the plurality of glints 108a...108f for the embodiment according to Figure 4b, wherein the number of the plurality of glints 108a...108f corresponds to the number of the plurality of scattering sections 702a...702f'. Figure 5 schematically illustrates a sensor chip of the event camera 300, which has an extension in a direction x and an extension in a direction y. The positions 304a...304f' of the plurality of glints 108a...108f are detected by means of this sensor chip. The time diagrams illustrated in Figure 5 show a time course and a position in the direction x and the direction y, as well as a temporal change in the radiation angle α and the second radiation angle β.

[0066] According to the explanations for Figure 4b, for the specific radiation angle a, two spatially separated positions 304a...304f' of the plurality of glints 108a...108f are simultaneously recorded by the event camera. These two positions 304a...304f' of the plurality of glints 108a...108f can be recorded and processed in parallel.

[0067] Figure 6 shows a temporal progression of the detected positions 304a...304f' of the plurality of glints 108a...108f for the exemplary embodiment according to Figure 4c, wherein the number of the plurality of glints 108a...108f corresponds to the number of the plurality of scattering sections 702a...702f'. Figure 6 schematically illustrates the sensor chip of the event camera 300. The positions 304a...304f' of the plurality of glints 108a...108f are detected by means of this sensor chip. The time diagrams illustrated in Figure 6 show a temporal progression and a position in the x-direction and the y-direction, as well as a temporal change in the radiation angle α and the second radiation angle β. According to the explanations for Figure 4c, the positions 304a...304f' of the majority of glints 108a...108f are recorded individually over time using the event camera. This simplifies the separation of the recorded positions 304a...304f' of the majority of glints 108a...108f.

[0068] Figure 7 shows a schematic representation of a reconstruction of the eye 2. It can be provided that the evaluation device 400 is designed to determine a position of a center 42 of the pupil 10 of the eye 2 captured by the event camera 300 or the photo camera and to determine a position of the center 36 of the spatial position 32 of the cornea 8. In addition, the evaluation device is designed to determine a gaze vector 40 of the eye 2 depending on the spatial position of the center 42 of the pupil 10 and the determined spatial position of the center 36 of the spatial position 32 of the cornea 8.

[0069] Figure 8 shows a schematic representation of eye 2 from the perspective of event camera 300. By way of example, it is shown that the majority of glints 108a...108f on eye 2 have a different shape depending on whether they originate on the cornea 8 or the sclera 6 of eye 2. Due to the different properties between the cornea 8 and the sclera 6, different volume scattering cones of the majority of glints 108a...108f result.

[0070] For example, glints on the cornea 8, designated in the example with reference numerals 108a and 108b, have a smaller volume scattering cone than glints on the sclera 6, designated in the example with reference numerals 108c and 108d. This difference in the volume scattering cones causes different event clusters to be detected by the event camera 300. Accordingly, a different number of pixels on the sensor chip of the event camera 300 are illuminated, and accordingly, a different number of events arise depending on the shape of the detected positions (304a, ..., 304f') of the majority of glints (108a, ..., 108f).

[0071] It can be provided that the evaluation device 400 is designed to classify the detected positions 304a, ... , 304f ' of the glints 108a, ... , 108f into at least two clusters as to whether a glint 108a, ... , 108f is to be assigned to the cornea 8 or the sclera 6 of the eye 2, in particular depending on the shape of the detected position 304a, ... , 304f ' of the glint 108a, ... , 108f on the event camera 300.

[0072] Figure 9 shows an embodiment of the reconstruction of the eye 2. It can be provided that the evaluation device 400 is designed to determine a spatial position 34 of the sclera 6 of the eye 2 depending on the detected positions of the glints 304a, ..., 304f ', which are assigned to the sclera 6, and to determine a position of a center 38 of the spatial position 34 of the sclera 6. In addition, the evaluation device 400 can be designed to determine a position of the center 36 of the spatial position 32 of the cornea 8. The evaluation device 400 determines the gaze vector 40 of the eye 2 depending on the position of the center 36 of the spatial position 32 of the cornea 8 and the position of the center 38 of the spatial position 34 of the sclera 6.

[0073] Figure 10 shows a method 800, in particular for operating the data glasses 500. The method 900 comprises generating 804 the laser beam 106, 106a, 106b, for example an infrared laser beam, in particular a modulated infrared laser beam. The method 800 also comprises illuminating 806 the eye 2, which comprises the eyeball 4, the cornea 8, the sclera 6, and the pupil 10, with diffusely scattered light 107a, 107b of the laser beam 106, 106a, 106b by means of the plurality of scattering sections 202a, 202b; 602a ... 606e 702a ... 702f ', wherein the scattering sections 202a, 202b; 602a ... 606e '; 702a ... 702f ' are irradiated by the laser beam 106, 106a, 106b for certain radiation angles a, whereby the laser beam 106, 106a, 106b is diffusely scattered.

[0074] The method 900 further comprises detecting 808 the position 304a, ... , 304f ' of the plurality of glints 108a, ... , 108f that the diffusely scattered laser beam 106, 106a, 106b generates on the eye 2, by means of the event camera 300, wherein the number of glints 108a, ... , 108f corresponds to the number of scattering sections 202a, 202b; 602a ... 606e '; 702a ... 702f '.

[0075] In addition, the method 800 comprises determining 810 the spatial position 32 of the cornea 8 of the eye 2, depending on the position 304a, ... , 304f ' of the detected glints 108a, 108f, the geometric arrangement of the event camera

[0076] 300 and the plurality of scattering sections 202a, 202b; 602a ... 606e 702a ... 702f ', a predetermined refractive power of the cornea 8, a predetermined radius 50 of the cornea 8 and a predetermined distance between a center 36 of the cornea 8 and a predetermined center point 38 of the sclera 6.

[0077] Alternatively or additionally, the method 800 comprises determining 812 a spatial position 32 of the cornea 8 of the eye 2, depending on the position 304a...304f ' of the detected glints 108a...108f as well as the geometric arrangement of the event camera 300 and the plurality of scattering sections 202a, 202b; 602a...606e '; 702a...702f ', wherein the plurality of scattering sections 202a, 202b; 602a...606e '; 702a...702f ' comprises at least four scattering sections 202a, 202b; 602a...606e '; 702a...702f '.

[0078] It can be provided that the holographic optical element 600 is segmented such that it forms the plurality of scattering sections 602a, ... , 602e ', for example by means of diffusely scattering holograms 602a, ... , 602e ', in particular wherein the holographic optical element 600 is designed such that it deflects rays emanating from the eye 2, in particular for a wavelength range of the laser beam 106... 106c, such that the eye 2 is captured by the event camera 300 as if from a central perspective relative to the eye 2.

[0079] It can be provided that the waveguide 700 comprises the coupling structure 706, by means of which the laser beam 106, 106a, 106b is coupled into the waveguide 700, wherein the waveguide 700 comprises the plurality of coupling-out structures 701a ... 701f ', by means of which the laser beam 106... 106c is coupled out for the respective determined radiation angle α, and wherein the waveguide 700 comprises the scattering element 712, wherein the plurality of scattering sections 702a ... 702f ' are formed by the plurality of coupling-out structures 701a ... 701f ' and the scattering element 712.

[0080] It is conceivable that the waveguide 700 is designed in such a way that the eye

[0081] 2, in particular the glints 108a, ..., 108f, are redirected to the event camera 300. It can be provided that at least one scattering section 608 of the plurality of scattering sections 202a, 202b; 602a ... 606e 702a ... 702f ' is arranged such that the eye 2 is illuminated by it from the central perspective relative to the eye 2 with diffusely scattered light of the laser beam 106, 106a, 106b.

[0082] For example, the method 800 according to Figure 11 comprises detecting 814 the pupil 10 by means of the event camera 300, determining 816 the position of the center 42 of the pupil 10 of the eye 2, determining 818 the position of the center 36 of the spatial position 32 of the cornea 8 and determining 820 the gaze vector 40 of the eye 2 depending on the position of the center 42 of the pupil 10 and the determined center 36 of the spatial position 32 of the cornea 8.

[0083] For example, the method 800 according to Figure 12 comprises classifying 822 the detected positions 304a, ... , 304f ' of the glints 108a, ... , 108f into at least two clusters as to whether a glint 108a, ... , 108f is to be assigned to the cornea 8 or the sclera 6 of the eye 2, for example depending on the shape of the detected position 304a, ... , 304f ' of the glint 108a, ... , 108f on the event camera 300, determining 824 the spatial position 34 of the sclera 6 of the eye 2 depending on the detected positions of the glints 304a, ... , 304f ' that are assigned to the sclera 6, determining 826 the position of the center 36 of the spatial position 32 of the Cornea 8, determining 828 the position of the center 38 of the spatial position 34 of the sclera 6 and determining 830 the gaze vector 40 of the eye 2 depending on the position of the center 36 of the spatial position 32 of the cornea 8 and the center 38 of the spatial position 34 of the sclera 6.

Claims

Claims 1. Method (800), in particular for operating data glasses (500), comprising: illuminating (806) an eye (2), which comprises an eyeball (4), a cornea (8), a sclera (6) and a pupil (10), with diffusely scattered light (107a, 107b) of a laser beam (106, 106a, 106b, 106c), for example an infrared laser beam, in particular a modulated infrared laser beam, by means of a plurality of scattering sections (202a, 202b; 602a ... 606e 702a ... 702f '), wherein the scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ') are emitted by the laser beam (106, 106a, 106b, 106c) are irradiated for specific radiation angles (a), whereby the laser beam (106, 106a, 106b, 106c) is diffusely scattered; detecting (808) a position (304a, ... , 304f ') of a plurality of glints (108a, ... , 108f) generated by the diffusely scattered laser beam (106, 106a, 106b, 106c) on the eye (2) by means of an event camera (300), wherein a number of the glints (108a, ..., 108f) corresponds to a number of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f '); determining (810) a spatial position (32) of the cornea (8) of the eye (2), depending on the position (304a, ... , 304f ') of the detected glints (108a, ... , 108f), a geometric arrangement of the event camera (300) and the plurality of scattering sections (202a, 202b; 602a . 606e '; 702a ... 702f '), a predetermined refractive power of the cornea (8), a predetermined radius (50) of the cornea (8) and a predetermined distance between a center (36) of the cornea (8) and a predetermined center point (38) of the sclera (6), and / or determining (812) a spatial position (32) of the cornea (8) of the eye (2), depending on the position (304a, ... , 304f ') of the detected glints (108a, ... , 108f) and a geometric arrangement of the event camera (300) and the plurality of scattering sections (202a, 202b; 602a ... 606e 702a ... 702f '), wherein the plurality of scattering sections (202a, 202b; 602a ... 606e 702a ... 702f ') comprises at least four scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ').

2. The method (800) according to claim 1, wherein a holographic optical element (600) is segmented such that it forms the plurality of scattering sections (602a, ..., 602e '), for example by means of diffusely scattering holograms (602a, ..., 602e '), in particular wherein the holographic optical element (600) is designed such that it deflects rays emanating from the eye (2), in particular for a wavelength range of the laser beam (106, 106a, 106b, 106c) such that the eye (2) is captured by the event camera (300) as if from a central perspective relative to the eye (2).

3. The method (800) according to claim 1, wherein a waveguide (700) comprises a coupling structure (706) by means of which the laser beam (106, 106a, 106b, 106c) is coupled into the waveguide (700), wherein the waveguide (700) comprises a plurality of coupling-out structures (701a ... 701f) by means of which the laser beam (106, 106a, 106b, 106c) is coupled out for the respective determined radiation angle (α), and wherein the waveguide (700) comprises a scattering element (712), wherein the plurality of scattering sections (702a ... 702f ') are formed by the plurality of coupling-out structures (701a ... 701f) and the scattering element (712).

4. The method (800) according to claim 3, wherein the waveguide (700) is designed such that rays emanating from the eye (2), in particular the glints (108a, ..., 108f), are deflected onto the event camera (300).

5. The method (800) according to one of claims 1 to 4, wherein at least one scattering section (608) of the plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ') is arranged such that the eye (2) is illuminated by it from the central perspective relative to the eye (2) with diffusely scattered light of the laser beam (106, 106a, 106b, 106c).

6. The method (800) of claim 5, comprising: detecting (814) the pupil (10) by means of the event camera (300); determining (816) a position of a center (42) of the pupil (10) of the eye (2); determining (818) a position of a center (36) of the spatial position (32) of the cornea (8); determining (820) a gaze vector (40) of the eye (2) depending on the position of the center (42) of the pupil (10) and the determined center (36) of the spatial position (32) of the cornea (8).

7. The method (800) according to one of the preceding claims 1 to 6, comprising: classifying (822) the detected positions (304a, , 304f') of the glints (108a, ..., 108f) into at least two clusters as to whether a glint (108a, ..., 108f) is to be assigned to the cornea (8) or the sclera (6) of the eye (2), for example, depending on a shape of the detected position (304a, ..., 304f') of the glint (108a, ..., 108f) on the event camera (300); determining (824) a spatial position (34) of the sclera (6) of the eye (2) depending on the detected positions of the glints (304a, ..., 304f ') which are assigned to the sclera (6); determining (826) a position of a center (36) of the spatial position (32) of the cornea (8); determining (828) a position of a center (38) of the spatial position (34) of the sclera (6); determining (830) a gaze vector (40) of the eye (2) depending on the position of the center (36) of the spatial position (32) of the cornea (8) and the center (38) of the spatial position (34) of the sclera (6).

8. Device (900) comprising: an illumination device (100) which is designed to generate (802) a laser beam (106, 106a, 106b, 106c), for example an infrared laser beam, in particular a modulated infrared laser beam, and to radiate this in at least one dimension at a radiation angle (α), in particular in two dimensions with a first radiation angle (α) and a second radiation angle (β), in particular by means of a Deflection device (104), for example a micro-electro-mechanical system mirror; an illumination device (200; 600; 700) which is designed to illuminate (806) an eye (2) comprising an eyeball (4), a cornea (8), a sclera (6) and a pupil (10) with diffusely scattered light (107a, 107b) of the laser beam (106, 106a, 106b, 106c), wherein the illumination device (200; 600; 700) comprises a plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f '), wherein the scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ') are designed to be emitted by the laser beam (106, 106a, 106b, 106c) for specific beam angles (a) and to diffusely scatter the laser beam (106, 106a, 106b, 106c); an event camera (300) configured to detect (808) a position (304a, ... , 304f ') of a plurality of glints (108a, ... , 108f) on the eye (2), wherein a number of the glints (108a, ..., 108f) corresponds to a number of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f '); an evaluation device (400) which is designed to determine (810) a spatial position (32) of the cornea (8) of the eye (2) depending on the position (304a, ... , 304f ') of the detected glints (108a, ... , 108f), a geometric arrangement of the event camera (300) and the plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f '), a predetermined refractive power of the cornea (8), a predetermined radius (50) of the cornea (8) and a predetermined distance between a center (36) of the cornea (8) and a predetermined center point (38) of the sclera (6), and / or a spatial position (32) of the cornea (8) of the eye (2) depending on the position (304a, ... , 304f ') of the detected glints (108a, ... , 108f) and a geometric arrangement of the event camera (300) and the plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a . 702f '), wherein the plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ') comprises at least four scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f '), to be determined (812).

9. The device (900) according to claim 8, wherein the illumination device (200; 600; 700) is designed as a holographic optical element (600) and the holographic optical element (600) is segmented such that it forms the plurality of scattering sections (602a, ... , 602e '), for example by means of diffusely scattering holograms (602a, ... , 602e '), in particular wherein the holographic optical element (600) is designed to deflect rays emanating from the eye (2), in particular for a wavelength range of the laser beam (106, 106a, 106b, 106c) such that the event camera (300) captures the eye (2) as if from a central perspective relative to the eye (2).

10. The device (900) according to claim 8, wherein the illumination device (200; 600; 700) is designed as a waveguide (700), wherein the waveguide (700) comprises a coupling structure (706) which couples the laser beam (106, 106a, 106b, 106c) into the waveguide (700), a plurality of coupling-out structures (701a ... 701f ) which are designed to couple out the laser beam (106, 106a, 106b, 106c) for the specific radiation angle (α), and a scattering element (712), wherein the plurality of coupling-out structures (701a ... 701f ) and the scattering element (712) form the plurality of scattering sections (702a ... 702f ').

11. The device (900) according to claim 10, wherein the waveguide (700) is designed to redirect rays emanating from the eye (2), in particular the glints (108a, ..., 108f) to the event camera (300).

12. The device (900) according to one of claims 8 to 11, wherein a scattering section (608) of the plurality of scattering sections (202a, 202b; 602a ... 606e '; 702a ... 702f ') is designed to illuminate the eye (2) from the central perspective relative to the eye (2) with diffusely scattered light of the laser beam (106, 106a, 106b, 106c).

13. The device (900) according to claim 12, wherein: the event camera (300) is designed to detect (814) the pupil (10); the evaluation device (400) is designed to determine (816) a position of a center (42) of the pupil (10) of the eye (2), to determine (818) a position of a center (36) of the spatial position (32) of the cornea (8), to determine a gaze vector (40) of the eye (2) depending on the spatial position of the center (42) of the pupil (10) and the determined spatial position of the center (36) of the spatial position (32) of the cornea (8) (820).

14. The device (900) according to one of claims 8 to 13, wherein the evaluation device (400) is configured to classify (822) the detected positions (304a, ..., 304f') of the glints (108a, ..., 108f) into at least two clusters as to whether a glint (108a, ..., 108f) is to be assigned to the cornea (8) or the sclera (6) of the eye (2), in particular depending on a shape of the detected position (304a, ..., 304f') of the glint (108a, ..., 108f) on the event camera (300); a spatial position (34) of the sclera (6) of the eye (2) depending on the detected positions of the glints (304a, ..., 304f ') assigned to the sclera (6); to determine (824) a position of a center (36) of the spatial position (32) of the cornea (8); to determine (826) a position of a center (338) of the spatial position (34) of the sclera (6); to determine (828) a gaze vector (40) of the eye (2) depending on the position of the center (36) of the spatial position (32) of the cornea (8) and the position of the center (38) of the spatial position (34) of the sclera (6).

15. Data glasses (500) comprising the device (900) according to one of claims 8 to 14, wherein the illumination device (100) and / or the event camera (300) and / or the evaluation device (400) is arranged on the data glasses (500), in particular in a frame and / or temple (502) of the data glasses (500) and / or that the illumination device (200; 600; 700) is arranged in a spectacle lens (504) of the data glasses (500).

Images