Method using a laser feedback interferometry sensor, virtual retinal display, and smart glasses

Abstract

The invention relates to a method using a laser feedback interferometry (LFI) sensor (10), in particular a virtual retinal display (12), preferably in smart glasses (14), wherein at least one laser beam (16) is emitted by the LFI sensor (10) and wherein reflection signals (18) which originate from the previously emitted laser beam (16) are recorded by the LFI sensor (10). According to the invention, a pulse frequency of a person to whom an eye (24) belongs and / or a blood flow direction of at least one blood vessel of an eye (24) are / is determined from the detected reflection signals (18) of the LFI sensor (10), which signals comprise a laser speckle reflection pattern (20) of a retina (22) of the eye (24).

Description

[0001] Description

[0002] Method with a laser feedback interferometry sensor, virtual retinal display and data glasses

[0003] State of the art

[0004] A method with a laser feedback interferometry (LF I) sensor in data glasses has already been proposed, wherein at least one laser beam is emitted by the LFI sensor and wherein reflection signals originating from the previously emitted laser beam are recorded by the LFI sensor.

[0005] Disclosure of the invention

[0006] The invention is based on a method with a laser feedback interferometry (LFI) sensor, in particular a virtual retinal display, preferably in data glasses, wherein at least one laser beam is emitted by the LFI sensor and wherein reflection signals originating from the previously emitted laser beam are recorded by the LFI sensor.

[0007] It is proposed that a pulse rate of an owner of the eye is determined from the recorded reflection signals of the LFI sensor, which comprise laser speckle reflection patterns of a retina of an eye. This advantageously makes it possible to obtain a variety of information about the user of data glasses. The information about the pulse rate can advantageously be obtained by using the LFI sensor with a particularly compact and / or highly integrable system, which in particular enables installation in data glasses, in particular AR and / or VR systems. The information about the pulse rate can advantageously be obtained by using the LFI sensor with a particularly energy-saving system, which in particular enables installation in wearable systems such as data glasses, in particular AR and / or VR systems.Advantageously, the information about the pulse frequency can be obtained by using the LFI sensor with a system that is particularly insensitive to stray light, which can particularly improve use in areas with uncontrolled ambient brightness, such as outdoors. In particular, the use of the LFI sensor can achieve at least these significant advantages over the use of a system based on a laser speckle flowgraphy (LSFG) method. In the LSFG method, a collimated and expanded laser beam is directed at an object to be examined, and its reflection is observed with imaging optics and a camera sensor, such as a CMOS sensor, that are separate from the laser system.

[0008] In an LFI method that can be performed by the LFI sensor, a laser beam, in particular an infrared laser beam, emitted by the LFI sensor is preferably collimated and directed onto a measurement object, for example the retina of the user of the data glasses with the LFI sensor. The laser beam, in particular an infrared laser beam, that strikes the measurement object is backscattered by the measurement object, in particular the retina. In particular, a portion of the backscattered light is reflected back into a laser cavity of the LFI sensor. In the LFI method, the laser cavity of the LFI sensor serves as an optical mixer between the (emitted) local light wave of the LFI sensor and the wave backscattered into the LFI sensor. Thus, constructive and / or destructive interference occurs within the laser cavity of the LFI sensor, which in particular modulates the power of the laser of the LFI sensor.This power modulation can be measured by a detector element, e.g., a photodiode, integrated into a back reflector of the laser cavity of the LFI sensor. The modulation signal measured by the detector element represents, in particular, the measurement signal used for the method according to the invention. To record an image, in the LFI method, the laser beam can be pivoted (preferably in 2D) over the surface of the measurement object to be examined, in particular tissue, preferably the retina to be examined, while the detector element is scanned. The laser beam of the LFI sensor is preferably scanned across the eye using a microscanner, in particular a MEMS micromirror system with at least one or at least two pivotable micromirrors. In this process, part of the laser beam, in particular the infrared laser beam, of the LFI sensor enters the eye through the pupil and strikes the retina.

[0009] The method for determining the pulse rate uses a so-called "bright pupil effect," which is based in particular on the relatively high infrared reflectivity of the retina of the eye compared to neighboring / other components of the eye. When using the "bright pupil effect," the pupil of the eye and thus a retinal signal of the eye can be determined directly from an image captured by the detector element. In particular, the retina of the eye produces the so-called laser speckle reflection pattern upon reflection of laser beams, especially a sufficiently coherent laser beam. The laser speckle reflection pattern (also called laser granulation or light granulation) is in particular a grainy-looking interference phenomenon that can be observed with sufficiently coherent illumination of optically rough object surfaces, such as the retina of the eye (cf. cones and rods of the retina).The roughness of the retina is related to the anatomy of the eye and behaves similarly for all people. Other areas of the eye, different from the retina, however, do not produce a laser speckle reflection pattern. The retina is rich in blood flow, so a pulse leads to slight movement, particularly up-and-down movement, of the retina. As a result, a reflection signal generated by the retina, e.g., the laser speckle reflection pattern, can include a modulation caused by the pulse.

[0010] The LFI sensor could be designed as a VCSEL, preferably a ViP-VCSEL (vertical-cavity surface-emitting laser with integrated photo diode). The LFI sensor could be integrated into the data glasses, e.g., into a frame, lens, or temple of the data glasses. In particular, the laser beam generated by the LFI sensor and used for the method according to the invention is an infrared laser beam that is invisible to the user and harmless to the eye. The LFI sensor could be integrated into a common laser module / laser projector with the data glasses' (RGB) laser diodes that generate the visible image content of a virtual retinal display. Alternatively, the LFI sensor could also be designed separately from the laser projector with the (RGB) laser diodes that generate the visible image content.In this case, the infrared laser beam of the LFI sensor could be coupled into the RGB laser beam of the (RGB) laser diodes generating the visible image content via a coupling structure of the data glasses, in particular one designed separately from the (RGB) laser projector. It is also conceivable for the data glasses to have more than one LFI sensor, e.g., two, three, or more than three LFI sensors. A preferred measurement interval / duration for determining a measurement signal, from which the pulse frequency is then determined, is approximately 300 images and / or 5 seconds. However, shorter or longer measurement intervals / durations are also conceivable.

[0011] “Data glasses” should be understood in particular as a wearable (head-mounted display) by means of which information can be added to a user’s field of vision. Data glasses preferably enable augmented reality, virtual reality and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses or VR glasses or AR glasses. The data glasses can form an AR or VR headset. In particular, the data glasses have the virtual retinal display (also called retinal scan display or light field display), which is particularly familiar to those skilled in the art. The virtual retinal display is particularly designed to sequentially project an image content by deflecting at least one visible laser beam, for example by means of optical elements integrated into a lens of the data glasses, such as holographic optical elements (HOE), prisms, mirrors or waveguides, at least one time-modulated light source, such asone or more (RGB) laser diodes of a laser projector, scanned and projected directly onto the retina of the user's eye by optical elements. The pulse frequencies and / or blood flow directions determined from the reflection signals can be used, in particular, for a health check of a user of the data glasses or for measuring a state of mind, a state of excitement, a stress level, a state of exhaustion, exertion, etc. of the user of the data glasses. "Set up" or "intended" should be understood in particular to mean specially programmed, designed and / or equipped. The fact that an object is intended or configured for a specific function should be understood in particular to mean that the object fulfills and / or executes this specific function in at least one application and / or operating state.

[0012] Furthermore, a method is proposed using the LFI sensor, in particular the virtual retinal display, preferably in the data glasses, wherein a blood flow direction of at least one blood vessel of the eye is determined from the detected reflection signals of the LFI sensor, which include the laser speckle reflection patterns of the retina of the eye. This advantageously allows a multitude of information about the user of the data glasses to be obtained. The information about the blood flow direction can advantageously be obtained by using the LFI sensor with a particularly compact and / or highly integrable system, which in particular enables installation in data glasses, in particular AR and / or VR systems.Advantageously, the information about the blood flow direction can be obtained by using the LFI sensor with a particularly energy-saving system, which in particular enables integration into wearable systems such as data glasses, in particular AR and / or VR systems. Advantageously, the information about the blood flow direction can be obtained by using the LFI sensor with a system that is particularly insensitive to stray light, which in particular can improve use in areas with uncontrolled ambient brightness, such as outdoors. In particular, each pulse beat generates a deformation of the surface of the blood vessel and the overlying tissue, e.g., the retina, that migrates along blood vessels. This migratory deformation modulates, in particular, the laser speckle reflection pattern reflected by the retina.Thus, the blood flow direction of the blood vessel can be advantageously read from the laser speckle reflection pattern detected by the detector element.

[0013] Furthermore, it is proposed that the pulse rate of the owner of the eye and / or the blood flow direction of the blood vessel of the eye be recorded using differential images of the laser speckle reflection patterns. This advantageously enables a reliable determination of the pulse rate and / or the blood flow direction by the LFI sensor. Since the measurement signal, i.e. the laser speckle reflection pattern, comes directly from the retina and is modulated there by interference, changes in the optical path length between the retina and the LFI sensor in the range of half an (infrared) laser beam wavelength are already sufficient to switch from constructive interference to destructive interference. This advantageously allows even the smallest movements of the blood-perfused fundus of the eye, in particular the retina, which are caused, for example, by the pulse or the periodic blood flow generated by the pulse, to be recorded.Each of the images captured by the retina at short intervals thus corresponds to a slightly different optical path length. By creating and evaluating difference images, information about the change, such as the rate of change or the migration of a reduced optical path length, can be determined.

[0014] It is also proposed that the reflection signals used to determine the pulse rate and / or the blood flow direction form at least two-dimensional images, which are generated by means of a scanning, in particular sinusoidal, of the laser beam of the LFI sensor over at least a part, preferably a large part, of a visible area of ​​the eye. This advantageously makes it possible to achieve a high level of reliability in the measurement results. Advantageously, a signal from the retina can be reliably distinguished from other signals. Advantageously, a measurement signal can be maximized. In particular, the pupil contour and in particular the part of the measurement signal originating from the retina can be determined from the two-dimensional images comprising the laser speckle reflection pattern using image processing algorithms, such as a Canny Edge detection algorithm.It is conceivable that a determined pupil contour could be included in an evaluation based on the pulse rate to determine the status of the data glasses user. For example, a change in the tension and / or excitement of the data glasses user could lead to a simultaneous change in the pulse rate and pupil size (dilation). If the data glasses user is startled, for example, the pupil contracts quickly and the pulse rate increases rapidly. A large part should be understood to mean 60%, preferably 75%, and most preferably 90%. The visible area of ​​the eye is the area that can be illuminated by visible light when the eyes are normally open and usually includes the pupil, the iris, and part of the sclera.In particular, the laser beam of the LFI sensor is scanned across the visible area of ​​the eye using the microscanner and other optical elements of the virtual retinal display. In particular, complete frames of the LFI sensor are used in the method to determine the pulse rate and / or blood flow direction.

[0015] Alternatively, it is proposed that the reflection signals used to determine the pulse rate and / or blood flow direction originate only from individual one-dimensional scan lines of a portion of the eye encompassing a pupil. This advantageously increases measurement speed. Furthermore, computational effort and / or energy consumption can be advantageously reduced. This advantageously reduces the computational complexity for calculating optical flow and / or the necessary storage capacity for buffering measurement signals, particularly within measurement intervals necessary to maintain sufficient signal-to-noise ratios (e.g., 5 seconds).

[0016] Furthermore, it is proposed that, to determine the pulse rate or the blood flow direction, a change rate of pixels, in particular pixel brightnesses, of a detected pupil region in the reflection signal, in particular a change rate of pixels / pixel brightnesses within a region of the laser speckle reflection pattern, is calculated, preferably using an optical flow algorithm. This advantageously enables a reliable determination of the pulse rate and / or the blood flow direction from differential images generated based on the measurement signals and / or from image sequences of the measurement signals. The differential images or image sequences of the laser speckle reflection patterns can be evaluated using a known optical flow algorithm, for example, a Lucas-Kanade algorithm.The optical flow algorithm is specifically designed to calculate the change rate of the pixels in the pupil region of the captured reflection signal images of the detector element, defined by the laser speckle reflection pattern and preferably detected by one of the image processing algorithms. In particular, the Lucas-Kanade algorithm can be used to calculate the optical flow from the raw signals of the detector element. Prior filtering of the raw signal is conceivable, but does not appear necessary.

[0017] If, in order to determine the pulse frequency or the blood flow direction, the determined change rate of the pixel / pixel brightnesses is transformed into the frequency domain by means of a fast Fourier transformation (FFT), it is advantageously possible to easily read out the pulse rate, in particular the pulse frequency, and / or the blood flow direction, preferably from a frequency spectrum generated by means of FFT.

[0018] Furthermore, in particular as an alternative or in addition to the previously described pulse rate determination method, it is proposed that the reflection signal be continuously determined / filtered from the time signal of the LFI sensor by means of a wavelet transformation, in particular by means of (wavelet) template matching, in order to determine the pulse rate. This advantageously makes it possible to achieve a reliable and / or low-computational determination of the pulse rate. In particular, the use of wavelet transformation represents an alternative to determining the pulse rate from two-dimensional images. Preferably, with wavelet transformation, the pulse rate is determined directly from the time course of the measurement signals of the detector element of the LFI sensor. The procedure preferably follows a method that is already known from ECG systems.In particular, a wavelet kernel is convolved over the time signal, and then a correlation is determined, e.g., with known patterns. This is then preferably repeated with different wavelet kernels (e.g., different in shape or temporal extent, etc.), especially to find similarities, preferably to be able to extract the pulse rate from the time signal. The wavelet method, known from the field of ECG measurement, is described, for example, in a thesis titled "Analysis of ECG Signals with the Wavelet Transformation" by Marcus Holland-Moritz, published in 2000.

[0019] Furthermore, it is proposed that, to determine the direction of blood flow, different parts of a detected pupil region, in particular the laser speckle reflection pattern, are each observed separately over a period of time and related to one another in time. This advantageously enables a reliable determination of the direction of blood flow. In particular, blood flow in blood vessels occurs in bursts. If a longer time period is recorded and sub-regions of the measurement signal from the pupil / retina are observed individually over this longer time period, the direction of blood flow can be determined from the change in the measurement signals over time in relation to the individual sub-regions of the measurement signal from the pupil / retina.

[0020] Furthermore, the virtual retinal display, in particular for data glasses, such as an AR headset and / or VR headset, and / or the data glasses, is proposed, comprising at least one laser feedback interferometry (LFI) sensor, preferably for carrying out the method, which is provided at least for emitting the laser beam in the direction of the user's eye and which is provided at least for detecting reflection signals, in particular reflection patterns, preferably laser speckle reflection patterns, which are reflected by the retina of the user's eye, and comprising a computing unit which is at least configured to determine the pulse frequency of the user and / or the blood flow direction of at least one blood vessel of the user's eye from the detected reflection signals, in particular reflection patterns, preferably laser speckle reflection patterns.This advantageously allows a variety of information about the user of data glasses to be obtained. Pulse rate and / or blood flow direction measurement can advantageously be integrated into data glasses, in particular AR and / or VR systems. Pulse rate and / or blood flow direction measurement in data glasses that is insensitive to stray light can advantageously be enabled. Pulse rate and / or blood flow direction measurement can advantageously be enabled even in uncontrolled ambient brightness of the data glasses' surroundings, such as outdoors. The virtual retinal display and / or the data glasses have the computing unit for carrying out the method according to the invention. A "computing unit" is to be understood in particular as a unit with an information input, an information processing unit, and an information output.The computing unit advantageously comprises at least one processor, a memory, input and output means, further electrical components, an operating program, control routines, control routines, and / or calculation routines. The components of the computing unit are preferably arranged on a common circuit board and / or advantageously arranged in a common housing. Alternatively, the computing unit can be designed as an external computing unit, e.g., a cloud or a mobile device, which is in communication with the virtual retinal display and / or the data glasses.

[0021] It is also proposed that the virtual retinal display, in particular the data glasses, be designed without camera sensors, such as CCD sensors or CMOS sensors. This advantageously allows for high cost, energy, weight, and / or installation space efficiency.

[0022] The method according to the invention, the virtual retinal display according to the invention, and the data glasses according to the invention are not intended to be limited to the application and embodiment described above. In particular, the method according to the invention, the virtual retinal display according to the invention, and the data glasses according to the invention may have a number of individual elements, components, units, and method steps that differs from the number stated herein to fulfill a functionality described herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0023] drawing

[0024] Further advantages will become apparent from the following description of the drawings. The drawings illustrate an exemplary embodiment of the invention. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will also expediently consider the features individually and combine them into useful further combinations.

[0025] Shown are: Fig. 1 a schematic representation of data glasses with a virtual retinal display,

[0026] Fig. 2 schematically shows a sinusoidal scanning scheme with which laser beams are scanned through a microscanner of the virtual retinal display,

[0027] Fig. 3 shows an image of an eye in inverted form recorded by a detector element of a laser feedback interferometry (LFI) sensor of the virtual retinal display using the laser beam,

[0028] Fig. 4a is a schematic diagram of the formation of a laser speckle reflection pattern on a retina of the eye,

[0029] Fig. 4b is an enlargement of a section of Fig. 4a,

[0030] Fig. 5 is a schematic flow diagram of a method using the LFI sensor of the virtual retinal display of the data glasses,

[0031] Fig. 6a an exemplary FFT spectrum / frequency spectrum of an artificial reference eye without pulse,

[0032] Fig. 6b an exemplary FFT spectrum / frequency spectrum of a living eye in a first viewing direction and

[0033] Fig. 6c shows an exemplary FFT spectrum / frequency spectrum of the living eye in a second viewing direction.

[0034] Description of the embodiment

[0035] Figure 1 shows a schematic representation of data glasses 14. The data glasses 14 form an AR headset. The data glasses 14 comprise a virtual retinal display 12. The data glasses 14 comprise a spectacle frame 54. The data glasses 14 have a spectacle frame 36. The spectacle frame 36 forms part of the spectacle frame 54. The data glasses 14 have spectacle temples 38. The spectacle temple 38 forms part of the spectacle frame 54. The data glasses 14 comprise a spectacle lens 40. The spectacle lens 40 is a transparent lens. Figure 1 shows an example of a user's eye 24. The virtual retinal display 12 has a laser projector 42. The virtual retinal display 12 is designed free of camera sensors, such as CCD sensors or CMOS sensors, in particular camera sensors arranged separately from the laser projector 42.The laser projector 42 is configured to at least generate and / or output at least one scanned visible laser beam 44. The laser projector 42 has at least one (RGB) laser source 46, e.g., a laser diode, for generating the visible laser beam 44. The scanned visible laser beam 44 generates an image display of the virtual retinal display 12 on a retina 22 of the user's eye 24, perceivable by the user. The laser projector 42 has a laser feedback interferometry (LFI) sensor 10. The LFI sensor 10 is configured to output a further laser beam 16. The further laser beam 16 output by the LFI sensor 10 is an infrared laser beam. The further laser beam 16 output by the LFI sensor 10 is a scanned laser beam. The laser projector 42 has a microscanner 52. The microscanner 52 forms a MEMS micromirror system.The microscanner 52 is designed to scan the additional laser beam 16. The microscanner 52 is designed to scan the laser beam 16 sinusoidally (see Fig. 2). The virtual retinal display 12 is configured to project the additional laser beam 16 onto the user's eye 24.

[0036] The laser projector 42 is at least partially integrated into the spectacle frame 54. The virtual retinal display 12 has an optical system with at least one deflection element 48, which is configured to deflect the laser beams 16, 44 emitted by the laser projector 42 in the direction of the eye 24. The deflection element 48 is integrated into the spectacle lens 40. As a result, the spectacle lens 40 forms part of the virtual retinal display 12. The deflection element 48 is designed as a holographic optical element (HOE). The deflection element 48 focuses the visible laser beam 44 onto a surface of the eye 24, preferably onto a pupil 28 of the eye 24. The deflection element 48 deflects the further laser beam 16 such that it impinges on the surface of the eye 24 as at least substantially parallel beams. The deflection element 48 does not focus infrared laser beams, but merely deflects them.The eye 24 reflects the further laser beam 16. The reflection of the further laser beam 16 by the eye 24 generates a reflection signal 18. The eye 24 reflects the further laser beam 16, passing through the optical system again, back to the laser projector 42. The reflection signal 18 thereby re-enters the LFI sensor 10. The portion of the further laser beam 16 that enters the eye 24 through the pupil 28 and strikes the retina 22 is reflected back by the retina 22. This creates a laser speckle reflection pattern 20 (see Fig. 3).

[0037] The LFI sensor 10 has a detector element 50. The detector element 50 is formed integrally with a laser module of the LFI sensor 10. The detector element 50 is designed to detect infrared light. The detector element 50 is configured to detect the reflection signal 18. The detector element 50 is configured to detect the laser speckle reflection pattern 20. Two-dimensional images 26 are generated from the detected reflection signal 18 based on the associated scan angles of the microscanner 52 (see Fig. 3). The two-dimensional images 26 include the laser speckle reflection pattern 20. The virtual retinal display 12 has a computing unit 80. The computing unit 80 is configured to determine a pulse rate of the user of the data glasses 14 from the detected reflection signal 18.The computing unit 80 is configured to determine a blood flow direction of at least one blood vessel of the user's eye 24 from the detected reflection signal 18.

[0038] Figure 2 schematically shows the sinusoidal scanning pattern with which the laser beams 16, 44 are scanned by the microscanner 52. Figure 3 shows an inverted exemplary image 26 of the reflection signal 18, which includes the laser speckle reflection pattern 20. In its unprocessed form, the laser speckle reflection pattern 20 would be recognizable as a collection of bright spots in a dark background.

[0039] Figures 4a and 4b outline the formation of the laser speckle reflection pattern 20 on the retina 22. If the further laser beam 16 falls on the retina 22, this leads to strong backscattering, since the reflectivity of the retina 22 is stronger than that of the other components of the eye 24, such as the iris or the sclera. In addition, the additional laser beam 16 incident on the eye 24 can be collimated such that the eye 24 itself focuses the additional laser beam 16 onto the retina 22 (see Fig. 4a), whereby the light backscattered by the retina 22 (quasi-Lambertian) is coherently directed backward onto the LFI sensor 10, where it leads to a significantly increased amplitude modulation of the optical power of a laser of the infrared laser diode of the LFI sensor 10 for a scan area in which the additional laser beam 16 is imaged onto the retina 22 through the pupil 28. The amplitude of the backscattered laser light is modulated by speckle.The speckling is generated by the optical roughness of the retina 22 (cf. Fig. 4b). The roughness of the retina 22 is related to the anatomy of the eye 24 and behaves roughly similarly for all people. Fig. 4a schematically shows a cross-section through the eye 24 illuminated by the additional laser beam 16. The additional laser beam 16 penetrates a cornea 56 of the eye 24 and is focused by a lens 58 of the eye 24 onto the retina 22. There, the additional laser beam 16 strikes a rough surface 60 of the retina 22 with its coherent wavefront. Part of the incoming wavefront is reflected directly by the surface 60 of the retina 22, whereby the wavefront 62 of the reflected wave is distorted by the signal propagation times through the rough surface 60 of the retina 22.A further portion of the incoming wavefront enters the upper tissue layers of the retina 22 and is reflected from there, which also leads to a distortion of the retroreflected wavefront 62. These two effects result in constructive or destructive interference in the LFI sensor 10, so that the laser speckle reflection pattern 20 with bright speckles and with dark speckles is formed in a normally distributed manner in the imaged area of ​​the retina 22 and thus of the pupil 28 in the obtained image 26. On the iris and surrounding structures of the eye 24, however, the reflectivity is low and the further laser beam 16 is not focused, so that a solid angle for the retroreflection in the direction of the LFI sensor 10 is comparatively much smaller. In addition, the phase position of neighboring positions on the iris varies greatly due to roughness, so that only minimal interference occurs in the LFI sensor 10 due to the reflection signal 18 of the iris.The image areas that cannot be assigned to the retina 22 thus appear dark in the recorded image 26. Essentially, only coherent infrared light, which has passed through the beam path between the LFI sensor 10 and the retina 22 in both directions, contributes to structures in the image 26 through interference in the laser resonator of an infrared photodiode of the LFI sensor 10, so that non-coherent stray light or other wavelengths cannot disrupt the signal relevant to the LFI sensor 10. This highly selective effect can now be exploited, for example, to determine the position of the pupil 28 in the image 26 using a Canny Edge algorithm. Figure 5 shows a schematic flow diagram of a method using the LFI sensor 10 of the virtual retinal display 12 of the data glasses 14. In at least one method step 64, the laser beams 16, 44 are generated and output by the laser projector 42.The additional infrared laser beam 16 is generated and output by the LFI sensor 10. In at least one further method step 66, the additional laser beam 16 is reflected by the eye 24. In the process, the reflection signal 18 is generated. The additional laser beam 16 is reflected by the surface of the eye 24 in areas outside the pupil 28 and by the retina 22 in an area inside the pupil 28. Upon reflection from the retina 22, the laser speckle reflection pattern 20 is generated. In at least one further method step 68, the reflection signal 18 with the laser speckle reflection pattern 20 is detected by the detector element 50 of the LFI sensor 10. The light detected by the LFI sensor 10 therefore originally originates from the LFI sensor 10 itself.In at least one further method step 70, the two-dimensional image 26 is generated from the reflection signal 18 detected by the LFI sensor 10, taking into account the scanning angles of the microscanner 52 associated with the detection times. In method step 70, the LFI sensor 10 detects differential images and / or image sequences from the images 26. The reflection signals 18 thus form the at least two-dimensional images 26 that were generated by the scanning, in particular sinusoidal, of the further laser beam 16 over a portion of a visible area of ​​the eye 24.

[0040] In at least one method step 72, a pulse rate of an owner of the eye 24 is determined. The pulse rate is determined from minimal movements of the retina 22 generated by a pulse of the owner of the eye 24, which can be read from the images 26 of the LFI sensor 10. The pulse rate is determined in a sub-step 74 of method step 72 from the laser speckle reflection patterns 20 of the difference images. In an alternative sub-step 76 of method step 72, the pulse rate is determined solely based on reflection signals 18 originating from individual one-dimensional scan lines of a portion of the eye 24 encompassing the pupil 28.In at least one further sub-step 78 of method step 72, a one-dimensional or two-dimensional pupil region 30 is detected in the reflection signal 18, in particular the image 26 or the line, depending in particular on which of the sub-steps 74, 76 preceded sub-step 78. In method step 72, in a further sub-step 82, in particular regardless of which of the sub-steps 74, 76 was previously performed, a change rate of pixels, in particular pixel brightnesses, of the detected pupil region 30 in the reflection signal 18, in particular a change rate of pixels / pixel brightnesses within a region of the laser speckle reflection pattern 20, is calculated to determine the pulse frequency. The calculation is performed using an optical flow algorithm of the computing unit 80.In method step 72, in a further sub-step 84 following sub-step 82, in particular regardless of which of the sub-steps 74, 76 was previously carried out, the determined change rate of the pixels / pixel brightnesses is transformed into the frequency domain by means of a fast Fourier transformation (FFT) in order to determine the pulse frequency.

[0041] In a method step 86 that is alternative or additional to method step 72, the reflection signal 18 is continuously determined / filtered from the time signal of the LFI sensor 10 using a wavelet transformation to determine the pulse frequency. Wavelet template matching of the determined signal with known patterns is performed.

[0042] In at least one further method step 88, a blood flow direction of at least one blood vessel of the eye 24 is determined. The blood flow direction is determined from minimal jerky movements of the retina 22 generated by a pulse of the owner of the eye 24, which can be read from the images 26 of the LFI sensor 10. The blood flow direction is determined in a sub-step 90 of method step 88 from the laser speckle reflection patterns 20 of the difference images. In an alternative sub-step 92 of method step 88, the blood flow direction is determined solely based on reflection signals 18 originating from individual one-dimensional scan lines of a portion of the eye 24 encompassing the pupil 28.In at least one further sub-step 94 of method step 88, the one-dimensional or a two-dimensional pupil region 30 is detected in the reflection signal 18, in particular the image 26 or the line, depending in particular on which of the sub-steps 90, 92 preceded the sub-step 94. To determine the direction of blood flow, various parts 32, 34 of the detected pupil region 30 (cf. Fig.4), in particular of the laser speckle reflection pattern 20, are each viewed separately over a period of time and related to one another in time. In method step 88, in a further sub-step 96, in particular regardless of which of the sub-steps 90, 92 was previously carried out, the change rate of pixels, in particular of pixel brightnesses, of the detected pupil region 30 in the reflection signal 18, in particular the change rate of pixels / pixel brightnesses within the area of ​​the laser speckle reflection pattern 20, is calculated in order to determine the blood flow direction. The calculation is carried out by means of an optical flow algorithm of the computing unit 80.In method step 88, in a further sub-step 98 following sub-step 96, in particular regardless of which of the sub-steps 90, 92 was previously performed, the determined change rate of the pixels / pixel brightnesses is transformed into the frequency domain using a fast Fourier transformation (FFT) to determine the blood flow direction. In at least one further method step 104, the determined pulse rate and / or the determined blood flow direction are output by the computing unit 80 and / or the data glasses 14.

[0043] Figures 6a, 6b, and 6c each show FFT spectra / frequency spectra. Figure 6a shows an FFT spectrum / frequency spectrum of an artificial reference eye without a pulse. No pulse frequency can be read out here. Figures 6b and 6c show FFT spectra / frequency spectra of a living eye 24 in two different viewing directions (to the side and straight ahead). Pulse frequencies can be read out here in each case. The owner of the eye 24 measured in Figures 6b and 6c, for example, has a resting pulse of approximately 70 to 80 beats per minute, which corresponds to approximately 1.2 Hz to 1.6 Hz. The FFT spectra / frequency spectra of Figures 6a to 6c were each determined from approximately 300 images, which corresponds to a measurement duration of the virtual retinal display 12 of approximately five seconds. On the abscissa 100 of the FFT spectra / frequency spectra, a frequency in Hertz is plotted.A frequency / intensity is plotted on the ordinates 102 of the FFT spectra / frequency spectra.

Claims

Claims 1. Method with a laser feedback interferometry (LFI) sensor (10), in particular a virtual retinal display (12), preferably in data glasses (14), wherein at least one laser beam (16) is emitted by the LFI sensor (10) and wherein reflection signals (18) originating from the previously emitted laser beam (16) are recorded by the LFI sensor (10), characterized in that a pulse frequency of an owner of the eye (24) is determined from the recorded reflection signals (18) of the LFI sensor (10), which comprise laser speckle reflection patterns (20) of a retina (22) of an eye (24).

2. Method, in particular according to claim 1, with a laser feedback interferometry (LFI) sensor (10), in particular a virtual retinal display (12), preferably in data glasses (14), wherein at least one laser beam (16) is emitted by the LFI sensor (10) and wherein reflection signals (18) originating from the previously emitted laser beam (16) are recorded by the LFI sensor (10), characterized in that a blood flow direction of at least one blood vessel of the eye (24) is determined from the recorded reflection signals (18) of the LFI sensor (10), which comprise laser speckle reflection patterns (20) of a retina (22) of an eye (24).

3. Method according to claim 1 or 2, characterized in that the pulse frequency of the owner of the eye (24) and / or the blood flow direction of the blood vessel of the eye (24) is detected using differential images of the laser speckle reflection patterns (20).

4. Method according to one of the preceding claims, characterized in that the reflection signals (18) used to determine the pulse frequency and / or the blood flow direction form at least two-dimensional images (26) which are generated by means of a, in particular sinusoidal, Scanning the laser beam (16) of the LFI sensor (10) over at least a part, preferably a large part, of a visible area of the eye (24).

5. Method according to one of claims 1 to 3, characterized in that the reflection signals (18) used to determine the pulse frequency and / or the blood flow direction originate only from individual one-dimensional scan lines of a part of the eye (24) comprising a pupil (28).

6. Method according to one of the preceding claims, characterized in that, in order to determine the pulse frequency or the blood flow direction, a change rate of pixels, in particular of pixel brightnesses, of a detected pupil region (30) in the reflection signal (18), in particular a change rate of pixels / pixel brightnesses within a region of the laser speckle reflection pattern (20), is calculated, preferably by means of an optical flow algorithm.

7. Method according to claim 6, characterized in that, in order to determine the pulse frequency or the direction of blood flow, the determined change rate of the pixel / pixel brightnesses is transformed into the frequency domain by means of a fast Fourier transformation (FFT).

8. Method according to one of claims 1 to 5, characterized in that, in order to determine the pulse frequency, the reflection signal (18) is continuously determined / filtered from the time signal of the LFI sensor (10) by means of a wavelet transformation.

9. Method according to one of the preceding claims, characterized in that, in order to determine the direction of blood flow, different parts (32, 34) of a detected pupil region (30), in particular of the laser speckle reflection pattern (20), are each observed separately over a period of time and brought into temporal relation to one another.

10. A virtual retinal display (12), in particular for data glasses (14), such as an AR headset and / or VR headset, with at least one laser feedback interferometry (LFI) sensor (10), preferably for carrying out a method according to one of the preceding claims, which is provided at least for emitting a laser beam (16) in the direction of an eye (24) of a user, and which is provided at least for detecting reflection signals (18), in particular laser speckle reflection patterns (20), which are reflected by a retina (22) of the eye (24) of the user, characterized by a computing unit (80) which is at least configured to determine a pulse rate of the user and / or a blood flow direction of at least one blood vessel of the eye (24) of the user from the detected reflection signals (18), in particular laser speckle reflection patterns (20).

11. Virtual retinal display (12) according to claim 10, characterized by a design free of camera sensors, such as CCD sensors or CMOS sensors.

12. Data glasses (14), in particular AR headset and / or VR headset, with a virtual retinal display (12) according to one of claims 10 or 11.

Images