Method for using laser feedback interference sensor, virtual retina display and data glasses

Abstract

The invention is based on a method of using a laser feedback interference (LFI) sensor (10), in particular a virtual retinal display (12), preferably in data glasses (14), in which at least one laser beam (16) is output by the LFI sensor (10), and in which a reflected signal (18) originating from a previously output laser beam (16) is recorded by the LFI sensor (10). According to the invention, the pulse rate of the owner of the eye (24) and / or the blood flow direction of at least one blood vessel of the eye (24) is determined from a detected reflected signal (18) of the LFI sensor (10), said reflected signal comprising a laser speckle reflection pattern (20) of the retina (22) of the eye (24).

Description

Background Technology

[0001] A method has been proposed for using a laser feedback interferometry (LFI) sensor in data glasses, wherein at least one laser beam is output by the LFI sensor, and wherein a reflected signal originating from a previously output laser beam is recorded by the LFI sensor. Summary of the Invention

[0002] The present invention is based on a method using a laser feedback interferometry (LFI) sensor, preferably in data glasses, especially a virtual retina display, wherein at least one laser beam is output by the LFI sensor, and wherein a reflected signal originating from a previously output laser beam is recorded by the LFI sensor.

[0003] It is proposed to extract the pulse rate of the eye owner from the reflected signal detected by the LFI sensor, which includes the laser speckle reflection pattern of the retina of the eye. This advantageously allows for the acquisition of a wealth of information about the user of the data glasses. Advantageously, by using an LFI sensor with a particularly small and / or highly integrable system, pulse rate information can be obtained, thereby enabling installation in data glasses, especially AR and / or VR systems. Advantageously, by using an LFI sensor with a particularly energy-efficient system, pulse rate information can be obtained, thereby enabling installation in portable systems such as data glasses, especially AR and / or VR systems. Advantageously, by using an LFI sensor with a system particularly insensitive to interfering light, pulse rate information can be obtained, thereby particularly improving usability in areas with uncontrolled ambient brightness (such as outdoors). In particular, these significant advantages can be achieved by using an LFI sensor compared to systems using laser speckle flow imaging (LSFG) methods. In the LSFG method, a collimated and broadened laser beam is guided onto the object to be inspected, and its reflection is observed using imaging optics and a camera sensor, such as a CMOS sensor, that are separate from the laser system.

[0004] In an LFI method that can be performed by an LFI sensor, it is preferable to collimate the laser beam, especially an infrared laser beam, output from the LFI sensor and point it toward the object being measured, for example, toward the retina of a user wearing data glasses with an LFI sensor. The laser beam, especially the infrared laser beam, incident on the object being measured is reflected back by the object, especially the retina. In particular, a portion of the back-reflected light is reflected back into the laser cavity of the LFI sensor. In the LFI method, the laser cavity of the LFI sensor acts as an optical mixer between the local light wave emitted by the LFI sensor and the wave reflected back into the LFI sensor. Therefore, constructive and / or destructive interference occurs within the laser cavity of the LFI sensor, which modulates, in particular, the power of the laser of the LFI sensor. This modulation of power can be measured by a detector element, such as a photodiode, integrated in the back reflector of the laser cavity of the LFI sensor. The modulation signal measured by this detector element particularly illustrates the measurement signal used in the method according to the invention. To capture an image, in the LFI method, while scanning the detector element, the laser beam can be oscillated (preferably in 2D) on the surface of the object to be examined, especially tissue, preferably the retina. Preferably, when using a miniature scanner, especially a MEMS micromirror system with at least one or two oscillable micromirrors, the laser beam of the scanning LFI sensor is swept across the eye. Here, a portion of the laser beam of the LFI sensor, especially the infrared laser beam, enters the eye through the pupil and is projected onto the retina there.

[0005] The method used to determine the pulse rate utilizes the so-called "bright pupil effect," which is based, in particular, on the relatively high infrared reflectivity of the retina of the eye relative to adjacent / other parts of the eye. When using the "bright pupil effect," the pupil of the eye can be directly determined from the image detected by the detector element, and thus the retinal signal of the eye. Specifically, the retina of the eye produces a so-called laser speckle reflection pattern when reflecting a laser beam, especially a sufficiently coherent one. This laser speckle reflection pattern (also called: laser grains or light grains) is, in particular, a granular interference phenomenon that can be observed when a sufficiently coherently illuminated optically rough surface, such as the retina of the eye (see cone and rod cells of the retina), is used. Here, the roughness of the retina is related to the anatomy of the eye and behaves similarly for all individuals. Conversely, other areas of the eye, different from the retina, do not produce laser speckle reflection patterns. The retina has a rich blood supply, so the pulse beat causes slight movements of the retina, especially up and down movements. Therefore, the reflected signal generated by the retina (e.g., the laser speckle reflection pattern) can include modulation generated by the pulse beat.

[0006] The LFI sensor can be constructed as a VCSEL, preferably a ViP-VCSEL (“Vertical-Cavity Surface-Emitting Laser with Integrated Photodiode”). The LFI sensor can be integrated into the data glasses, for example, into the frame, lenses, or temples of the glasses. Specifically, the laser beam generated by the LFI sensor and used in the method according to the invention is an infrared laser beam that is invisible to the user and harmless to the eyes. The LFI sensor can be integrated with the (RGB) laser diode of the data glasses that generates visible image content for the virtual retina display in a common laser module / common laser projector. Alternatively, the LFI sensor can also be constructed separately from the laser projector having the (RGB) laser diode that generates visible image content. In this case, the infrared laser beam of the LFI sensor can be coupled to the RGB laser beam of the (RGB) laser diode that generates visible image content via a coupling structure of the data glasses, particularly separately constructed from the (RGB) laser projector. Furthermore, it is conceivable that the data glasses have more than one LFI sensor, for example, two, three, or more than three LFI sensors. The preferred measurement interval / preferred measurement duration for obtaining the measurement signal (from which the pulse rate is subsequently determined) is approximately 300 images and / or 5 seconds. However, shorter or longer measurement intervals / durations are also conceivable.

[0007] "Data glasses" should be understood in particular as wearable devices (head-mounted displays) that can add information to a user's field of vision. Data glasses preferably enable augmented reality, virtual reality, and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses, VR glasses, or AR glasses. Data glasses can be constructed as AR or VR head-mounted displays. In particular, data glasses feature a virtual retinal display (also known as a retinal scanning display or light field display), which is particularly familiar to those skilled in the art. The virtual retinal display is specifically configured to, for example, scan image content sequentially by deflecting at least one visible laser beam from at least one time-modulated light source (e.g., one or more (RGB) laser diodes of a laser projector) using optical elements (such as holographic optical elements (HOEs), prisms, mirrors, or waveguides) integrated into the lenses of the data glasses, and image it directly onto the retina of the user's eye via the optical elements. The pulse rate and / or blood flow direction derived from the reflected signals can be used, in particular, for health checks of the user of the data glasses, or for measuring the user's emotional state, excitement level, stress level, fatigue level, and degree of fatigue. "Configuration" or "setting" should be understood in particular as specifically programmed, designed and / or equipped. An object is set up or configured for a specific function, and should be understood in particular as the object implementing and / or performing that specific function in at least one application and / or runtime state.

[0008] Furthermore, a method is proposed using an LFI sensor, preferably used in data glasses, especially in virtual retinal displays, wherein the blood flow direction of at least one blood vessel in the eye is determined from the reflected signal of a laser speckle reflection pattern of the retina detected by the LFI sensor. This advantageously provides a wealth of information about the user of the data glasses. Advantageously, information about the blood flow direction can be obtained by using an LFI sensor with a particularly small and / or highly integrable system, thereby enabling installation in data glasses, especially AR and / or VR systems. Advantageously, information about the blood flow direction can be obtained by using an LFI sensor with a particularly energy-efficient system, thereby enabling installation in portable systems such as data glasses, especially AR and / or VR systems. Advantageously, information about the blood flow direction can be obtained by using an LFI sensor with a system particularly insensitive to interfering light, thereby particularly improving usability in areas with uncontrolled ambient brightness (such as outdoors). In particular, each pulse beat produces a deformation along the blood vessel surface and the tissue above it (e.g., the retina). This shifting deformation particularly modulates the laser speckle reflection pattern reflected by the retina. Therefore, the direction of blood flow can be advantageously read from the laser speckle reflection pattern detected by the detector elements.

[0009] Furthermore, it is proposed to detect the pulse rate and / or blood flow direction of the eye's vessels based on differential images of laser speckle reflection patterns. This advantageously enables the LFI sensor to reliably determine the pulse rate and / or blood flow direction. Since the measurement signal (i.e., the laser speckle reflection pattern) originates directly from the retina and is modulated there by interference, the change in the optical path length between the retina and the LFI sensor within half the wavelength range of the (infrared) laser beam is sufficient to transform from constructive interference to destructive interference. Thus, the smallest movements of blood flow through the fundus, especially the retina, can be advantageously detected, such as those caused by pulse beats or by periodic blood flow generated by pulse beats. Each image captured from the retina within a short temporal interval therefore corresponds to a slightly different optical path length. Therefore, by forming and evaluating differential images, information about changes, such as the rate of change of the decreasing optical path length or movement, can be obtained.

[0010] Furthermore, it is proposed that a two-dimensional image be formed from the reflected signals used to determine the pulse rate and / or blood flow direction. This two-dimensional image is generated by scanning, particularly sinusoidal scanning, at least a portion, preferably most of, the visible area of ​​the eye using a laser beam from an LFI sensor. This advantageously allows for high reliability of the measurement results. Advantageously, the retinal signal can be reliably distinguished from other signals. Advantageously, the measurement signal can be maximized. In particular, the pupil contour, and especially the portion of the measurement signal originating from the retina, can be determined from the two-dimensional image including the laser speckle reflection pattern via an image processing algorithm, such as the Canny edge detection algorithm. It is conceivable that the determined pupil contour be incorporated into an assessment of the data glasses user's state based on pulse rate. For example, when the user's level of tension and / or excitement changes, a simultaneous change in pulse rate and pupil size (dilation) may occur. For example, if the user is startled, the pupil will rapidly constrict and the pulse rate will increase sharply. "Most of" should be understood in particular as 60%, preferably 75%, and more preferably 90%. The visible area of ​​the eye, especially the area illuminated by visible light when the eye is normally open, typically includes the pupil, iris, and part of the sclera. Specifically, the laser beam of the LFI sensor is used to scan this portion of the visible area of ​​the eye using a miniature scanner and other optical elements of a virtual retina display. In particular, the entire frame of the LFI sensor is used in this method to determine the pulse rate and / or blood flow direction.

[0011] As an alternative, it is proposed that the reflected signals used to determine pulse rate and / or blood flow direction originate only from the individual one-dimensional scan lines of the eye, including the pupil. This advantageously increases measurement speed. Furthermore, it advantageously reduces computational cost and / or energy consumption. Advantageously, this reduces the computational complexity of optical flow calculation and / or reduces the required storage capacity for intermediate storage of measurement signals, especially within the measurement interval (e.g., 5 seconds) required to obtain a sufficient signal-to-noise ratio.

[0012] Furthermore, it is proposed that, in order to determine the pulse rate or blood flow direction, an optical flow algorithm is preferably used to calculate the pixel change rate in the detected pupil region of the reflected signal, particularly the pixel / pixel brightness change rate within the region of the laser speckle reflection pattern. This advantageously enables reliable determination of the pulse beat and / or blood flow direction from differential images generated based on measurement signals and / or image sequences based on measurement signals. Differential images or image sequences of laser speckle reflection patterns can be evaluated using known optical flow algorithms, such as the Lucas-Kanade algorithm. Here, the optical flow algorithm is specifically configured to calculate the pixel change rate in the pupil region of the detected reflected signal image of the detector element, which is bounded by the laser speckle reflection pattern and preferably detected by one of the image processing algorithms. In particular, the Lucas-Kanade algorithm can be used to calculate the optical flow from the raw signal of the detector element. Prior filtering of the raw signal is conceivable, but does not appear to be necessary.

[0013] If, in order to obtain the pulse rate or blood flow direction, the desired rate of change of pixel / pixel brightness is transformed into the frequency domain using a Fast Fourier Transform (FFT), it is advantageous to simply read out the pulse beat, especially the pulse rate, and / or blood flow direction, from the spectrum generated by the FFT.

[0014] Furthermore, in particular as an alternative to or addition to the aforementioned pulse rate determination method, it is proposed that, in order to obtain the pulse rate, the reflected signal is continuously extracted / filtered from the time signal of the LFI sensor using wavelet transform, especially using (wavelet) template matching. This advantageously allows for reliable and / or computationally inefficient determination of the pulse rate. In particular, the use of wavelet transform provides an alternative to obtaining the pulse rate from a two-dimensional image. Preferably, the pulse rate is determined directly from the temporal variation of the measurement signal of the detector element of the LFI sensor in the wavelet transform. Here, the method preferably follows methods already known from ECG (electrocardiogram) systems. In particular, a trial function (wavelet kernel) is folded onto the time signal and subsequently correlated (e.g., with known patterns). This is then preferably repeated with different wavelet kernels (e.g., different in shape or temporal extension, etc.), especially to find consistency so that the pulse beat can preferably be read from the time signal. For example, Marcus Holland-Moritz's 2000 dissertation, entitled "Analysis of ECG Signals with Wavelet Transformation," describes wavelet methods known from the field of ECG measurement.

[0015] Furthermore, it is proposed that, in order to determine the blood flow direction, different portions of the detected pupil region, particularly the laser speckle reflection pattern, should be observed individually over a time period and correlated temporally. This allows for a reliable determination of the blood flow direction. In particular, blood flow in blood vessels is stoßartig. If a longer time range is recorded, and portions of the measurement signal from the pupil / retina are observed individually over this longer time range, the blood flow direction can be determined from the change in the measurement signal over time relative to each portion of the measurement signal from the pupil / retina.

[0016] Furthermore, a virtual retinal display, and / or data glasses, is proposed, particularly for data glasses (e.g., AR and / or VR headsets), having at least one laser feedback interferometry (LFI) sensor, preferably used to perform the method. This LFI sensor is at least configured to output a laser beam in a direction toward the user's eyes, and is at least configured to detect reflected signals projected back from the user's retina, particularly a reflection pattern, preferably a laser speckle reflection pattern. It also has a computing unit configured to calculate the user's pulse rate and / or the blood flow direction of at least one blood vessel in the user's eye from the detected reflected signals, particularly the reflection pattern, preferably the laser speckle reflection pattern. This advantageously provides a wealth of information about the user of the data glasses. Advantageously, pulse rate measurement and / or blood flow direction measurement can be integrated into the data glasses, particularly in AR and / or VR systems. Advantageously, pulse rate measurement and / or blood flow direction measurement insensitive to interfering light can be implemented in the data glasses. Advantageously, pulse rate measurement and / or blood flow direction measurement can be achieved even when the ambient brightness of the environment surrounding the data glasses is uncontrolled (e.g., outdoors). The virtual retina display and / or the data glasses have a computing unit for performing the method according to the invention. The term "computing unit" should be understood in particular as a unit having an information input section, an information processing section, and an information output section. Advantageously, the computing unit has at least one processor, memory, input and output devices, other electrical components, operating procedures, adjustment routines, control routines, and / or calculation routines. Preferably, the components of the computing unit are arranged on a common circuit board and / or advantageously arranged in a common housing. Alternatively, the computing unit may be configured as an external computing unit, such as a cloud or mobile device, in communicative connection with the virtual retina display and / or data glasses.

[0017] Furthermore, it is proposed that virtual retinal displays, especially data glasses, are constructed without camera sensors, such as CCD or CMOS sensors. This advantageously enables high cost efficiency, energy efficiency, weight efficiency, and / or installation space efficiency.

[0018] Hereinafter, the method according to the invention, the virtual retina display according to the invention, and the data glasses according to the invention should not be limited to the applications and embodiments described above. In particular, in order to achieve the functional modes described herein, the method according to the invention, the virtual retina display according to the invention, and the data glasses according to the invention may have a different number of individual elements, components, units, and method steps than those mentioned herein. Furthermore, for the numerical ranges given in this disclosure, values ​​within the limits should also be considered disclosed and can be used arbitrarily. Attached Figure Description

[0019] Further advantages are illustrated in the following figures, which depict embodiments of the invention. The figures, description, and claims contain features in numerous combinations. Those skilled in the art will also find it appropriate to consider these features individually and combine them into other meaningful combinations.

[0020] The attached diagram shows: Figure 1 A schematic diagram of data glasses with a virtual retina display is shown. Figure 2 This schematically illustrates a sinusoidal scanning scheme of a laser beam scanned by a miniature scanner on a virtual retina display. Figure 3 This image shows an inverted image of the eye recorded by the detector element of a laser feedback interferometry (LFI) sensor using a laser beam from a virtual retina display. Figure 4a This diagram illustrates the laser speckle reflection pattern produced on the retina of the eye. Figure 4b Show Figure 4a A magnified view of a specific area. Figure 5 A schematic flowchart illustrating a method for using an LFI sensor in a virtual retinal display with data glasses. Figure 6a An exemplary FFT spectrum / spectrum of an artificial reference eye without a pulse is shown. Figure 6b An exemplary FFT spectrum / spectrum of a living eye in the first line of sight is shown, and Figure 6c An exemplary FFT spectrum / spectrum of a living eye in the second line of sight is shown. Detailed Implementation

[0021] Figure 1A schematic diagram of data glasses 14 is shown. Data glasses 14 constitutes an AR head-mounted display. Data glasses 14 includes a virtual retinal display 12. Data glasses 14 includes an eyeglass frame 54. Data glasses 14 has eyeglasses rims 36. Eyeglasses rims 36 constitute part of eyeglasses frame 54. Data glasses 14 has eyeglasses temples 38. Eyeglasses temples 38 constitute part of eyeglasses frame 54. Data glasses 14 includes eyeglasses lenses 40. Eyeglasses lenses 40 are see-through lenses. Figure 1 The image exemplarily illustrates a user's eye 24. The virtual retina display 12 has a laser projector 42. The virtual retina display 12 does not have a camera sensor, such as a CCD sensor or CMOS sensor, arranged separately from the laser projector 42. The laser projector 42 is at least configured to generate and / or output at least one scanning visible laser beam 44. The laser projector 42 has at least one (RGB) laser source 46, such as a laser diode, for generating the visible laser beam 44. The scanning visible laser beam 44 produces a user-perceptible image display of the virtual retina display 12 on the retina 22 of the user's eye 24. The laser projector 42 has a laser feedback interferometry (LFI) sensor 10. The LFI sensor 10 is configured to output another laser beam 16. The other laser beam 16 output by the LFI sensor 10 is an infrared laser beam. The other laser beam 16 output by the LFI sensor 10 is a scanning laser beam. The laser projector 42 has a miniature scanner 52. The miniature scanner 52 constructs a MEMS micromirror system. The miniature scanner 52 is configured to scan the other laser beam 16. Miniature scanner 52 is configured for sinusoidal scanning laser beam 16 (see...) Figure 2 The virtual retina display 12 is configured to shine another laser beam 16 onto the user's eyes 24.

[0022] The laser projector 42 is at least partially integrated into the eyeglass frame 54. The virtual retinal display 12 has an optical system with at least one steering element 48 configured to deflect laser beams 16, 44 output by the laser projector 42 in the direction toward the eye 24. The steering element 48 is integrated into the eyeglass lens 40. Thus, the eyeglass lens 40 forms part of the virtual retinal display 12. The steering element 48 is configured as a holographic optical element (HOE). The steering element 48 focuses the visible laser beam 44 onto the surface of the eye 24, preferably onto the pupil 28 of the eye 24. The steering element 48 deflects another laser beam 16 such that it strikes the surface of the eye 24 as a substantially parallel beam. The steering element 48 does not focus the infrared laser beam, but merely deflects it. The eye 24 reflects the other laser beam 16. A reflection signal 18 is generated by the reflection of the other laser beam 16 by the eye 24. The eye 24 causes the other laser beam 16 to be reflected back through the optical system to the laser projector 42. The reflected signal 18 thus re-enters the LFI sensor 10. The portion of the other laser beam 16 that passes through the pupil 28, enters the eye 24, and strikes the retina 22 is reflected / projected back by the retina 22. This produces the laser speckle reflection pattern 20 (see...). Figure 3 ).

[0023] The LFI sensor 10 has a detector element 50. The detector element 50 is integrally constructed with the laser module of the LFI sensor 10. The detector element 50 is configured to detect infrared light. The detector element 50 is configured to detect a reflected signal 18. The detector element 50 is configured to detect a laser speckle reflection pattern 20. A two-dimensional image 26 (see [reference]) is generated from the detected reflected signal 18 based on the relevant scanning angle of the miniature scanner 52. Figure 3 The two-dimensional image 26 includes a laser speckle reflection pattern 20. The virtual retinal display 12 has a computing unit 80. The computing unit 80 is configured to determine the pulse rate of the user of the data glasses 14 from the detected reflection signal 18. The computing unit 80 is also configured to determine the blood flow direction of at least one blood vessel in the user's eye 24 from the detected reflection signal 18.

[0024] Figure 2 The diagram schematically illustrates a sinusoidal scanning scheme of laser beams 16 and 44 scanned by a miniature scanner 52. Figure 3 An exemplary image 26 showing an inverted reflection signal 18 including laser speckle reflection pattern 20 is shown. In its unprocessed form, laser speckle reflection pattern 20 would be identified as a collection of bright spots against a dark background.

[0025] Figure 4a and 4bThe diagram illustrates the generation of the laser speckle reflection pattern 20 on the retina 22. If another laser beam 16 strikes the retina 22, this results in strong backscattering because the retina 22 has a higher reflectivity than other parts of the eye 24, such as the iris or sclera. Furthermore, the other laser beam 16 striking the eye 24 can be collimated such that the eye 24 itself focuses the other laser beam 16 onto the retina 22 (see [reference]). Figure 4a The (quasi-Lambertian) light reflected back from the retina 22 is thus coherently guided backward onto the LFI sensor 10, where another laser beam 16 passes through the pupil 28 and images onto the scanning area of ​​the retina 22. This results in a significant increase in the amplitude modulation of the optical power of the laser in the infrared laser diode of the LFI sensor 10. The amplitude of the back-reflected laser is modulated by speckle. The speckle is produced by the optical roughness of the retina 22 (see...). Figure 4b Here, the roughness of the retina 22 is related to the anatomical structure of the eye 24, and behaves roughly similarly for all people. Figure 4a The diagram schematically shows a cross-section of an eye 24 illuminated by another laser beam 16. This other laser beam 16 penetrates the cornea 56 of the eye 24 and is focused onto the retina 22 by the lens 58 of the eye 24. There, the other laser beam 16 strikes the rough surface 60 of the retina 22 with its coherent wavefront. A portion of the incident wavefront is reflected directly by the surface 60 of the retina 22, where the wavefront 62 of the reflected wave is distorted due to the signal travel time across the rough surface 60 of the retina 22. Another portion of the incident wavefront enters the upper tissue layer of the retina 22 and is reflected from there, also resulting in distortion of the back-reflected wavefront 62. Through these two effects, constructive or destructive interference occurs in the LFI sensor 10, causing a laser speckle reflection pattern 20 with bright and dark speckles to be normally distributed in the imaging region of the retina 22 and thus the pupil 28 in the acquired image 26. Conversely, the reflectivity of the structures surrounding the iris and eye 24 is low, and the other laser beam 16 is unfocused, resulting in a relatively small solid angle of retroreflection towards the LFI sensor 10. Furthermore, the phase of adjacent locations on the iris differs significantly due to roughness, thus the reflected signal 18 from the iris in the LFI sensor 10 produces only minor interference. Therefore, image areas that cannot be assigned to the retina 22 appear dark in the recorded image 26. Thus, essentially only the coherent infrared light that has already passed through the optical path between the LFI sensor 10 and the retina 22 in both directions affects the structures in image 26 through interference in the laser resonator of the infrared photodiode of the LFI sensor 10, preventing incoherent interference light or other wavelengths from interfering with the signal associated with the LFI sensor 10. This high selectivity effect can now be utilized, for example, by means of the Canny edge algorithm, to determine the position of the pupil 28 in image 26.

[0026] Figure 5 A schematic flowchart of a method for using an LFI sensor 10 on a virtual retinal display 12 with data glasses 14 is shown. In at least one method step 64, laser beams 16, 44 are generated and output by a laser projector 42. Here, another infrared laser beam 16 is generated and output by the LFI sensor 10. In at least one additional method step 66, another laser beam 16 is reflected by the eye 24. A reflection signal 18 is generated here. The other laser beam 16 is reflected by the surface of the eye 24 in the region outside the pupil 28 and by the retina 22 in the region inside the pupil 28. A laser speckle reflection pattern 20 is generated at the reflection at the retina 22. In at least one additional method step 68, the reflection signal 18 with the laser speckle reflection pattern 20 is detected by the detector element 50 of the LFI sensor 10. The light detected by the LFI sensor 10 therefore originates initially from the LFI sensor 10 itself. In at least one additional method step 70, a two-dimensional image 26 is generated from the reflection signal 18 detected by the LFI sensor 10 combined with the scanning angle of the micro-scanner 52 at the detection time. In method step 70, the LFI sensor 10 detects differential images and / or image sequences from image 26. Thus, the reflected signal 18 constructs at least a two-dimensional image 26, which is generated by causing another laser beam 16 to perform a particularly sinusoidal scan over a portion of the visible area of ​​the eye 24.

[0027] In at least one method step 72, the pulse rate of the owner of eye 24 is determined. The pulse rate is determined from the minute movements of retina 22 caused by the pulse beats of the owner of eye 24, which can be read from image 26 of LFI sensor 10. Here, in sub-step 74 of method step 72, the pulse rate is determined from the laser speckle reflection pattern 20 of the differential image. In an alternative sub-step 76 of method step 72, the pulse rate is determined only from the reflection signals 18 originating from the respective one-dimensional scan lines of the portion of eye 24 including pupil 28. In at least one additional sub-step 78 of method step 72, particularly depending on which of sub-steps 74, 76 precedes sub-step 78, a one-dimensional or two-dimensional pupil region 30 is detected in the reflection signals 18, particularly in image 26 or lines. In method step 72, in a further sub-step 82, particularly independent of which of the previously performed sub-steps 74 and 76 was implemented, the change-rate of pixel brightness in the pupil region 30 detected in the reflected signal 18 was calculated to obtain the pulse rate, specifically the change-rate of pixel / pixel brightness within the region of the laser speckle reflection pattern 20. This calculation was performed using the optical flow algorithm of the calculation unit 80. In method step 72, in a further sub-step 84 following sub-step 82, particularly independent of which of the previously performed sub-steps 74 and 76 was implemented, the obtained change rate of pixel / pixel brightness was transformed into the frequency domain using a Fast Fourier Transform (FFT) to obtain the pulse rate.

[0028] In method step 86, which is an alternative to or additional to method step 72, the reflected signal 18 is continuously extracted / filtered from the time signal of LFI sensor 10 by means of wavelet transform in order to obtain the pulse rate. Wavelet template matching is performed on the extracted signal using a known pattern.

[0029] In at least one additional method step 88, the blood flow direction of at least one blood vessel of the eye 24 is determined. The blood flow direction is determined from the minute pulse-like movements of the retina 22 generated by the pulse of the owner of the eye 24, which can be read from the image 26 of the LFI sensor 10. Here, in sub-step 90 of method step 88, the blood flow direction is determined from the laser speckle reflection pattern 20 of the differential image. In an alternative sub-step 92 of method step 88, the blood flow direction is determined only from the reflection signals 18 originating from the respective one-dimensional scan lines of the eye 24, including the pupil 28. In at least one additional sub-step 94 of method step 88, particularly depending on which of sub-steps 90, 92 precedes sub-step 94, a one-dimensional or two-dimensional pupil region 30 is detected in the reflection signals 18, particularly in the image 26 or the lines. To determine the blood flow direction, the detected pupil region 30 (see Figure 4), and especially different portions 32 and 34 of the laser speckle reflection pattern 20, are observed individually within a time period and correlated temporally. In method step 88, in a further sub-step 96, particularly independent of which of the previously performed sub-steps 90 and 92, the change-rate of pixel brightness in the detected pupil region 30 in the reflected signal 18 is calculated, particularly the change-rate of pixel / pixel brightness within the region of the laser speckle reflection pattern 20, in order to determine the blood flow direction. This calculation is performed using the optical flow algorithm of the calculation unit 80. In method step 88, in a further sub-step 98 following sub-step 96, particularly independent of which of the previously performed sub-steps 90 and 92, the obtained change rate of pixel / pixel brightness is transformed into the frequency domain using a Fast Fourier Transform (FFT) to determine the blood flow direction. In at least one additional method step 104, the calculated pulse rate and / or the calculated blood flow direction are output by the computing unit 80 and / or the data glasses 14.

[0030] Figure 6a , 6b 6c and 6c each show the FFT spectrum / spectrum. Figure 6a The FFT spectrum / spectrum of an artificial reference eye without a pulse is shown. The pulse rate cannot be read here. Figure 6b and 6c The FFT spectrum / spectrum of a living eye 24 is shown in two different viewing directions (lateral and forward). The pulse rate can be read from each. Figure 6b and 6c The owner of the eye 24 being measured exemplarily has a resting pulse of about 70 to 80 beats per minute, which is equivalent to about 1.2 to 1.6 Hz. Figures 6a to 6cThe FFT spectra / spectral values ​​were each derived from approximately 300 images, which corresponds to a measurement duration of approximately five seconds on the virtual retinal display 12. The frequencies, in Hertz, are plotted on the x-axis (100) of the FFT spectra / spectral values. The occurrence frequency / intensity is plotted on the y-axis (102).

Claims

1. A method of using a laser feedback interferometry (LFI) sensor (10) with, preferably in, a virtual retinal display (12), especially in data glasses (14), wherein, At least one laser beam (16) is output by the LFI sensor (10), and wherein the LFI sensor (10) records a reflected signal (18) originating from the previously output laser beam (16), characterized in that the pulse rate of the owner of the eye (24) is obtained from the reflected signal (18) detected by the LFI sensor (10), the reflected signal including a laser speckle reflection pattern (20) of the retina (22) of the eye (24).

2. A method of using a laser feedback interferometry (LFI) sensor (10), preferably in a data eyeglass (14), especially a virtual retinal display (12), especially according to claim 1, wherein, The LFI sensor (10) outputs at least one laser beam (16), and wherein the LFI sensor (10) records a reflection signal (18) originating from the previously output laser beam (16), characterized in that the blood flow direction of at least one blood vessel of the eye (24) is determined from the reflection signal (18) detected by the LFI sensor (10), the reflection signal including a laser speckle reflection pattern (20) of the retina (22) of the eye (24).

3. The method according to claim 1 or 2, characterized in that, The pulse rate of all persons in the eye (24) and / or the direction of blood flow in the blood vessels of the eye (24) are detected based on the differential image of the laser speckle reflection pattern (20).

4. The method according to any of the preceding claims, characterized in that, The reflected signal (18) used to determine the pulse rate and / or the direction of blood flow is considered to form at least a two-dimensional image (26), which is generated by scanning, in particular a sinusoidal scan, over at least a portion, preferably most of, the visible area of ​​the eye (24) by means of the laser beam (16) of the LFI sensor (10).

5. The method according to any one of claims 1 to 3, characterized in that, Consider that the reflected signal (18) used to determine the pulse rate and / or the direction of blood flow originates only from the individual one-dimensional scan lines of the eye (24), including the portion of the pupil (28).

6. The method according to any of the preceding claims, characterized in that, In order to determine the pulse rate or the blood flow direction, it is preferable to use an optical flow algorithm to calculate the pixel change rate of the detected pupil region (30) in the reflected signal (18), especially the pixel change rate of the pixel brightness in the region of the laser speckle reflection pattern (20).

7. The method of claim 6, wherein, To determine the pulse rate or the blood flow direction, the calculated change rate of the pixel / pixel brightness is transformed into the frequency domain using a Fast Fourier Transform (FFT).

8. The method according to any one of claims 1 to 5, characterized in that, In order to obtain the pulse rate, the reflected signal (18) is continuously obtained / filtered from the time signal of the LFI sensor (10) by means of wavelet transform.

9. The method according to any of the preceding claims, characterized in that, In order to determine the direction of blood flow, the detected pupil area (30), especially different parts (32, 34) of the laser speckle reflection pattern (20), were observed separately over a time period and correlated with each other in time.

10. A virtual retinal display (12) particularly for data glasses (14), such as AR headsets and / or VR headsets, the virtual retinal display having at least one laser feedback interferometry (LFI) sensor (10), preferably for performing the method according to any one of the preceding claims, the laser feedback interferometry sensor being at least configured to output a laser beam (16) in a direction toward the user's eye (24), and the laser feedback interferometry sensor being at least configured to detect a reflected signal (18), in particular a laser speckle reflection pattern (20), reflected from the retina (22) of the user's eye (24). characterized in that The calculation unit (80) is configured to determine the pulse rate of the user and / or the blood flow direction of at least one blood vessel in the user's eye (24) from the detected reflection signal (18), especially the laser speckle reflection pattern (20).

11. The virtual retina display (12) of claim 10, characterized by It does not have a camera sensor, such as a CCD sensor or a CMOS sensor.

12. A data glasses (14), particularly an AR headset and / or a VR headset, having a virtual retinal display (12) according to any one of claims 10 or 11.

Images