Pupil detection method, pupil detection device and data spectacles

Abstract

The invention relates to a pupil detection method, in particular for data spectacles (10), comprising at least one scanning step (30) in which a laser beam (12), in particular an infrared laser beam, of at least one laser feedback interferometry (LFI) sensors (14) is scanned, in particular by means of an MEMS micromirror system (16), over at least the majority of a visible part of a user eye (18), and comprising at least one detection step (32) in which a part (20) of the laser beam (12) scattered back from the user eye (18) is detected by the LFI sensor (14) and converted into an electric signal. It is proposed that in at least one recording step (34), for a detection of a pupil (22) of the user eye (18), signal excursions (24, 50, 52) in the electric signal, which in particular exceed a specifiable threshold value, are recorded together with scan coordinates (α, β), in particular of the MEMS mirror signal (16), belonging to the respective signal excursions (24).

Description

[0001] Description and data glasses

[0002] State of the art

[0003] A pupil detection method for data glasses has already been proposed, comprising at least one scanning step in which a laser beam from a laser feedback interferometry (LFI) sensor is scanned across at least a large part of a visible part of a user's eye, and at least one detection step in which a portion of the laser beam scattered back by the user's eye is detected by the LFI sensor and converted into an electrical signal. For example, an infrared laser beam scanned across the user's eye is backscattered depending on the reflectivity of the respective eye components of the user's eye, and a two-dimensional image is created from the reflection signal, in which the pupil appears bright due to increased reflectivity of the retina ("bright pupil effect"). Using an image recognition algorithm, such asA Canny Edge Detector detects a bright pupil contour in the image and then determines a pupil ellipse. This pupil detection method requires continuous sampling to capture the complete two-dimensional images, which results in relatively high power consumption.

[0004] Disclosure of the invention

[0005] The invention is based on a pupil detection method, in particular for data glasses, with at least one scanning step in which a laser beam, in particular an infrared laser beam, of at least one laser feedback interferometry (LFI) sensor, in particular by means of a MEMS micromirror system, is scanned over at least a large part of a visible part of a user's eye, in particular two-dimensionally, and with at least one detection step in which a part of the laser beam scattered back by the user's eye is at least partially detected by the LFI sensor and, preferably with the aid of an electronic unit, converted into an electrical signal.

[0006] It is proposed that, in at least one recording step for detecting a pupil of the user's eye, preferably exclusively, signal deflections in the electrical signal which in particular exceed a definable threshold are recorded together with scan coordinates associated with the respective signal deflections, in particular of the MEMS micromirror system. This advantageously increases efficiency. Advantageously, the power consumption required for pupil detection can be significantly reduced. Advantageously, continuous sampling of photodiode signals, e.g. of the LFI sensor, can be dispensed with. Furthermore, the storage requirement for signals / images recorded by the LFI sensor can advantageously be significantly reduced, in particular since only individual signal deflections need to be recorded instead of entire continuous images.In addition, image processing algorithms can advantageously be dispensed with or at least the power consumption of image processing algorithms can be significantly reduced, in particular since the signals / images to which these image processing algorithms are applied are significantly simpler and / or smaller in their data size.

[0007] “Data glasses” should be understood in particular as a wearable (head-mounted display) by means of which information can be added to a user’s field of vision. Data glasses preferably enable augmented reality, virtual reality and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses or VR glasses or AR glasses. In particular, the data glasses have a virtual retinal display (also called a retinal scan display or light field display), which is particularly familiar to those skilled in the art. The virtual retinal display is particularly designed to sequentially scan an image content by deflecting at least one visible laser beam from at least one temporally modulated light source, such as one or more (RGB) laser diodes of a laser projector, and to project it directly onto the retina of the user’s eye using optical elements.The pupil detection method can be used in particular for eye tracking using an eye-tracking system that is preferably integrated into the data glasses. An eye-tracking system is particularly configured to track and / or record a movement of at least the user's eye. In particular, the LFI sensor forms a, particularly integral, component of the eye-tracking system of the data glasses. The eye-tracking system could be provided for detecting an eye and / or pupil position of the user's eye using the so-called "dark pupil effect." Preferably, however, the eye-tracking system is provided at least for detecting the eye and / or pupil position using the so-called "bright pupil effect," which is based in particular on the relatively high infrared reflectivity of the retina of the eye compared to neighboring / other components of the user's eye."Configured" or "intended" should be understood in particular to mean specifically programmed, designed, and / or equipped. The fact that an object is intended or configured for a specific function should be understood in particular to mean that the object fulfills and / or performs this specific function in at least one application and / or operating state.

[0008] The LFI sensor can be designed, for example, as a VCSEL, preferably ViP-VCSEL (vertical-cavity surface-emitting laser with integrated photo diode). The LFI sensor is integrated in particular into the data glasses, e.g., into a frame, lens, or temple of the data glasses. The LFI sensor can be integrated into a common laser module / laser projector with the (RGB) laser diodes of the data glasses that generate the projected image. Alternatively, the LFI sensor can also be designed separately from the laser projector with the (RGB) laser diodes that generate the projected image. It is also conceivable for the data glasses to have more than one LFI sensor, e.g., two, three, or more than three LFI sensors. The LFI sensor is based on an interferometric measuring method. The LFI sensor can also be capable of detecting a distance to a target (e.g., the user's eye) as well as a surface speed of the target.In particular, the LFI sensor emits the laser beam in the infrared spectrum, which then strikes a surface with a reflectivity R at an angle y. The light of the laser beam is then backscattered from this surface, so that it re-enters a laser cavity of the LFI sensor. In the laser cavity of the LFI sensor, the backscattered light interferes with a locally oscillating field of the LFI sensor. This leads, in particular, to a modulation of the laser power of the laser source, which can be detected either by a photodiode integrated into a back reflector of the laser cavity or by a direct measurement of a voltage of the laser source. In particular, the photodiode of the LFI sensor, which is integrated into the laser module, is designed to detect the backscattered part of the laser beam and convert it into an electrical signal.Furthermore, a current from the laser source can be modulated with a modulation signal, preferably a triangular one, to achieve a cyclic shift in the laser wavelength. Given known laser parameters, a beat frequency and a Doppler frequency can then be determined. From this, the surface velocity of the eye, the distance of the LFI sensor from the eye, and other eye parameters can be determined using known equations.

[0009] The MEMS micromirror system preferably forms at least part of a microscanner. The microscanner is, in particular, a micro-opto-electromechanical system from the class of micromirror actuators for the dynamic modulation of light. A modulating movement of individual mirrors of the MEMS micromirror system can be rotational about at least one or two axes, which in particular deflects light beams incident on the individual mirrors. In particular, the scanned laser beam is scanned line by line. However, scanning the scanned laser beam column by column is also conceivable. Typical dimensions of individual mirrors of a MEMS micromirror system are in the range between one and three millimeters. However, larger mirror apertures (e.g., 10 mm x 3 mm) can also be manufactured. Scanning frequencies of the MEMS micromirror system are in the range of several kilohertz, in particular up to 50 kHz.The visible part of the user's eye, through which the laser beam is to be scanned, is formed at least predominantly by the pupil of the user's eye, an iris of the user's eye and a sclera of the user's eye. In particular, the retina produces a so-called speckle pattern upon reflection of a laser beam, in particular a sufficiently coherent laser beam. The speckle pattern (also called laser granulation or light granulation) is in particular a grainy-looking interference phenomenon which can be observed with sufficiently coherent illumination of optically rough object surfaces, such as the retina of the eye (cf. cones and rods of the retina). In particular, the individual speckles of the speckle pattern are detected by the photodiode of the LFI sensor. In particular, individual speckles of the speckle pattern that hit the photodiode of the LFI sensor each produce at least one signal deflection of the electrical signal there.In particular, in the recording step, speckles of the speckle pattern generated by the retina through (back)reflection are recorded.

[0010] In particular, the data glasses comprise an electronics unit and / or a computing unit, which is at least intended to receive the electrical signal of the detection step and / or to record at least part of the electrical signal of the detection step, in particular the signal deflections. Preferably, the respective recorded signal deflections are linked by the electronics unit and / or the computing unit to the scan coordinates current at the time of the measurement, preferably stored in a linked manner. Preferably, only the signal deflections are recorded in the pupil detection method, while in particular all other parts of the electrical signal remain disregarded and / or even unrecorded. In particular, the threshold value that determines whether a signal is recorded or not can be specified.Preferably, the electrical signal of an event to be identified as a signal deflection is at least twice as high, advantageously at least three times as high, preferably at least five times as high, and particularly preferably at least ten times as high as the electrical signal immediately before and immediately after the event and / or as an average value or noise of the electrical signal. A "computing unit" should be understood in particular as a unit with an information input, an information processing unit, and an information output. Advantageously, the computing unit has at least one processor, a memory, input and output means, further electrical components, an operating program, control routines, control routines, and / or calculation routines. Preferably, the components of the computing unit are arranged on a common circuit board and / or advantageously arranged in a common housing.It is further proposed that, in at least one contour determination step, at least part of a pupil contour of the pupil of the user's eye is determined from a plurality of initial signal deflections in different scan lines of the laser beam scanned, in particular, by means of the MEMS micromirror system, and / or from a plurality of final signal deflections in different scan lines of the laser beam scanned, in particular, by means of the MEMS micromirror system. This advantageously enables particularly efficient pupil detection, in particular pupil contour recording. Advantageously, the required data volume and / or power consumption for pupil detection can be kept low. Furthermore, this advantageously enables an increase in the spatial (two-dimensional) resolution of the pupil contour, in particular by enabling analog (quasi-direct) detection of the pupil edge.In addition, direct "on-the-fly" detection of the pupil contour and / or pupil center, e.g., through (adaptive) fitting, could be advantageously enabled while the image acquisition / scanning step / scan of the MEMS micromirror system is still in progress. This can advantageously minimize system latency. Advantageously, pupil parameters such as the pupil contour and / or pupil center, etc., can be provided immediately at the end of each scanning process. In particular, LFI sensors are comparatively less susceptible to stray light, so the number of false triggers during recording is low / there are particularly few false triggers during recording. A first signal deflection represents, in particular, the first time the electrical signal from the photodiode of the LFI sensor exceeds the definable threshold value in a scan line or scan column of a scan scheme of the MEMS micromirror system.A final signal deflection represents, in particular, the final exceeding of the definable threshold by the electrical signal of the photodiode of the LFI sensor in a scan line or scan column of a scan scheme of the MEMS micromirror system. In particular, in the scan area overlapping the retina, each scan line comprises an initial signal deflection and a final signal deflection. Between the initial signal deflection and the final signal deflection of a scan line, further signal deflections can be recorded / lie. The remaining signal deflections of a scan line between the initial signal deflection of the scan line and the final signal deflection of the scan line, in particular the further signal deflections of the scan line, can be filtered out and / or discarded.The pupil contour represents, in particular, an outer outline of the pupil in the scan coordinates, in particular in a coordinate system of the data glasses.

[0011] Furthermore, it is proposed that in at least one pupil determination step, a pupil shape and / or a pupil position of the pupil of the user's eye is determined from a merely partial, in particular approximately half, pupil contour or from a completely circumferential pupil contour, for example via an ellipse fitting algorithm (e.g., least squares) of the computing unit and / or the electronics unit. This advantageously enables fast and / or energy-saving pupil determination. If a complete pupil contour and / or pupil shape of the user's eye and in particular their relationship to the coordinate system of the data glasses is known, a gaze vector of the user's eye can be determined using an algorithm known from the prior art, such as a Swirski algorithm. The partial, e.g.A partial pupil contour can be determined, for example, from an upper or lower half, each with at least two recorded signal deflections per scan line. A partial pupil contour, e.g., a half, can be determined, for example, from a lateral half, each with only one signal deflection, in particular, only one initial signal deflection or only one final signal deflection per line.

[0012] Furthermore, it is proposed that the signal deflections recorded in the recording step correspond to (individual) speckles, in particular the speckle pattern, which are generated by (back)scattering of the laser beam irradiated in the scanning step on a retina of the user's eye. This advantageously allows reliable detection of the pupil by exploiting a retinal property. In particular, the speckles, formed as bright light points, generate the signal deflections in the electrical signal of the LFI sensor's photodiode upon re-entry into the LFI sensor.

[0013] In addition, it is proposed that the electrical signal in the recording step be monitored for exceeding a threshold value, in particular by means of a Schmitt trigger, wherein an interrupt is triggered when the threshold value is exceeded, and wherein the interrupt triggers a query of the current scan coordinates, in particular the position of the micromirror system assumed at the time of the interrupt, preferably by means of a digital processor. This advantageously enables a secure and / or reliable assignment of speckles generated by the retina to scan coordinates, in particular in the coordinate system of the data glasses. The scan coordinates are preferably stored together with a binary Schmitt trigger signal in a memory of the data glasses, in particular in the form (α, β, binary variable).Here, α and β are preferably angles / angular positions, in particular instantaneous pivot angles / instantaneous pivot angle positions, of the MEMS micromirror system, in particular of the relevant individual mirror of the MEMS micromirror system. The binary variable preferably represents a binary signal associated with the Schmitt trigger, which can, for example, be 0 if no threshold is exceeded and 1 if a threshold is exceeded, or vice versa.

[0014] If the threshold value is iteratively adjusted in at least one learning step, in particular with each new frame generated by the scanned laser beam, preferably until the number of signal deflections per frame falls below a definable maximum number, e.g., 10, 20, or 30, a particularly well-optimized balance of sensitivity and power consumption can advantageously be achieved. Advantageously, high efficiency can be achieved. Preferably, the threshold values, in particular of the Schmitt trigger, are set digitally via a control signal. In particular, the threshold value, e.g., of the Schmitt trigger, can be set once for a respective user in the learning step, in particular since the threshold value advantageously does not change due to insensitivities of the LFI sensor to ambient light. The iterative adjustment of the threshold value can be automated, e.g.,by means of the computing unit and / or the electronics unit, or at least partially monitored, e.g. by confirmation from the user of the data glasses. Alternatively, to set the threshold value of the Schmitt trigger, the Schmitt trigger can be bypassed during the teach-in step and the complete signal (back-reflection signal) can be recorded for an entire frame (full scan of the MEMS micromirror system), with the highest electrical signals, in particular voltages, (e.g. 5% or 10%) then being used to set the threshold value. The highest electrical signals, in particular voltages, preferably correspond to the brightest speckles / amplitudes of the measured back-reflected part of the scanned laser beam. In particular, the Schmitt trigger at least partially forms the electronics unit and / or the computing unit.Alternatively, the electronics unit and / or the computing unit can also have other programmable threshold detectors for electrical signals, in particular voltage signals, which preferably perform identical tasks as described above for the Schmitt trigger by way of example. As soon as an overwriting of the threshold value is detected, the threshold detector can trigger an interrupt on a connected digital processor of the electronics unit or the computing unit.

[0015] Furthermore, it is proposed that the laser beam be scanned in alternating directions, in particular sinusoidally, at least over the majority of the visible part of the user's eye during the scanning step. This advantageously makes it possible to determine the pupil contour directly from an analog signal without filtering out and / or discarding events (signal deflections). This advantageously makes it possible to further reduce power consumption and / or data rate / data volume. In particular, in this case only the initial signal deflection is required to determine the pupil contour directly from the analog signal. Ellipse fitting algorithms can advantageously be dispensed with. The alternating scanning can begin from the left (“even line”) or from the right (“odd line”).From the "even lines", in particular from the scan lines running from left to right, a left pupil edge of the user's pupil can be determined. From the "odd lines", in particular from the scan lines running from right to left, a right pupil edge of the user's pupil can be determined. For this purpose, the electronics unit, preferably the Schmitt trigger, can comprise an interruption module for an output signal of the Schmitt trigger, which is designed to open an output of the Schmitt trigger after the Schmitt trigger has been triggered for the first time by exceeding the threshold value, so that preferably no further events can be determined for the current scan line. The interruption by the interruption module can then be reset for each new line, e.g. by an h-sync signal from the MEMS micromirror system.A majority should be understood to mean in particular 51%, preferably 66%, advantageously 75%, preferably 85% and particularly preferably 95%.

[0016] If at least one electronics unit, in particular of the electronics unit which is provided for detecting the signal deflections in the electrical signal, for example a Schmitt trigger and / or a photodiode of the LFI sensor, is deactivated for the remainder of the scan line after detecting an initial signal deflection in a scan line of the laser beam scanned in alternating directions, e.g. by the interruption module, and is in particular only reset and / or reactivated upon transition to a subsequent scan line, a particularly high energy efficiency / particularly low power consumption can advantageously be achieved.

[0017] It is further proposed that the electrical signal be a voltage signal, wherein, during the conversion of the portion of the laser beam scattered back into the LFI sensor into the electrical signal in the detection step, a photocurrent of a photodiode of the LFI sensor is converted, in particular by means of a transimpedance amplifier of the electronics unit, into a voltage signal proportional to the photocurrent. This advantageously enables simple and reliable detection of signal deflections. A simple design is advantageously enabled.

[0018] Furthermore, a pupil detection device, in particular for data glasses, with at least one projector unit comprising at least one laser feedback interferometry (LF I) sensor, which is at least configured to scan a laser beam, in particular an infrared laser beam, in particular by means of a MEMS micromirror system of the projector unit, over at least a large part of a visible part of a user's eye, and which is at least configured to at least partially detect a part of the laser beam scattered back by the user's eye with the LFI sensor and, in particular with the aid of the electronics unit, to convert it into an electrical signal, and with the electronics unit, which is at least configured to detect a pupil of the user's eye, signal deflections in the electrical signal, which in particular exceed a definable threshold value,and record them together with the scan coordinates associated with the respective signal deflections, particularly of the MEMS micromirror system, as well as data glasses with the pupil detection device. This can advantageously increase efficiency. Advantageously, the power consumption required for pupil detection can be significantly reduced.

[0019] The pupil detection method according to the invention, the pupil detection device according to the invention, and the data glasses according to the invention are not intended to be limited to the application and embodiment described above. In particular, the pupil detection method according to the invention, the pupil detection device according to the invention, and the data glasses according to the invention may have a number of individual elements, components, units, and method steps that differs from the number stated herein in order to fulfill a functionality described herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0020] drawing

[0021] Further advantages will become apparent from the following description of the drawings. The drawings illustrate exemplary embodiments of the invention. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will also expediently consider the features individually and combine them into useful further combinations.

[0022] They show:

[0023] Fig. 1 is a schematic representation of a part of a data glasses with a pupil detection device for carrying out a pupil detection method, Fig. 2 is a schematic circuit diagram with a projector unit of the pupil detection device and with an electronic unit of the pupil detection device,

[0024] Fig. 3 shows a schematic detailed view of part of the electronic unit,

[0025] Fig. 4 is a schematic flow diagram of the pupil detection method,

[0026] Fig. 5 shows three quantities plotted against a common time: an h-sync signal of a MEMS micromirror system of the pupil detection device (top), a v-sync signal of the MEMS micromirror system (middle) and a binary signal determined from an electrical signal of the electronics unit, which indicates signal deflections (bottom),

[0027] Fig. 6 shows a frame generated by the projector unit with the signal deflections registered in this frame at certain scan coordinate pairs,

[0028] Fig. 7 the same frame as in Fig. 6 with a filtering of the signal deflections,

[0029] Fig. 8 shows an example in which only initial and final signal deflections of a part of all scan lines of a frame were determined,

[0030] Fig. 9 schematically shows a scanning pattern of the MEMS micromirror system in which scan lines are scanned alternately in opposite directions, and

[0031] Fig. 10 is a schematic representation of the scan lines in the frame created with the alternating counter-rotating scan pattern.

[0032] Description of the embodiment

[0033] Figure 1 schematically shows a pair of data glasses 10. The data glasses 10 comprise a virtual retinal display 94 (retina scan display). The data glasses 10 comprise at least one lens 96. The virtual retinal display 94 comprises a combiner 98. The combiner 98 is embodied, for example, as a holographic optical element (HOE). The combiner 98 is intended to introduce an image generated by a projector unit 76 of the data glasses 10 into a field of view of a user of the data glasses 10. In the illustrated case, the combiner 98 is integrated into the lens 96.

[0034] The data glasses 10 are provided for carrying out a pupil detection method. The data glasses 10 comprise a pupil detection device 74. The pupil detection device 74 has the projector unit 76. The data glasses 10 comprise a glasses frame 100. The projector unit 76 is at least partially integrated into the glasses frame 100. The projector unit 76 comprises at least one laser feedback interferometry (LFI) sensor 14. The LFI sensor 14 is provided for generating and outputting a laser beam 12. The laser beam 12 is coherent. The laser beam 12 is an infrared laser beam. The laser beam 12 is provided at least for pupil detection of a pupil 22 of a user eye 18 of the user of the data glasses 10. The LFI sensor 14 comprises at least one infrared laser diode 80 for generating the laser beam 12.The LFI sensor 14 comprises at least one photodiode 70 for detecting and / or capturing at least infrared light, preferably at least a portion of a portion 20 of the laser beam 12 scattered back by the user's eye 18. The LFI sensor 14 is embodied, for example, as part of a laser module 82 of the projector unit 76, which additionally has further laser diodes 84, 86, 88 for generating and outputting visible laser light. The further laser diodes 84, 86, 88 form an RG B laser system. The further laser diodes 84, 86, 88 are each provided for outputting visible red laser beams, visible green laser beams, or visible blue laser beams. The further laser diodes 84, 86, 88 are provided for generating / projecting an image, e.g., of an augmented reality object, into the user's field of vision.

[0035] The projector unit 76 is provided for scanning the laser beam 12 over at least a large part of a visible part of the user's eye 18. The projector unit 76 has a MEMS micromirror system 16. The MEMS micromirror system 16 comprises at least one movable, preferably pivotable, individual mirror 90. Preferably, the MEMS micromirror system 16 comprises at least two individual mirrors 90, 92 that can be pivoted orthogonally to one another. The portion 20 of the laser beam 12 scattered back into the LFI sensor 14 generates a photocurrent in the photodiode 70. The pupil detection device 74, in particular the projector unit 76, has an electronics unit 78. The pupil detection device 74, in particular the projector unit 76, is provided, in particular with the aid of the electronics unit 78, to convert the detected backscattered portion 20 of the laser beam 12 into an electrical signal. The electrical signal is a voltage signal.The pupil detection device 74, in particular the projector unit 76, is provided, in particular with the aid of the electronics unit 78, preferably with the aid of a transimpedance amplifier 72 (cf. Fig. 2) of the electronics unit 78, to convert the photocurrent into the electrical signal. The electrical signal, embodied as a voltage signal, is proportional to the strength of the generated photocurrent. The electronics unit 78 is configured at least to detect signal deflections 24, 50, 52 (cf., among others, Fig. 6) in the electrical signal for detecting the pupil 22 of the user's eye 18 and to record them together with scan coordinates α, β (cf., among others, Fig. 6) of the MEMS micromirror system 16 associated with the respective signal deflections 24, 50, 52. The electronic unit 78 is designed to record only the signal deflections 24, 50, 52 that exceed a definable threshold value.

[0036] Fig. 2 shows a schematic circuit diagram of the projector unit 76 and the electronics unit 78. The LFI sensor 14 with the integrated photodiode 70 emits the infrared laser beam 12 with a known wavelength in the direction of the user's eye 18. This laser beam 12 is deflected two-dimensionally by the MEMS micromirror system 16. Alternative deflection units to a MEMS micromirror system 16 for deflecting the laser beam 12 are also conceivable. The deflected laser beam 12 reaches the user's eye 18 and is backscattered by the user's eye 18. A portion of the backscattered laser beam 12 is backscattered by a retina 56 of the user's eye 18 and, due to a surface condition of the retina 56, generates a speckle pattern. The backscattered light of the laser beam 12, which enters the LFI sensor 14, creates an interference with a locally oscillating field within the LFI sensor 14.Positive interference results in an increase in the laser's optical power, which is detected as a photocurrent by the integrated photodiode 70 of the LFI sensor 14. The transimpedance amplifier 72 amplifies the photocurrent signal and converts it into the corresponding proportional voltage signal. This measurement method is sensitive only to the emitted light itself and can therefore be performed independently of optical filters (e.g., laser line filters) or electronic filters (e.g., bandpass filters).

[0037] The electronics unit 78 has a Schmitt trigger 58. The Schmitt trigger 58 is provided for monitoring analog signals, e.g., the electronic signal embodied as a voltage signal, for whether a threshold value has been exceeded. Alternatively, the electronics unit 78 could also have another programmable threshold detector. The voltage signal generated by the transimpedance amplifier 72 is fed to the Schmitt trigger 58. The electronics unit 78 has a digital processor 60 or is connected to a digital processor 60 of a computing unit of the data glasses 10. If the threshold value is detected to be exceeded, an interrupt is triggered at the digital processor 60. The digital processor 60 is connected to the MEMS micromirror system 16. When the interrupt occurs, the digital processor 60 queries the current position of the individual mirror(s) 90, 92 of the MEMS micromirror system 16.The electronics unit 78 has a memory unit 102 or is connected to a memory unit 102 of the computing unit of the data glasses 10. The digital processor 60 is provided to store the scan coordinates a, ß obtained by the query triggered by the interrupt in the memory unit 102 as a detected event / as a speckle.

[0038] Fig. 3 shows a detailed view of part of the electronics unit 78. The electronics unit 78 has an interrupt switch 104. The interrupt switch 104 is arranged at an input of the Schmitt trigger 58. The interrupt switch 104 is closed at the beginning of each scan line 42, 44, 46 (see Fig. 10) and is opened, e.g. triggered by the digital processor 60, after the threshold value is exceeded for the first time, so that no further events for this scan line 42, 44, 46 can be detected. The interrupt switch 104 is then reset, i.e. closed, after the end of the current scan line 42, 44, 46 and before the start of the next scan line 42, 44, 46. This can occur, for example, by receiving an h-sync signal 62 (see Fig. 5) from the MEMS micromirror system 16.

[0039] Figure 4 shows a schematic flow diagram of the pupil detection method. In at least one method step 68, the laser beam 12 is generated by the LFI sensor 14. In at least one scanning step 30, the laser beam 12 of the LFI sensor 14 is scanned by the MEMS micromirror system 16 across at least a large portion of the visible portion of the user's eye 18. In the scanning step 30, the laser beam 12 can always be scanned in the same direction (always from right to left or always from left to right). However, the laser beam 12 can also be scanned in alternating directions 64, 66, e.g., sinusoidally, in the scanning step 30 (see Fig. 9). In at least one method step 106, a portion 20 of the laser beam 12 is backscattered by the user's eye 18. Different regions of the user's eye 18 generate different backscattering patterns.The retina 56 of the user's eye 18 generates a speckle pattern upon the back reflection of the coherent laser beam 12. In at least one sub-step 108 of a detection step 32, at least a portion of the portion 20 of the laser beam 12 backscattered by the user's eye 18 is detected by the photodiode 70 of the LFI sensor 14. In at least one sub-step 110 of the detection step 32, a photocurrent signal from the photodiode 70 is converted into the electrical signal. In at least one speckle detection step 112, speckles of the speckle pattern generated by the retina 56 are detected. The speckles each generate signal peaks 24, 50, 52 in the electrical signal. By monitoring a definable threshold, the signal deflections 24, 50, 52 are separated from the rest of the signal of the photodiode 70 and identified as speckles. The threshold is set in a learning step 40, which is particularly performed initially.This can be done manually or automatically. During the automated execution of the learning step 40, the threshold value is iteratively adjusted with each new frame 28 generated by the scanned laser beam 12 (see, among others, Fig. 6). This continues until the number of signal deflections 24, 50, 52 per frame 28 falls below a definable maximum number, e.g., 10, 20, or 30.

[0040] In at least one recording step 34, the detected signal excursions 24, 50, 52 in the electrical signal that exceed the threshold value are recorded together with the scan coordinates α, β of the MEMS micromirror system 16 associated with the respective signal excursions 24, 50, 52. The signal excursions 24, 50, 52 recorded in the recording step 34 thus correspond to speckles of the speckle pattern generated by the backscattering of the laser beam 12 irradiated in the scanning step 30 from the retina 56 of the user's eye 18. The electrical signal is monitored in the recording step 34 for whether the threshold value has been exceeded using the Schmitt trigger 58. If the threshold value is exceeded, the interrupt is triggered by the Schmitt trigger 58.The triggering of the interrupt in turn triggers a query of the current scan coordinates a, ß of the position of the micromirror system 16 taken at the time of the interrupt by means of the digital processor 60.

[0041] In at least one contour determination step 36, at least part of a pupil contour 48, 54 of the pupil 22 of the user's eye 18 is determined from the signal deflections 24, 50, 52. In this case, only a partial, e.g., half, pupil contour 54 (cf. Fig. 8) or a completely circumferential pupil contour 48 (cf. Fig. 6) can be determined from the signal deflections 24, 50, 52. The partial pupil contour 54 can be determined from a plurality of initial signal deflections 50 in different scan lines 42, 44 of the scanned laser beam 12. The partial pupil contour 54 can be determined from a plurality of final signal deflections 52 in different scan lines 42, 46 of the scanned laser beam 12. The completely circumferential pupil contour 54 can be determined from the respective initial signal deflections 50 and the respective final signal deflections 52 in different scan lines 42, 46 of the scanned laser beam 12.In at least one pupil determination step 38, a pupil shape 124 and / or a pupil position of the pupil 22 of the user's eye 18 is then determined from the only partial, in particular approximately half, pupil contour 54 or from the completely circumferential pupil contour 48. However, it is also conceivable that the completely circumferential pupil contour 48 is completely determined only by initial signal deflections 50. For this purpose, each successive scan line 42, 44, 46 is scanned in a different direction. To limit the recorded and stored signal deflections 24, 50, 52 to only the initial signal deflections 50, an electronic system (see Fig.3), by means of which the Schmitt trigger 58 is deactivated for the remainder of the scan line 42, 44, 46 after detecting an initial signal deflection 50 in a scan line 42, 44, 46 of the scanned laser beam 12, and is only reset and reactivated upon transition to a subsequent scan line 42, 44, 46. The described pupil detection method can also be used for a stereo approach, in particular by arranging multiple pupils 22 in the scan area of ​​the LFI sensor 14 or when using stereo holograms as combiners 98.

[0042] Fig. 5 shows three quantities plotted against a common time 116: the h-sync signal 62 of the MEMS micromirror system 16 (top), a v-sync signal 114 of the MEMS micromirror system 16 (middle), and a binary signal 26 determined from the electrical signal (bottom). The h-sync signal 62 describes the reaching of a maximum / minimum horizontal scan range by the MEMS micromirror system 16. A signal deflection 118 of the h-sync signal 62 thus marks the beginning of a new scan line 42, 44, 46 in a frame 28. The v-sync signal 114 describes the reaching of a last scan line of a frame 28. The v-sync signal 114 describes the maximum deflection angle of the MEMS micromirror system 16 in the vertical direction. A signal deflection 120 of the v-sync signal 114 thus marks the beginning of a new frame 28. The binary signal 26 describes the determined and stored signal deflections 24, 50, 52 of the electrical signal.If the electrical signal is below the threshold, the binary signal 26 assumes the value 0. If the electrical signal exceeds the threshold, the binary signal 26 assumes the value 1. The binary signal 26 thus indicates the triggering of the Schmitt trigger 58 by speckles of the speckle pattern and thus the times at which the pupil 22 of the user's eye 18 was hit by the laser beam 12.

[0043] Fig. 6 shows a frame 28 generated by the projector unit 76 with the scan coordinates a, ß and the signal deflections 24, 50, 52 registered in this frame 28 at certain scan coordinate pairs. From the h-sync signals 62 and the v-sync signals 114 or from the times of occurrence of the signal deflections 24, 50, 52 with respect to the instantaneous h-sync and v-sync signals 62, 114, a current scan position of the MEMS micromirror system 16 in the scan coordinates a, ß is determined and displayed two-dimensionally in the frame 28 in particular. Fig. 7 shows the same measurement / frame 28 as Fig. 6, but all signal excursions 24 which are not initial signal excursions 50 of a scan line 42, 44, 46 or final signal excursions 52 of a scan line 42, 44, 46 have been filtered out and / or discarded.If the scan of the MEMS micromirror system 16 is performed line by line, the signal deflections 24, 50, 52 within a scan line 42, 44, 46 can be filtered so that, as in the illustrated case, only the initial signal deflections 50 and the final signal deflections 52 per scan line 42, 44, 46 are retained. All signal deflections 24 in between can be easily discarded. Based on the remaining signal deflections 50, 52, the completely circumferential pupil contour 54 can then be determined. Fig. 8 shows an example in which only the initial and final signal deflections 50, 52 of a portion of all scan lines 42, 44, 46 of a frame 28 were determined. A partial pupil contour 48 is then determined from these signal deflections 50, 52. An ellipse fitting algorithm is then used to determine a center 122 of the pupil 22 in the scan coordinates a, ß and / or a pupil shape 124 in the scan coordinates a, ß.

[0044] Fig. 9 schematically shows a scan pattern of the MEMS micromirror system 16, in which the scan lines 42, 44, 46 are scanned alternately in opposite directions 64, 66. The scan pattern shown in Fig. 9 is sinusoidal. However, alternative scan patterns, e.g., rectangular scan patterns, are also conceivable. Fig. 10 schematically shows the scan lines 42, 44, 46 of a frame 28 created with such an alternating, opposite scan pattern. In this case, it is sufficient to merely record initial signal deflections 50 in order to be able to determine the completely circumferential pupil contour 48. A left edge of the pupil 22 is determined from scan lines 42 that run from left to right ("even lines"). From scan lines 44 running from right to left (“odd lines”), a right edge of the pupil 22 is determined.

Claims

Claims 1 . Pupil detection method, in particular for data glasses (10), with at least one scanning step (30), in which a laser beam (12), in particular an infrared laser beam, of at least one laser feedback interferometry (LFI) sensor (14), in particular by means of a MEMS micromirror system (16), is scanned over at least a large part of a visible part of a user's eye (18), and with at least one detection step (32), in which a part (20) of the laser beam (12) scattered back by the user's eye (18) is at least partially detected by the LFI sensor (14) and converted into an electrical signal, characterized in that in at least one recording step (34) for detecting a pupil (22) of the user's eye (18), signal deflections (24, 50, 52) in the electrical signal, which in particular exceed a definable threshold value, are recorded together with the respective signal Scan coordinates (a, ß) corresponding to deflections (24, 50, 52),in particular the MEMS micromirror system (16).

2. Pupil detection method according to claim 1, characterized in that in at least one contour determination step (36) at least a part of a pupil contour (48, 54) of the pupil (22) of the user's eye (18) is determined from a plurality of initial signal deflections (50) in different scan lines (42, 44) of the scanned laser beam (12) and / or from a plurality of final signal deflections (52) in different scan lines (42, 46) of the scanned laser beam (12).

3. Pupil detection method according to claim 2, characterized in that in at least one pupil determination step (38) a pupil shape (124) and / or a pupil position of the pupil (22) of the user eye (18) is determined from a merely partial, in particular approximately half, pupil contour (54) or from a completely circumferential pupil contour (48) becomes.

4. Pupil detection method according to one of the preceding claims, characterized in that the signal deflections (24, 50, 52) recorded in the recording step (34) correspond to speckles which are generated by a (back)scattering of the laser beam (12) irradiated in the scanning step (30) on a retina (56) of the user's eye (18).

5. Pupil detection method according to one of the preceding claims, characterized in that the electrical signal in the recording step (34) is monitored for exceeding a threshold value, in particular by means of a Schmitt trigger (58), wherein an interrupt is triggered when the threshold value is exceeded, and wherein the interrupt triggers a query of the current scan coordinates (α, β), in particular of the position of the micromirror system (16) assumed at the time of the interrupt, preferably by means of a digital processor (60).

6. Pupil detection method according to claim 5, characterized in that in at least one learning step (40) the threshold value is iteratively adjusted, in particular with each new frame (28) generated by the scanned laser beam (12), preferably until a number of signal deflections (24, 50, 52) per frame (28) reaches a definable maximum number, e.g. 10, falls below.

7. Pupil detection method according to one of the preceding claims, characterized in that the laser beam (12) is scanned in the scanning step (30) in alternating directions (64, 66), in particular sinusoidally, at least over the majority of the visible part of the user's eye (18).

8. Pupil detection method according to claim 7, characterized in that at least one electronic unit which is provided for detecting the signal deflections (24, 50, 52) in the electrical signal, for example a Schmitt trigger (58) and / or a photodiode (70) of the LFI sensor (14), is deactivated for the remainder of the scan line (42, 44, 46) after detecting a first signal deflection (50) in a scan line (42, 44, 46) of the laser beam (12) scanned in alternating directions (64, 66) and is in particular only reset and / or reactivated upon transition to a subsequent scan line (42, 44, 46).

9. Pupil detection method according to one of the preceding claims, characterized in that the electrical signal is a voltage signal, wherein during the conversion of the part (20) of the laser beam (12) scattered back into the LFI sensor (14) into the electrical signal in the detection step (32), a photocurrent of a photodiode (70) of the LFI sensor (14) is converted, in particular by means of a transimpedance amplifier (72), into a voltage signal proportional to the photocurrent.

10. Pupil detection device (74), in particular for data glasses (10), preferably for carrying out a method according to one of the preceding claims, with at least one projector unit (76) comprising at least one laser feedback interferometry (LFI) sensor (14), which is at least configured to scan a laser beam (12), in particular an infrared laser beam, in particular by means of a MEMS micromirror system (16) of the projector unit (76), over at least a large part of a visible part of a user's eye (18), and which is at least configured to at least partially detect a part (20) of the laser beam (12) scattered back by the user's eye (18) with the LFI sensor (14) and to convert it into an electrical signal, characterized by an electronic unit (78) which is at least configured to generate signal deflections (24, 50,52) in the electrical signal, which in particular exceed a definable threshold value, and to record them together with scan coordinates (α, β) associated with the respective signal deflections (24, 50, 52), in particular of the MEMS micromirror system (16).

11. Data glasses (10) which are configured to carry out a pupil detection method according to one of claims 1 to 9 and / or which have a pupil detection device (74) according to claim 10.

Images