Oculography method, oculography device, ar or VR data glasses comprising the oculography device, and control unit
The proposed method for AR and VR data glasses dynamically selects the best LFI sensor configuration for eye movement tracking based on parameter comparisons, addressing anatomical variations and enhancing user-friendliness and tracking precision while ensuring eye safety.
Patent Information
- Application Number
- PCT/EP2024/056897
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-19
- Filing Date
- 2024-03-15
- Publication Date
- 2024-11-28
AI Technical Summary
Existing AR and VR data glasses oculography methods using laser feedback interferometry (LFI) sensors face challenges in accommodating varying user anatomies and head geometries, leading to inconsistent laser beam alignment on the eye, which affects eye movement tracking accuracy and user-friendliness.
The method involves a selection step before oculography where multiple LFI sensors are tested to determine the best configuration for eye movement tracking, allowing for a dynamic adjustment of sensor selection based on parameter comparisons, ensuring optimal eyebox enlargement and maintaining measurement capability even with changes in sensor position relative to the eye.
This approach enhances user-friendliness and tracking precision by optimizing eye and pupil detection, maintaining measurement accuracy across different user anatomies and head positions, and ensuring eye safety through controlled laser intensity.
Smart Images

Figure EP2024056897_28112024_PF_FP_ABST
Abstract
Description
[0001] Description
[0002] AR or VR data glasses with
[0003] State of the art
[0004] An oculography method, in particular a data glasses oculography method, with at least one oculography step in which at least one eye movement of an eye, in particular of a data glasses user's eye, is determined, in particular tracked, by a subset of laser feedback interferometry (LFI) sensors of an LFI sensor array assigned to the eye, has already been proposed. For practical operation of data glasses with a corresponding oculography method, it is essential that the laser beams of the LFI sensor actually hit the user's eye, namely at an angle that is not perpendicular to the surface velocity of the eyeball. Different people have fundamentally different physiologies. Due to individually different interocular distances, individually different head and nose geometries, or simply due to the data glasses slipping, the point of impact of the laser beams on the eye can vary.Therefore, it will usually not be possible to find a single laser arrangement that is equally suitable for all users.
[0005] Disclosure of the invention
[0006] The invention is based on an oculography method, in particular a data glasses oculography method, e.g. for augmented reality (AR) data glasses and / or for virtual reality (VR) data glasses, with at least one oculography step in which at least one eye movement of an eye, in particular of a data glasses user's eye, is determined, in particular tracked, by a subset of laser feedback interferometry (LFI) sensors of an LFI sensor array assigned to the eye, preferably by all LFI sensors of the LFI sensor array assigned to the eye, in particular a data glasses.
[0007] It is proposed that in at least one selection step, in particular one which precedes the oculography step in time, a selection of at least one LFI sensor provided for carrying out the oculography step, in particular the oculography step which follows it in time, is made based on a parameter comparison of test measurements of several, preferably all, LFI sensors of the LFI sensor array. This can advantageously increase the user-friendliness and / or usability of data glasses per se. Advantageously, an eyebox of data glasses can be significantly enlarged, whereby the system advantageously remains capable of measuring even if a position of the data glasses, in particular of the LFI sensors of the data glasses, changes relative to the user's eye. In particular, an oculography method for displaying, detecting and / or recording eye movements is provided.Preferably, the oculography method is applicable to different types and styles of data glasses, for example, see-through AR data glasses and / or pure VR data glasses.
[0008] “Data glasses” should be understood in particular to mean a wearable (head-mounted display) by means of which information can be added to a user’s field of vision. Data glasses preferably enable augmented reality, virtual reality and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses or VR glasses or AR glasses. In particular, the optical display system is provided for generating and outputting virtual content or video content using a laser source, in particular one different from the LFI sensor array. The laser source could, for example, be arranged in a side region (e.g., temple area) of the data glasses. The data glasses preferably comprise at least imaging optics, such as a lens, in particular a pancake lens of a VR headset or a spectacle lens of an AR headset.In particular, the LFI sensor array forms a, particularly integral, component of an eye-tracking system of the data glasses. The eye-tracking system could be provided for detecting the eye and / or pupil position of the user's eye using the so-called "dark pupil effect." Preferably, however, the eye-tracking system is provided for detecting the eye and / or pupil position using the so-called "bright pupil effect," which is based in particular on the high infrared reflectivity of the eye's retina. "Provided" and / or "configured" should be understood in particular as specifically programmed, designed, and / or equipped. The fact that an object is provided and / or configured for a specific function should be understood in particular to mean that the object fulfills and / or executes this specific function in at least one application and / or operating state.
[0009] In particular, the LFI sensor array comprises a plurality, preferably more than two, advantageously more than three, particularly advantageously more than four, preferably more than five, and particularly preferably fewer than 10 individual LFI sensors, in particular per eye of the user. In particular, the LFI sensors of the LFI sensor array are arranged statically at least relative to the data glasses, in particular at least relative to one lens of the data glasses. The LFI sensors can be arranged, for example, in a frame of the data glasses that at least partially surrounds the lens, or in one of the temples of the data glasses. The LFI sensors preferably emit their laser beams at different angles onto the eye / face of the user and / or onto different sub-regions of the eye / face of the user.The test measurements of the various LFI sensors of the LFI sensor array can be performed sequentially or at least partially simultaneously (e.g., in groups at the same time). However, in partially simultaneous measurements with multiple LFI sensors, it is preferable to activate only as many LFI sensors simultaneously as necessary to ensure eye safety and protect the user's eyes. The total laser intensity emitted simultaneously by the LFI sensor array thus always remains below a definable limit, in particular always below a danger threshold.
[0010] To perform the parameter comparison, individual measurements are first taken from several, preferably all, LFI sensors of the LFI sensor array in a measurement step. These individual measurements form, in particular, the test measurements. These individual measurements are then preferably compared (parameter comparison), and one or more of the LFI sensors are selected for the subsequent oculography based on predeterminable criteria, in particular those readable from the individual measurements. In particular, the method for determining an eye and / or pupil position based on measurement data from LFI sensors is known to those skilled in the art.
[0011] The LFI sensors can be designed, for example, as VCSELs, preferably ViP-VCSELs (“vertical-cavity surface-emitting laser with integrated photo diode”). The LFI sensors of the AR data glasses can be integrated either in the frame, the lens, or the temples. The LFI sensors of a VR data glasses / VR headset can, for example, be integrated into an imaging optic, e.g., in the edge area of the pancake lens. Furthermore, the LFI sensors in the VR data glasses / VR headset could be arranged next to an image-generating display or, in the case of a partially transparent display, such as an OLED display, even behind the image-generating display. LFI sensors are based on an interferometric measuring method and are preferably capable of detecting a distance to a target (e.g., the eye of the data glasses user) as well as a surface speed of the target.In particular, the LFI sensor emits the laser beam in the infrared spectrum, which then strikes a surface with a reflectivity R at an angle y. The light of the laser beam is then backscattered from this surface, so that it re-enters a laser cavity of the LFI sensor. In the laser cavity of the LFI sensor, the backscattered light interferes with a locally oscillating field of the LFI sensor. This leads, in particular, to a modulation of the laser power of the laser source, which can be detected either by a photodiode integrated into a back reflector of the laser cavity or by a direct measurement of a voltage of the laser source. Furthermore, a current of the laser source can be modulated with a preferably triangular modulation signal in order to achieve a cyclic shift of the laser wavelength. If the laser parameters are known, a beat frequency and a Doppler frequency can then be determined.From this, the surface speed of the eye, the distance of the LFI sensor to the eye and other eye parameters can be determined using known equations.
[0012] It is further proposed that, in the selection step, the selection of the at least one LFI sensor of the LFI sensor array intended for carrying out the oculography step is carried out based on a parameter comparison of speed test measurements, in particular eye movement speed test measurements, with the plurality of LFI sensors of the LFI sensor array. This advantageously makes it possible to achieve a high oculography sensitivity. This advantageously makes it possible to optimize, in particular make more precise, eye tracking. In particular, the LFI sensors do not yet directly measure the eye movement speed in the test measurements, but rather a "raw speed" from which the actual eye movement speed could then be determined via further method steps, e.g. for a method for determining an eye and / or pupil position.
[0013] If at least the LFI sensor with the highest speed determined in the selection step is selected for performing the oculography step, a high oculography sensitivity can advantageously be achieved. This can advantageously optimize, and in particular, make eye tracking more precise. It is also conceivable that a group of LFI sensors from the LFI sensor array with the highest speeds determined in the selection step are selected for performing the oculography step.
[0014] It is also proposed that the selection step be repeated when a time interval, which can in particular be specified, e.g. several seconds or several minutes, has elapsed, when a minimum measurement data quality is determined to have been undershot when carrying out the oculography step and / or when a significant change in an optical path length between the LFI sensor and the scattering surface, e.g. the eye of the data glasses user, determined by the currently selected LFI sensor is detected. This can advantageously increase the user-friendliness and / or usability of data glasses themselves. Advantageously, optimized eye and / or pupil tracking can be achieved. In particular, after repeating the selection step, the previously made selection of LFI sensors of the LFI sensor array for the oculography step is confirmed or adjusted / changed. The minimum measurement data quality can be measured, for example, using a signal-to-noise ratio.For example, a failure to meet the minimum measurement data quality requirements during the oculography step is detected if the amplitude of a beat frequency in an amplitude-frequency spectrum of the currently selected LFI sensor falls below a definable limit value compared to a background intensity. The optical path length between the LFI sensor and the scattering surface / eye can be determined by frequency-modulating the currently selected LFI sensor. The corresponding method for this is known to those skilled in the art. By determining the optical path length, LFI sensors could also be excluded from the current selection in the selection step, e.g. due to the thickness of an eyelid, which allows a distinction to be made regarding the point of impact of the laser beam on the eye or on the eyelid.In addition, a significant abrupt substantial increase in the optical path length indicates that a pupil of the eye was hit, which could indicate a good momentary alignment of this sensor to the eye.
[0015] Furthermore, an oculography device for carrying out the oculography method is proposed, comprising at least one, in particular statically arranged, LFI sensor array, comprising a plurality of LFI sensors, each configured to detect, in particular track, at least one eye movement of an eye, in particular the eye of a smart glasses user, wherein at least a majority of all LFI sensors of the LFI sensor array, in particular each LFI sensor of the LFI sensor array, each have (relative to one another) unique radiation angles and / or unique radiation directions. This can advantageously increase the user-friendliness and / or usability of smart glasses per se. Advantageously, an eyebox of smart glasses can be significantly enlarged, whereby the system advantageously remains capable of measuring even if a position of the smart glasses, in particular of the LFI sensors of the smart glasses, changes relative to the user's eye.In particular, the LFI sensor array is designed such that at least a large portion of the LFI sensors of the LFI sensor array, preferably all LFI sensors of the LFI sensor array, lie in a common plane. In particular, the LFI sensors of the LFI sensor array are arranged statically / immovably relative to one another. The static arrangement of the LFI sensor array should be understood in particular to mean that the LFI sensor array is immovably arranged relative to other components of the data glasses (e.g., lens, frame, temples, apart from any possible folding movement of the temples). A "large portion" should be understood in particular to mean 60%, preferably 75%, and more preferably 90%.The fact that an LFI sensor of the LFI sensor array has a unique radiation angle / direction is to be understood in particular as meaning that the LFI sensor array, preferably the data glasses, is free of other LFI sensors with an identical radiation angle / direction. In particular, the unique radiation angle / direction occurs only once in the LFI sensor array, preferably in the data glasses.
[0016] It is further proposed that the oculography device comprise a common lens that is simultaneously assigned to several LFI sensors of the LFI sensor array, preferably simultaneously to all LFI sensors of the LFI sensor array, wherein at least a majority of the LFI sensors assigned to the common lens, preferably each LFI sensor assigned to the common lens, have a different distance and / or a different distance direction from an optical axis of the common lens. This advantageously makes it possible, in a simple and / or cost-effective manner, to ensure that the LFI sensors of the LFI sensor array have unique beam angles. This advantageously improves oculography.In particular, in this configuration, light from at least a majority of the LFI sensors associated with the common lens, preferably from each LFI sensor associated with the common lens, exits the common lens at a different angle.
[0017] Alternatively, it is proposed that the oculography device has a common (micro) lens array with a plurality of (micro) lenses, wherein the individual (micro) lenses of the (micro) lens array are each assigned to one of the plurality of LFI sensors of the LFI sensor array, and wherein in particular at least a majority of the (micro) lenses, preferably all (micro) lenses, have different radiation angles from one another. This advantageously makes it possible to achieve, in a simple, cost-effective and / or particularly compact manner, that the LFI sensors of the LFI sensor array have unique radiation angles. This advantageously makes it possible to improve oculography. In particular, the (micro) lenses of the (micro) lens array lie in a common plane, which is preferably parallel to the plane in which the LFI sensors lie.In particular, a micro-lens has a diameter (perpendicular to the intended beam direction of the lens) of less than 1 mm, preferably less than 0.5 mm, and more preferably less than 0.1 mm. In particular, exactly one (micro-)lens of the (micro-)lens array is assigned to exactly one LFI sensor of the LFI sensor array. The (micro-)lens array can be formed separately from the LFI sensor array or as a chip-integrated (micro-)lens array integrated with the LFI sensor array.
[0018] If the LFI sensors of the LFI sensor array feature chip-integrated optics, especially chip-integrated lenses, a particularly compact design can be achieved. Furthermore, the number of parts to be assembled during the manufacture of the data glasses can be kept to a minimum. The microlenses, in particular, can be directly integrated with the corresponding LFI sensors.
[0019] If the chip-integrated optics are also formed by a metal lens, a particularly space- and / or weight-saving design of the oculography device can be advantageously achieved. In particular, the optical function of the metal lens is based on a nanostructuring of a surface.
[0020] Furthermore, it is proposed that at least two of the LFI sensors of the LFI sensor array, preferably at least three of the LFI sensors of the LFI sensor array, preferably all LFI sensors of the LFI sensor array, be formed by a common multi-cavity laser chip. This can advantageously reduce production costs, in particular by requiring only a single laser chip comprising multiple LFI sensors to be positioned relative to an optical system of the data glasses, e.g., relative to the lens or the (micro) lens array. Furthermore, the overall size can advantageously be reduced. It is also conceivable for the LFI sensor array to comprise multiple multi-cavity laser chips, each with multiple LFI sensors.
[0021] Furthermore, it is proposed that at least two of the LFI sensors of the LFI sensor array, preferably at least three of the LFI sensors of the LFI sensor array, preferably all LFI sensors of the LFI sensor array, are configured to emit light of different polarization, and that the oculography device has polarization-dependent optics which are assigned to the LFI sensors emitting differently polarized light. This advantageously makes it possible to achieve, in a simple, cost-effective and / or particularly compact manner, that the LFI sensors of the LFI sensor array have unique radiation angles. In particular, at least in this case, the LFI sensors of the LFI sensor array are configured to generate and output differently polarized, preferably differently linearly polarized, light.For example, several linearly polarized VCSELs could be arranged in the LFI sensor array, correspondingly rotated relative to one another. It is also conceivable to use VCSELs that each have two closely spaced laser cavities with mutually orthogonal polarization. Chips with more than two laser cavities are also conceivable, in which, for example, the polarization directions are alternately distributed similar to a checkerboard pattern. The polarization-dependent optics preferably generate different emission angles depending on the polarization of the light passing through the polarization-dependent optics. The polarization-dependent optics can be designed as a birefringent crystal, a prism, a polarization-dependent metamaterial, or polarization-dependent liquid crystals.
[0022] Furthermore, data glasses, in particular AR data glasses and / or VR data glasses / VR headsets, are proposed with the oculography device, wherein the data glasses have a hologram unit integrated into a lens of the data glasses, which hologram unit has a plurality of reflection holograms, each with different optical functions, which are each configured to differently deflect light from different LFI sensors of the LFI sensor array at different angles of incidence. This enables advantageous embodiments of data glasses having the oculography device, for example with regard to construction and / or design. The light from the LFI sensor array incident from the different angles of incidence is preferably deflected by the different reflection holograms in the direction of the data glasses user's eye / an eye positioning region of the data glasses.The reflection holograms can be arranged in the edge areas of the lenses. This advantageously ensures good visibility through the lenses. In another variant, a material forming the reflection hologram is fully integrated into the lens, but only in the edge areas of the lens is provided with an optical function that causes reflection of the laser beams from the LFI sensors. To prevent the perception of the boundaries of these active areas, the holographic function can be implemented in transition areas according to a grayscale gradient, so that no discrete edge is perceptible.
[0023] The hologram unit comprises one or more holographic optical elements (HOEs). The HOEs of the hologram unit can be formed jointly or separately. The reflection holograms of the hologram unit are formed by volume holograms. The volume hologram can be implemented as a continuous optical function or as a piecemeal discrete optical function, for example, as four quadrants that redirect incident laser beams in different directions toward the eye. The reflection holograms of the hologram unit are specifically tuned to the wavelength(s) of the LFI sensors. The reflection holograms of the hologram unit specifically reflect, in a wavelength-selective manner, only light of the wavelength(s) of the LFI sensors. Light of other wavelengths preferably remains unaffected by the reflection holograms of the hologram unit and passes through them essentially unhindered.
[0024] This installation position can be particularly advantageous if the LFI sensor array is designed in such a way that at least one LFI sensor radiates at a wide angle in different directions.
[0025] As a result, the laser beam from this LFI sensor preferentially hits very different positions of the hologram unit, especially the reflection holograms, in the lens. Different optical functions are then imprinted at the different positions of the hologram unit, each deflecting this laser beam toward the eye. This makes it possible to irradiate the eye with a laser beam from different directions, so that (in contrast to a fixed installation position of this LFI sensor, e.g., in the lower area of the eyeglass frame) multiple, ideally linearly independent, projection directions of a surface velocity vector of the eye can be generated.
[0026] Furthermore, a control unit for the oculography device and / or for the data glasses, which is configured to carry out the oculography method, is proposed. This can advantageously increase the user-friendliness and / or usability of data glasses per se. A “control unit” should be understood in particular as a unit with at least one control electronics unit. A “control electronics unit” should be understood in particular as a unit with a processor and with a digital memory as well as with an operating program stored in the digital memory. In particular, the control unit comprises the operating program with a program code which includes instructions which, when executed by the processor, cause the control unit to carry out the oculography method.
[0027] The oculography method according to the invention, the oculography device according to the invention, the data glasses according to the invention, and the control unit according to the invention are not intended to be limited to the application and embodiment described above. In particular, the oculography method according to the invention, the oculography device according to the invention, the data glasses according to the invention, and the control unit according to the invention may, in order to fulfill a functionality described herein, have a number of individual elements, components, units, and method steps that differs from the number stated herein. Furthermore, in the value ranges specified in this disclosure, values within the stated limits are also to be considered disclosed and can be used arbitrarily. Drawing
[0028] Further advantages will become apparent from the following description of the drawings. The drawings illustrate seven exemplary embodiments of the invention. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will also expediently consider the features individually and combine them into useful further combinations.
[0029] They show:
[0030] Fig. 1 schematically shows an exemplary data glasses with an oculography device having an LFI sensor array,
[0031] Fig. 2 schematically shows an exemplary partial array of the LFI sensor array with two LFI sensors,
[0032] Fig. 3 schematically shows a top view of an exemplary embodiment of the sub-array / the LFI sensor array,
[0033] Fig. 4 schematically shows an eye of a user of the data glasses with impact points of laser beams of the LFI sensor array,
[0034] Fig. 5 is a schematic flow diagram of the oculography procedure,
[0035] Fig. 6 shows a partial array of an LFI sensor array or an LFI sensor array of an embodiment of an alternative oculography device,
[0036] Fig. 7 is a top view of a first further embodiment of the alternative oculography device,
[0037] Fig. 8 is a top view of a second further embodiment of the alternative oculography device,
[0038] Fig. 9 is a schematic side view of a partial array of an LFI sensor array or an LFI sensor array of another alternative oculography device,
[0039] Fig. 10 is a schematic side view of a partial array of an LFI sensor array or of an LFI sensor array of a second further alternative oculography device, Fig. 11 is a schematic side view of a partial array of an LFI sensor array or of an LFI sensor array of a third further alternative oculography device,
[0040] Fig. 12 is a schematic side view of a partial array of an LFI sensor array or an LFI sensor array of a fourth further alternative oculography device,
[0041] Fig. 13 is a schematic side view of the partial array of the LFI sensor array or the LFI sensor array of the fourth further alternative oculography device and
[0042] Fig. 14 shows schematically an alternative data glasses with one of the oculography devices described above.
[0043] Description of the embodiments
[0044] Figure 1 schematically shows an example of a pair of data glasses 44a. The data glasses 44a are designed as AR data glasses. The data glasses 44a have a projector unit 52a. The projector unit 52a is configured to output a virtual image using a visible laser beam 54a. The data glasses 44a have a spectacle lens 48a. In the data glasses 44a shown as an example in Fig. 1, the spectacle lens 48a comprises a holographic optical element configured to redirect the visible laser beam 54a toward a user's eye 12a. The data glasses 44a comprise a spectacle frame 56a. The spectacle frame 56a supports the spectacle lens 48a. The data glasses 44a have an oculography device 46a.
[0045] The oculography device 46a is configured to perform an oculography method. Based on the oculography method, the oculography device 46a is configured to track a position and / or posture of the eye 12a within the data glasses 44a. The oculography device 46a has a control unit 96a. The control unit 96a is configured to perform the oculography method. The control unit 96a is, for example, integrated into the data glasses 44a. Alternatively, the control unit 96a could also be configured externally to the data glasses 44a (e.g., as a mobile device such as a smartphone, a tablet, or a smartwatch, or as a cloud) and be communicatively connected to the data glasses 44a. The oculography device 46a has a laser feedback interferometry (LFI) sensor array 18a. In the case shown in Fig. 1, the LFI sensor array 18a is integrated into the spectacle frame 56a.The LFI sensor array 18a is statically arranged in the data glasses 44a. In the case illustrated in Fig. 1, the LFI sensor array also comprises two sub-arrays 58a, 60a. Configurations with only a single array or more than two arrays are also conceivable. The LFI sensor array 18a comprises a plurality of LFI sensors 14a, 16a. Each of the sub-arrays 58a, 60a comprises a plurality of LFI sensors 14a, 16a. The LFI sensors designated here with the reference numerals 14a and 16a are assigned, by way of example, to a first sub-array 58a of the sub-arrays 58a, 60a. The LFI sensors 14a, 16a are each configured to detect and / or track at least one eye movement of the eye 12a. Each of the LFI sensors 14a, 16a of the LFI sensor array 18a is provided for outputting a laser beam 62a, 64a. The laser beams 62a, 64a of the LFI sensors 14a, 16a are invisible to the eye 12a. The laser beams 62a, 64a of the LFI sensors 14a, 16a are infrared laser beams (e.g.with a wavelength of approximately 850 nm or approximately 940 nm). The laser beams 62a, 64a have an intensity of less than 1 mW (optical output power). The laser beams 62a, 64a are eye-safe. The laser beams 62a, 64a can be collimated or focused to an optical path length 22a between the respective LFI sensor 14a, 16a and a scattering surface 24a, e.g., the eye 12a. The LFI sensors 14a, 16a of the LFI sensor array 18a each have unique emission angles. The LFI sensors 14a, 16a of the LFI sensor array 18a each have unique emission directions. The radiation angles and / or radiation directions of the laser beams 62a, 64a of the LFI sensors 14a, 16a are each unique.
[0046] The laser beams 62a, 64a of the LFI sensors 14a, 16a impinge on a facial surface of the user of the data glasses 44a or on an ocular surface of the eye 12a of the user of the data glasses 44a and are scattered there. The ocular surface and / or the facial surface thus each form a scattering surface 24a. Due to the scattering at the scattering surface 24a, a portion of the respective laser beam 62a, 64a is scattered back into the respective LFI sensor 14a, 16a, in particular into a laser cavity of the respective LFI sensor 14a, 16a, thereby generating a sensor signal usable for the oculography method.
[0047] Figure 2 schematically shows an exemplary partial array 58a of the LFI sensor array 18a with two LFI sensors 14a, 16a. The LFI sensor array 18a could also be formed entirely by the one shown in Fig. 2. The oculography device 46a has a common lens 26a. The common lens 26a is simultaneously assigned to several LFI sensors 14a, 16a of the LFI sensor array 18a. The common lens 26a is simultaneously assigned to the two sensors 14a, 16a shown in Fig. 2. The common lens 26a has an optical axis 28a. The optical axis 28a forms an axis of symmetry of the rotationally symmetric optical system lens 26a. The two LFI sensors 14a, 16a, to which the common lens 26a is assigned, have a different distance 30a, 30'a to the optical axis 28a of the common lens 26a.The two LFI sensors 14a, 16a, to which the common lens 26a is assigned, have a different distance direction 68a, 68'a to the optical axis 28a of the common lens 26a. The distances 30a, 30'a run in a plane parallel to an array plane 66a of the LFI sensor array 18a. The distance directions 68a, 68'a run in a plane parallel to an array plane 66a of the LFI sensor array 18a. A first LFI sensor 14a of the LFI sensors 14a, 16a has a first distance 30a to the optical axis 28a that is perpendicular to the optical axis 28a. The first LFI sensor 14a of the LFI sensors 14a, 16a has a first distance direction 68a perpendicular to the optical axis 28a. A second LFI sensor 16a of the LFI sensors 14a, 16a has a second distance 30'a from the optical axis 28a that is perpendicular to the optical axis 28a.The second LFI sensor 16a of the LFI sensors 14a, 16a has a second distance direction 68a pointing perpendicular to the optical axis 28a. The first distance 30a and the second distance 30'a are different. The first distance direction 68a and the second distance direction 68'a point in different directions.
[0048] The subarray 58a / the LFI sensor array 18a comprises a carrier substrate 70a. The carrier substrate 70a is designed as an electrical conductor track / submount. The subarray 58a / the LFI sensor array 18a comprises a housing 72a. The common lens 26a is enclosed in the housing 72a. The LFI sensors 14a, 16a are positioned relative to the common lens 26a such that their laser beams 62a, 64a both pass through the common lens 26a, but produce significantly different exit angles due to their different angles of incidence on a surface of the common lens 26a.
[0049] Figure 3 schematically shows a top view of an exemplary embodiment of the sub-array 58a / the LFI sensor array 18a with five LFI sensors 14a, 16a arranged on a common carrier substrate 70a under a single common lens 26a. In this exemplary embodiment, the centrally arranged LFI sensor 14a radiates at a right angle to the carrier substrate 70a, while the off-center LFI sensors 14a, 16a each radiate at different exit angles to the optical axis 28a. Due to the distance (optical path length 22a) between the installation location of the LFI sensors 14a, 14'a, 16a (e.g. in the spectacle frame 56a) and the eye 12a, the laser beams 62a, 64a of the partial array 58a / the LFI sensor array 18a hit the eye 12a at different locations due to their different exit angles.
[0050] This is schematically illustrated in Fig. 4, which images the eye 12a and impact points 74a, 76a, 78a, 80a, 82a of the laser beams 62a, 64a of the subarray 58a / the LFI sensor array 18a on and around the eye 12a. A first impact point 74a lies on an eyelid 84a of the eye 12a. A second impact point 76a lies on an iris 86a of the eye 12a. A third impact point 78a, a fourth impact point 80a, and a fifth impact point 82a lie predominantly on a sclera 88a of the eye 12a.
[0051] Figure 5 shows a schematic flow diagram of the oculography method. In at least one activation step 90a, the data glasses 44a and / or the oculography device 46a, in particular the LFI sensor array 18a, are activated. In at least one selection step 20a, a selection of at least one LFI sensor 14a, 14'a, 16a of the LFI sensor array 18a intended for performing an oculography step 10a following the selection step 20a is made based on a parameter comparison of test measurements from multiple LFI sensors 14a, 14'a, 16a of the LFI sensor array 18a. For this purpose, the LFI sensors 14a, 14'a, 16a are operated sequentially or simultaneously to each perform at least one test measurement. The test measurements are each speed test measurements.During the speed test measurements, raw speeds of movements of the scattering surfaces 24a at the impact points 74a, 76a, 78a, 80a, 82a of the individual laser beams 62a, 64a are determined. The observed measurement signal from the speed test measurements provides information about whether the associated LFI sensor 14a, 14'a, 16a is currently hitting the eye 12a well or not. In the example in Fig. 4, no speed would be observed at the first impact point 74a, since this point lies on the lower eyelid 84a of the eye 12a and thus no speed can be determined. At the fourth impact point 80a in the example shown, the associated laser beam 62a, 64a could hit the scattering surface 24a, in particular an eyeball surface of the eye 12a, at a right angle, so that a surface velocity component in the laser beam direction is zero. The LFI sensor 14a, 14'a, 16a associated with the fourth impact point 80a would therefore also be unsuitable.In the example shown in Fig. 4, one of the LFI sensors 14a, 14'a, 16a of the LFI sensor array 18a, whose laser beam 62a, 64a generates one of the impact points 76a, 78a, 82a, would therefore generate the largest measurement signal. The LFI sensor 14a, 14'a, 16a with the highest speed determined in the selection step 20a is selected for performing the following oculography step 10a. In the oculography step 10a, at least one eye movement of the eye 12a is determined, in particular tracked, by one or more of the LFI sensors 14a, 14'a, 16a that delivered the highest speed in the selection step 20a.
[0052] The selection step 20a is repeated following a first execution of the oculography step 10a if a, in particular definable, time interval has elapsed, if a fall below a minimum measurement data quality is determined during the execution of the oculography step 10a and / or if a significant change in the optical path length 22a determined by the currently selected LFI sensor 14a, 14'a, 16a is determined.
[0053] Figures 6 to 14 show further embodiments of the invention.
[0054] The following descriptions and the drawings are essentially limited to the differences between the exemplary embodiments, whereby with regard to components with the same designation, in particular with regard to components with the same reference numerals, reference can also be made to the drawings and / or the description of the other exemplary embodiments, in particular Figures 1 to 5. To distinguish the exemplary embodiments, the letter a is placed after the reference numerals of the exemplary embodiment in Figures 1 to 5. In the exemplary embodiments in Figures 6 to 14, the letter a is replaced by the letters b to g.
[0055] Figure 6 shows a partial array 58b of an LFI sensor array 18b or an LFI sensor array 18b of an alternative oculography device 46b. The partial array 58b or the LFI sensor array 18b comprises, for example, at least two LFI sensors 14b, 16b. The at least two LFI sensors 14b, 16b are formed by a common multi-cavity laser chip 40b.
[0056] Figure 7 shows a top view of a further embodiment of the alternative oculography device 46'b. This view shows that in this embodiment, three LFI sensors 14b, 16b are arranged in a common multi-cavity laser chip 40b. The sub-array 58b or the LFI sensor array 18b also comprises a plurality of multi-cavity laser chips 40b, which are arranged together under a common lens 26b of the alternative oculography device 46'b. In a second further embodiment of the alternative oculography device 46'b shown in Figure 8, all LFI sensors 14b, 16b assigned to the common lens 26b are combined in a common multi-cavity laser chip 40b. In this case, the multi-cavity laser chip 40b comprises nine LFI sensors 14b, 16b.
[0057] Figure 9 shows a schematic side view of a partial array 58c of an LFI sensor array 18c or of an LFI sensor array 18c of another alternative oculography device 46c. The LFI sensor array 18c comprises a plurality of LFI sensors 14c, 16c. This embodiment addresses the problem that, when using a common lens 26a, 26b, relatively large distances between the laser cavities of the LFI sensors 14c, 16c are required for large angular differences between the emerging laser beams 62c, 64c. The resulting large chip area can impact the efficiency of semiconductor manufacturing, as many more laser cavities could potentially be produced on the same area. In order to be able to place the laser cavities of the LFI sensors 14c, 16c of the LFI sensor array 18c as close to each other as possible, the LFI sensors 14c, 16c of the LFI sensor array 18c of the further alternative oculography device 46c have chip-integrated optics 92c.The chip-integrated optics 92c of the LFI sensors 14c, 16c of the LFI sensor array 18c of the further alternative oculography device 46c form chip-integrated lenses. The chip-integrated optics 92c modify the beam angles of each laser cavity separately.
[0058] In the embodiment illustrated by way of example in Fig. 9, a rear side of each of the LFI sensors 14c, 16c was provided with chip-integrated micro-lens arrays 32c using grayscale lithography. However, it is generally irrelevant for the invention whether the LFI sensors 14c, 16c are front- or rear-emitting. The further alternative oculography device 46c thus has a (common) micro-lens array 32c. The micro-lens array 32c comprises a plurality of micro-lenses 34c, 36c. Each of the individual micro-lenses 34c, 36c of the micro-lens array 32c is assigned to a respective one of the LFI sensors 14c, 16c of the LFI sensor array 18c. The micro-lenses 34c, 36c of the micro-lens array 32c have different radiation angles from each other.It is conceivable that after the adjustment of the radiation angle by the micro-lens array 32c, a further lens (not shown) is connected downstream to adjust a collimation state of the deflected laser beams 62c, 64c.
[0059] Fig. 10 shows a schematic side view of a partial array 58d of an LFI sensor array 18d or of an LFI sensor array 18d of a second, further alternative oculography device 46d. The LFI sensor array 18d comprises a plurality of LFI sensors 14d, 16d. The second, further alternative oculography device 46d has a (common) microlens array 32d. The microlens array 32d comprises a plurality of microlenses 34d, 36d. Each of the individual microlenses 34d, 36d of the microlens array 32d is assigned to a respective one of the LFI sensors 14d, 16d of the LFI sensor array 18d. The microlenses 34d, 36d of the microlens array 32d are embodied as chip-integrated optics 92d. The microlenses 34d, 36d of the microlens array 32d are embodied as metal lenses 38d. The chip-integrated optics 92d are formed by metal lenses 38d. The metal lens 38d is embodied, for example, by a binary column structure.Alternative metal lens shapes known from the state of the art that enable laser beam deflection are also conceivable.
[0060] Fig. 11 shows a schematic side view of a partial array 58e of an LFI sensor array 18e or of an LFI sensor array 18e of a third further alternative oculography device 46e. The LFI sensor array 18e comprises a plurality of LFI sensors 14e, 16e. The third further alternative oculography device 46e has a (common) microlens array 32e. The microlens array 32e comprises a plurality of microlenses 34e, 36e. Each of the individual microlenses 34e, 36e of the microlens array 32e is assigned to a respective one of the LFI sensors 14e, 16e of the LFI sensor array 18e. The microlenses 34e, 36e of the microlens array 32e are formed separately from the LFI sensors 14e, 16e. The microlens array 32e is formed as a separate lens plate 94e. The separate lens plate 94e comprises the plurality of microlenses 34e, 36e. The microlens array 32e is manufactured separately from the LFI sensors 14e, 16e.The separate lens plate 94e is combined with the LFI sensors 14e, 16e using a wafer-to-wafer process.
[0061] Figure 12 shows a schematic top view of a partial array 58f of an LFI sensor array 18f or of an LFI sensor array 18f of a fourth further alternative oculography device 46f. The LFI sensor array 18f comprises a plurality of LFI sensors 14f, 16f. At least two of the LFI sensors 14f, 16f are combined in common multi-cavity laser chips 40f. The LFI sensors 14f, 16f of a common multi-cavity laser chip 40f each emit mutually orthogonally polarized laser beams 62f, 64f (indicated by double arrows). Thus, two of the LFI sensors 14f, 16f of the LFI sensor array 18f are configured to emit light of different polarization. The fourth further alternative oculography device 46f has a polarization-dependent optics 42f. The polarization-dependent optics 42f are provided for a polarization-dependent deflection of incident light.The polarization-dependent optics 42f are assigned to the differently polarized light-emitting LFI sensors 14f, 16f. Although the respective laser beams 62f, 64f enter the polarization-dependent optics 42f approximately uniformly, they exit the polarization-dependent optics 42f at substantially different exit angles. Figure 13 shows a schematic side view of the sub-array 58f of the LFI sensor array 18f or the LFI sensor array 18f of the fourth further alternative oculography device 46f with, for example, two double-cavity chips, each emitting two laser beams 62f, 64f of different polarization, which are then deflected differently by the polarization-dependent optics 42f.
[0062] The optical arrangements shown in Figures 1 to 13 (individually or in combination) enable virtually any beam manipulation, particularly since a combination of polarization state, position of the laser chip relative to the lens, and lens area is possible. This allows the angular and parallel displacement of the laser beams 62a-f, 64a-f to be individually adjusted to achieve a particularly advantageous laser spot distribution over the user's eye region.
[0063] Fig. 14 schematically shows an alternative pair of data glasses 44g with one of the previously described oculography devices 46a, 46b, 46c, 46d, 46e, 46f. The alternative pair of data glasses 44g has a spectacle lens 48g. A hologram unit 50g is integrated into the spectacle lens 48g. The hologram unit 50g has a plurality of reflection holograms, each with different optical functions. The reflection holograms of the hologram unit 50g with the different optical functions are each configured to deflect light from different LFI sensors 14g, 16g of an LFI sensor array 18g of the data glasses 44g differently with different beam widths. The LFI sensor array 18g is integrated into a temple piece 98g of the data glasses 44g. The reflection holograms of the hologram unit 50g can be arranged only in a region of an edge of the spectacle lens 48g or extend over a large part of the spectacle lens 48g.In the latter case, an efficiency of the holographic function of the reflection hologram of the hologram unit 50g could increase from a center of the spectacle lens 48g to an edge of the spectacle lens 48g.
Claims
Claims 1 . Oculography method, in particular data glasses oculography method for augmented reality (AR) data glasses and / or for virtual reality (VR) data glasses / VR headsets, with at least one oculography step (10a-g), in which at least one eye movement of an eye (12a-g), in particular of a data glasses user's eye, is determined, in particular tracked, by a subset of laser feedback interferometry (LFI) sensors (14a-g, 16a-g) of an LFI sensor array (18a-g) assigned to the eye (12a-g), characterized in that in at least one selection step (20a-g), in particular preceding the oculography step (10a-g), a selection of at least one LFI sensor (14a-g, 16a-g) provided for carrying out the oculography step (10a-g), in particular following the oculography step (10a-g) based on a parameter comparison of test measurements of several LFI sensors (14a-g, 16a-g) of the LFI sensor array (18a-g).
2. Oculography method according to claim 1, characterized in that in the selection step (20a-g) the selection of the at least one LFI sensor (14a-g, 16a-g) of the LFI sensor array (18a-g) provided for carrying out the oculography step (10a-g) is carried out on the basis of a parameter comparison of speed test measurements, in particular eye movement speed test measurements, with the plurality of LFI sensors (14a-g, 16a-g) of the LFI sensor array (18a-g).
3. Oculography method according to claim 2, characterized in that at least the LFI sensor (14a-g, 16a-g) with the highest speed determined in the selection step (20a-g) is selected for carrying out the oculography step (10a-g).
4. Oculography method according to one of the preceding claims, characterized in that the selection step (20a-g) is repeated is when a, in particular definable, time interval has expired, when a fall below a minimum measurement data quality is determined when carrying out the oculography step (10a-g) and / or when a significant change in an optical path length (22a-g) between the LFI sensor (14a-g, 16a-g) and the scattering surface (24a-g), e.g. the eye of the data glasses user, is determined by the currently selected LFI sensor (14a-g, 16a-g).
5. Oculography device (46a-g) for carrying out an oculography method according to one of the preceding claims, with at least one, in particular statically arranged, LFI sensor array (18a-g), comprising a plurality of LFI sensors (14a-g, 16a-g), each of which is configured to determine, in particular track, at least one eye movement of an eye (12a-g), in particular an eye of a data glasses user, characterized in that at least a majority of all LFI sensors (14a-g, 16a-g) of the LFI sensor array (18a-g), in particular each LFI sensor (14a-g, 16a-g) of the LFI sensor array (18a-g), each has unique radiation angles and / or unique radiation directions.
6. Oculography device (46a; 46b; 46g) according to claim 5, characterized by a common lens (26a; 26b; 26g) which is simultaneously assigned to a plurality of LFI sensors (14a, 16a; 14b, 16b; 14g, 16g) of the LFI sensor array (18a; 18b; 18g), wherein at least a majority of the LFI sensors (14a, 16a; 14b, 16b; 14g, 16g) assigned to the common lens (26a; 26b; 26g), preferably each LFI sensor (14a, 16a; 14b, 16b; 14g, 16g) assigned to the common lens (26a; 26b; 26g), has a different distance (30a, 30'a; 30b, 30'b; 30g, 30'g) and / or a different distance direction (68a, 68'a; 68b, 68'b; 68g, 68'g) to an optical axis (28a; 28b; 28g) of the common lens (26a; 26b; 26g).
7. Oculography device (46c-e) according to claim 5, characterized by a common (micro) lens array (32c-e) with a plurality of (micro) lenses (34c-e, 36c-e), wherein the individual (micro) lenses (34c-e, 36c-e) of the (micro) lens array (32c-e) each correspond to one of the plurality of LFI Sensors (14c-e, 16c-e) of the LFI sensor array (18c-e) are assigned, and wherein in particular at least a majority of the (micro) lenses (34c-e, 36c-e), preferably all (micro) lenses (34c-e, 36c-e) have different radiation angles from one another.
8. Oculography device (46c-d) according to one of claims 5 to 7, characterized in that the LFI sensors (14c-d, 16c-d) of the LFI sensor array (18c-d) have chip-integrated optics (92c-d), in particular chip-integrated lenses.
9. Oculography device (46d) according to claim 8, characterized in that the chip-integrated optics (92d) is formed by a metal lens (38d).
10. Oculography device (46b-d; 46f-g) according to one of claims 5 to 9, characterized in that at least two of the LFI sensors (14b-d, 16b-d; 14f-g, 16f-g) of the LFI sensor array (18b-d; 18f-g), preferably at least three of the LFI sensors (14b-d, 16b-d; 14f-g, 16f-g) of the LFI sensor array (18b-d; 18f-g), preferably all LFI sensors (14b-d, 16b-d; 14f-g, 16f-g) of the LFI sensor array (18b-d; 18f-g) are formed by a common multi-cavity laser chip (40b-d; 40f-g).
11. Oculography device (46f) according to one of claims 5 to 10, characterized in that at least two of the LFI sensors (14f, 16f) of the LFI sensor array (18f), preferably at least three of the LFI sensors (14f, 16f) of the LFI sensor array (18f), preferably all LFI sensors (14f, 16f) of the LFI sensor array (18f), are designed to emit light of different polarization, and characterized by a polarization-dependent optics (42f) which is assigned to the differently polarized light-emitting LFI sensors (14f, 16f).
12. Data glasses (44a-g), in particular AR data glasses and / or VR data glasses / VR headset, with an oculography device (46a-g) according to one of claims 5 to 11.
13. Data glasses (44g) according to claim 12, characterized by a hologram unit (50g) integrated into a spectacle lens (48g) of the data glasses (44g), which hologram unit has a plurality of reflection holograms each with different optical functions, which are each configured to reflect light from different LFI sensors (14g, 16g) of the LFI sensor array (18g) with different Einstrah Iwin no different to deflect.
14. Control unit (96a-g) for an oculography device (46a-g) according to one of claims 5 to 11 and / or for data glasses (44a-g) according to one of claims 12 or 13, which is configured to carry out an oculography method according to one of claims 1 to 4.
Citation Information
Patent Citations
Method for determining the gaze direction of an eye
DE102020206821A1
Methods for detecting human activity
DE102021207058A1
Relative Position Based Eye-tracking System
US20200026350A1
Method and apparatus for data capture and evaluation of ambient data
WO2017025483A1
