Method for determining current depth of focus of user of data glasses and data glasses

Abstract

The invention relates to a method for determining a current depth of focus (10, 10 ', 10' ') of a user of data glasses (12), in particular at least by means of a monocular tracking system, in which in at least one method step (20) a current Listing plane (16) of the user's eyes (18, 18') is measured, and in at least one further method step (22) a current Listing plane (16) of the user's eyes (18, 18 ') is measured. A current convergence of the user eye (18, 18 ') is ascertained as a function of a measured current Listing plane (16) of the user eye (18, 18'), and wherein, in at least one additional further method step (24), a user-specific current depth of focus (10, 10 ', 10' ') of the user eye (18, 18') is ascertained as a function of the ascertained current convergence of the user eye (18, 18 ').

Description

Background Art

[0001] Methods for determining the depth of focus have been proposed that are based on the use of a world camera with depth information. The use of a world camera may lead to increased costs and increased energy consumption for the data glasses. Summary of the Invention

[0002] The present invention proposes a method for determining the current depth of focus of a user wearing data glasses, in particular using at least a monocular tracking system. In at least one method step, the current Listing plane of the user's eye is measured. In at least one further method step, the current vergence of the user's eye is determined based on the measured current Listing plane of the user's eye. In at least one additional further method step, the user-specific current depth of focus of the user's eye is determined based on the determined current vergence of the user's eye. This advantageously allows for low costs and / or low energy consumption, in particular because the use of a costly and / or energy-intensive world camera can be advantageously omitted. Advantageously, the total number of components required for determining the current depth of focus in the data glasses, in particular in the form of a monocular system, can be kept low. Advantageously, the proposed method for determining depth of focus enables applications based on knowledge of the depth of focus, such as depth-of-focus adaptive control of a display, in particular a virtual retinal display of the data glasses, and calibration of the display, in particular a virtual retinal display of the data glasses, based on the real-time vergence of the user's eye. Furthermore, a second measurement channel can advantageously be provided in the stereo system, which can be used, for example, to compensate for eye disorders such as nystagmus, which cannot be taken into account in a simple estimation of the intersection of the gaze vectors ascertained from the stereo system.

[0003] The depth of focus of a user should be understood, in particular, as the distance from the focal point of the user's eye to the user's eye. The focal point of the user's eye should be understood, in particular, as the point in space in front of the user's eye at which the line of sight / gaze vectors of the user's eye intersect. "Data glasses" should be understood, in particular, as wearable devices (head-mounted displays) that can be used to add information to the user's field of view. Preferably, data glasses enable augmented reality applications and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses, VR glasses, or AR glasses. Data glasses particularly include a virtual retinal display (also known as a retinal scanning display or light field display), which is particularly well known to those skilled in the art. The virtual retinal display is particularly configured to sequentially scan image content by deflecting at least one visible laser beam of at least one time-modulated light source, such as one or more (RGB) laser diodes of a laser projector, and to project this image content directly onto the retina of the user's eye via optical elements. Monocular tracking systems are particularly configured to track and / or record the movements and / or velocity of a single user's eye. "Configured" or "set" should particularly be understood to mean specifically programmed, designed, and / or equipped. In this context, “an object is provided or configured for a certain function” is to be understood to mean, in particular, that the object performs and / or executes the certain function in at least one application state and / or operating state.

[0004] According to the so-called Listing's law, all axes of rotation of all glances and slow tracking movements of the user's eyes starting from the main position are located in a plane, the so-called Listing plane of the user's eyes. Therefore, when gazing at a distant object through the user's eyes, not all possible eye positions in three-dimensional space are occupied, but only eye positions that are limited to rotations around the rotation axes in the Listing plane. The Listing plane is particularly constructed as a plane in which all axes of rotation of all possible eye positions that can be reached by rotating the eyes starting from the main position of the user's eyes are located in this plane. Therefore, the position of the Listing plane depends in particular on the corresponding main position. For example, when looking straight ahead (main position = straight ahead), the Listing plane is approximately perpendicular to the visual axis of the user's eyes. For example, when looking straight ahead (main position = straight ahead), the Listing plane corresponds to the equatorial plane of the user's eyes. When the gaze is diverted from an eye position different from the main position, for example 30° upward, the rotation axis of the user's eye is also located in the Listing plane, but the Listing plane is no longer perpendicular to the gaze direction compared to direct vision, but is tilted in the direction of the gaze diversion, in particular tilted by half the angle of the gaze direction (here: 15°) relative to the direct vision main position. Therefore, the Listing plane is also tilted according to the convergence, in particular proportionally to the depth of focus. Therefore, the current convergence of the user's eyes can be advantageously determined based on the measured Listing plane. As a result, the convergence, in particular the tilt of the Listing plane when the eye focus changes, has an impact on the exact movement trajectory of the user's eyes, for example the movement trajectory of the pupil, during the eye movement. Knowledge of the convergence, in particular knowledge of the opposite movements of the two user's eyes, preferably allows the intersection of the gaze vectors and / or the current depth of focus of the user's eyes to be determined. The current convergence is in particular the current opposite eye movement of the user's eyes.

[0005] Furthermore, it is proposed that, in the method steps, the current listing plane is determined based on at least a measurement of the rotation axis of the user's eye movement, in particular using a laser feedback interferometer (LFI) sensor assembly. This advantageously keeps costs and / or energy consumption low, in particular because the use of a costly and / or energy-intensive world camera can be advantageously omitted. Advantageously, the LFI sensor can be used to measure the depth of focus.

[0006] The LFI sensor assembly particularly includes at least a plurality, for example, three or four, individual LFI sensors. Preferably, the LFI laser sensors of the LFI sensor assembly are integrated into the data glasses, for example, into the frames, lenses, and / or temples of the data glasses. The LFI sensors of the LFI sensor assembly are particularly arranged such that their laser beams, particularly infrared laser beams, strike the eyes of a user wearing the data glasses. Optical deflection elements may be provided, if necessary, to deflect one or more of the laser beams toward the user's eyes. Using the known laser feedback interferometer measurement method implemented by the LFI sensors, which may include wavelength modulation of the laser light of the LFI sensors, the Doppler shift caused by rotation of the respective user's eyes can be advantageously measured. Furthermore, the distance d between the respective user's eyes and the respective LFI sensors of the LFI sensor assembly can advantageously be determined using known LFI measurement methods. Preferably, the surface velocity v of the user's eyes in the direction of the respective laser beams and the distance d are determined from these measurement data of the LFI measurement method for each LFI sensor of the LFI sensor assembly. Therefore, by fusing the measurement results of at least three LFI sensors of the LFI sensor assembly, it is possible to determine, in particular, the rotation speed of the user's eye. Therefore, by fusing the measurement results of at least three LFI sensors of the LFI sensor assembly, it is possible to determine, in particular, the position of the user's eye relative to the data glasses. Furthermore, by fusing the measurement results of the LFI sensors of the LFI sensor assembly, it is possible to determine, in particular, the real-time rotation axis e. Furthermore, by fusing the measurement results of the LFI sensors of the LFI sensor assembly, it is possible to determine, in particular, the starting gaze vector and / or the target gaze vector of the monitored eye movement. Alternatively, an eye observation camera device can be integrated into the system to determine the starting gaze vector and / or the target gaze vector, which is preferably configured for video eye tracking. In this alternative, the fusion can then be performed based on the measurement values ​​of the eye observation camera device and the (static) measurement values ​​of the LFI sensor assembly.

[0007] In particular, in order to be able to determine the axis of rotation, in particular by means of measurement data from the LFI sensor assembly, the relationship between the coordinate system of the user's eye and the coordinate system of the data glasses must first be known (first step of the algorithm for determining the axis of rotation). In particular, the position and orientation of the laser sensor in the coordinate system of the data glasses are known from production, for example from end-of-line measurements performed in production or from the assembly accuracy specified in production. The position and orientation of the laser sensor in the coordinate system of the data glasses are in particular assumed to be constant. In order to determine the center of the user's eye and / or the center of the coordinate system of the user's eye, the user's eye is preferably assumed to be a sphere with a defined and known diameter r. The relationship between the coordinate system of the data glasses and the coordinate system of the user's eye is preferably expressed as a combination of a rotation and a translation. Therefore, in order to determine the relationship between the coordinate system of the data glasses and the coordinate system of the user's eye, a triangulation approach is preferably used, see in particular equation (1).

[0008]

[0009] Here, x g Indicates the point in the coordinate system of the data glasses, R indicates the rotation of the user's eyes, t indicates the translation and r indicates the radius. Point x g In particular, taking into account the radius of the user's eye, the coordinate system of the user's eye is transformed by a rotation R and a translation t, wherein the corresponding point is represented in particular by x h This results in the spherical coordinates θ, and r equation (2):

[0010]

[0011] The parameters R and t are preferably determined by means of known triangulation methods based on distance measurements of the three LFI sensors of the LFI sensor assembly.

[0012] After the relationship of the coordinate systems is known, in a further step (the second step of the algorithm for determining the rotation axis), the surface velocity of the user's eye at the point of incidence of the laser light of the LFI sensor is determined. The surface velocity measured by the nth LFI sensor can in particular be based on the point of incidence x on the spherical surface of the user's eye. n , based on the previously obtained sensor orientation (vector p n ) and the velocity v measured by Doppler shift n To find , see equation (3).

[0013]

[0014] The surface velocity is defined as the change in angular velocity about the eye's axis of rotation. A change in radius, i.e., a displacement of the point of incidence, can lead in particular to a change in velocity that must be compensated (this does not apply to the retina, however, since a rotation of the sphere does not cause a change in radius).

[0015] In particular, it is possible to construct a matrix containing the rotation axis e and the angular velocity and The set of equations (4).

[0016]

[0017] In this set of equations (4), only the velocity v measured by the nth LFI sensor at the point of incidence of the sensor on the user's eye can be substituted n The measured value and the incident point x n Then the equation group (4) can be solved to obtain the unknown rotation axis e. If the angular velocity is integrated at this time, the rotation of the user's eye, especially the high-resolution motion trajectory of the user's eye along the rotation axis e of the user's eye, can be determined. In particular, the determination is based solely on measurements of the LFI sensor assembly.

[0018] Furthermore, it is proposed that in the method steps, the current Listing plane is determined (the third step of the algorithm for determining the rotation axis) based on at least a measurement of the starting position of the user's eye movement around the rotation axis and / or based on a measurement of the end position of the user's eye movement around the rotation axis, in particular using a laser feedback interferometer (LFI) sensor assembly of the data glasses or using an eye observation camera of the data glasses. This advantageously allows for low costs and / or low energy consumption, in particular because the use of a costly and / or energy-intensive world camera can be advantageously omitted. Advantageously, only the LFI sensor can be used to measure the starting position and / or the end position. Alternatively, however, it is also conceivable to determine the starting position and / or the end position using an eye observation camera and video eye tracking, but this may not fully achieve the advantages described. The user's eye movements associated with the starting position and the end position can be intentional or performed by saccades.

[0019] In particular, in order to determine the (absolute) starting position and / or ending position of the gaze vector during its movement along the trajectory of the user's eye, the iris plane of the iris of the user's eye is triangulated in space. To this end, in particular, the laser beams of at least three LFI sensors must be directed onto the iris of the user's eye. Then, in particular, the absolute orientation of the iris of the user's eye can be determined. In the underlying model, the iris is preferably assumed to be disk-shaped ("iris disk"). Then, the absolute gaze angles θ0 and θ can be determined using equations (5) to (7).

[0020]

[0021] Then, the normal vector n pointing in the direction of the view Iris It is preferably derived from a reconstructed plane, which can be generated from the triangulation of the distance measurements of three LFI sensors whose laser beams illuminate the irises of the user's eyes. Alternatively, the absolute position of the user's eyes can also be determined using a video eye tracking algorithm based on a possibly additionally installed eye observation camera. Thus, from the three aforementioned steps of the algorithm for determining the rotation axis, at least the following parameters can be derived: a) the trajectory θ of the user's eyes during eye movements; (t) , b) the rotation axis e about which the user's eye rotates during the eye movement, and c) the absolute position of the user's eye at the start and end of the eye movement

[0022] Furthermore, it is proposed that, in the method steps, a trajectory of the user's eye movement is determined based on the detected rotation axis of the user's eye movement and the detected starting and ending positions of the user's eye movement. This advantageously allows the determination of the Listing plane of the user's eye, and thus in particular also the depth of focus of the user's eye, using a particularly cost-effective and / or energy-efficient system. The trajectory of the movement particularly forms a great circle of the respective associated Listing plane or half-angle plane.

[0023] Furthermore, it is proposed that in the method steps, the surface velocity, in particular the surface rotation velocity, of the user's eye is detected by means of the LFI sensor assembly of the data glasses, preferably in the manner described above (see in particular equation (3)), and evaluated to determine the rotation axis and / or the movement path, in particular in the manner described above (see in particular equation (4)). This advantageously allows the determination of the Listing plane of the user's eye, and thus in particular also the determination of the focal depth of the user's eye, by means of a particularly cost-effective and / or energy-efficient system.

[0024] Furthermore, it is proposed that in the further method step, the current convergence is determined based on the Listing plane angle between the measured current Listing plane and the (untilted) Listing plane for the main position of the user's eye looking towards infinity, in particular initially determined and / or calibrated. As a result, the depth of focus of the user's eye can be advantageously determined with the aid of a system that results in particularly low costs and / or particularly low energy consumption. The position of the main position is determined in particular by an initial calibration. The correlation between the position of the current Listing plane and the convergence is determined in particular by an initial calibration. For a given starting position, there can be in particular a set of half-angle planes, which each result from a determined rotation of the Listing plane. If, in particular, the axis of rotation is known, it is usually possible to clearly determine the half-angle plane containing this axis of rotation from the set of half-angle planes, and thus the position of the current Listing plane or the position of the actual main position can be clearly determined.

[0025] Furthermore, it is proposed that in the further method step, the current convergence is read out from a known characteristic curve or a lookup table, which has been calibrated in a preparatory phase, in which the convergence is plotted over the change in the angle of the Listing plane. This makes it possible to advantageously determine the depth of focus of the user's eye with the aid of a system that results in particularly low costs and / or particularly low energy consumption. The characteristic curve or the lookup table describes in particular the (possibly to be calibrated) correlation between the current Listing plane of the user's eye and the real-time convergence of the user's eye. In this regard, it is pointed out as an additional consideration that if the convergence should be known (for example due to simultaneous measurement of both eyes), this information can in turn be used as a constraint for determining the rotation axis of the user's eye.

[0026] Furthermore, in the additional further method step, the current user-specific depth of focus of the user's eye is determined using initial user calibration data, by which different convergences are associated with different user-specific depths of focus of the user's eye. This advantageously allows the depth of focus of the user's eye to be determined with the aid of a particularly low-cost and / or energy-efficient system. It can be seen from the above part of the method that the convergence of the user's eye can be determined based on the measured movement trajectory and the starting and target positions of the gaze vector and the rotation axis. In order to ultimately be able to convert this into the depth of focus of the user's eye, a (one-time) system calibration is particularly required (for each user), in particular because the data glasses can be located at different positions on the user's face depending on the user, and therefore the coordinate system of the data glasses, i.e. the user's head, can be different (e.g. depending on the head shape, nose shape, pupil distance, etc.).

[0027] In this regard, it is proposed that, in a user calibration step that temporally precedes the additional further method step, initial user calibration data be determined by observing at least one moving real calibration object via a world camera of the data glasses and simultaneously measuring the vergence of the user's eye, which follows and focuses on the moving real calibration object, via an LFI sensor assembly of the data glasses or an eye-viewing camera of the data glasses. The determination is particularly performed by evaluating known geometric parameters of the moving real calibration object at different positions and distances from the world camera and simultaneously assigning these parameters to the measured vergence, and / or by determining different positions and distances of the moving real calibration object from the world camera using a stereo system and / or a structured light system and simultaneously assigning these positions and distances to the measured vergence. This advantageously enables a depth of focus determination for the data glasses. The movable real calibration object can in particular be a finger of the user's hand, such as the index finger. For example, the user extends their hand and points upwards with their index finger. In this example, the fingertip can then be detected by the world camera, and the focal depth between the world camera (which is fixedly positioned in the coordinate system of the data glasses) and the user's eyes or the data glasses can be determined based on the geometry of the fingertip (e.g., the shape or thickness of the finger or the nail of the finger) or by using the stereo system or structured light system of the data glasses (e.g., multiple controllable light sources arranged at different points on the data glasses). In this example, the user can then slowly move their finger in the direction of the world camera toward the data glasses while their eyes are fixed on the finger. For example, information output via a display on the data glasses or acoustic signals from the data glasses can prompt the user to stop moving the finger toward the data glasses at a certain focal depth and then move the finger to the left and right parallel to the user's head at the corresponding focal depth, particularly to induce so-called "smooth pursuit eye movements" in the user's eyes at the corresponding focal depth. These movements of the user's eyes can then be detected and / or recognized by the data glasses, and the depth information from the world camera can then be mapped accordingly to the position of the current Listing plane, in particular relative to the main position of the user's eyes, after the shift. This process is preferably repeated until enough calibration points have been collected (for example, at least two calibration points are required for a linear two-point calibration). The real calibration object can be a finger, as described above, but can also be another object, preferably with approximately known dimensions. The user calibration step also includes, in particular, taking measurements when the eyes are looking towards infinity, i.e. at zero convergence.A "world camera" is to be understood in particular as a camera of the data glasses that is oriented away from the user's face. The field of view of the world camera of the data glasses preferably overlaps at least largely with the field of view of the user as seen through the eyeglasses of the data glasses.

[0028] Additionally, it is proposed that during the user calibration step, a real calibration object is moved at different distances (e.g., two, three, or four) from the world camera in a distance plane that is at least substantially parallel to the image plane of the world camera. This advantageously enables the aforementioned "smooth pursuit eye movement" of the user's eye at focal depths corresponding to the different distances. This advantageously enables the depth information from the world camera to be mapped onto the offset, current position of the Listing plane, particularly relative to the main position of the user's eye. The image plane of the world plane is particularly oriented at least substantially perpendicular to the direction of sight of the user's eye in the main position. "Substantially parallel" is to be understood here as particularly an orientation of a direction relative to a reference direction, particularly within a plane, wherein the deviation of the direction from the reference direction is particularly less than 8°, advantageously less than 5°, and particularly advantageously less than 2°. The term "substantially perpendicular" is to be defined here as an orientation of a direction relative to a reference direction, wherein the direction and the reference direction, particularly considered in the projection plane, enclose a 90° angle, and wherein the maximum deviation of the angle is particularly less than 8°, advantageously less than 5°, and particularly advantageously less than 2°.

[0029] Furthermore, it is additionally provided that, in a user calibration step, a real calibration object is manually moved by the user, wherein the movement of the real calibration object is monitored by a world camera, and movement instructions and / or movement commands are determined based on the monitoring data acquired thereby and output to the user acoustically and / or optically by the data glasses. This advantageously enables a simple and reliable initial user calibration. The optical output of these movement instructions and / or movement commands can in particular be performed using a virtual retinal display of the data glasses.

[0030] Alternatively or additionally, it is proposed that, in an alternative or additional user calibration step, which precedes the additional further method step, initial user calibration data is determined by display-integrating a moving virtual calibration object via the display of the data glasses, in particular via a virtual retinal display of the data glasses, and simultaneously measuring the convergence of the user's eyes following and focusing on the moving virtual calibration object via an LFI sensor assembly of the data glasses or an eye-viewing camera of the data glasses. This advantageously enables the determination of the focal depth for the data glasses. Furthermore, this advantageously enables the determination of the focal depth of the user's eyes using a particularly cost-effective and / or energy-efficient system, which in particular does not require a world camera. For example, for this purpose, embedded stimuli (e.g., moving dots, text, etc.) can be virtually displayed in different focus planes, for example via the virtual retinal display of the data glasses, and the eye movements can be correspondingly measured back using the system of the data glasses, in particular via the LFI sensor assembly of the data glasses. This advantageously enables a mapping of the current (shifted) listing plane relative to the main position to the corresponding focus plane. This embodiment is particularly advantageous because it enables the focus depth to be calibrated directly for the display content of the virtual retinal display of the data glasses.

[0031] As described above, if in the alternative or additional user calibration step a virtual calibration object is presented and moved via the display of the data glasses, in particular via a virtual retinal display, at different focusing distances, in particular produced by the optical system of the data glasses, such as the lens system of the data glasses or a laser projector, then the depth of focus can advantageously be calibrated directly depending on the display content of the virtual retinal display of the data glasses and / or for the display content of the virtual retinal display of the data glasses.

[0032] The present invention also proposes a calculation unit for determining the current depth of focus of a user wearing data glasses, wherein the calculation unit is configured at least to determine the current vergence of the user's eye based on a current listing plane measured by the user's eye, in particular, at least measured using a monocular tracking system, and wherein the calculation unit is configured at least to determine the user-specific current depth of focus of the user's eye based on the current vergence of the user's eye. This advantageously allows for low costs and / or low energy consumption. Advantageously, the total number of components required for determining the current depth of focus in the data glasses, in particular in data glasses configured as a monocular system, can be kept low. The calculation unit may in particular be a microcontroller or similar component of the data glasses. In this context, a calculation unit may be understood to mean, in particular, a unit having at least one control electronics component. "Control electronics" should in particular be understood to mean a unit having a processor, a storage medium, and an operating program stored in the storage unit. Alternatively, however, the calculation unit may also be formed at least partially by the cloud or another centralized computing infrastructure (e.g., a smartphone coupled to the data glasses), or by a decentralized computing infrastructure separate from the data glasses. In this case, the data glasses preferably have a (wireless) communication interface for communicating with external parts of the computing unit.

[0033] Furthermore, the present invention proposes data glasses having at least one internal or external computing unit and a laser feedback interferometer (LFI) sensor assembly, wherein the data glasses, in particular the computing unit, are configured at least to determine the current depth of focus of a user of the data glasses. This advantageously allows for low costs and / or low energy consumption, in particular because the use of a costly and / or energy-intensive world camera can be advantageously omitted. Advantageously, the total number of components required for determining the current depth of focus in the data glasses, in particular in data glasses designed as a monocular system, can be kept low.

[0034] The method according to the invention and the data glasses according to the invention are not intended to be limited to the aforementioned applications and embodiments. In particular, the method according to the invention and the data glasses according to the invention may have a number of individual elements, components, and units, as well as method steps, that differs from the numbers mentioned herein, in order to fulfill the functionalities described herein. Furthermore, within the value ranges given in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Further advantages emerge from the following description of the drawings. The drawings illustrate exemplary embodiments of the invention. The accompanying drawings, the specification, and the claims contain numerous combinations of features. A person skilled in the art will also examine these features individually for specific purposes and combine them into other meaningful combinations.

[0036] The accompanying drawings show:

[0037] Figure 1 Schematic partial view of data glasses,

[0038] Figure 2 schematically illustrates the convergence of the user's eyes of a user of data glasses,

[0039] Figure 3 A schematic flow chart of a method for determining the current depth of focus of a user of data glasses,

[0040] Figure 4 Schematic side view of the data glasses and the real calibration object during the user calibration step of the method,

[0041] Figure 5a Schematically shows the user's eyes looking towards infinity and the corresponding (untilted) Listing plane,

[0042] Figure 5b schematically illustrates the current Listing plane of the user's eye that is not looking towards infinity, i.e., is focused, and

[0043] Figure 6 Exemplary movement trajectories of a user's eyes with different primary positions are schematically visualized. DETAILED DESCRIPTION

[0044] Figure 1A schematic partial view of data glasses 12 is shown. The data glasses 12 include a virtual retinal display for visually presenting virtual content, such as augmented reality content, in the field of view of a user 54 of the data glasses 12. The data glasses 12 are designed to be comparable to known data glasses 12 with a virtual retinal display in terms of their basic functionality and their basic components. The data glasses 12 include a monocular tracking system ("Mono-Eye-Tracking-System"). The data glasses 12 include a computing unit 58. The computing unit 58 is exemplarily configured as an internal computing unit. Alternatively, however, the computing unit 58 can also be configured to be at least partially separate from the data glasses 12 (for example, as a mobile terminal device coupled to the data glasses 12, such as a smartphone or similar terminal device). The data glasses 12 include a laser feedback interferometer (LFI) sensor assembly 26. The LFI sensor assembly 26 exemplarily includes five LFI sensors 60, 60', 60", 60'", 60". Alternatively, however, more or fewer than five LFI sensors 60, 60', 60", 60'", 60"" are also conceivable. The data glasses 12 include an eye observation camera 32. The eye observation camera 32 has a field of view oriented in the direction of the user eye 18 of the user 54. The eye observation camera 32 is configured to observe at least one user eye 18, in particular the position of the user eye in the coordinate system of the data glasses 12. The data glasses 12, in particular the computing unit 58, is configured to determine the current depth of focus 10 (see Figure 2 The data glasses 12 are configured to carry out the method for determining the current depth of focus 10 of a user 54 of the data glasses 12. The data glasses 12 may have a world camera 46 (see Figure 4 ) or is constructed without a world camera device.

[0045] Figure 2 The convergence of the user eyes 18, 18' of the user 54 of the data glasses 12 is schematically shown. Figure 2 In FIG. 1 , the user's eyes 18 and 18' are shown as an example at three different focal depths 10, 10', and 10". When the focal depths 10, 10', and 10" decrease, the two sight lines of the two user's eyes 18 and 18' move closer to each other. When the focal depths 10, 10', and 10" increase, the two sight lines of the two user's eyes 18 and 18' move away from each other. Each focal depth 10, 10', and 10" is assigned a focal point (at Figure 2 marked by a circle or diamond in the diagram).

[0046] Figure 3A schematic flow chart of a method for determining the current depth of focus 10, 10', 10" of a user 54 of data glasses 12 is shown. A computing unit 58, in particular the data glasses 12, is configured to carry out the method for determining the current depth of focus 10, 10', 10" of the user 54. In at least one user calibration step 42, user-specific initial user calibration data are determined. At least one moving real calibration object 44 (see FIG. 4 ) is imaged by means of a world camera 46 of the data glasses 12. Figure 4 ) and simultaneously measure the convergence of the user's eyes 18, 18' that follow the moving real calibration object 44 and focus on the moving real calibration object 44 by means of the LFI sensor assembly 26 of the data glasses 12 or the eye observation camera 32 of the data glasses 12 to obtain initial user calibration data. Figure 4 In the example, real calibration object 44 is designed as a finger of user 54. To determine user calibration data, in particular user-specific convergence behavior, in user calibration step 42, known geometric parameters of the moving real calibration object 44 are evaluated at different positions and different distances 62, 62' from world camera 46 of the moving real calibration object 44, and these parameters are simultaneously assigned to the measured convergence. Alternatively, different positions and different distances 62, 62' of the moving real calibration object 44 from world camera 46 can be determined using a stereo system and / or structured light system 48 of data glasses 12, and these positions and distances can be simultaneously assigned to the measured convergence. In the example of a finger, the known geometric parameter is the width of the fingernail of the finger. Furthermore, in user calibration step 42, the real calibration object 44 is moved at different distances 62, 62' from world camera 46 in a distance plane 52 that is at least substantially parallel to the image plane 50 of world camera 46. In this case, during user calibration step 42, the movement of real calibration object 44 is manually performed by user 54. Furthermore, during user calibration step 42, the movement of real calibration object 44 is monitored by world camera 46, and movement instructions and / or movement commands are determined based on the monitored data. Furthermore, during user calibration step 42, the determined movement commands are output to user 54 acoustically and / or optically by data glasses 12.

[0047] In at least one alternative or additional user calibration step 56, initial user calibration data are determined by displaying a moving virtual calibration object on a virtual retinal display of the data glasses 12 and simultaneously measuring the convergence of the user's eyes 18, 18' following and focusing on the moving virtual calibration object 44 via the LFI sensor assembly 26 of the data glasses 12 or the eye-viewing camera 32 of the data glasses 12. In this alternative user calibration step 56, the virtual calibration object is displayed and moved on the display of the data glasses 12 at different focal distances, for example, generated by the optical system (not shown) of the data glasses 12.

[0048] In at least one method step 20, the current Listing plane 16 of the user's eye 18 is measured (see Figure 5b In method step 20, at least the rotation axis 14 (see Figure 5a and Figure 5b ) is used to determine the current listing plane 16. In method step 20, the surface velocity of the user's eye 18 is detected by means of the LFI sensor assembly 26 of the data glasses 12 and evaluated to determine the rotation axis 14. In method step 20, at least a starting position 28 (see FIG. 1 ) of the movement of the user's eye 18 about the rotation axis 14 is determined based on the LFI sensor assembly 26 of the data glasses 12 or the eye observation camera 32 of the data glasses 12. Figure 6 ) is measured to determine the current listing plane 16. In method step 20, at least the end position 30 of the movement of the user's eye 18 around the rotation axis 14 is obtained by means of the LFI sensor assembly 26 of the data glasses 12 or the eye observation camera 32 of the data glasses 12 (see Figure 6 ) is used to determine the current Listing plane 16. In method step 20, based on the detected rotation axis 14 of the movement of the user's eye 18 and based on the detected starting position and end position 28, 30 of the movement of the user's eye 18, a movement trajectory 34, 34' (see Figure 6 In method step 20 , the surface velocity of the user's eye 18 is detected by means of the LFI sensor assembly 26 of the data glasses 12 and evaluated to determine the movement trajectory 34 , 34 ′.

[0049] In at least one further method step 22, the current convergence of the user's eye 18 is determined based on the measured current Listing plane 16 of the user's eye 18. In this further method step 22, the current convergence of the user's eye 18 is determined based on the Listing plane angle 36 (see ) between the measured current Listing plane 16 and the initially determined and / or calibrated Listing plane 38 for the main position 40 of the user's eye 18 looking towards infinity. Figure 5b In this further method step 22 , the current convergence is read out from a known characteristic curve calibrated in a preliminary stage, in which the convergence is plotted against the variation of the Listing plane angle 36 .

[0050] In at least one additional further method step 24, the user-specific current depth of focus 10, 10', 10" of the user eye 18 is determined based on the current convergence of the user eye 18. In this additional further method step 24, the user-specific current depth of focus 10, 10', 10" of the user eye 18 is determined by applying the user calibration data determined in one of the user calibration steps 42, 56 or in the two initial user calibration steps 42, 56, and different convergences are associated with different user-specific depths of focus 10, 10', 10" of the user eye 18 via these user calibration data.

[0051] In at least one further method step 70, the initial calibration from the user calibration step 42, 56 is continuously tracked. This is conceivable, for example, during the reading of content in different distance planes 52, 52', in order to advantageously enable tracking of the calibration function when the data glasses 12 are sliding. Conceivable applications of the above method include, for example, operating the user interface (UI) of the data glasses 12 in a monocular tracking system; redundantly measuring the focal depth 10 in a stereo system to account for visual defects such as nystagmus or strabismus; or adapting a virtual retinal display of the data glasses 12, such as a VR headset, based on the eye position (glass misalignment).

[0052] Figure 4 A schematic side view of the data glasses 12 and a real calibration object 44 is shown during a user calibration step 42. Two different distances 62, 62' of the real calibration object 44 from the data glasses 12 are shown by way of example.

[0053] Figure 5aThe user's eye 18 is shown schematically looking towards infinity and the associated (untilted) listing plane 38. All possible rotation axes 14 of the user's eye 18 when moving from an eye position looking towards infinity lie in this listing plane 38. Figure 5a In FIG, the user's eye 18 is in the primary position 40. The primary position 40 is determined by Figure 5a 38. The vector of the primary position 40 corresponds to the direction of sight 64 of the user's eye 18 when looking towards infinity.

[0054] Figure 5b The current Listing plane 16 of a user eye 18 that is not looking towards infinity, i.e., a focused user eye 18, is schematically shown. The current Listing plane 16 is tilted compared to the Listing plane 38 of the eye 18 that is looking towards infinity. The tilt angle of the Listing plane 16 compared to the Listing plane 38 of the main position 40 is half the angle between the gaze direction 64' of the focused eye 18 and the main position 40 (the vector representing the main position 40). All possible rotation axes 14' of eye movements starting from this point are located in the current Listing plane 16 of the focused user eye 18. The Listing plane 16 of the focused user eye 18 forms a half-angle plane of the angle between the gaze direction 64' of the focused user eye 18 and the gaze direction 64 of the user eye 18 that is looking towards infinity.

[0055] Figure 6The schematic diagram shows exemplary movement trajectories 34, 34' of a user's eye 18 with different main positions 40, 40'. The rotation axes 14, 14' differ depending on the associated main position 40, 40'. The movement trajectories 34, 34' differ depending on the associated main position 40, 40'. In the illustrated case, the eye center points 66 of the two main positions 40, 40' are superimposed one on top of the other. The rotation of the associated user's eye 18 takes place around each eye center point 66. In the case of the first movement trajectory 34, the main position 40 corresponds to the starting point for the movement along this movement trajectory 34 and the starting point for the direction of sight 64 of the user's eye 18 looking toward infinity. The movement trajectory 34 to the target point 68 of the movement of the user's eye 18 follows from the associated rotation axis 14. In this case, the movement trajectory 34 corresponds to a great circle of the user's eye 18, which is assumed to be spherical. In the case of the second movement trajectory 34', the main position 40' does not correspond to the starting point for the movement along this movement trajectory 34'. However, the starting point for the movement along both illustrated movement trajectories 34, 34' is the same. In the case of the second movement trajectory 34', the rotation axis 14' lies in the half-angle plane (angle bisector plane) between the main position 40' and the starting point for the movement along the second movement trajectory 34'. Therefore, the second movement trajectory 34' describes a different route to reach the same destination point 68 as the first movement trajectory 34. Thus, the movement trajectories 34, 34' for the same starting point and the same destination point 68 vary depending on the convergence of the user's eyes 18, and thus depending on the main positions 40, 40'.

Claims

1. A method for determining the current depth of focus (10, 10', 10") of a user (54) of data glasses (12), in particular by means of at least a monocular tracking system, wherein: In at least one method step (20), a current listing plane (16) of a user's eye (18, 18') is measured, wherein, in at least one further method step (22), a current convergence of the user's eye (18, 18') is determined based on the measured current listing plane (16) of the user's eye (18, 18'), and wherein, in at least one additional further method step (24), a user-specific current depth of focus (10, 10', 10") of the user's eye (18, 18') is determined based on the determined current convergence of the user's eye (18, 18').

2. The method according to claim 1, wherein: In the method step (20), the current listing plane (16) is determined based on at least a measurement of the rotation axis (14) of the movement of the user's eyes (18, 18'), in particular by means of a laser feedback interferometer (LFI) sensor assembly (26).

3. The method according to claim 2, wherein: In the method step (20), the current listing plane (16) is determined at least based on a measurement of a starting position (28) of the movement of the user's eyes (18, 18') around the rotation axis (14) and / or based on a measurement of an ending position (30) of the movement of the user's eyes (18, 18') around the rotation axis (14), in particular by means of the laser feedback interferometer (LFI) sensor assembly (26) of the data glasses (12) or by means of an eye observation camera device (32) of the data glasses (12).

4. The method according to claim 2 or 3, characterized in that: In the method step (20), a movement trajectory (34, 34') of the movement of the user's eye (18, 18') is determined based on the detected rotation axis (14) of the movement of the user's eye (18, 18') and based on the detected starting position and end position (28, 30) of the movement of the user's eye (18, 18').

5. The method according to any one of claims 2 to 4, characterized in that: In the method step (20), the surface velocity, in particular the surface rotation velocity, of the user's eye (18, 18') is detected by means of the LFI sensor assembly (26) of the data glasses (12), and this surface velocity is evaluated to determine the rotation axis (14) and / or the movement trajectory (34, 34').

6. The method according to any one of the preceding claims, characterized in that: In the further method step (22), the current convergence is determined based on a Listing plane angle (36) between a measured current Listing plane (16) and a Listing plane (38) for the main position of the user's eyes (18, 18') looking towards infinity, in particular initially determined and / or calibrated.

7. The method according to claim 6, characterized in that: In the further method step (22), the current convergence is read out from a known, in particular calibrated in a preparatory phase, characteristic curve, in which the convergence is plotted against the change in the listing plane angle (36).

8. The method according to any one of the preceding claims, characterized in that: In the additional further method step (24), a current user-specific depth of focus (10, 10', 10") of the user's eye (18, 18') is determined using initial user calibration data, by which different convergences are associated with different user-specific depths of focus (10, 10', 10") of the user's eye (18, 18').

9. The method according to claim 8, characterized in that: In a user calibration step (42) chronologically preceding the additional further method step (24), the initial user calibration data are determined by observing at least one moving real calibration object (44) by means of a world camera (46) through the data glasses (12) and by simultaneously measuring the convergence of the user's eyes (18, 18') following the moving real calibration object (44) and focusing on the moving real calibration object (44) by means of an LFI sensor assembly (26) of the data glasses (12) or an eye observation camera (32) of the data glasses (12), in particular The determination is performed by evaluating known geometrical parameters of the moving real calibration object (44) at different positions of the moving real calibration object (44) and at different distances (62, 62') from the world camera (46) and simultaneously assigning these parameters to the measured convergence, and / or by determining different positions of the moving real calibration object (44) and at different distances (62, 62') from the world camera (46) with the aid of a stereo system and / or a structured light system (48) and simultaneously assigning these positions and distances to the measured convergence.

10. The method according to claim 9, characterized in that: In the user calibration step (42), the real calibration object (44) is moved at different distances (62, 62') from the world camera (46) in a distance plane (52) at least substantially parallel to an image plane (50) of the world camera (46).

11. The method according to claim 10, characterized in that: The movement of the real calibration object (44) is performed manually by the user (54) in the user calibration step (42), wherein the movement of the real calibration object (44) is monitored by the world camera device (46) and movement instructions and / or movement commands are determined based on the monitoring data detected therein and are output to the user (54) acoustically and / or optically by the data glasses (12).

12. The method according to any one of claims 8 to 11, characterized in that: In an alternative or additional user calibration step (56) which temporally precedes the additional further method step (24), the initial user calibration data are determined by displaying a moving virtual calibration object through the display of the data glasses (12) and by simultaneously measuring the convergence of the user's eyes (18, 18') following the moving virtual calibration object (44) and focusing on the moving virtual calibration object (44) through the LFI sensor component (26) of the data glasses (12) or the eye observation camera (32) of the data glasses (12).

13. The method according to claim 12, wherein: In the alternative or additional user calibration step (56), a virtual calibration object is displayed and moved via the display of the data glasses (12) at different focal lengths, in particular those generated by the optical system of the data glasses (12).

14. A calculation unit (58) for determining a current depth of focus (10, 10', 10") of a user (54) of data glasses, wherein: The calculation unit (58) is at least configured to determine a current convergence of the user's eye (18, 18') based on a current listing plane (16) measured by the user's eye (18, 18'), in particular at least measured by means of a monocular tracking system, wherein the calculation unit (58) is at least configured to determine a current user-specific depth of focus (10, 10', 10") of the user's eye (18, 18') based on the determined current convergence of the user's eye (18, 18').

15. Data glasses (12) having at least one internal or external computing unit (58), in particular a computing unit according to claim 14, and having a laser feedback interferometer (LFI) sensor assembly (26), characterized in that: The data glasses (12), in particular the computing unit (58), are provided at least for determining a current depth of focus (10, 10', 10") of a user (54) of the data glasses (12) by means of a method according to any one of claims 1 to 13.

Images