Apparatus and method for determining pupil position

By employing optical equipment and methods, utilizing laser feedback interferometry and tilted photodetectors, the problem of eye tracking interference caused by user head movement was solved, achieving robust pupil position and line-of-sight vector determination, which is suitable for stable tracking in data glasses.

CN119053899APending Publication Date: 2024-11-29ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380034896.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-20
Filing Date
2023-03-02
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively compensate for interference caused by user head movements when using optical methods to determine the pupil and line-of-sight vectors, especially in data glasses where eye-tracking interference caused by slippage is difficult to resolve.

Method used

An optical system comprising a laser, first and second photodetectors, an adjustable beam deflection device, and a control unit is employed. By using laser feedback interferometry and a tilted second photodetector, combined with an analysis and processing unit, robust determination of the pupil position and line-of-sight vector is achieved.

Benefits of technology

It achieves robust determination of pupil position, can compensate for the relative motion of optical devices with respect to the user's head, is suitable for long-term stable eye tracking in data glasses, reduces computational workload, and improves the accuracy and stability of eye tracking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119053899A_ABST
    Figure CN119053899A_ABST
Patent Text Reader

Abstract

The invention relates to an optical device (100) for determining a pupil position of an eye (10, 210, 410), comprising a laser (20), a first photodetector (30), a second photodetector (40), an adjustable beam deflection device (50), and a control unit (150) for actuating the adjustable beam deflection device (50), the optical device (100) is designed to guide light generated by the laser (20) at least partially as illumination light (60) via an illumination beam path (110) to the beam deflection device (50) and to deflect the illumination light (60) by adjusting the beam deflection device (50) by the control unit (150). Illuminating a plurality of illumination points (90, 290) on the eye (10, 210, 410) and an eye environment (70, 270, 417) of the eye (10, 210, 410) by means of the illumination light (60), during illumination of at least one first portion (90a) of the illumination points (90, 290), the laser power of the laser (20) during illumination of the illumination point (90, 290) is measured by means of the first photodetector (30a, 30c) and / or illumination light (60a) which is reflected at the interface of the eye (10, 210, 410) or of the eye environment (70, 270, 417) and enters the illumination beam path (110) via the beam deflection device (50) is detected by means of the first photodetector (30a, 30b), during illumination of at least one second portion (90b) of the illumination point (90, 290), illumination light (60b) scattered on the eye (10, 210, 410) or on the eye environment (70, 270, 417) is detected by means of the second photodetector (40). The invention further relates to a method for determining a pupil position of an eye (10, 210, 410) using an optical device (100) and to data glasses (130) having a corresponding optical device (100) for determining a pupil position of an eye (10, 210, 410).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an optical device and a method for determining the position of a pupil. In addition, the invention relates to data glasses based thereon. Background Art

[0002] Different schemes for optically determining the position of the eye, pupil and sight vector are known from the prior art. Thus, for example, cameras and optical sensors are used, possibly in combination with software for image processing. The purpose of application of these methods is to use them in the field of sight detection (eye tracking) in data glasses (smart glasses) in order to monitor the eye movements of users of such data glasses. A big problem when using an optical system to determine the eye position, pupil position and / or sight vector is the movement of the user's head relative to the optical system used by the user. For example, the use of eye tracking in data glasses. In this case, unintentional slipping of data glasses can cause strong interference during eye tracking. For this reason, schemes for determining and compensating for such slipping are known from the prior art. Thus, a detection and correction of possible slipping of data glasses by means of a video camera is proposed in US10,264,964B1, and a detection and correction of possible slipping of data glasses using a motion sensor is proposed in US2015 / 0995A1. Summary of the invention

[0003] According to the invention, an optical device for determining the position of a pupil, a method for determining the position of a pupil and a pair of data glasses having an optical device according to the invention are provided. Advantageous embodiments are the subject matter of the dependent claims and of the following description.

[0004] According to a first aspect of the present invention, an optical device for determining the pupil position of an eye is proposed, wherein the eye has a retina, wherein the device comprises a laser, a first photodetector, a second photodetector, an adjustable beam deflection device and a control unit, the control unit being used to control the adjustable beam deflection device. The control unit may comprise digital components and / or analog components, such as a computer, a microprocessor, an ASIC (application-specific integrated circuit) and / or an FPGA (Field Programmable Gate Array). The pupil position in the sense of the present invention is understood to be the spatial position of the pupil of the eye. In particular, the pupil position may be the relative position of the pupil relative to the eye environment. In the scope of the present invention, the eye environment of the eye is an area of ​​the head associated with the eye, and may include any element of the head, the position of which is independent of the movement of the eye, i.e., for example, the skin, fibers and / or hair of the head. The device according to the present invention and the method according to the present invention are not limited to the determination of the pupil position of a person, but may also be used for animals.

[0005] Here, the optical device is designed to guide the light generated by the laser at least partially as illumination light via an illumination beam path to a beam deflection device, and by adjusting the beam deflection device by a control unit, a plurality of illumination points on the eye and the eye environment (Augenumgebung) of the eye are illuminated by the illumination light. That is, the eye and the eye environment are scanned by means of the beam deflection device. The beam deflection device can be designed in such a way that it can move the illumination light in one or preferably two spatial directions. Not only pulsed lasers but also continuous wave lasers (CW lasers) can be used as lasers. The density of the illumination points can be arbitrary, i.e., a continuous line can also be formed. This can be the case, for example, when a corresponding high-frequency pulsed laser or a CW laser is used as the laser.

[0006] Furthermore, the optical device is designed to measure the laser power of the laser by means of the first photodetector during the illumination of at least a first part of the illumination points, for example the illumination points on the eye or all illumination points. For this purpose, it is not necessary to determine the absolute value of the laser power: Advantageously, the difference in the laser power between the different illumination points is detected, i.e. the change in power when the illumination switches from the first illumination point to the second illumination point. In other words, the device is advantageously designed to determine the change in the laser power when switching from the first illumination point to the second illumination point by evaluating the electrical output signal of the first photodetector.

[0007] In particular, by measuring the laser power of the laser, it can be determined whether (in the case of substantially vertical incidence) the illumination light of the illumination point is reflected on the retina of the eye: if in the case of substantially vertical light incidence the illumination light is reflected on the interface of the eye, for example on the retina of the eye, and then reaches the laser for illumination via the beam deflection device and the illumination beam path (beam path for illumination) in the opposite direction to the illumination direction (i.e. the direction from which the illumination is carried out) and there enters the laser resonator of the laser, then the reflected light interferes with the light generated in the laser. Typically, this leads to amplitude changes and frequency changes of the emitted laser light. Therefore, the occurrence of such interference can be detected based on the changes in the laser power. Such a measurement method is called laser feedback interferometry or self-mixing interferometry. In a variant of the device according to the invention, a first photodetector is used for this purpose, which measures the intensity of the light generated by the laser and thus measures the laser power.

[0008] Since the retina (compared to other possibly illuminated interfaces of the eye and the eye environment) reflects light, in particular infrared light, particularly well, it is possible to identify with high reliability those illumination points in which light is reflected from the retina back into the illumination beam path. From this information, the current position of the pupil of the eye can be inferred. Alternatively or additionally, the optical device can also be designed so that when illuminating at least the first part of the illumination point, the illumination light reflected on the eye or the eye environment (depending on the orientation of the respective illumination point) and entering the illumination beam path via the beam deflection device is detected directly by means of the first photodetector. Since the retina has a certain optical roughness, a speckle pattern typically appears when illuminating with the aid of laser light, correspondingly, light is not reflected from every point of the illuminated retina back into the illumination beam path. However, the manifestation of such a pattern For the solution according to the invention, this is not disruptive but, on the contrary, can even help to identify the position of the pupil. In addition, the device is designed so that during the illumination of at least a second part of the illumination points (e.g., the illumination points on the eye environment or all illumination points), the illumination light scattered on the eye (illumination points on the eye) or on the eye environment (illumination points on the eye environment) can be detected by means of a second photodetector.

[0009] The laser used as a light source can be an infrared laser, which is preferably used for light with a wavelength in the range of 800 nm to 1050 nm, particularly preferably 850 nm or 940 nm, and the infrared laser is preferably in the form of a laser diode, such as an infrared laser diode and / or a surface emitter (VCSEL, Vertical-Cavity Surface-Emitting Laser). Lasers and laser diodes that emit in the infrared are advantageous for the implementation of the present invention, since the retina reflects particularly well in the infrared. For example, the first photodetector and / or the second photodetector can be a photodiode, a CMOS sensor (CMOS: Complementary metal-oxide-semiconductor) and / or a CCD sensor (CCD: charge-coupled device). It is particularly advantageous that, in the case where a surface emitter is used as a light source, the surface emitter has a monitor diode, such as a vertically integrated monitor diode, which is integrated in a rear reflector ( Spiegel). In this case, the integrated monitoring diode can be identical to the first photodetector. In addition, the adjustable beam deflection device can include one or more mirrors, in particular micromirror actuators and / or galvanometer scanners and / or resonant scanners (scanning mirrors operated at a resonant frequency).

[0010] The optical device according to the invention may also include at least one further optical element, such as a reflector and / or a prism and / or a HOE (holographic optical element), which is designed to redirect the illumination light after passing through the beam deflection device. The further optical element can be designed in such a way that the illumination light from the beam deflection device is deflected in such a way that the eye and the eye environment can be scanned as desired by means of the beam deflection device, i.e., for example, the desired position of the illumination point can be reached. In this way, it is conceivable that the further optical element is a curved reflector or a curved HOE, which is adapted to the shape of the eye and the optical properties of the eye in such a way that for the desired illumination point, the illumination light is reflected on the retina and enters the illumination beam path. Therefore, the further optical element allows the optimization of the optical device to improve the detection quality and also allows the beam deflection device to be prevented more independently of the position of the eye and the eye environment, which allows a freer design of the optical device. In particular, when the optical device according to the invention is used in data glasses, it is conceivable that such a further optical element is part of a typically curved lens of the data glasses or a corrective lens for such data glasses and / or is embedded in the lens or the corrective lens.

[0011] It is also advantageous that the optical device includes an analysis unit, which is configured to process and analyze (i.e. perform signal analysis) the electrical output signals of the first photodetector and the second photodetector (hereinafter referred to as detector signals or simply signals) and determine the pupil position and / or the sight vector from the signals. Here, the analysis unit can form a unit with the control unit or be identical to the control unit. The term signal analysis in the sense of the present invention should be understood as all types of analog signal processing and digital signal processing and analysis of signals. In this way, the signal analysis of the photodetector can include signal amplification, digitization, filtering and / or data fusion, but can also include steps for generating data, information and other signals derived from the data thus obtained. The analysis unit may include digital components and / or analog components. For example, the analysis unit may include one or more amplifiers, bandpass filters, A / D converters and / or computing units, such as computers, microprocessors, ASICs and / or FPGAs. In particular, the evaluation unit can include a multiplexer, by means of which it is possible to switch between the signal of the first photodetector and the signal of the second photodetector for the purpose of evaluating the signals. Likewise, the evaluation unit can include an image processing device, for example in the form of a computer, which is set up to generate a first image from the possibly preprocessed signal of the first photodetector and a second image from the signal of the second photodetector. Based on these images, a data fusion can be performed in order to determine the pupil position and in particular also the gaze vector. In contrast to many other related methods in the prior art, it is not necessary to determine a coordinate transformation based on consistent image features between the images generated from the signals of the two photodetectors, since both images are based on the same illumination point.

[0012] It is also particularly advantageous if the first photodetector is arranged in such a way that a portion of the light generated by the laser is detected by the first photodetector. Such a configuration of the device according to the invention enables laser feedback interferometry and thus enables the detection of fluctuations in the power of the laser, which fluctuations are generated by the illumination light entering the resonator of the laser, which illumination light is reflected back from the retina of the eye into the illumination beam path, for example. In the case of the second photodetector, it is also particularly advantageous if the second photodetector is arranged in such a way that for each illumination point, the optical axis of the second photodetector is at an angle of ≥5°, preferably ≥10°, particularly preferably ≥20° to the direction from which the illumination point is illuminated by the optical device, i.e., the illumination direction. By such an inclined arrangement, it is particularly avoided that light reflected by the retina is detected by means of the detector, which light could falsify the signal of the second photodetector. In this case, the optical axis of the second photodetector corresponds to the detection direction for the light to be detected and is typically perpendicular to the photosensitive surface of the second photodetector, which is used to detect the light. By such an arrangement of the second photodetector, the pupil of the eye appears dark in the image generated based on the signal data of the second photodetector, because the retina is shaded for the second photodetector due to the aperture of the pupil. Illumination light reflected or scattered on the retina cannot reach the second photodetector.

[0013] According to a second aspect of the present invention, a method for determining the pupil position of an eye using an optical device is provided, wherein the optical device comprises a laser, a first photodetector, a second photodetector, an adjustable beam deflection device, a control unit and an analysis unit, the control unit being used to control the adjustable beam deflection device, and the eye having a pupil and a retina. Preferably, the optical device is an optical device for determining the pupil position according to the present invention as described above. The method according to the present invention comprises the following steps:

[0014] Light is generated by a laser and is guided at least partially as illumination light via an illumination beam path to a beam deflection device. Thus, by adjusting the beam deflection device by a control unit, a plurality of illumination points on the eye and the ocular environment of the eye are illuminated by the illumination light, wherein the beam deflection device assumes different positions for different illumination points. Here, for a first portion of the illumination points, the illumination light is reflected back to the beam deflection device at the retina of the eye (i.e. opposite to the corresponding illumination direction). From the beam deflection device, the light may return via the illumination beam path to the laser and enter the resonator of the laser. For a second portion of the illumination points, the illumination light is scattered on at least one reference mark of the ocular environment. Here, a reference mark of the ocular environment is any feature of the ocular environment (ocular feature) that can be identified by evaluating the signal of the second photodetector and is suitable for determining a relative position in the ocular environment relative to the position of the ocular feature. The reference mark can be a single site, but also a larger structure. For example, skin blemishes, eyelids, eyelashes, lacrimal sacs or specific sites within these elements can be used as reference marks. It is advantageous if the reference marks used are as static as possible relative to the rest of the head, i.e. the position of the reference marks is the best possible indicator of the position of the head. Advantageously, either non-rotationally symmetrical reference marks can be used and / or, in particular in the case of point-shaped or rotationally symmetrical reference marks, two, three or more reference marks can be used. This has the advantage, on the one hand, that even possible rotations of the optical device and thus reference marks (not just translations) can be identified. On the other hand, errors and inaccuracies in the position of the individual reference marks can also be compensated or minimized with the aid of a plurality of reference marks.

[0015] Furthermore, when illuminating at least the first part of the illumination point, the laser power of the laser is measured by means of a first photodetector so that the reflection of the illumination light can be determined by means of laser feedback interferometry. Alternatively or additionally, the illumination light reflected at the interface of the eye or the eye environment is detected by means of the first photodetector. Subsequently, the signal of the photodetector is evaluated by means of an evaluation unit. Here, the corresponding position of the beam deflection device is also assigned to the signal in order to achieve a spatial distribution. When illuminating at least the second part of the illumination point, the illumination light scattered on the eye or the eye environment is detected by means of a second photodetector, and the signal of the second photodetector is evaluated by means of an evaluation unit. Here, the signal is also respectively assigned to the position (Stellungen) of the beam deflection device. In the scope of the signal evaluation for different positions of the beam deflection unit, in particular, an image of the eye and / or the eye environment can be created. In this way, when the position of the beam deflection unit is known, an image can be generated not only based on the signal data of the first photodetector, but also based on the signal of the second photodetector. The position of the beam deflection device for an illumination point can either be derived from a control signal of a control unit, which is used to control the beam deflection device. Alternatively or additionally, it is also conceivable that the beam deflection device directly provides this information, i.e., it can be measured during illumination, for example by means of a PSD (Position Sensitive Device) mounted on the back side of a reflector of the beam deflection device in combination with a light source such as a laser diode. In this case, the position of the beam deflection device can be given in any units and in any form.

[0016] Finally, the position for at least one reference mark is determined by the evaluation unit based on the evaluated signal of the second photodetector. Taking into account the position of the at least one reference mark, the pupil position is determined by the evaluation unit based on the evaluated signal of the second photodetector. Thus, in the simplest case, for example within the scope of signal evaluation, the uncorrected pupil position p' is determined from the signal of the first photodetector taking into account the position of the beam deflection device, and the position r of the reference mark is determined from the signal of the second photodetector, likewise taking into account the position of the beam deflection device. The uncorrected pupil position is now shifted by the position of the reference mark in order to obtain the sought pupil position p, which specifies the position of the pupil relative to the reference mark: p=p'-r. In other words, the pupil position defined in this way specifies the position relative to a fixed point of the head, i.e. the reference mark. If, during repeated illumination of the illumination point, i.e. during repeated scanning of the eye and the eye environment, the position of the optical device used relative to the head and therefore relative to the eye environment changes, not only the uncorrected position p' of the pupil but also the position r of the reference mark changes, but the vector p remains the same for identical pupil positions relative to the eye environment. Preferably, in a more complex implementation of the method according to the invention, a plurality of reference marks are used or a rotation of the reference marks is taken into account in addition to their positions. This makes it possible to take into account not only possible movements between the optical device and the eye environment but also rotations relative to one another.

[0017] Therefore, since there is not only information about the pupil but also about the head, within the scope of the method according to the invention, for example, eye movements and / or the sight vector of the eye can also be calculated (given the relationship between the position of the pupil relative to the reference mark and the sight vector). Within the scope of the invention, a sight vector is an arbitrary vector that indicates the direction of the sight, which, for example, extends through the center of the pupil and is oriented away from the eye. Eye movement is an arbitrary movement of the eye relative to the eye environment. Typically, an eye movement corresponds to a change in the direction of the sight of the eye. During an eye movement, the eye rotates in the eye socket. Therefore, the invention is based on the fusion of two detector signals generated in different ways, i.e., this involves a correlation method: on the one hand, the pupil position is determined from the signal of a first photodetector, and on the other hand, the position of the reference mark is determined from the signal of a second photodetector. Since both methods use the same illumination beam path and the same beam deflection device, the signals can be spatially correlated with each other in an uncomplicated manner in order to determine the pupil position relative to the reference mark. This also applies when the signals of the first photodetector and the signals of the second photodetector are not recorded simultaneously for all illumination points or for a part of the illumination points.

[0018] Taking into account the respective positions of the beam deflection device for each detector signal (signal of the photodetector), the detector signals can be spatially correlated with one another so that the following position can be determined for the pupil of the eye: this position indicates the relative position of the pupil relative to the reference mark. Thus, here, the position of the reference mark can be used to define a reference point so that after a translation as a relative movement of the optical device relative to the head of the user of the optical device, the pupil position can be determined independently of this relative movement. In the case of a relative movement that also includes rotation, it is also necessary to detect the rotation of a reference mark or to use multiple reference marks so that the pupil position relative to the reference mark can be clearly and uniquely identified. In the context of analyzing the signal of the first photodetector, a first image can be generated based on the signal of the first photodetector. The same applies to the signal of the second photodetector: here, image generation can also be achieved. That is, the analyzed image includes these images or is identical to these images. In the case of images generated at least partially for the same position of the beam deflection device from the signals of the first photodetector and the second photodetector (i.e., for at least a portion of the illumination points, the signals are recorded not only by the first photodetector but also by the second photodetector), these images can be superimposed accordingly in order to merge the information from the two images. An example of this is to determine the pupil position and / or the pupil diameter of the pupil while taking into account both the signals of the first photodetector and the second photodetector in order to reduce inaccuracies, for example by superimposing the corresponding images. However, without generating an image from the signals, it is also conceivable to cancel out the signals of the two photodetectors for one or more illumination points.

[0019] In a particularly advantageous embodiment of the method according to the invention, the beam deflection device is adjusted in such a way that the illumination points are arranged along a substantially sinusoidal or meandering or sawtooth illumination pattern. The resulting line-shaped scanning of the eye and the eye surroundings makes it particularly easy to create an image from the signals of the photodetector, since dense coverage of the eye and the eye surroundings by the illumination points can thereby be achieved with a uniform distribution.

[0020] Advantageously, the method according to the invention is used in such a way that for each determined illumination point the signal of the first photodetector is recorded. That is, if the illumination point covers the entire eye and a determined area around the eye and if the illumination point is repeatedly illuminated by means of a beam deflection device, the pupil position can be tracked continuously. The signal evaluation required for this is computationally inexpensive, since the retina can be easily identified by corresponding signal changes due to its high reflectivity. Typically, the movement of the optical device relative to the head of the user of the optical device occurs slowly and rarely. That is, the position of the reference mark relative to the optical device also changes correspondingly slowly or rarely. That is, it is advantageous to detect and evaluate the signal of the second photodetector only in certain time intervals and / or for certain illumination points (for example after a plurality of full scans, i.e. after a plurality of complete scans of all determined illumination points), and on this basis to (possibly re) correct the uncorrected pupil position determined by means of the signal of the first photodetector.

[0021] According to a third aspect, the present invention relates to data glasses with an optical device according to the invention for determining the pupil position of an eye. Preferably, such eyes are equipped with additional laser light sources, such as laser diodes, for generating images for the user of the data glasses. These additional laser light sources can be arranged in a laser module together with the laser of the device according to the invention. The image construction of the data glasses can also be carried out via the same optical illumination beam path, wherein the same beam deflection device can also be used. In this regard, the present invention enables a particularly robust solution for correcting the slipping of the data glasses when determining the pupil position, for example in the context of eye tracking. In the case of the data glasses according to the invention, the optical device according to the invention or a part of the optical device (such as a second photodetector) can be integrated, for example, in the glasses frame, in one of the glasses legs, in the bridge of the nose or in the nose pads and / or arranged there. It is also conceivable to integrate at least part of the components into the glasses of the data glasses.

[0022] Advantages of the present invention

[0023] By evaluating the illumination light reflected on the retina of the eye by means of a first photodetector, the pupil can be directly recorded and simply detected in the detector signal. Unlike, for example, video recording, no complex image processing methods are required, and in the simplest case, due to the particularly sharp reflectivity of the retina, it is sufficient to simply consider the signal intensity. Since only the light entering the illumination beam path via the deflection device is detected, the image is particularly robust to interfering light. This is the case in particular when using laser feedback interferometry, because here, on the one hand, the light must enter the resonator of the laser in order to be detected, and on the other hand, there must also be coherence of the light in order to be able to cause corresponding interference effects. Therefore, with this processing method, an extremely simple and efficient determination of the (uncorrected) pupil position is achieved. However, such a solution has the disadvantage that no information is obtained about where the pupil is located relative to the eye. On the one hand, a change in the pupil position can be achieved by changing the eye position, but on the other hand, it can also be achieved by a movement of the optical device relative to the eye environment, which optical device is, for example, data glasses that implement this solution. Therefore, an error-free calculation of the sight vector is also not possible.

[0024] It is now recognized that the procedure according to the invention provides a surprisingly simple and elegant possibility: by means of the second photodetector arranged obliquely according to the invention, information about the eye environment is obtained in such a way that it can be fused in a simple manner with the information obtained by the first photodetector in order to compensate for undesired relative movements between the optical device and the eye environment. In this context, it should be pointed out that the use of such a second photodetector alone would be non-targeted, since for such an implementation, a computationally complex method for detecting the pupil is required.

[0025] By means of the solution according to the invention, a robust determination of the pupil position with respect to interfering light can be achieved, which determination is at the same time stable with respect to the relative movement of the optical device according to the invention with respect to the head of the user of the device. This is particularly advantageous, for example, when the invention is used in data glasses, since the data glasses may easily slip off, but at the same time stable and error-free eye tracking over a longer period of time is generally required. Since it is not necessary to continuously perform position corrections with the aid of the second photodetector, but rather to perform them at larger intervals, there is no need to increase the computational workload in particular. An essential point of the invention is that the same beam deflection device is used for the two measurement methods used. As a result, the acquired data can be easily correlated with each other without, for example, having to determine the spatial correlation of the data in a complex manner (for example, by identifying consistent image features in the images of the first and second photodetectors and then performing a coordinate transformation). On the contrary, a relatively small data set obtained from the second photodetector signal is sufficient for identifying the reference mark, so that the reference mark can be reliably identified and its position and possible rotation can be determined. There is no need to detect and analyze the entire eye and the larger environment of the eye with the aid of the second photodetector. The present invention therefore discloses a particularly robust and at the same time structurally simple approach for determining the pupil position and the gaze vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Embodiments of the present invention are explained in more detail with reference to the drawings and the following description.

[0027] The accompanying drawings show:

[0028] Figure 1 Schematic diagram showing an optical device according to the present invention and data glasses according to the present invention

[0029] Figure 2 Schematic diagram showing a possible arrangement of lighting points

[0030] Figure 3 Flow chart showing a method including signal analysis processing according to the present invention

[0031] Figure 4 Schematic diagram showing the determination of a sight vector using the method according to the invention DETAILED DESCRIPTION

[0032] In the following description of embodiments of the invention, identical or similar elements are denoted by the same reference numerals, wherein a repeated description of these elements is omitted in individual cases. The figures merely schematically illustrate the subject matter of the invention.

[0033] Figure 1Purely schematically, an exemplary embodiment of an optical device 100 according to the invention for determining the pupil position of an eye 10 is shown, and data glasses 130 according to the invention having such an optical device 100 are shown, the optical device having a laser 20, which is part of a laser module 120, a first photodetector 30, which is, for example, a photodiode, a CMOS sensor and / or a CCD sensor, a second photodetector 40, which is, for example, a photodiode, a CMOS sensor and / or a CCD sensor, a control unit for actuating the adjustable beam deflection device 50. Figure 1 Components are shown in dashed lines in the figure: these components can form an alternative to certain other components shown, or can also be used in addition to these components. Details in this regard can be found in the following description of the figures.

[0034] To generate an image, a further laser 22 can be located in the laser module 120. For example, the laser 20 can be an infrared laser diode. The light generated by the laser 20 is guided via a mirror 24 through three beam splitters 26 and then via a further mirror 142 via the illumination beam path 110 to the beam deflection device 50. Here, the mirror 142 also represents other possible optical elements in the beam path, such as mirrors, lenses, filters, prisms, HOEs, etc. The eye 10 shown includes a sclera 16, a cornea 18, a choroid 19, a lens 12, an iris 13 surrounding the pupil 12, and a retina 14. The optical device 100 shown can be installed, for example, in data glasses 130, wherein in the data glasses 130 shown, the components of the optical device (laser module 120, first photodetector 30, control unit 150 and evaluation unit 180) are placed in one of the temples 132. These and further elements, such as the beam deflection device 50 , the second photodetector 40 and the further optical element 42 , can, for example, also be integrated in and / or on the spectacle frame 134 , the nose bridge 136 , the nose pads and / or the spectacle lenses 138 .

[0035] The light used as illumination light 60 is redirected by the beam deflection device 50 to the eye 10 and the eye environment 70 with the aid of an optical element 140. The optical element 140 can be, for example, a reflector or an HOE. Purely exemplary beam paths are shown for a simple case of an optical element 140, which works like a plane reflector. These beam paths are shown in a greatly simplified manner and should only be used to illustrate the principle of the present invention. The optical element 140 can also have a non-planar shape. For example, in the case of a curved HOE as the optical element 140, it is also possible to redirect the beam in a converging manner to the eye 10 and the eye environment 70 in the case of an oblique beam incidence. In particular, it is conceivable that the optical element 140 is part of a typical curved eyeglass 138 or a corrective lens for data glasses 130 and / or is embedded in the eyeglass 138 or the corrective lens.

[0036] The beam deflection device 50 is controlled via a control unit 150, for which the signals required are sent from the control unit 150 to the beam deflection device 50 via a cable 52. Figure 1 In the embodiment of the present invention, the beam deflection device 50 is represented as a single mirror, but it can include a plurality of mirrors. By correspondingly adjusting the beam deflection device 50 (indicated by the double-headed arrow 55), a plurality of illumination points 90 can be implemented on the eye 10 and the eye surroundings 70. In particular, it is possible to realize, for example, a complete scan of the eye 10 with the aid of the beam deflection device 50. Figure 1 , only a one-dimensional scan is shown in a simplified manner; however, the beam deflection device 50 is advantageously designed in such a way that a two-dimensional scan of the eye 10 and its eye surroundings 70 can be achieved.

[0037] If the illumination light 60 is incident on the illumination point 90 on the eye 10, it is possible that the illumination point is such that (this is given for the illustrated illumination point 90a) a portion of the illumination light 60 passes through the cornea 18, the pupil 12 and the eye lens 15 to the retina 14 of the eye 10 and is reflected there again in the opposite direction to the illumination direction. The illumination light 60a reflected on the retina 14 enters the illumination beam path 110 at least partially via the optical element 140 and the beam deflection device 50. Information about the position of the beam deflection device 50 is transmitted to an evaluation unit 180, which can form a unit together with the control unit 150 or can be identical to the control unit. This information can come from the control unit 150 itself (cable connection 54a) and correspond to target values, with the aid of which the beam deflection device is also actuated. Alternatively or additionally, the position of the beam deflection device 50 can also be determined by measurement (ie, the actual value can be determined) and transmitted to the evaluation unit 180 (cable connection 54 b ).

[0038] exist Figure 1 In the figure, a number of possibilities are outlined in order to be able to draw conclusions about the intensity of the reflected illumination light 60a. The laser 20 outlined can be a laser diode with an integrated monitoring diode 30a, for example a surface emitter with a longitudinally integrated monitoring diode. The monitoring diode 30a can be used on the one hand to measure the reflected illumination light 60a, but can also be used to monitor the power of the laser 20, which fluctuates due to interference of the reflected light in the laser 20. Alternatively, a beam splitter 28 can also be used to branch off a portion of the reflected light and redirect it to a detector, for example a photodetector 30b. Power fluctuations of the laser 20 can also be detected by means of a photodetector 30c located outside the laser 20, wherein in this case the element 24 symbolizes a beam splitter: a portion of the light generated by the laser 20 is branched off by the beam splitter 24 and redirected to the photodetector 30c. If the illumination light is incident on the eye 10 or the eye environment 70 including the reference marks 80 of the eye environment 70, the illumination light is typically also partially scattered. The scattered light 60b can be detected by the second photodetector 40 (at Figure 1 140, which has an optical axis 45 perpendicular to the photosensitive surface of the second photodetector 40 for detection, wherein the photodetector 40 sends its signal to the evaluation unit 180. In this case, in particular, the signal for the illumination point 90b is recorded, which is arranged on the reference mark 80. In this case, the second photodetector 40 is arranged so that for each illumination point 90, the optical axis 45 of the second photodetector 40 is at an angle of ≥5° with the direction from which the illumination point is illuminated by the optical device. That is, in any case, the angle between the illumination light 60 reflected on the optical element 140 and the optical axis 45 of the second photodetector 40 is ≥5°. The signals of the first photodetector 30 and the second photodetector 40 arrive at the multiplexer 182 of the evaluation unit 180. In this case, the signal to be analyzed can be selected and further processed in the evaluation unit 180. Finally, images are created by an image processing device 184 (which in the example is part of the evaluation unit 180) based on the evaluated signals of the first and second photodetectors 30, 40 and the information about the position of the beam deflection device 50. Based on these images, the pupil position relative to the optical device 100 and also the line of sight vector can be determined taking into account the position of the reference marker 80.

[0039] exist Figure 22 shows schematically an exemplary possibility of scanning an eye 210 with an iris 213 and a pupil 212 and an environment 270 of the eye. Here, the beam deflection device 50 of the optical device 100 according to the invention is adjusted so that the illumination light describes a substantially sawtooth-shaped illumination pattern 220. Illumination points 290 are located on this trajectory 220, of which only a small part is drawn here purely by way of example.

[0040] Figure 3The flowchart of the method according to the present invention for determining the pupil position of the eye 10 is shown as an example, and a variant of the signal evaluation is explained. In step 310, light is generated by the laser 20. In step 320, the light is guided through the eye 10 and the eye environment 70 by adjusting the beam deflection device 50. In this case, different illumination points 90 are illuminated on the eye 10 and the eye environment 70. Subsequently, in step 330, the illumination light 60a reflected on the retina 14 is either detected directly by the first photodetector 30 or indirectly measured by means of laser feedback interferometry. Similarly, in step 340, the illumination light 60b scattered on the eye 10 or the eye environment 70 is detected by the second photodetector 40. The signals of the detectors 30, 40 are now provided to the evaluation unit 300 and there first reach the multiplexer 182, by means of which in step 350, it can be switched between the two signals for image generation. Next, the corresponding signal is preprocessed in step 355 via an amplifier, for example a transimpedance amplifier, and in step 360 via a filter, for example a bandpass filter, and finally digitized by an ADC (Analog-to-digital converter) in step 365. In this case, the gain factor of the amplifier and the angular frequency of the filter and the setting of the ADC can be adapted according to the signal characteristics of the corresponding detector signal, so that an image of the highest possible quality can be generated from the detector signal. In step 370, the signal is now assigned to the state of the beam deflection device 50, which is also transmitted to the evaluation unit 300 (arrow 325), and in step 372 (in the case of the detector signal of the first photodetector 30) or step 374 (in the case of the detector signal of the second photodetector 40), an image is generated from these data in the image processing device 184. Depending on which detector signal the image thus generated is based on, for example, the uncorrected position of the pupil 12 (in the case of an image from the signal of the first photodetector 30, step 372) or the position of the reference mark 80 (in the case of an image from the signal of the second photodetector 40, step 374) can be generated from the image. Finally, by combining the information of the two images, an eye model and a gaze vector for the eye 10 can be determined in step 380, the eye model including the (corrected) pupil position, which describes the position of the pupil 12 relative to the reference mark 80. Alternatively, with Figure 3 Unlike the illustration, the two possible signal paths may not be connected via the multiplexer 182 in step 350 , but may also be implemented separately from one another and not be combined until a final step 380 .

[0041] Finally Figure 4, an image is shown for explaining the determination of an eye model 429 and in particular a sight vector 428 for an eye 419 by means of the method according to the invention. When carrying out the method according to the invention, images 400 and 430 are generated from the detector signals of the first photodetector 30 and the second photodetector 40 in the context of signal evaluation, together with knowledge of the associated position of the beam deflection device 50 used. In this case, the image 430 is based on the signal of the first photodetector 30, which is based on the intensity of the light 60a reflected on the retina 14. The image 400 is created from the signal of the second photodetector 40, which detects the light 60b scattered on the eye 410 or the eye surroundings 417. Both images 400, 430 are created based on the same illumination point 90 on the eye 410 or the eye surroundings 417. Accordingly, the eye 410 and its eye surroundings 417, the iris 413 and the pupil 412 of the eye 410 can be seen in the image 400. The pupil 412 of the eye 410 can be seen in the image 430 , which can be seen in the image 430 as a speckle pattern 432 , which is caused by the optical roughness of the retina of the eye 410 .

[0042] From the image 400, the positions of two reference markers 419a, 419b can be discerned, wherein Figure 4 In the example, the angle of eye 410 is used as a reference mark. Although the pupil position can also be determined from image 400, since pupil 412 can be seen here. However, due to the greater information density compared to image 430, this requires significantly more computing power than the computing power required to determine the pupil position from image 430. Figure 4 In the specific case of , the two images 400 and 430 have exactly the same size, i.e. coincide, because the two images 400, 430 are based on the same illumination point 90 in this embodiment. However, this is not important for the implementation of the invention. However, according to the invention, the illumination of the illumination points must be realized via the same beam deflection device so that the illumination points can be correlated with each other. Figure 44. In the embodiment of the invention, an eye model 420 can be determined from the two images 400, 430. For this purpose, the positions of the reference markers 419a, 419b are extracted from the image 400 and referenced to the eye model 420 (arrows 422a, 422b). Since the two images 400, 430 have the same size, the positions of the reference markers 419a, 419b can also be transferred to the image 430 (arrows 424a, 424b). Therefore, a pupil position 436 of the pupil 412 in the image 430 relative to the two reference markers 419a, 419b can be output and can also be transferred to the eye model 420 (arrow 426). For further clarification, a dashed line 434 is drawn in the image 430, which schematically reproduces the shape of the eye 410 in the image 400. Finally, with the knowledge of the position of the reference markers 419a, 419b relative to the eye 410 (here: the position on the rotation axis 429a of the eye 410), and therefore with the knowledge of the relationship between the position of the pupil 412 (relative to the reference markers 419a, 419b) and the gaze vector, the gaze vector 428 can be calculated, which can be calculated, for example, with the help of the angle θ shown in the figure. It is noted that the angles describe the rotation of the eye 410 about two of its rotation axes 429a, 429b.

[0043] The invention is not limited to the embodiments described herein and the aspects emphasized therein. On the contrary, numerous modifications within the scope of a person skilled in the art can be realized within the scope given by the claims.

Claims

1. An optical device (100) for determining the pupil position of an eye (10, 210, 410), the optical device comprising a laser (20), a first photodetector (30), a second photodetector (40), an adjustable beam deflection device (50) and a control unit (150), the control unit being used to control the adjustable beam deflection device (50), characterized in that: The optical device (100) is configured to: guiding the light generated by the laser (20) at least partially as illumination light (60) via an illumination beam path (110) to the beam deflection device (50), and illuminating a plurality of illumination points (90, 290) on the eye (10, 210, 410) and an eye environment (70, 270, 417) of the eye (10, 210, 410) with the illumination light (60) by adjusting the beam deflection device (50) by the control unit (150), during illumination of at least one first portion (90a) of the illumination point (90, 290), measuring by means of the first photodetector (30a, 30c) the laser power of the laser (20) when illuminating the illumination point (90, 290) and / or detecting by means of the first photodetector (30a, 30b) illumination light (60a) which is reflected at an interface of the eye (10, 210, 410) or of the eye environment (70, 270, 417) and which enters the illumination beam path (110) via the beam deflection device (50), During illumination of at least one second portion (90b) of the illumination point (90, 290), illumination light (60b) scattered on the eye (10, 210, 410) or on the eye surroundings (70, 270, 417) is detected by means of the second photodetector (40).

2. The optical device (100) according to claim 1, characterized in that The laser (20) is an infrared laser and / or a laser diode, and / or the first photodetector (30) and / or the second photodetector (40) is a photodiode, a CMOS sensor and / or a CCD sensor, and / or the adjustable beam deflection device (50) comprises a micromirror actuator and / or a galvanometer scanner and / or a resonant scanner.

3. The optical device (100) according to any one of the preceding claims, characterized in that The laser (20) is a surface emitter with an integrated monitor diode (30a), and the first photodetector (30) is the monitor diode (30a) of the surface emitter.

4. The optical device (100) according to any one of the preceding claims, characterized in that The optical device (100) comprises a further optical element (140), such as a mirror and / or a prism and / or an HOE, which is designed to redirect the illumination light (60) after passing through the beam deflection device (50).

5. The optical device (100) according to any one of the preceding claims, characterized in that The optical device (100) comprises an analysis unit (180, 300) which is designed to analyze the signal of the first photodetector (30) and the signal of the second photodetector (40) and to determine the pupil position (436) and / or the gaze vector (428) from the analyzed signals.

6. The optical device (100) according to claim 5, characterized in that The analysis unit (180) includes a multiplexer (182), by means of which switching can be performed between the signal of the first photodetector (30) and the signal of the second photodetector (40) for analyzing the signals, and / or the analysis unit (180) includes an image processing device (184) which is configured to generate a first image from the analyzed signal of the first photodetector (30) and a second image from the signal of the second photodetector (40).

7. The optical device (100) according to any one of the preceding claims, characterized in that The first photodetector (30) is arranged in such a way that a portion of the light generated by the laser (20) can be detected by the first photodetector (30a, 30c), and / or the second photodetector (40) is arranged in such a way that, for each illumination point (90, 290), the optical axis (45) of the second photodetector (40) makes an angle of ≥5°, preferably ≥10°, particularly preferably ≥20° with the direction from which the illumination point (90, 290) is illuminated by the optical device (100).

8. Method for determining the pupil position of an eye (10, 210, 410) using an optical device (100), preferably an optical device according to any one of claims 1 to 7, wherein: The optical device (100) comprises a laser (20), a first photodetector (30), a second photodetector (40), an adjustable beam deflection device (50), a control unit (150) and an analysis unit (180), wherein the control unit is used to control the adjustable beam deflection device (50), and the eye (10, 210, 410) has a pupil (12) and a retina (14), and the method comprises the following steps: a) generating light by means of the laser (20) and guiding the light as illumination light (60) via an illumination beam path (110) at least partially to the beam deflection device (50), b) by means of adjusting the beam deflection device (50) by means of the control unit (150), illuminating a plurality of illumination points (90, 290) on the eye (10, 210, 410) and on the eye environment (70, 270, 417) of the eye (10, 210, 410) with the illumination light (60), wherein for different illumination points (90, 290) the beam deflection device (50) assumes different positions and for a first part (90a) of the illumination points (90, 290) the illumination light (60) is reflected back to the beam deflection device (50) on the retina (14) of the eye (10, 210, 410) and for a second part (90b) of the illumination points (90, 290) the illumination light (60) is scattered on at least one reference mark (80) of the eye environment (70, 270, 417), c) when illuminating at least the first part (90a) of the illumination point (90, 290), measuring the laser power of the laser (20) by means of the first photodetector (30) and / or detecting the illumination light (60a) reflected at the interface of the eye (10, 210, 410) or the eye environment (70, 270, 417) by means of the first photodetector (30), and analyzing the signal of the first photodetector (30) by means of the analysis unit (180), the analysis comprising assigning the signal to the position of the beam deflection device (50), d) when illuminating at least the second part (90b) of the illumination point (90, 290), detecting illumination light (60b) scattered on the eye (10, 210, 410) or on the eye environment (70, 270, 417) by means of the second photodetector (40), and analyzing the signal of the second photodetector (40) by means of the analysis unit (180), the analysis comprising assigning the signal to the position of the beam deflection device (50), e) determining the position of the at least one reference mark (80) by means of the evaluation unit (180) based on the evaluated signal of the second photodetector (40), f) determining the pupil position (436) by means of the evaluation unit (180) based on the evaluated signal of the first photodetector (30) taking into account the position of the at least one reference marker (80).

9. The method according to claim 8, characterized in that The analysis and processing of the signal of the first photodetector (30) includes generating a first image (430) based on the signal of the first photodetector (30), and / or the analysis and processing of the signal of the second photodetector (40) includes generating a second image (400) based on the signal of the second photodetector (40), and / or for at least a part of the illumination point (90, 290), not only step c) but also step d) is performed.

10. The method according to claim 8 or 9, characterized in that: The beam deflection device (50) is adjusted in such a way that the illumination points (90, 290) are arranged along a substantially sinusoidal or meandering or sawtooth-shaped illumination pattern (220).

11. The method according to any one of claims 8 to 10, characterized in that The method comprises calculating a gaze vector (428) of the eye (10, 210, 410) and / or an eye movement of the eye (10, 210, 410).

12. Data glasses (130) having an optical device (100) for determining a pupil position (436) of an eye (10, 210, 410) according to any one of claims 1 to 7.