Method for detecting a cataract in the eye of a driver in a vehicle

Abstract

The invention relates to a method for detecting a cataract in the eye of a driver in a vehicle (1). According to the invention, the degree of a cataract is passively determined in the vehicle (1) based on a projected and reflected display content of a head-up display unit (2) and signals from a driver observation camera (3) relating to at least one eye of the driver, with respect to an idealized transmission curve of the optical media.

Description

[0001] The invention relates to a method for detecting a cataract in the eye of a driver in a vehicle.

[0002] German patent application DE 10 2019 004 816 A1 discloses a method for calibrating a vehicle's head-up display. In this method, a virtual image is projected into the driver's field of vision via the head-up display, and the virtual image is superimposed on a calibration pattern located in front of the vehicle. Additionally, the calibration pattern is projected onto a projection surface in front of the vehicle by means of a headlight, and the driver manually aligns the virtual image with the calibration pattern.

[0003] WO 2017 / 171257 A1 describes a vehicle control device for a vehicle comprising a display for showing image information connected to the vehicle, a light source for emitting light to form a reflected light in an area of ​​a user's pupil and eyeball looking at an area on the display, a memory for storing coordinate information for each area within the vehicle and the single light source, a camera for capturing an image encompassing the single area of ​​the user's pupil and eyeball, and a controller for calculating a first coordinate from the center point of the pupil and calculating a second coordinate from the single reflected light contained in the captured image, and calculating a coordinate of a point within the vehicle as a reference coordinate from pre-stored coordinate information of the single light source.if the distance between the first coordinate and the second coordinate is less than a preset distance, and includes performing a calibration of the reference coordinate for a coordinate corresponding to a user's viewing direction.

[0004] German patent DE 10 2018 008 243 A1 describes a method for detecting visual opacity in a vehicle driver. Using an infrared light source integrated into the vehicle, at least one of the driver's eyes is illuminated. The direction of the eye's gaze is determined by means of at least one camera integrated into the vehicle, based on the position of the reflection of infrared light in the eye and the center of the pupil. From the camera images, the light transmission of the eye's lens is determined by measuring the black or red value of the pupil's center. Changes in light transmission are detected by comparing the black or red value with previously measured black or red values ​​of the pupil's center.

[0005] US Patent 2003 / 0142041A1 describes an eye-tracking device comprising a lighting unit and a reflector unit. The reflector unit reflects light from the lighting unit onto the surface of a vehicle's windshield in such a way that the light is reflected onto the eye of a person inside the vehicle, creating an image of that eye. Furthermore, a sensor is provided that receives the reflection of the image of the eye from the reflector unit and generates a corresponding output signal.

[0006] The invention is based on the objective of providing a method for detecting cataracts in a driver in a vehicle.

[0007] The problem is solved according to the invention by a method which has the features specified in claim 1.

[0008] Advantageous embodiments of the invention are the subject of the dependent claims.

[0009] The inventive method for detecting a cataract in a driver's eye in a vehicle provides that the degree of a cataract is passively determined in the vehicle based on a projected and reflected display content of a head-up display unit and captured signals relating to at least one eye of the driver from a driver observation camera with respect to an idealized transmission curve of the optical media (Optical-Media-Transmittance-Curve).

[0010] By applying this method, it is possible to cyclically detect cataracts in the driver's eye, thereby enabling the determination of gradual eye degeneration. This detection is performed without the driver's conscious involvement. The method can be carried out independently of the driver and regardless of the prevailing light and weather conditions, using components already present in the vehicle. Therefore, essentially only a software modification of an algorithm is required. The method also functions independently of the installation position of the head-up display unit and the driver observation camera.

[0011] For example, the process builds on standard components typically found in the vehicle and represents a comparatively innovative, specific approach for a manufacturer of these standard components.

[0012] The method, which uses vehicle-side means to non-invasively and passively quantify the cataract, in particular to automatically initiate a measure on the vehicle side when a cataract is detected, to increase driving safety and optimize driving comfort for the driver of the vehicle, especially to compensate for a loss of visual acuity.

[0013] An epipolar geometry of the reflection from specific pixels of the displayed content is determined as an epiline relative to the vehicle's driver. An image of the driver's eye is captured using the driver observation camera, measuring the reflection of incoming light at the interface between the lens and cornea of ​​the eye. This is used to detect lens opacities and determine the degree of cataracts. The reflection is then used to determine which area of ​​the head-up display illuminates which area of ​​the eye's lens. The reflection of the projected display content on the windshield is used as a "second camera," allowing the epipolar geometry to be utilized. This results in an epiline for each pixel of the projected display content relative to the driver observation camera.This makes it possible to determine which pixel needs to be projected to illuminate a defined area of ​​the lens.

[0014] In one implementation, the head-up display unit and the driver observation camera are calibrated with respect to a coordinate system, in particular a vehicle coordinate system, and color values ​​projected onto a windshield by the head-up display unit are measured in relation to a windshield position. This provides information about which wavelength range is reflected by the vehicle's windshield.

[0015] In one possible implementation, the driver's eye, as captured by the driver-monitoring camera, is divided into a predetermined number of sub-areas, and the center of each sub-area and its respective neighbors are determined. This division into sub-areas is particularly important to enable the recurring projection of pixels of different wavelengths at the same position. Thus, for each center of a sub-area, the neighborhood is determined, and the epiline with the smallest orthogonal distance—that is, which projected pixel illuminates this center—is identified.

[0016] In a further embodiment, the projection position of a pixel of the displayed content is determined based on a calculated minimum orthogonal distance between the center of a sub-area and an epiline. The epilines indicate which pixel must be addressed to illuminate a specific area, particularly a sub-area, of the eye.

[0017] In one embodiment, a sequence projection of the display content is projected onto the windshield with respect to a center of the sub-area, whereby this projection lies above a so-called flicker frequency, also referred to as flicker fusion frequency or critical flicker frequency, so that this projection is not perceived by the driver and, in particular, is not considered disturbing.

[0018] In a further step, the valid mapping of the center of each sub-area to the epilines is cyclically checked, and if the mapping is found to be valid, an optical media transmission curve for the driver's eye is determined and compared with the idealized optical media transmission curve to determine the degree of cataract in the driver's eye. Thus, the degree of cataract in a driver's eye can be determined essentially without the driver noticing, and, if necessary, a measure can be initiated to compensate for the reduced visual acuity.

[0019] Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

[0020] This shows: Fig. 1 schematically a vehicle with a head-up display unit and a driver observation camera and Fig. 2 schematically illustrates a procedure for detecting a cataract in the eye of a driver in a vehicle.

[0021] Corresponding parts are marked with the same reference symbols in all figures.

[0022] In Fig. Figure 1 is an example and is highly simplified representation of a vehicle 1 with a head-up display unit 2 and a driver observation camera 3.

[0023] Fig. Figure 2 shows a procedure for detecting a cataract in the eye of a driver in a vehicle 1.

[0024] It is generally known that assistance systems of a vehicle 1, for example for distance control, lane keeping, lateral control and based on a sensory all-round view for parking operations and sensor data visualization, support a driver of the vehicle 1 in many ways.

[0025] Such assistance systems are designed to support the driver of vehicle 1 in their driving task and are usually activated manually by the driver when needed. However, it would be desirable for assistance systems to be activated automatically when a driver's performance impairment is detected. A typical example of this is the detection of driver fatigue by signals recorded by a driver monitoring camera 3.

[0026] It is also known that cataracts, also known as clouding of the lens of the eye, lead to a deterioration of vision, particularly due to clouding of the lens. This means that if a driver of a vehicle (1) has cataracts, their driving safety may be impaired, for example, in certain driving scenarios. In particular, it has been found that contrast vision is reduced in people with cataracts, and thus the recognition of objects in traffic, especially at night and / or in fog, can significantly increase the risk of accidents. Since cataracts occur primarily in older age and vision deteriorates gradually over a longer period, it is difficult for people, especially drivers of vehicles (1), to assess the deterioration of their eyesight.A specific assessment is carried out by a doctor, although a preceding period poses risks to the driver, especially in road traffic.

[0027] The following describes a method for detecting a cataract in the eye of a driver in a vehicle 1, wherein the method provides for determining the degree of the cataract non-invasively and essentially unnoticed by a driver of the vehicle 1.

[0028] For example, after a cataract is detected in the driver of vehicle 1, it is possible to automatically activate the aforementioned assistance systems of vehicle 1. It is also conceivable that the vehicle could recommend to the driver that they consult an ophthalmologist and / or that the degree of the cataract, as determined, could be recorded, for example, using an event data recorder, in order to be able to provide evidence in the event of a legal dispute.

[0029] Cataracts are typically detected using a slit lamp, which illuminates and magnifies the lens of the eye, allowing opacities along the lens to be seen. This and other well-known methods for detecting cataracts are active procedures, and their application is conscious.

[0030] The method described below aims to determine the cataract of a driver of vehicle 1, in particular the degree of cataract, by combining a head-up display unit 2, which projects a display content into the driver's field of vision directed towards the windshield of vehicle 1, and a driver observation camera 3. For this purpose, existing research, specifically an idealized optical media transmission curve, is used. Based on a projection of display content, a so-called augmented reality projection, an optical media transmission curve is determined with respect to one of the driver's eyes. A visual impairment of the driver, in particular a cataract, can then be detected based on a comparison between the determined optical media transmission curve and the idealized optical media transmission curve.

[0031] A first procedural step S1 of the procedure provides that the head-up display unit 2 and the driver observation camera 3 are (extrinsically) calibrated with respect to a common coordinate system, in particular a vehicle coordinate system.

[0032] In a second process step S2, a color adjustment is performed on the display content projected by the head-up display unit 2, whereby the color values ​​of the projected pixels are measured in relation to a windshield position. This provides information about which wavelength range of the projected pixels is reflected by the windshield. Furthermore, the execution of the process requires information about which area of ​​the head-up display unit 2 illuminates which area of ​​a lens in the driver's eye.

[0033] In a third process step S3, epilines, in particular stereo epilines of an epipolar geometry, are determined. This is achieved by using a reflection of the display content projected by the head-up display unit 2 onto the windshield as a "second camera," thus utilizing the epipolar geometry. This results in an epiline for each projected pixel relative to the driver observation camera 3. Therefore, it can be determined which pixel must be projected to illuminate a defined area on the lens of the driver's eye.

[0034] In a fourth process step S4, an image of one of the driver's eyes is captured using the driver observation camera 3. This image is then subdivided into sub-areas, known as areas of interest, in a fifth process step S5. These sub-areas all have the same rectangular shape. This subdivision is performed to enable the repeated projection of pixels of different wavelengths at the same position.

[0035] A sixth procedure step S6 of the procedure provides that a neighborhood is determined for a center of the respective sub-area and in a seventh procedure step S7 it is determined which epiline to a center of a sub-area has the smallest orthogonal distance, i.e. which projected pixel illuminates this center of a sub-area.

[0036] In particular, using process steps S5 and S6, an epiline is determined for each projected pixel in order to determine the area that is illuminated by the projected pixel.

[0037] Based on this, in a seventh process step S7, a projection position of a pixel of the display content is determined, whereby the epiline indicates which pixel must be addressed. Subsequently, in an eighth process step S8, a sequence projection is projected onto a position of the center of the sub-areas, whereby the projection lies above a flicker fusion frequency, so that this projection is not perceived by the driver. The sequence projection for the center of each sub-area comprises a sweep through a defined wavelength range, for example, from 400 nm to 780 nm.

[0038] Alternatively or additionally, it is possible, for example, to use and slightly modify the color spaces of an existing projection when the distance control of a leading vehicle 1 is activated, so that the color green with a wavelength of 480 nm is projected in the range of 440 nm to 520 nm.

[0039] In a ninth process step, S9, the validity of the pixel projection position determined in the seventh process step, S7, is cyclically verified. This involves cyclically checking, and / or based on a trigger criterion, the valid assignment of the centers of the sub-areas to the epilines. This is because any assignment can be flawed due to movement, resulting in a constant projection, while the driver moves, and different positions being projected onto the lens during the sequence projection, potentially leading to inaccurate measurement results. Once the sequence projection is complete and the validity of the centers to the epilines is continuously ensured, the optical media transmission curve for the driver's eye is determined in a tenth process step, S10.

[0040] In an initial application, an idealized, specifically "clinically healthy," optical media transmission curve with a corresponding error variance is known. The determined optical media transmission curve is compared with the idealized optical media transmission curve, and an initial cataract probability is derived based on the degree of similarity. Furthermore, process steps S4 to S10 are repeatedly executed cyclically, for example, with each ignition cycle of vehicle 1, and any changes in the idealized optical media transmission curve over a specified period are evaluated.

[0041] Depending on the deviation of the determined optical media transmission curve from the idealized optical media transmission curve, an action can be performed to support the driver of vehicle 1. For example, with regard to sensor data visualization, a zoom level can be adjusted to a close-up view and / or an interior color and / or a color of icons can be changed, or the color can be adjusted with respect to a color value whose wavelength has the highest transmission.

[0042] The procedure is applied cyclically, for example with each ignition change, so that any eye degradation of the driver, especially in relation to a cataract, which affects driving and cannot necessarily be recognized by the driver, is detected by the vehicle.

[0043] Furthermore, the procedure enables a comparatively high number of actions according to the determined degree of cataract in order to increase road safety and driving comfort for the driver of vehicle 1. Reference symbol list 1 vehicle 2 Head-up display unit 3 Driver observation cameras S1 to S10 process step

Claims

[1] Method for detecting a cataract in the eye of a driver in a vehicle (1) wherein a degree of cataract is passively determined in the vehicle (1) based on a projected and reflected display content of a head-up display unit (2) and captured signals relating to at least one eye of the driver from a driver observation camera (3) with respect to an idealized transmission curve of the optical media and - an epipolar geometry of a reflection of certain pixels of the display content is determined as an epiline in relation to the driver of the vehicle (1) and an image of one eye of the driver is taken using the driver observation camera (3). [2] Method according to claim 1, characterized by, that the head-up display unit (2) and the driver observation camera (3) are calibrated with respect to a common coordinate system and that color values ​​projected onto a windshield by means of the head-up display unit (2) are measured with respect to a windshield position. [3] Method according to any one of the preceding claims, characterized by , that the driver's eye is divided into a predetermined number of sub-areas and a center of each sub-area and its respective neighborhood is determined. [4] Method according to claim 3, characterized by , that based on a determined minimum orthogonal distance between a center of a sub-area and an epiline, a projection position of a pixel of the display content is determined. [5] Method according to claim 4, characterized by, that a sequence projection of the display content is projected onto the windshield with respect to a center of the sub-area. [6] Method according to any one of the preceding claims, characterized by , that a valid assignment of the center of the respective sub-area to the epilines is cyclically checked and if it is determined that the assignment is valid, a transmission curve of the optical media for the driver's eye is determined and compared with the idealized transmission curve of the optical media to determine a degree of cataract of the driver's eye.

Images