Procedure for the on-demand issuance of a hazard warning concerning an object and vehicle located in a blind spot of the vehicle

Abstract

A method for the on-demand output of a hazard warning concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, in the event of a hazard situation detected by the vehicle, the hazard warning is issued, and an area of ​​the object leading to the blind spot hazard warning is determined, characterized in that: reflections occurring in the cornea of ​​the driver's eye are determined by the driver observation camera and are represented in a predefined representation; the area of ​​the object leading to the blind spot hazard warning is superimposed with the representation depicting the reflections in the cornea of ​​the driver's eye; and the hazard warning is suppressed.when the area of ​​the object leading to the blind spot hazard warning is represented in the corneal area of ​​the eye, whereby, before suppressing the hazard warning, it is checked whether the object represented in the corneal area of ​​the eye is a vehicle.

Description

The invention relates to a method for the on-demand output of a hazard message concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, and in the event of a hazard situation detected by the vehicle, the hazard message is output, as well as a vehicle with a driver assistance system for carrying out the method. German patent DE 10 2023 002 261 A1 discloses a method for operating a vehicle assistance system in which the vehicle's surroundings are monitored by sensors. When an object is detected in the vehicle's blind spot, a hazard warning is issued to the driver. This warning is issued via an acoustic and / or visual signal, with the visual signal being emitted by a light source in the exterior mirror or by the ambient lighting. The hazard warning is canceled as soon as the object is within the driver's field of vision. However, the warning is also suppressed if an object is detected in the blind spot due to system malfunctions or sensor errors, which could lead to dangerous traffic situations. The free encyclopedia Wikipedia lists devices and methods for tracking gaze direction under the keyword "eyetracking", for example, determining gaze direction by evaluating reflections in the cornea area of ​​an eye. US Patent 2011 / 0 169 625 A1 discloses a device for supporting safe vehicle operation comprising an environmental sensor system for detecting hazards in the vehicle's surroundings, a driver monitor that provides data on driver perception such as eye movements, and an attention assessment module that, by comparing the hazard data with the eye movements, determines whether the driver perceives hazards sufficiently or insufficiently. A warning signal can be issued if hazards are not perceived. The object of the invention is to provide a method for the demand-based output of a hazard message, with which a reliable and error-free detection of an object in the blind spot area of ​​the vehicle is achieved. The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims. Further features, applications, and advantages of the invention will become apparent from the following description and the explanation of exemplary embodiments of the invention illustrated in the figures. The problem is solved by the subject matter of claims 1 and 9. In the method described above for the on-demand issuance of a hazard warning concerning an object located in a vehicle's blind spot, in which a three-dimensional model of the vehicle's environment is determined and coupled with image data from a driver observation camera, and the hazard warning is issued when a hazard situation is detected by the vehicle, an area of ​​the object leading to the blind spot hazard warning is determined, and the driver observation camera detects reflections occurring in the cornea of ​​the driver's eye, which are represented in a predefined representation, whereby the area of ​​the object leading to the blind spot hazard warning is superimposed with the representation depicting the reflections in the cornea of ​​the driver's eye, and the hazard warning is suppressed.The warning is only suppressed if the area of ​​the object triggering the blind spot warning is represented in the corneal area of ​​the eye. The warning is only suppressed if it is certain that the driver has actually perceived the object in the vehicle's blind spot. A reliable comparison can be made even if the driver only perceives the object peripherally. This comparison is independent of the type of object triggering the warning. The advantage lies in the individual and therefore adaptive adjustment of the warning to the driver's actual visual acuity. This increases road safety. At the same time, suppressing the warning improves the comfort of the vehicle occupants. Human intervention is no longer necessary. Before suppressing the hazard warning, the system checks whether the object depicted in the corneal area of ​​the eye is a vehicle. Distinguishing between a vehicle and another type of obstacle increases the safety of the procedure. In a further implementation, the hazard warning is suppressed only if the object depicted in the cornea of ​​the eye is a vehicle. If the detected object is an arm, a vehicle's A-pillar, or a similar object, the hazard warning is maintained. In a further refinement, the object imaged in the corneal area of ​​the eye is checked using a pre-trained machine learning model. This allows the blind spot functionality to be individually adapted to the respective driver, thereby improving the test result. In a further development, the review is repeated cyclically. Through adaptive cyclical repetition, the quality of the review result is progressively improved. In a further refinement, the identified area of ​​the object triggering the blind spot warning and the captured corneal images are transformed into a common coordinate system. This transformation ensures that both the detected object and the captured corneal images are subject to the same conditions for comparison. In a further refinement, the corneal area of ​​the eye is rectified onto a 2D image plane, onto which the 3D representation of the object triggering the blind spot warning is superimposed. Such a superimposition provides a simple method to determine whether the object detected by the sensor is identical to the object perceived by the driver. In a further embodiment, the hazard warning is retained even if the check reveals that the object depicted in the cornea of ​​the eye is not a vehicle. This allows it to be assumed, for example, that a vehicle is obscured by the object perceived by the driver's eye. In a further implementation, the hazard warning is also issued if the image data acquired by the driver observation camera indicates an object obscuring the cornea of ​​the eye. This assumes that no comparison with an object detected within the cornea of ​​the driver's eye is possible. A further aspect of the invention relates to a vehicle with a driver assistance system for carrying out the method according to at least one feature described in this patent application, comprising sensors for determining the vehicle's surroundings, a driver observation camera for capturing the driver's eye in the three-dimensional vehicle interior, a warning unit, and an evaluation unit for superimposing the area of ​​the object triggering the blind spot warning with the representation depicting the reflections in the cornea of ​​the driver's eye, and for outputting a deactivation signal to the warning unit when the area of ​​the object triggering the blind spot warning is depicted in the representation of the cornea of ​​the eye. Before suppressing the warning by outputting the deactivation signal, it is checked whether the object depicted in the cornea of ​​the eye is a vehicle.Distinguishing between a vehicle and another type of obstacle increases the safety of the procedure. The driver assistance system enables the described procedure to be carried out using sensors and computing devices already present in the vehicle. Therefore, no hardware modifications are necessary. Furthermore, it is independent of the sensor modality (radar, vision, HDMap), which means it is independent of prevailing weather and lighting conditions. This allows the driver assistance system to be used in vehicles with varying configurations. Further advantages, features, and details will become apparent from the following description, in which at least one embodiment is described in detail. The described features can, individually or in any meaningful combination, constitute the subject matter of the invention, optionally also independently of the claims, and can, in particular, also be the subject matter of one or more separate applications. Figure 1 shows an embodiment of the vehicle according to the invention, Figure 2 shows an embodiment of the method according to the invention, Figure 3 shows a pictorial illustration of individual method steps. Figure 1 shows an embodiment of the vehicle according to the invention. The vehicle 1 comprises a driver assistance system 3, which has an evaluation unit 5 connected to at least one radar sensor 7 arranged at the rear of the vehicle 1, which detects the area behind the vehicle 1. An interior camera 9, also connected to the driver assistance system 3, is directed towards one eye of the driver 11 (Fig. 1a). A light source 15 is arranged in an exterior mirror 13 of the vehicle 1, which can be controlled by the driver assistance system (Fig. 1b). An embodiment of the method according to the invention is stored in the evaluation unit 5, which will be explained with reference to Fig. 2. In step S1, the area surrounding the vehicle 1 is detected using the data supplied by the radar sensor 7. The data supplied by the radar sensor 7 are checked to determine whether an object, for example, another vehicle, is located in the blind spot of the vehicle 1 (step S2). If this is the case, the driver assistance system 3 activates a hazard warning for the driver by activating the light source 15. With this warning, the object that triggered the warning and its relative position to the vehicle 1 are known. In step S3, the interior camera 9, which is aimed at the driver's eye 11, is calibrated online in a higher-level coordinate system. Subsequently, in step S4, the calibrated interior camera 9 captures the driver's eye 11 in three-dimensional space.In step 5, a so-called corneal reflective image of the eye is detected and rectified onto a two-dimensional image plane. This results in a representation of the driver's vision, both directly and indirectly, on a two-dimensional image plane. In step S6, the three-dimensional position of the detected object is projected onto the two-dimensional image plane of the corneal reflective image. This is possible because, due to calibration, the data from the object detected by radar sensor 7 and the data from the interior monitoring camera 9 are in the same reference system. Thus, a monitoring area 17 can be easily created, which, in step S7, is continuously searched for characteristic features of the vehicle 1 and the object detected by vehicle 1 using a correspondence analysis. This can be done using artificial intelligence, for example, with a pre-trained machine learning model.To increase the accuracy of the correspondence analysis, the system repeatedly switches between steps S6 and S7. If a correspondence between the object detected by the vehicle and the image of the driver's eye is established, the hazard warning is suppressed in step S8 by deactivating light source 15. If this is not the case, the system returns to step S2 to execute the entire process chain while the warning is active. If the hazard warning has been deactivated, steps S1 to S8 are repeated after a predefined cycle time, for example, 10 Hz (step 9). Figure 3 provides a visual illustration of individual process steps to better understand the described process steps. Figure 3a shows the corneal reflective image derived from the image captured by the interior camera 9. Figure 3b shows the rectification of the spherical projection onto the two-dimensional image plane. The projection of the three-dimensional environmental area 19, including the object detected by the radar sensor 7, onto the two-dimensional image plane 21 of the corneal reflective image is shown in Figure 3c. The monitoring area 17, which is to be checked for correspondence, is identified, and the correspondence analysis is performed. In Figure 3d, a correspondence is found between the object detected by the vehicle and the image of the driver's eye, whereupon the hazard warning is aborted.3e shows the driver's eye being covered by an arm or glasses 23 of the driver 11, which leads to the conclusion that there is no correspondence between the object detected by the vehicle and the image of the driver's eye and the hazard warning is maintained.

Claims

A method for the on-demand output of a hazard warning concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, in the event of a hazard situation detected by the vehicle, the hazard warning is issued, and an area of ​​the object leading to the blind spot hazard warning is determined, characterized in that: reflections occurring in the cornea of ​​the driver's eye are determined by the driver observation camera and are represented in a predefined representation; the area of ​​the object leading to the blind spot hazard warning is superimposed with the representation depicting the reflections in the cornea of ​​the driver's eye; and the hazard warning is suppressed.when the area of ​​the object leading to the blind spot hazard warning is represented in the corneal area of ​​the eye, whereby, before suppressing the hazard warning, it is checked whether the object represented in the corneal area of ​​the eye is a vehicle. Method according to claim 1, characterized in that the suppression of the danger warning only occurs if the object depicted in the cornea area of ​​the eye is a vehicle. Method according to claim 1 or 2, characterized in that the verification of the object imaged in the corneal area of ​​the eye is carried out using a pre-trained machine learning model. Method according to claim 1, 2 or 3, characterized in that the verification is repeated cyclically. Method according to at least one of the preceding claims, characterized in that the determined area of ​​the object leading to the blind spot hazard warning and the captured cornea images are transformed into a common coordinate system. Method according to claim 5, characterized in that the presentation of the corneal area of ​​the eye is rectified onto a 2D image plane, onto which the 3-dimensionally formed area of ​​the object leading to the blind spot hazard warning is superimposed. Method according to claim 1, characterized in that the output of the hazard warning is maintained if the verification shows that the object depicted in the corneal area of ​​the eye is not a vehicle. Method according to at least one of the preceding claims, characterized in that the hazard warning is further issued if the image data obtained from the driver observation camera indicate an object covering the cornea of ​​the eye. Vehicle with a driver assistance system for carrying out the method according to at least one of the preceding claims, with sensor technology (7) for determining a vehicle environment, characterized by: - ​​a driver observation camera (9) for capturing one eye of the driver (11) in the three-dimensional vehicle interior, - a warning unit (15) and - an evaluation unit (5) for superimposing the area of ​​the object leading to the blind spot hazard warning with the representation depicting the reflections in the cornea area of ​​the driver's eye (11) and for outputting a deactivation signal to the warning unit (15) when the area of ​​the object leading to the blind spot hazard warning is depicted in the representation of the cornea area of ​​the eye, wherein the evaluation unit (5) checks, before suppressing the hazard warning by outputting the deactivation signal, whether the object depicted in the cornea area of ​​the eye is a vehicle.

Images