Procedure for the on-demand issuance of a hazard warning concerning an object and vehicle located in a blind spot of the vehicle

Abstract

The invention relates to a method for the on-demand output of a hazard message concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, and the hazard message is output when a hazard situation is detected by the vehicle.In a method that enables reliable and error-free detection of an object in the vehicle's blind spot, an area of ​​the object triggering the blind spot warning is determined. The driver observation camera detects reflections occurring in the cornea of ​​the driver's eye, which are mapped in a predefined representation. The area of ​​the object triggering the blind spot warning is superimposed with the representation depicting the reflections in the cornea of ​​the driver's eye, and the warning is suppressed if the area of ​​the object triggering the blind spot warning is mapped in the representation of the cornea of ​​the eye.

Description

[0001] The invention relates to a method for the on-demand output of a hazard message concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, and in the event of a hazard situation detected by the vehicle, the hazard message is output, as well as a vehicle with a driver assistance system for carrying out the method.

[0002] German patent DE 10 2023 002 261 A1 discloses a method for operating a vehicle assistance system in which the vehicle's surroundings are monitored by sensors. When an object is detected in the vehicle's blind spot, a hazard warning is issued to the driver. This warning is issued via an acoustic and / or visual signal, with the visual signal being emitted by a light source in the exterior mirror or by the ambient lighting. The hazard warning is canceled as soon as the object is within the driver's field of vision. However, the warning is also suppressed if an object is detected in the blind spot due to system malfunctions or sensor errors, which could lead to dangerous traffic situations.

[0003] The object of the invention is to provide a method for the on-demand output of a hazard warning, with which a reliable and error-free detection of an object in the blind spot area of ​​the vehicle is achieved.

[0004] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims. Further features, applications, and advantages of the invention will become apparent from the following description and the explanation of exemplary embodiments of the invention illustrated in the figures.

[0005] The problem is solved by the subject matter of claims 1 and 10.

[0006] In the method described above for the on-demand issuance of a hazard warning concerning an object located in a vehicle's blind spot, in which a three-dimensional model of the vehicle's environment is determined and coupled with image data from a driver observation camera, and the hazard warning is issued when a hazard situation is detected by the vehicle, an area of ​​the object leading to the blind spot hazard warning is determined, and the driver observation camera detects reflections occurring in the cornea of ​​the driver's eye, which are represented in a predefined representation, whereby the area of ​​the object leading to the blind spot hazard warning is superimposed with the representation depicting the reflections in the cornea of ​​the driver's eye, and the hazard warning is suppressed.The warning is only suppressed if the area of ​​the object triggering the blind spot warning is represented in the corneal area of ​​the eye. The warning is only suppressed if it is certain that the driver has actually perceived the object in the vehicle's blind spot. A reliable comparison can be made even if the driver only perceives the object peripherally. This comparison is independent of the type of object triggering the warning. The advantage lies in the individual and therefore adaptive adjustment of the warning to the driver's actual visual acuity. This increases road safety. At the same time, suppressing the warning improves the comfort of the vehicle occupants. Human intervention is no longer necessary.

[0007] In one implementation, before suppressing the hazard warning, the system checks whether the object depicted in the corneal area of ​​the eye is a vehicle. Distinguishing between a vehicle and another type of obstacle increases the safety of the procedure.

[0008] In a further implementation, the hazard warning is suppressed only if the object depicted in the cornea of ​​the eye is a vehicle. If the detected object is an arm, a vehicle's A-pillar, or a similar object, the hazard warning is maintained.

[0009] In a further refinement, the object imaged in the corneal area of ​​the eye is checked using a pre-trained machine learning model. This allows the blind spot functionality to be individually adapted to the respective driver, thereby improving the test result.

[0010] In a further development, the review is repeated cyclically. Through adaptive cyclical repetition, the quality of the review result is progressively improved.

[0011] In a further refinement, the identified area of ​​the object triggering the blind spot warning and the captured corneal images are transformed into a common coordinate system. This transformation ensures that both the detected object and the captured corneal images are subject to the same conditions for comparison.

[0012] In a further refinement, the presentation of the corneal area of ​​the eye is rectified onto a 2D image plane, onto which the 3D representation of the object triggering the blind spot warning is superimposed. Such a superimposition provides a simple method for determining whether the object detected by the sensor is identical to the object perceived by the driver.

[0013] In a further embodiment, the hazard warning is retained even if the check reveals that the object depicted in the cornea of ​​the eye is not a vehicle. This allows it to be assumed, for example, that a vehicle is obscured by the object perceived by the driver's eye.

[0014] In a further implementation, the hazard warning is also issued if the image data acquired by the driver observation camera indicates an object obscuring the cornea of ​​the eye. This assumes that no comparison with an object detected within the cornea of ​​the driver's eye is possible.

[0015] A further aspect of the invention relates to a vehicle with a driver assistance system for carrying out the method according to at least one feature described in this patent application, comprising sensors for determining the vehicle's surroundings, a driver observation camera for capturing the driver's eye in the three-dimensional vehicle interior, a warning unit, and an evaluation unit for superimposing the area of ​​the object triggering the blind spot warning with the representation depicting the reflections in the cornea of ​​the driver's eye, and for outputting a deactivation signal to the warning unit when the area of ​​the object triggering the blind spot warning is depicted in the representation of the cornea of ​​the eye. The driver assistance system enables the described method to be carried out using sensors and computing devices already present in the vehicle. Therefore, no hardware modifications are necessary.Furthermore, the system is independent of the sensor modality (radar, vision, HDMap), which means it is independent of prevailing weather and lighting conditions. This allows the driver assistance system to be used in vehicles with varying levels of technical specifications.

[0016] Further advantages, features, and details will become apparent from the following description, in which at least one embodiment is described in detail. The described features can, individually or in any meaningful combination, constitute the subject matter of the invention, optionally also independently of the claims, and can, in particular, also be the subject matter of one or more separate applications.

[0017] They show: Fig. 1 an embodiment of the vehicle according to the invention, Fig. 2 an embodiment of the method according to the invention, Fig. 3. A visual illustration of individual process steps.

[0018] In Fig. Figure 1 shows an embodiment of the vehicle according to the invention. The vehicle 1 comprises a driver assistance system 3, which has an evaluation unit 5 connected to at least one radar sensor 7 arranged at the rear of the vehicle 1, which detects the environment behind the vehicle 1. An interior camera 9, also connected to the driver assistance system 3, is directed towards one eye of the driver 11 ( Fig. 1a). A light source 15 is arranged in an exterior mirror 13 of the vehicle 1, which can be controlled by the driver assistance system ( Fig. 1b).

[0019] An embodiment of the method according to the invention is stored in the evaluation unit 5, which is based on the Fig. This will be explained in section 2. In step S1, the area surrounding vehicle 1 is detected using the data supplied by radar sensor 7. The data supplied by radar sensor 7 is then checked to determine whether an object, such as another vehicle, is located in the blind spot of vehicle 1 (step S2). If so, the driver assistance system 3 activates a hazard warning for the driver by activating the light source 15. This warning is based on the known object that triggered the warning and its relative position to vehicle 1. In step S3, the interior camera 9, which is aimed at the driver's eye 11, is calibrated online to a higher-level coordinate system. Subsequently, in step S4, the calibrated interior camera 9 captures the driver's eye 11 in three-dimensional space.In step 5, a so-called corneal reflective image of the eye is detected and rectified onto a two-dimensional image plane. This results in a representation of the driver's vision, both directly and indirectly, on a two-dimensional image plane. In step S6, the three-dimensional position of the detected object is projected onto the two-dimensional image plane of the corneal reflective image. This is possible because, due to calibration, the data from the object detected by radar sensor 7 and the data from the interior monitoring camera 9 are in the same reference system. Thus, a monitoring area 17 can be easily created, which, in step S7, is continuously searched for characteristic features of the vehicle 1 and the object detected by vehicle 1 using a correspondence analysis. This can be done using artificial intelligence, for example, with a pre-trained machine learning model.To increase the accuracy of the correspondence analysis, the system repeatedly switches between steps S6 and S7. If a correspondence between the object detected by the vehicle and the image of the driver's eye is established, the hazard warning is suppressed in step S8 by deactivating light source 15. If this is not the case, the system returns to step S2 to execute the entire process chain while the warning is active. If the hazard warning has been deactivated, steps S1 to S8 are repeated after a predefined cycle time, for example, 10 Hz (step 9).

[0020] In Fig. Figure 3 is a visual illustration of individual process steps to better understand the explained process steps. Fig. Figure 3a shows the Corneal Reflective Image derived from the image taken by interior camera 9. Fig. Figure 3b shows the rectification of the spherical projection onto the two-dimensional image plane. The projection of the three-dimensional environment 19 with the object detected by the radar sensor 7 onto the two-dimensional image plane 21 of the Corneal Reflective Image is shown in Fig. Figure 3c is shown. The monitoring area 17, which is to be checked for correspondence, is identified and the correspondence analysis is carried out. Fig. In 3D, a correspondence is detected between the object detected by the vehicle and the image of the driver's eye, whereupon the hazard warning is aborted. Fig. 3e shows the driver's eye being covered by an arm or glasses 23 of the driver 11, which leads to the conclusion that there is no correspondence between the object detected by the vehicle and the image of the driver's eye and the hazard warning is maintained. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2023 002 261 A1

[0002]

Claims

[1] Method for the on-demand output of a hazard warning concerning an object located in a blind spot of the vehicle, in which a three-dimensional environment model of the vehicle is determined, which is coupled with image data from a driver observation camera, and in the event of a hazard situation detected by the vehicle, the hazard warning is issued, characterized by , that -an area of ​​the object that triggers the blind spot hazard warning is determined, -Reflections occurring in the cornea area of ​​the driver's eye, as determined by the driver observation camera, are displayed in a predefined representation, -the area of ​​the object leading to the blind spot warning is superimposed with the representation that reproduces the reflections in the cornea area of ​​the driver's eye and -the hazard warning is suppressed if the area of ​​the object leading to the blind spot hazard warning is represented in the corneal area of ​​the eye. [2] Method according to claim 1, characterized by , that before suppressing the danger signal, it is checked whether the object depicted in the corneal area of ​​the eye is a vehicle. [3] Method according to claim 2, characterized by that the suppression of the danger warning only occurs if the object depicted in the corneal area of ​​the eye is a vehicle. [4] Method according to claim 2 or 3, characterized by that the verification of the object imaged in the corneal area of ​​the eye is carried out using a pre-trained machine learning model. [5] Method according to claim 2, 3 or 4, characterized by that the review is repeated cyclically. [6] Method according to at least one of the preceding claims, characterized by, that the determined area of ​​the object leading to the blind spot hazard warning and the captured cornea images are transformed into a common coordinate system. [7] Method according to claim 6, characterized by , that the presentation of the corneal area of ​​the eye is rectified onto a 2D image plane, onto which the 3-dimensionally formed area of ​​the object leading to the blind spot hazard warning is superimposed. [8] Method according to claim 2, characterized by , that the hazard warning will be retained if the check reveals that the object depicted in the corneal area of ​​the eye is not a vehicle. [9] Method according to at least one of the preceding claims, characterized by , that the hazard warning will continue to be issued if the image data obtained from the driver observation camera suggests an object covering the cornea of ​​the eye. [10] Vehicle with a driver assistance system for carrying out the method according to at least one of the preceding claims, characterized by -a sensor system (7) for determining a vehicle's surroundings, -a driver observation camera (9) for capturing one eye of the driver (11) in the three-dimensional vehicle interior, -a warning unit (15) and -an evaluation unit (5) for superimposing the area of ​​the object leading to the blind spot hazard warning with the representation depicting the reflections in the cornea area of ​​the driver's eye (11) and for outputting a deactivation signal to the warning unit (15) when the area of ​​the object leading to the blind spot hazard warning is reflected in the representation of the cornea area of ​​the eye.

Images