Vehicle guidance system taking dark objects into account at night

Abstract

The invention relates to a system for a vehicle (1) for preventing a collision with an obstacle, comprising a primary sensor unit with a camera and a secondary sensor unit with a radar unit and / or lidar unit, and a computing unit configured to specify a maximum permissible speed for the vehicle (1) such that the vehicle (1) can be brought to a standstill within a line of sight (3) of the secondary sensor unit, to reduce the maximum permissible speed in the event of object detection without object classification, based on the data of the secondary sensor unit in front of the vehicle (1), so that the vehicle (1) can be brought to a standstill within a line of sight (5) of the primary sensor unit, and to specify a deceleration of the vehicle (1) in the event of object detection and object classification of a detected object as an obstacle, based on the data of the primary sensor unit.to bring the vehicle (1) to a standstill in front of the obstacle.

Description

[0001] The invention relates to a system for a vehicle to prevent a collision with an obstacle, a vehicle with such a system, and a method for preventing a collision with an obstacle.

[0002] A vehicle's camera is typically the central sensor for reliably and automatically classifying objects in the vehicle's vicinity, particularly for assessing whether an object can be driven over. Detecting dark objects like tires or tire debris poses a challenge when driving at night. A conventional camera system, in particular, relies on sufficient illumination from the vehicle's lighting system, such as low beams and / or high beams. For example, high beams are recommended when driving on an unlit highway with little traffic. However, even with high beams, dark objects are often not sufficiently illuminated at the necessary distances of approximately 150 meters; thus, they may not be visible to a camera, or at least not sufficiently so.In such a case, the resulting short braking distances required from the moment an object becomes visible, which represents the lower limit of the reaction time, limit the maximum driving speed in automated driving.

[0003] In the prior art, it is known to determine lighting information regarding a sub-area of ​​a camera image in order to be able to provide an estimate of the camera's visibility range based on this, and to adjust automated driving operation depending on the visibility range.

[0004] In this context, DE 10 2019 134 539 A1 relates to a device for determining the viewing range of a vehicle camera, which is configured to capture images relating to the vehicle's surroundings; wherein the vehicle includes a lighting element designed to illuminate at least part of the surroundings captured by the camera; wherein the device is designed to determine illumination information relating to a partial area of ​​a camera image that is illuminated by the lighting element; and to determine an estimated value of the camera's viewing range based on the illumination information.

[0005] The reliability of lidar and radar systems for object detection is typically lower than that of camera-based object detection. Therefore, it may be required that an automated decision regarding whether a detected object can be driven over be made solely based on camera data, and not on data from a lidar or radar system. This necessitates that, in principle, the vehicle speed be limited to a level that allows for stopping within the camera's field of view, which is particularly limited at night due to lighting conditions.

[0006] German patent DE 10 201 8 128 350 A1 discloses a system for detecting an object in the vicinity of a motor vehicle, wherein the system comprises a first sensor for classifying the object and wherein the classification capability of the first sensor depends on the brightness of the object. The system further comprises a second sensor for detecting the object and an actuator for increasing the brightness of the object. The actuator is configured to increase the brightness of the object when the second sensor detects the object and the first sensor cannot classify it.

[0007] From DE 10 201 5 003 960 A1, a method for determining the visibility of a vehicle in fog is known, in which at least one camera captures at least one image of at least part of the vehicle's surroundings and the visibility is determined based on this at least one image, comprising the following steps: statistical evaluation of at least one parameter of the image; determination of the presence of fog in the vehicle's surroundings as a function of the statistical evaluation; performance of object recognition by which at least one object in the image is identified; determination of at least one distance between the identified object and at least one other object; determination of the current time of day and / or the current position of the sun; determination of the visibility as a function of the distance, wherein the visibility is also determined as a function of the time of day and / or the position of the sun.

[0008] German patent DE 10 2013 210 928 A1 discloses a method for distinguishing between real obstacles and apparent obstacles in a driver assistance system for motor vehicles. This system includes a positioning system for determining the vehicle's own location and a radar sensor for measuring distances and relative speeds of radar targets. Location data for radar targets identified as apparent obstacles are stored in a database. If the driver assistance system detects a stationary radar target at a specific location, a query is sent to the database to determine whether an apparent obstacle is stored for that location.

[0009] German patent DE 10 2007 049 516 A1 discloses a system for the multimodal detection of objects in the field of view in front of and / or behind a vehicle. In this system, a radar sensor is used to detect the surroundings, and the radar sensor output signal is fed to a radar signal analysis method. A second set of surroundings is detected using a video sensor, and the video output signal is fed to a video analysis method. Object confirmation is achieved by feeding an object identified by the radar signal analysis method and an object identified by the video analysis method to an object confirmation and situation analysis module for verification and determination of a relevant object. Upon detection of a hazardous situation, measures to increase passive safety are initiated.

[0010] The object of the invention is to improve automated and / or assisted vehicle control at night in such a way that, compared to the problem described above, it is possible to drive at generally higher speeds.

[0011] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0012] A first aspect of the invention relates to a system for a vehicle for preventing a collision with an obstacle, comprising a primary sensor unit and a secondary sensor unit, wherein the primary sensor unit comprises a camera and the secondary sensor unit comprises a radar unit and / or a lidar unit, and comprising a computing unit configured to specify a maximum permissible speed for the vehicle such that the vehicle can be brought to a standstill within a line of sight of the secondary sensor unit, wherein, according to the invention, in the event of object detection without object classification, the maximum permissible speed is reduced in front of the vehicle based on the data from the secondary sensor unit to such an extent that the vehicle can be brought to a standstill within a line of sight of the primary sensor unit.and, in the case of object detection and object classification of a detected object as an obstacle based on the data of the primary sensor unit, to specify a deceleration of the vehicle in order to bring the vehicle to a standstill in front of the obstacle, wherein the computing unit is configured to perform an object classification of the data acquired by the primary sensor unit and / or secondary sensor unit, wherein the result of the object classification includes the categories "obstacle" and "drive-over object", wherein the computing unit is configured to continuously recalculate the visibility range (5) of the primary sensor unit, and wherein the computing unit is configured to determine the length of a respective light cone by evaluating an object detection of an infrastructure element carried out on the basis of the primary sensor unit.

[0013] The system is particularly suitable for use during night driving to detect dark objects at great distances from the vehicle, thus enabling the driver to drive quickly and safely, for example in automated driving mode, or in another application to protect the driver from dark obstacles during assisted driving.

[0014] The system helps to adapt the driving strategy during night driving. A two-stage procedure is used for this purpose. In the first stage, the secondary sensor unit, equipped with a radar unit and / or lidar unit, is used to detect objects as early as possible and outside the line of sight of the primary sensor unit and its camera unit. These objects are potentially recognized by the camera unit and the subsequent object classification as genuine obstacles, which by definition are considered "non-driveable".

[0015] The sensor data from the secondary sensor unit typically covers a greater range than that of the camera unit. However, it is less reliable, and if, as is typically the case, it is stipulated that reliable object classification may only be performed based on data from a camera unit, then braking the vehicle to a standstill in front of an obstacle that has been identified as such by the camera unit and the object classification system is unavoidable.

[0016] Accordingly, the processing unit sets the vehicle's maximum permissible speed so that, without successful object classification, particularly as an obstacle, the vehicle's braking distance is short enough to bring it to a complete stop within the line of sight of the secondary sensor unit. For example, if the secondary sensor unit reliably detects an obstacle, appropriate braking can be initiated. However, if an object is detected, but due to uncertain or nonexistent object classification it is unclear whether the obstacle is impassable or passable, the processing unit reduces the maximum permissible speed so that the vehicle travels at a lower speed, sufficient to allow it to brake to a complete stop before reaching the obstacle within the line of sight of the primary sensor unit.

[0017] Accordingly, if the vehicle's speed is reduced due to object detection by the secondary sensor unit, the vehicle advantageously has such a low speed at the exact moment the object enters the field of view of the primary sensor unit that, at the moment of the first object classification as an obstacle based on the data from the primary sensor unit, it is possible to brake the vehicle to a standstill even before the vehicle hits the obstacle.

[0018] The respective braking distances are preferably determined depending on current environmental conditions, so that a longer braking distance is expected to be factored in for a full braking maneuver in wet, snowy or slippery conditions.

[0019] Preferably, object classification is performed using an artificial neural network. This network is pre-trained and therefore typically has a finite set of possible object classification results. In a first embodiment, the result set can be restricted to the results "traversable" and "not traversable," where the result "not traversable" is the same as the result "obstacle."

[0020] The processing unit preferably comprises a microprocessor implemented as an integrated circuit, which performs arithmetic operations using data and / or programs stored, for example, in ROM (Read Only Memory), RAM (Random Access Memory), and / or FlashROM memory. The processing unit is configured to read and process sensor data and to control devices such as a cruise control system for setting the vehicle speed, an output unit for issuing warning signals, a headlight, or a driver protection device.

[0021] This approach offers a solution for increasing driving speed at night while avoiding collisions even with dark objects that might not be detected by a highly attentive driver. The critical scenario for night driving involves dark objects that, in extreme cases, cannot be detected by the camera and high beams. Therefore, according to the invention, sensors are used in the far range that are capable of detecting objects without the aid of the lighting system, albeit with higher uncertainties, which are compensated for by reducing the driving speed.

[0022] The processing unit is designed to perform an object classification of the data acquired by the primary and / or secondary sensor units, with the result of the object classification comprising the categories "obstacle" and "object that can be driven over." In one embodiment, only these two categories can be defined; in another embodiment, further categories can also be defined.

[0023] The processing unit is designed to continuously recalculate the line of sight of the primary sensor unit. Therefore, the definition of the primary sensor unit's line of sight is dynamic and depends in particular on weather conditions.

[0024] The processing unit is further designed to determine the line of sight of the primary sensor unit from the respective length of the vehicle's light cone and that of other road users. How far the length of the line of sight extends visibly, i.e., how far it preferably lies above a certain light intensity, provides information about the line of sight of the primary sensor unit.

[0025] According to a further advantageous embodiment, the computing unit is designed to perform a driving maneuver assigned to a respective result, depending on the result of the object classification based on the data from the secondary sensor unit.

[0026] The maneuver is preferably performed alongside decelerating the vehicle to a lower speed or bringing it to a standstill. For example, if the end of a traffic jam is reached or people are detected on the road, stopping and / or activating the hazard warning lights is preferred. If a moving object is detected, swerving around it is preferred.

[0027] According to a further advantageous embodiment, the processing unit is designed to continuously recalculate the viewing range of the primary sensor unit. Thus, the definition of the viewing range of the primary sensor unit is dynamic and depends in particular on weather-related visibility.

[0028] According to another advantageous embodiment, the viewing range of the secondary sensor unit is always greater than the viewing range of the primary sensor unit.

[0029] In a further embodiment, the processing unit is configured to control the vehicle's headlights based on data from the secondary sensor unit in front of the vehicle during object detection without object classification. This allows the headlights to direct a beam of light onto the object detection area, thus increasing the range of the primary sensor unit. The headlights are preferably designed as LED matrix headlights that direct a focused beam of light onto a selected area. In addition to focusing, the power of the LEDs directed onto the selected area, i.e., the area encompassing the detected object, is increased.

[0030] This ensures that other road users, such as oncoming traffic, are not dazzled, while simultaneously increasing the visibility range of the primary sensor unit, such as a camera. This allows for a higher maximum permissible speed, at which the vehicle may have to be brought to a standstill before the detected object, compared to a system without increased visibility.

[0031] According to a further advantageous embodiment, the computing unit is designed to control an output unit for issuing a warning signal to other road users and / or to prepare a device for driver protection when an obstacle is detected by object detection and object classification based on the data of the secondary sensor unit.

[0032] The driver protection device includes, for example, a seatbelt tensioner and / or an airbag.

[0033] Another aspect of the invention relates to a vehicle with a system as described above and below.

[0034] Advantages and preferred further developments of the proposed vehicle result from an analogous and substantive transfer of the above statements made in connection with the proposed system.

[0035] Another aspect of the invention relates to a method for preventing a collision with an obstacle, wherein a primary sensor unit comprising a camera and a secondary sensor unit comprising a radar unit and / or lidar unit are used, and a maximum permissible speed for the vehicle is specified by a computing unit such that the vehicle can be brought to a standstill within a line of sight of the secondary sensor unit, in the event of object detection without object classification based on the data of the secondary sensor unit in front of the vehicle the maximum permissible speed is reduced to such an extent that the vehicle can be brought to a standstill within a line of sight of the primary sensor unit, and in the event of object classification as an obstacle based on the data of the primary sensor unit a deceleration of the vehicle is specified in order to bring the vehicle to a standstill in front of the obstacle.

[0036] Advantages and preferred further developments of the proposed procedure result from an analogous and substantive transfer of the above statements made in connection with the proposed system.

[0037] Further advantages, features, and details will become apparent from the following description, in which – possibly with reference to the drawing – at least one embodiment is described in detail. Identical, similar, and / or functionally equivalent parts are identified by the same reference numerals.

[0038] It shows: Fig. 1: A situation in road traffic involving the application of a system according to an embodiment of the invention.

[0039] Fig.Figure 1 shows a driving situation of vehicle 1 in darkness. A primary sensor unit comprising a camera unit and a secondary sensor unit comprising a radar unit and / or lidar unit are active. Their sensor data is transmitted to a processing unit in vehicle 1 to perform object detection. The result of object detection is simply information about whether an object, in the abstract sense of the term, has been detected on the road ahead. Only object classification categorizes the detected object into a specific category, such as pedestrian, cyclist, vehicle tire, boulder, cargo, etc. A dark object is characterized by its low reflectivity. For example, the rubber of a tire is particularly poorly reflective and therefore harder for cameras to detect in the dark.The limiting factor in object classification detection is the lighting system, which, due to energy consumption and the risk of dazzling other road users, can only illuminate the road to a limited extent. Two detection ranges are used to identify dark objects, which then determine the driving strategy. Up to the full range of the secondary sensor unit, a camera unit in conjunction with high beams cannot detect dark objects. Therefore, measurement principles such as radar and lidar are used. These are capable of detecting dark objects, such as a black tire, at long distances. They do not rely on the headlight beam. However, these sensors have a lower resolution than a camera unit.This leads to significant uncertainties in classifying objects as drivable, particularly whether they are obstacles or easily driven over, such as a plastic bag or a road marking. Thus, an object detected within the range of the secondary sensor unit but outside the range of the primary sensor unit could also be small pieces of tire debris on the road, which the system vehicle could then easily drive over. This misclassification must be accepted and addressed through driving strategy. The range of the primary sensor unit with a camera unit is essentially limited by sufficient illumination of dark objects. The camera unit achieves higher accuracy in object measurement, and objects can be more precisely classified as drivable or non-drivable (i.e., obstacles) using object classification techniques such as "scene labeling."The detection range 5 of the primary sensor unit is preferably adjusted dynamically while the vehicle 1 is driving. For objects with high reflectivity, such as a wooden Euro pallet, this detection range 5 increases, prompting a corresponding adjustment. The driving strategy for dark objects is therefore implemented as follows: If a dark object is detected within the detection range 3 of the secondary sensor unit, the processing unit reduces the maximum permissible speed of the vehicle 1, and during the implementation of this speed reduction, a moderate deceleration of, for example, a maximum of 3 to 5 m / s is applied. 2This is done to prevent rear-end collisions with other road users, especially those behind the vehicle 1. If an object is detected within the camera unit's field of view 5 and identified as an obstacle (defined as a non-crossable object), the vehicle 1 brakes until it comes to a complete stop in front of the detected object. If, as the vehicle 1 approaches the object, repeated object classifications reveal that the object is crossable and therefore not an obstacle, the vehicle 1 can stop decelerating and maintain its speed or accelerate again. If necessary, subclasses of crossability can be defined as possible outcomes of the object classification, allowing the vehicle to pass over the object at a residual speed.If the visibility range 5 of the primary sensor unit is too great for the camera unit to require support from the secondary sensor unit, for example, because at the current driving speed a complete stop of the vehicle 1 up to the visibility range 5 of the primary sensor unit is possible anyway, then object classification can be performed with sufficient reliability and the intermediate step of reducing the speed of the vehicle 1 for objects detected by the secondary sensor unit can be skipped; in this case, only object classification based on the data from the camera unit is necessary, whereupon a decision is made as to whether it is an obstacle that cannot be driven over or an object that can be driven over.

[0040] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description. Reference symbol list 1 vehicle 3. Viewing range of the secondary sensor unit 5. Viewing range of the primary sensor unit

Claims

[1] System for a vehicle (1) for preventing a collision with an obstacle, comprising a primary sensor unit and a secondary sensor unit, wherein the primary sensor unit comprises a camera and the secondary sensor unit comprises a radar unit and / or a lidar unit, and comprising a computing unit configured to specify a maximum permissible speed for the vehicle (1) such that the vehicle (1) can be brought to a standstill within a line of sight (3) of the secondary sensor unit, In the case of object detection without object classification, the maximum permissible speed is reduced based on the data from the secondary sensor unit in front of the vehicle (1) to such an extent that the vehicle (1) can be brought to a standstill within a line of sight (5) of the primary sensor unit, and in the case of object detection and object classification of a detected object as an obstacle based on the data from the primary sensor unit, a deceleration of the vehicle (1) is specified in order to bring the vehicle (1) to a standstill in front of the obstacle, wherein the computing unit is configured to perform an object classification of the data acquired by the primary sensor unit and / or secondary sensor unit, wherein the result of the object classification includes the categories "obstacle" and "object that can be driven over". wherein the computing unit is designed to continuously recalculate the visibility range (5) of the primary sensor unit and wherein the computing unit is designed to determine the length of a respective light cone by evaluating an object detection of an infrastructure element carried out on the basis of the primary sensor unit. [2] System according to claim 1, wherein the computing unit is configured to perform a driving maneuver assigned to a respective result depending on the result of the object classification based on the data of the secondary sensor unit. [3] System according to claim 1, wherein the computing unit is configured to determine the length of a respective light cone by evaluating an object detection of an infrastructure element carried out on the basis of the primary sensor unit. [4] System according to one of the preceding claims, wherein the viewing range of the secondary sensor unit is always greater than the viewing range of the primary sensor unit. [5] System according to one of the preceding claims, wherein the computing unit is configured to control headlights of the vehicle on the basis of the data of the secondary sensor unit in front of the vehicle (1) in the event of object detection without object classification, so that these direct a beam of light onto the area of ​​object detection to increase the visibility range (5) of the primary sensor unit. [6] System according to one of the preceding claims, wherein the computing unit is configured to control an output unit for issuing a warning signal to other road users and / or to prepare a device for driver protection when an obstacle is detected by object detection and object classification based on the data of the secondary sensor unit. [7] Vehicle (1) with a system according to any of the preceding claims. [8] Method for preventing a collision with an obstacle, wherein a primary sensor unit comprising a camera and a secondary sensor unit comprising a radar unit and / or lidar unit are used, and a maximum permissible speed for the vehicle (1) is specified by a computing unit such that the vehicle (1) can be brought to a standstill within a line of sight (3) of the secondary sensor unit, in the event of object detection without object classification based on the data of the secondary sensor unit in front of the vehicle (1) the maximum permissible speed is reduced to such an extent that the vehicle (1) can be brought to a standstill within a line of sight (5) of the primary sensor unit, and in the event of object classification as an obstacle based on the data of the primary sensor unit a deceleration of the vehicle (1) is specified in order to bring the vehicle (1) to a standstill in front of the obstacle, wherein by means of the processing unit, an object classification of the data acquired by the primary sensor unit and / or secondary sensor unit is carried out, the result of the object classification comprising the categories "obstacle" and "object to be driven over", The viewing range (5) of the primary sensor unit is continuously recalculated by means of the computing unit, and the length of each light cone is determined by evaluating an object detection of an infrastructure element based on the primary sensor unit.

Images