Method and device for determining the viewing range of a vehicle camera
The method and device use vehicle headlights and a neural network to determine camera visibility in low-light conditions, addressing reduced illumination issues, ensuring safe adaptive vehicle operations.
Patent Information
- Application Number
- DE102023003121
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-07-28
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2043-07-28
AI Technical Summary
Existing methods for determining the visibility of a vehicle camera in low-light conditions, such as at night, are inadequate, particularly when headlight components fail, leading to reduced illumination and impaired camera detection range, which compromises safety in automated or autonomous driving.
A method and device utilizing vehicle headlights with light-emitting diodes to illuminate the camera's detection area, coupled with a voltage monitoring system and data processing unit, including a neural network, to determine the camera's visibility based on captured image data and electrical voltage, enabling adaptive vehicle operations.
Enables accurate, reliable, and cost-effective determination of camera visibility, allowing vehicle functions to adapt to reduced visibility, enhancing safety in automated and autonomous driving scenarios.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for determining a visibility range of a camera of a vehicle.
[0002] The invention further relates to a device for determining a visibility range of a camera of a vehicle.
[0003] The invention further relates to a method for operating a vehicle.
[0004] DE 10 2022 131 400 A1 discloses a processor having one or more circuits for calculating data using a neural network and based on image data of an environment generated using one or more image sensors. The data is representative of one or more parameters of an operating range corresponding to the environment. Based on the data, an operating level of an ego machine corresponding to the environment is determined, wherein operation of the ego machine is controlled according to the operating level.
[0005] Furthermore, it is known from Chaabani, H. et al.: A Neural network approach to visibility range estimation under foggy weather conditions; International Workshop on Connected & Intelligent Mobility (CIM 2017); ScienceDirect Procedia Computer Science 113 (2017) 466-471, available at www.sciencedirect.com, that deterioration in visibility due to foggy weather conditions is a frequent trigger for traffic accidents. A neural network approach for estimating visibility range using a camera mounted on a moving vehicle is described. It uses a single camera and a neural network learning phase based on a hybrid global feature descriptor.
[0006] From DE 10 2007 030 825 A1 a method for determining the visibility range for a vehicle is known, in which it is provided that, when the headlights of the vehicle are switched on, an area not illuminated by the headlights is detected by means of a camera, the brightness of this area is determined and, based on this, the visibility is determined.
[0007] DE 10 2011 081 392 A1 discloses a method for determining the headlight range of a vehicle's headlights. Road marking features are illuminated by the headlights and detected by a vehicle camera. The headlight range is then determined based on the distance to the road marking features.
[0008] From DE 10 2019 134 539 A1 a method for determining the visibility of a camera of a vehicle is known, whereby it is provided that the size of an illuminated area is determined by means of the camera by means of the headlights of the vehicle and that the visibility is determined from this.
[0009] The invention is based on the object of providing a novel method and a novel device for determining a visibility range of a camera of a vehicle as well as a novel method for operating a vehicle.
[0010] The object is achieved according to the invention by a method for determining a visibility range of a camera of a vehicle, which has the features specified in claim 1, by a device which has the features specified in claim 5, and by a method for operating a vehicle which has the features specified in claim 6.
[0011] Advantageous embodiments of the invention are the subject of the subclaims.
[0012] In the method for determining a visibility range of a camera of a vehicle in the dark, an area of the vehicle's surroundings that is at least partially within the detection range of the camera is illuminated by at least one headlight arranged on the vehicle, which headlight comprises light-emitting diodes as a light source, while the vehicle is being operated in the dark. An electrical voltage applied to at least one electrical connection of the headlight is detected, with image data detected by the camera and the detected electrical voltage being fed to an on-board data processing unit. The data processing unit automatically determines the visibility range of the camera based on the image data and the electrical voltage.
[0013] In the dark, especially at night, cameras require a light source to illuminate their detection area in order to capture it. This light source is provided by at least one headlight of the vehicle, for example a headlight. For example, on highways or other unlit traffic routes, when there is no external light source and no other vehicles in the immediate vicinity of the vehicle, the camera can only capture the vehicle's surroundings in those areas illuminated by the light emitted by the headlight. Thus, the camera's field of view correlates directly with the light distribution generated by the headlight, in particular the headlight's beam range.
[0014] If individual LEDs fail in a headlight that uses light-emitting diodes as a light source, for example, a so-called LED array, the light distribution generated by the headlight is impaired. This can result in insufficient illumination of the vehicle's surroundings within the camera's detection range, for example, a reduced beam range, which impairs the camera's ability to detect the surroundings in the dark. Such a failure of individual LEDs results in a reduction in the electrical voltage applied to the headlight's electrical connection.
[0015] By monitoring this voltage, the present method can be used to accurately determine the camera's visibility in the dark in a simple, reliable, and cost-effective manner. This, in turn, allows vehicle functions, such as driver assistance functions, to be automatically adapted to the camera's visibility, thereby increasing vehicle safety during driving, for example, in automated, particularly highly automated or autonomous, driving modes.
[0016] According to one possible embodiment of the method, the data processing unit comprises a trained artificial neural network, which is used to determine the camera's field of view. This enables particularly reliable automatic determination of the camera's field of view. Alternatively, the field of view can also be determined using other algorithms executed by the data processing unit.
[0017] According to another possible embodiment of the method, if the electrical voltage falls below a predetermined threshold, a reduced camera visibility is inferred. By specifying such a threshold, a specific deviation of the light distribution actually generated by the headlight from a predetermined light distribution, and thus a fall below a certain illumination range, can be easily specified as an indicator of a reduced camera visibility.
[0018] According to another possible embodiment of the method, several voltage thresholds are specified, with each voltage threshold being assigned a camera viewing distance. This allows for stepwise detection of the camera's viewing distance and, if a sufficiently large number of voltage thresholds are specified, for virtually stepless detection of the viewing distance.
[0019] The device for determining the visibility of a camera of a vehicle in the dark is characterized according to the invention by - at least one headlight arranged on the vehicle, which comprises light-emitting diodes as lighting means and is designed to illuminate an area of the vehicle's surroundings that is at least partially within the detection range of the camera during operation of the vehicle in the dark, - a voltage monitoring device which is designed to detect an electrical voltage applied to at least one electrical connection of the headlight, and - a data processing unit coupled to the camera and the voltage monitoring device, which is designed to automatically determine the field of view of the camera based on image data captured by the camera and the detected electrical voltage.
[0020] By monitoring the voltage of at least one headlight, the present device can be used to accurately determine the camera's visibility in the dark in a simple, reliable, and cost-effective manner. This, in turn, allows vehicle functions, such as driver assistance functions, to be automatically adapted to the camera's visibility, resulting in increased vehicle safety during driving, for example, in automated, particularly highly automated or autonomous, driving modes.
[0021] In the method for operating a vehicle, according to the invention, the visibility of the vehicle's camera in the dark is determined by means of an aforementioned method and if a visibility limit value is undershot, - a warning is issued inside the vehicle and / or - in an automated driving mode of the vehicle, a maximum permissible speed of the vehicle is reduced and / or - in an automated driving mode of the vehicle, a request to take over a driving task is transmitted to a vehicle user and / or - in an automated driving mode of the vehicle, the vehicle is brought to a standstill.
[0022] Driver assistance systems and automated driving functions require various sensors to detect the vehicle's surroundings. A particularly inexpensive and reliable sensor is the camera. By determining the camera's visibility in the dark, its visibility can be monitored during vehicle operation. If at least one headlight degrades, this allows, for example, the vehicle's speed to be adjusted to the visibility during driving or the driver to be transferred to the vehicle operator if visibility falls below a certain visibility threshold, at which safe automated operation of the vehicle is no longer possible.
[0023] Embodiments of the invention are explained in more detail below with reference to drawings.
[0024] Showing: Fig. 1 schematically shows a plan view of a vehicle on a roadway with full light output from headlights, Fig. 2 schematically shows a plan view of a vehicle on a roadway with a reduced light output of headlights and Fig. 3 schematically shows a block diagram of a device for determining a visibility range of a camera of a vehicle.
[0025] Corresponding parts are provided with the same reference numerals in all figures.
[0026] In Fig. 1 shows a plan view of a vehicle 1 located on a roadway FB with full light output from headlights 2, 3 of the vehicle 1 designed as headlights. Fig. 2 shows the vehicle 1 on the roadway FB with a reduced light output of the headlights 2, 3.
[0027] The vehicle 1 includes a camera 4, which is designed to capture the surroundings of the vehicle 1. The camera 4 is directed, in particular, in the longitudinal direction of the vehicle 1, toward an area located in front of the vehicle.
[0028] In particular, the vehicle 1 is designed for automated, for example highly automated or autonomous driving, which, among other things, depends on the data recorded by the camera 4 and stored in Fig. 3 image data D shown in more detail.
[0029] When driving in the dark, for example at night, the detection range of the camera 4 is limited by a light distribution LV generated by the headlights 2, 3. In the illustrated embodiments, this means that a visibility range SW of the camera 4 at least substantially corresponds to a light range of the headlights 2, 3 and is reduced in the example with reduced light output compared to the example with full light output of the headlights 2, 3.
[0030] The headlights 2, 3 comprise a plurality of light-emitting diodes as illuminating means, for example, arranged in a so-called light-emitting diode array. Reduced light output of the headlights 2, 3 can result, for example, from a failure, i.e., a malfunction, of individual light-emitting diodes.
[0031] In the event of such a failure, there is a reduction in the electrical voltage U1, U2 applied to an electrical connection of the respective headlight 2, 3 (shown in Fig. 3) compared to a standard voltage with non-failing LEDs.
[0032] In Fig. 3 is a possible embodiment of a device 5 for determining a visibility range SW of a camera 4 of a vehicle 1, for example the camera 4 of the vehicle 1 according to the Fig. 1 and Fig. 2, shown in the dark.
[0033] The device 5 comprises the headlights 2, 3 arranged on the vehicle 1, a voltage monitoring device 6 and a data processing unit 7.
[0034] To detect the visibility SW of camera 4, the behavior of the voltage U1, U2 at the electrical connection of the respective headlight 2, 3 is monitored as described above.
[0035] In this case, the voltage monitoring device 6 detects the electrical voltages U1, U2 present at the electrical connections of the headlights 2, 3 while the vehicle 1 is traveling in the dark and transmits them to the data processing unit 7.
[0036] Furthermore, image data D captured by the camera 4 are transmitted to the data processing unit 7.
[0037] Based on the image data D and the electrical voltages U1, U2, the data processing unit 7 automatically determines the viewing range SW of the camera 4. In particular, the data processing unit 7 comprises a trained artificial neural network, for example, a deep artificial neural network. Another suitable algorithm can also be executed by the data processing unit 7 for this purpose.
[0038] In particular, the data processing unit 7 concludes that the viewing range SW of the camera 4 is reduced if the electrical voltages U1, U2 fall below a predetermined voltage threshold. For example, several voltage thresholds are predetermined, with each voltage threshold being assigned a viewing range SW of the camera 4.
[0039] The data processing unit 7 is coupled to a device 8 for operating the vehicle 1, wherein by means of the device 8, when the visibility SW of the camera 4 falls below a predetermined visibility limit value, for example - a corresponding warning is issued within vehicle 1 and / or - in an automated driving mode of the vehicle 1, a maximum permissible speed of the vehicle 1 is reduced and / or - in the automated driving mode of vehicle 1, a request to take over a driving task is transmitted to a vehicle user and / or - in the automated driving mode of vehicle 1, vehicle 1 is brought to a standstill, for example in a safe location.
Claims
[1] Method for determining a visibility range (SW) of a camera (4) of a vehicle (1) in the dark, characterized by , that - during operation of the vehicle (1) in the dark, an area of the surroundings of the vehicle (1) which is at least partially within the detection range of the camera (4) is illuminated by means of at least one headlight (2, 3) arranged on the vehicle (1) which comprises light-emitting diodes as lighting means, - an electrical voltage (U1, U2) applied to at least one electrical connection of the headlight (2, 3) is detected, - image data (D) captured by the camera (4) and the detected electrical voltage (U1, U2) are fed to a vehicle-specific data processing unit (7) and - the visibility range (SW) of the camera (4) is determined automatically by means of the data processing unit (7) on the basis of the image data (D) and the electrical voltage (U1, U2). [2] Method according to claim 1, characterized by that the data processing unit (7) comprises a trained artificial neural network by means of which the field of view (SW) of the camera (4) is determined. [3] Method according to claim 1 or 2, characterized by that if the electrical voltage (U1, U2) falls below a predetermined voltage limit, it is concluded that the field of view (SW) of the camera (4) is reduced. [4] Method according to claim 3, characterized by that several voltage limit values are specified, each voltage limit value being assigned a viewing range (SW) of the camera (4). [5] Device (5) for determining a visibility range (SW) of a camera (4) of a vehicle (1) in the dark, characterized by - at least one headlight (2, 3) arranged on the vehicle (1), which comprises light-emitting diodes as lighting means and is designed to illuminate an area of the surroundings of the vehicle (1) that is at least partially located within the detection range of the camera (4) during operation of the vehicle (1) in the dark, - a voltage monitoring device (6) which is designed to detect an electrical voltage (U1, U2) applied to at least one electrical connection of the headlight (2, 3), and - a data processing unit (7) coupled to the camera (4) and the voltage monitoring device (6), which is designed to automatically determine the field of view (SW) of the camera (4) on the basis of image data (D) acquired by the camera (4) and the acquired electrical voltage (U1, U2). [6] Method for operating a vehicle (1), wherein by means of a method according to one of claims 1 to 4 the visibility range (SW) of the camera (4) of the vehicle (1) is determined in the dark and when a visibility limit value is undershot - a warning is issued inside the vehicle (1) and / or - in an automated driving mode of the vehicle (1), a maximum permissible speed of the vehicle (1) is reduced and / or - in an automated driving mode of the vehicle (1) a request to take over a driving task is transmitted to a vehicle user and / or - in an automated driving operation of the vehicle (1), the vehicle (1) is brought to a standstill.
Citation Information
Patent Citations
vision determination device for a vehicle
DE102007030825A1
Method for calibrating the light emission of at least one headlight of a vehicle
DE102011081392A1
Method and device for determining the viewing range of a camera
DE102019134539A1
Deep learning-based verification of operating areas using camera-based inputs for autonomous systems and applications
DE102022131400A1
