Method for predictive exposure control of a vehicle camera
The predictive exposure control method for vehicle cameras addresses exposure control issues by anticipating shadow entry and adjusting exposure values, stabilizing image quality and recognition performance.
Patent Information
- Application Number
- DE102017206247
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2017-04-11
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2037-04-11
AI Technical Summary
Current exposure control methods for vehicle cameras struggle to maintain ideal image exposure during rapid transitions between bright and shadowy areas, leading to performance issues in object and lane recognition.
A predictive exposure control method that detects shadow areas in advance, calculates the time until the vehicle enters the shadow, and adjusts the exposure value accordingly to compensate for the expected change in illumination.
Ensures stable image exposure during light-shadow transitions, enhancing the performance of object and lane recognition algorithms.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for predictive exposure control of a vehicle camera.
[0002] Today's driver assistance systems typically include one or more cameras mounted on a vehicle that capture its surroundings. During image capture, the dynamic range of brightness, especially in bright sunlight, is very high. Current camera sensors cannot capture this full dynamic range and therefore require exposure control to adapt to the specific situation. In shade, this typically involves longer exposure times or greater amplification of the raw signal compared to brighter areas, such as direct sunlight. This ensures that the images capture the maximum amount of information.
[0003] Typically, exposure control works by evaluating the currently recorded image and, depending on whether the image is too bright or too dark, adjusting the exposure time of the next image in the sequence accordingly. Usually, a time-based filter is also active, meaning that if the vehicle enters a shadowed area, it can take several images (up to approximately 200 ms) until the images are ideally exposed for the new lighting conditions.
[0004] When switching between direct sunlight and a shaded area (and vice versa), there are short periods (100 ms to 200 ms) during which the images are not ideally exposed, and therefore the algorithms processing these images cannot deliver optimal performance. This can lead to failures in object detection, traffic sign recognition, or lane detection, for example.
[0005] US Patent 2012 / 0162424A1 discloses an exposure control system for a vehicle in which exposure values are determined as a function of the distance to the vehicle within a predefined area of an image captured by a camera. If the difference between these exposure values and a current exposure value exceeds a threshold, the exposure is adjusted based on the assumption that the modified exposure value will be suitable for the overall image. This approach can be problematic if the vehicle does not approach the predefined area uniformly. With more complex vehicle movements, such as cornering, or in cases of irregular shadows, reliable exposure control is either impossible or extremely difficult.
[0006] The object of the present invention is to provide a method for controlling the exposure of a vehicle camera which overcomes the aforementioned disadvantages.
[0007] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments are the subject matter of the dependent claims, the following description, and the figures.
[0008] According to a first aspect of the invention, a method for predictive exposure control of a vehicle camera is provided.
[0009] The method comprises providing a camera mounted on a vehicle and capturing at least one image of the vehicle's surroundings with the camera. Within the at least one image, a shadow area is detected, and the remaining time until the vehicle with the camera is expected to enter the shadow area is determined. Furthermore, a necessary change in the exposure value is determined to compensate for any darkening that occurs when the vehicle with the camera is within the shadow area.
[0010] At least one image contains the vehicle's external surroundings, in which the vehicle is potentially moving. In particular, the area in front of the vehicle can be captured by the camera and included in the image. A shadow area can be detected within this area. Known methods for detecting shadows in digital images can be applied to identify this shadow area.
[0011] The invention takes advantage of the fact that large areas of shadow or cast shadows on the road typically lead to a change in the exposure of the camera sensor when it moves into the shadow area from a brighter environment.
[0012] Depending on the distance between the detected shadow area and the vehicle or camera, and the vehicle's movement (especially direction, speed, acceleration, braking, etc.), a remaining time until the vehicle enters the shadow area can be calculated. Based on this calculated remaining time and a reference time or the current time, a time can be determined at which the vehicle is expected to enter the shadow area.
[0013] The required change in exposure value can be automatically set at the precise moment when the vehicle is expected to enter the shadow area, for example, using the image processing unit. This ensures that the exposure value is already set when the vehicle enters the detected shadow area, compensating for the darkening that occurs when the vehicle and camera are within the shadow area.
[0014] The method according to the first aspect of the invention combines the detection of cast shadows, the determination of the time until entry into this area, the change in illumination, and the exposure control of the sensor into a predictive method. Thus, the exposure control can be made more robust in situations with alternating light and shadows.
[0015] The method according to the first aspect of the invention can further be used, for example, in dashcams, surround-view systems, video cameras or smartphones.
[0016] According to one embodiment, the required change in exposure value is determined by identifying local brightness differences within the at least one image between the detected shadow area and unshaded areas outside the detected shadow area.
[0017] Furthermore, the vehicle's movement values can be determined, and based on these values, the remaining time until the vehicle is expected to enter the shadow area can be calculated. This method of calculating the remaining time is particularly efficient because movement values such as the vehicle's speed and direction are typically determined anyway and can be used for the method according to the present invention with little or no additional effort.
[0018] According to a further embodiment, it is provided that the detected shadow area is scaled within several images taken successively over time, wherein the images are taken by means of the camera from the external environment of the vehicle, and based on the result of this scaling of the shadow area over time, the remaining period is determined until the vehicle with the camera is expected to enter the shadow area.
[0019] According to a second aspect of the invention, a device for predictive exposure control is provided. The device comprises a camera and an image processing unit, wherein the device is configured by means of the camera and the image processing unit to carry out a method according to the first aspect of the invention.
[0020] According to a third aspect of the invention, a vehicle is provided which includes a device according to the second aspect of the invention. The vehicle is, for example, a motor vehicle such as a car, bus, or truck.
[0021] According to a fourth aspect of the invention, a program element is provided which, when executed on a processor, directs the processor to carry out the process steps of the method according to the first aspect of the invention, wherein all process steps are carried out which take place after the provision of the camera which is arranged on the vehicle.
[0022] According to a fifth aspect of the invention, a computer-readable medium is provided on which a program element is stored which, when executed on a processor, directs the processor to carry out the process steps of the method according to the first aspect of the invention, wherein all process steps are carried out which take place after the provision of the camera which is arranged on the vehicle.
[0023] Exemplary embodiments of the invention are explained in more detail below with reference to the schematic drawing. This shows Fig. 1 a side view of a motor vehicle with an embodiment of a device according to the invention and Fig. 2. a camera of the device Fig. 1. Image taken showing a shadow cast on a road, onto which the vehicle moved after Fig. 1 moves.
[0024] Fig. Figure 1 shows a vehicle 1 in the form of an automobile. The vehicle 1 comprises a device 2 for predictive exposure control of a vehicle camera, wherein the device 2 in turn comprises an image-generating sensor in the form of a camera 3 and an image processing unit 4. The image processing unit 4 can, in particular, be software-implemented. In the illustrated embodiment, a software-implemented image processing unit 4 can, for example, be stored on a memory unit of the camera 3.
[0025] The camera 3 is arranged in an interior 5 of the vehicle 1, specifically in an area behind a windshield 6 of the vehicle 1. By means of the camera 3, one or more external shots or images of an external environment 7 of the vehicle 1, in particular in a forward direction of the vehicle 1, can be taken through the windshield 6, whereby in the case of several external shots the external shots can be taken successively.
[0026] Fig. Figure 2 shows an example image 8, which can be captured by camera 2. Image 8 shows a forward area in the outer environment 7 of vehicle 1 and includes, in particular, a road 9 on which vehicle 1 is moving in its forward direction, an object in the form of a house 10 to the side of road 9, and a shadow 11 cast by the house 10, especially onto road 9.
[0027] The device 2 is configured to detect the shadow 11 within the image 8, for example, using known methods that can be applied by the image processing unit 4. In the illustrated embodiment, the vehicle 1 moves along the road 9 in the direction of the shadow 11. The vehicle 1 travels, for example, at a speed v, which can be determined by the vehicle 1 in a known manner. Using known image processing methods, the image processing unit 4 can, for example, calculate the distance by which the shadow 11 is currently separated from the vehicle 1.
[0028] Based on the calculated distance and the movement (direction and speed) of vehicle 1, the remaining time until vehicle 1 with camera 3 enters the shadow area 11 can be calculated. If the current time is known, the time at which vehicle 1 with camera 3 will enter the shadow area 11 can be calculated based on this current time and the calculated remaining time. Alternatively, the remaining time and the aforementioned time at which vehicle 1 with camera 3 will enter the shadow area can also be calculated based on a scaling of the shadow area 11 over time.The detected shadow area 11 can be scaled across several images taken sequentially, with the images being captured by camera 3 from the external surroundings 7 of vehicle 1. Based on the result of this scaling over time, the remaining time until vehicle 1, as shown by camera 3, is expected to enter the shadow area 11 can then be determined.
[0029] The area of shadow 11 is less bright than the unshaded area. The difference in brightness between the area of shadow 11 and the unshaded area allows for the determination of a necessary change in the exposure value. This ensures that when vehicle 1 with camera 3 enters the area of shadow 11, the exposure value is increased accordingly to compensate for the lower brightness in this area. In this way, the camera sensor 3 can be predictively set and exposed correctly when entering the area of shadow 11. This allows the camera algorithms to run more reliably.
Claims
[1] Method for predictive exposure control of a vehicle camera (3), the method comprising the steps: - Providing a camera (3) which is arranged on a vehicle (1), - taking at least one image (8) of an external environment (7) of the vehicle (1) by means of the camera (3), - detecting a shadow area (11) within the at least one image (8), - determining a remaining period of time until the vehicle (1) with the camera (3) is expected to enter the shadow area (11), and - Determining a required change in an exposure value to compensate for a darkening that occurs when the vehicle (1) with the camera (3) is located within the shadow area (11), wherein the required change in the exposure value to compensate for a lower brightness in the shadow area (11) is determined by determining, within the at least one image (8), local brightness differences between the detected shadow area (11) and non-shadowed areas outside the detected shadow area (11) and a difference in brightness in the detected shadow area (12) and non-shadowed areas outside the detected shadow area (11) [2] Method according to claim 1, wherein movement values of the vehicle (1) are determined, and based on the determined movement values, the remaining period of time is determined until the vehicle (1) with the camera (3) is expected to enter the shadow area (11). [3] Method according to one of the preceding claims, wherein the detected shadow area (11) is scaled within a plurality of images taken successively in time, the images being taken by means of the camera (3) of the external environment (7) of the vehicle (1), and based on the result of this scaling over time, the remaining period of time is determined until the vehicle (1) with the camera (3) is expected to enter the shadow area (11). [4] Device (2) for predictive exposure control of a vehicle camera (3), the device (2) comprising a camera (3) for a vehicle (1) and an image processing unit (4), wherein the device (2) is configured by means of the camera (3) and the image processing unit (4) to carry out the steps of a method according to one of the preceding claims. [5] Vehicle (1) comprising a device (2) according to claim 4. [6] Program element which, when executed on a processor, instructs the processor to perform the method steps according to any one of claims 1 to 3. [7] A computer-readable medium having stored thereon a program element which, when executed on a processor, instructs the processor to perform the method steps according to any one of claims 1 to 3.
Citation Information
Patent Citations
Exposure control apparatus for vehicle-mounted camera
US20120162424A1
