Device and method for identifying objects and states in the region from a vehicle

Abstract

The invention relates to a device for detecting objects and states in an area (8) in front of a vehicle. The invention relates to a device for analyzing and evaluating an area in front of a vehicle, comprising a headlight (1) for emitting light into the area in front of the vehicle in accordance with a predefined light distribution, a camera (2) for capturing an image (13) of the area in front of the vehicle, and an image evaluation device (3) for evaluating the image captured by the camera, smoothing the acquired image as an original image by means of the smoothing process to form an intermediate image; a normalization process by means of which the original image and the intermediate image are mathematically normalized into a target image, such that the target image has a normalized light intensity value; and an image analysis and evaluation process by means of which the light intensity values of the target image are analyzed and evaluated in order to identify structures and / or objects in the area in front of the vehicle.

Description

Technical Field

[0001] The present invention relates to a device for object and state recognition in an area in front of a vehicle, the device comprising a headlamp for emitting light into the area in front of the vehicle according to a predetermined light distribution, a camera for capturing an image of the area in front of the vehicle, and an image analysis and evaluation device for analyzing and evaluating the image captured by the camera.

[0002] The invention further relates to a method for object and state recognition in an area in front of a vehicle, wherein headlights generate a light distribution in the area in front of the vehicle and wherein a camera records an image of the area in front of the vehicle, wherein the image recorded by the camera is evaluated. Background Art

[0003] DE 10 2016 210 379 A1 discloses a device for state recognition in the area ahead of a vehicle, which device has a lidar sensor. The sensor signal provided by the lidar sensor is evaluated so that the properties of the road surface are determined.

[0004] EP3138731B1 discloses a device and a method for evaluating the state of a headlamp. The device comprises a headlamp for emitting light into the area in front of the vehicle, a camera for capturing an image of the area in front of the vehicle, and an image analysis and evaluation device for analyzing and evaluating the image captured by the camera. It is desirable to further increase the information content during image analysis and evaluation. Summary of the invention

[0005] It is therefore an object of the present invention to further develop a device and a method for object and state recognition in the area ahead of a vehicle in such a way that the quality of the detection of radiation sources is improved in a simple manner.

[0006] In order to achieve the above-mentioned purpose, the present invention is characterized in that, in combination with the preamble of claim 1, the image analysis and evaluation device comprises: a smoothing process, by means of which the acquired image is smoothed as an original image to form an intermediate image; a standardization process, by means of which the original image and the intermediate image are standardized into a target image by mathematical operations, so that the target image has a standardized light intensity value; and an image analysis and evaluation process, by means of which the light intensity value of the target image is analyzed and evaluated for identifying structures and / or objects in the area in front of the vehicle.

[0007] The information content of the image is significantly increased by the image processing according to the present invention, so that objects and / or state parameters in the area in front of the vehicle can be detected or recognized more reliably by means of a temporally subsequent algorithm. According to the present invention, the raw image captured by the camera is smoothed and then standardized into a target image. Only the standardized target image is used for the analysis and evaluation or for the analysis and evaluation process, which, compared with the raw image, also allows the identification of structures in the area in front of the vehicle or in the far area of ​​the light distribution. The light distribution emitted by the headlight usually has such a characteristic that the illumination intensity decreases as the distance from the vehicle increases, which means that the far area in the area in front of the vehicle is illuminated with a weaker intensity than the near area in front of the vehicle. This disadvantage can be compensated by means of the image processing or image analysis and evaluation according to the present invention, so that structures can be recognized not only in the near field but also in the far field. The structure is generated due to the light intensity difference within the target image, which can be determined by a suitable analysis and evaluation algorithm. The present invention can especially achieve better and more accurate identification of objects not only in the near area in front of the vehicle but also in the far area. In particular, objects or objects with a reflectivity similar to that of the roadway in the area in front of the vehicle can be identified. According to the present invention, the image processing according to the present invention can more accurately extract the projected object or subject.

[0008] According to a preferred embodiment of the present invention, the smoothing process includes a Gaussian smoothing filter, by means of which the original image is blurred, thereby the original image is smoothed to eliminate any structure in the image.

[0009] According to a further development of the invention, the smoothed intermediate image is normalized, wherein the image data or pixel data of the original image are used for the normalization. This results in a normalization of the light intensity values. Compared to the original image, the calculated target image now has an increased contrast, in particular in the distant areas of the recorded area in front of the vehicle, so that the structure of the target image can be evaluated comprehensively and more simply.

[0010] According to a further development of the invention, the emitted light distribution can be designed as a line projection with a plurality of illuminated strips extending in the direction of travel or in the main radiation direction of the headlight. By means of the image processing or evaluation according to the invention, the illuminated strips extend in a straight line and with a constant light intensity both in the near region and in the far region of the image. Advantageously, the angular change of the illuminated strips relative to a standard strip or to another illuminated strip can be determined as a function of the pitch angle of the vehicle. Advantageously, the current pitch angle of the vehicle can thereby be determined simply and quickly.

[0011] To achieve this object, the method according to the invention has the features of claim 10 .

[0012] A particular advantage of the method according to the invention is that the image recorded by the camera is transformed on a software basis to such an extent that structures based on light intensity differences can be detected within the image both in the near region of the recorded area in front of the vehicle and in the far region. Advantageously, the light distribution can be formed more evenly by the image evaluation according to the invention. The determined light intensity values ​​of the target image are distributed more evenly over the relatively large far region and the near region.

[0013] Further advantages of the invention are revealed in the further dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The embodiments of the present invention are described in more detail below with the aid of the accompanying drawings.

[0015] Figure 1 A block diagram showing a device according to the present invention;

[0016] Figure 2 showing a raw image of the area in front of the vehicle with structures in the near field;

[0017] Figure 3 An intermediate image showing the area in front of the vehicle calculated by smoothing the original image;

[0018] Figure 4 showing a normalized target image calculated from the intermediate image;

[0019] Figure 5 a schematic diagram showing line projections when a vehicle is traveling at a pitch angle of zero; and

[0020] Figure 6 The diagram shows a line projection when the vehicle is traveling at a pitch angle greater than zero. DETAILED DESCRIPTION

[0021] according to Figure 1 The device according to the invention essentially comprises a headlight 1 , a camera 2 and an image analysis and evaluation device 3 .

[0022] The headlamp 1 is designed as a high-resolution headlamp having a plurality of individually controllable light sources 4, which are arranged on a carrier plate 5. The headlamp 1 also has an optical unit 6 with a plurality of lenses, by means of which a predetermined light distribution 7 is projected into a vehicle front area 8. The light sources 4 are preferably designed as LED light sources or LED pixels.

[0023] The camera 2 is preferably arranged on the windshield of the vehicle in a region of the vehicle 9 close to the roof.

[0024] The image analysis and evaluation device 3 is a component of an analysis and evaluation unit which has a processor and a program memory in which at least one analysis and evaluation program or analysis and evaluation algorithm is stored.

[0025] Furthermore, an actuation unit 10 is provided for actuating the light source 4 . The actuation signal 11 generated by the actuation unit 10 for the light source 4 is realized as a function of the type of object recognition or state recognition in the area 8 in front of the vehicle.

[0026] According to a first embodiment of the present invention, the control unit 10 generates such a control signal 11 that the headlamp 1 generates a rectangular light distribution 12 on the measuring screen. Such a light distribution may be, for example, a city light distribution or a low beam distribution.

[0027] The camera 2 detects the area 8 in front of the vehicle and generates a recording of an image 13 , which is provided as a raw image to the image analysis and evaluation device 3 for use.

[0028] The image analysis and evaluation device 3 is software-based and has one or more algorithms, with the aid of which the raw image 13 is further processed.

[0029] The image analysis and evaluation device 3 has a smoothing process 14 as a mathematical operation, by means of which the acquired raw image 13 is smoothed and a Figure 3 Intermediate image 15 shown.

[0030] The smoothing process 14 includes a Gaussian smoothing filter, by means of which the pixel values ​​of the original image 13 are weighted according to the distance between the pixel value and the central pixel according to a two-dimensional discrete Gaussian function (bell curve). The Gaussian function is:

[0031]

[0032] Among them, x and y are coordinates, and sigmaσ is a factor.

[0033] For the center pixels arranged in the matrix below, for example, the following evaluation of the brightness value is obtained:

[0034]

[0035] The weights of the 25 pixel values ​​arranged in the filter matrix are not equal, but depend on their distance from the center pixel.

[0036] The larger the sigmaσ value is, the higher the smoothness is. For example, σ can take a minimum value of 8, preferably 18.

[0037] In the analysis and evaluation Figure 2 In the raw image 13 shown, only structures 16 can be determined in the near region 17 of the area 8 in front of the vehicle, which structures can be identified as potholes, for example. In order to be able to analyze reliably also in the far region 21 of the area 8 in front of the vehicle, the raw image 13 is smoothed with a Gaussian filter matrix, so that according to Figure 3 Intermediate image 15. It can be seen here that structures 16 can be detected as larger white spots, which are caused by soft edges. Such intermediate images cannot be used for state recognition or object status.

[0038] In a further step of the image analysis evaluation, a normalization process 18 is now applied, by means of which a standardization process is generated from the intermediate image 15 according to Figure 4 A target image 19 is generated which forms the basis for a final structural analysis evaluation by an image analysis evaluation process 20 .

[0039] In this embodiment, the normalization process 18 is formed such that the quotient of the brightness value of the original image 13 and the brightness value of the intermediate image 15 is calculated, as shown in the following formula:

[0040]

[0041] from Figure 4 As can be seen in the figure, the target image 19 has not only the structure 16 in the near region 17, but also the structure 16' in the far region 21 of the vehicle front region 8. By applying the image analysis and evaluation process 20 to the target image 19, it is now possible to identify not only the potholes 16 in the near region 17, but also the potholes 16' in the far region 21, whereas when the original image 13 was analyzed and evaluated, only the potholes 16 in the near region 17 of the vehicle front region 8 could be identified. With the help of the present invention, the dark regions in the far region and the bright regions in the near region 17 can be balanced with each other by normalizing the intensity values ​​of the dark regions in the (far away) far region 21 that are less illuminated. This results in a more homogeneous target image 19 compared to the original image 17, which leads to better recognizability of objects in the far region 21.

[0042] Due to the higher light intensity difference or higher contrast, structures 16' can be detected in the far region 21. The smoothing and normalization algorithms according to the invention achieve a "shift" of the light from the near region 17 of the vehicle front region 8 or of the image into the far region 21, so that structures in the far region 21 can also be detected.

[0043] Additionally or alternatively, the object image 19 may have a higher contrast in the near region 17 of the area 8 in front of the vehicle than in the near region 17 of the raw image 13 , so that objects located in the near region 17 can be better recognized.

[0044] The analysis and evaluation program including the smoothing process 14, the standardization process 18 and the image analysis and evaluation process 20 is stored in a memory 22 of the image analysis and evaluation device 3. The image analysis and evaluation device 3 can have archived, for example, reference image data or comparison data in the memory 22, which the image analysis and evaluation process 20 can access. By comparing the target image with a predetermined reference image or reference data, the state or structure 16, 16' in the area 8 in front of the vehicle can be determined. Preferably, a comparison is performed between the target image 19 and the reference image.

[0045] according to Figure 5 and Figure 6 According to the second embodiment of the present invention, the control unit 10 can generate such a control signal 11 that the light source 4 in combination with the optical unit 6 generates a light distribution 12' in the form of a line projection or illuminated strip 23, which extends along the main radiation direction H of the headlamp 1, that is, along the longitudinal direction of the vehicle or the driving direction. Figure 5 In the horizontal position shown, the illuminated strips 23 have a length L1 and the outer illuminated strips 23 enclose an opening angle α1. is 0°.

[0046] For example, if the vehicle 9 brakes (where the pitch angle greater than 0°), see Figure 6 , then each illuminated strip 23 diverges, wherein each illuminated strip has a reduced length L2 and the opening angle α2 is greater than the opening angle α1 in the horizontal position of the vehicle 9. By means of the image analysis evaluation according to the present invention by means of smoothing and normalizing the illuminated strips 23, it is possible to reliably identify the changed opening angle α2 compared to the initial opening angle α1 and determine the current pitch angle of the vehicle based on the correlation rule.

[0047] The image evaluation according to the invention thus enables the state of vehicle 9 or of the area in front of the vehicle 8 to be detected. In addition, the image evaluation can also be used for improved detection of objects such as pedestrians, signs, etc. in the area in front of the vehicle 8.

[0048] According to an alternative embodiment of the present invention (not shown), an infrared radiation source or a radiation source in another wavelength range may be used instead of the LED light source 4 as the radiation source.

[0049] For example, the invention can be used to detect a road surface in the form of snow, rain or moisture as a structure.

[0050] Reference numerals list

[0051] 1 headlight

[0052] 2 Camera

[0053] 3 Image analysis and evaluation device

[0054] 4 Light Source

[0055] 5 Support plate

[0056] 6 Optical unit

[0057] 7 Light Distribution

[0058] 8 Area in front of the vehicle

[0059] 9 Vehicles

[0060] 10 Control unit

[0061] 11 Control Signals

[0062] 12, 12' light distribution

[0063] 13 images / original images

[0064] 14 Smooth Process

[0065] 15 Intermediate Image

[0066] 16, 16' structure

[0067] 17 Nearby area

[0068] 18 Standardized Process

[0069] 19 Intermediate Image

[0070] 20 Image Analysis and Evaluation Process

[0071] 21 Distant Areas

[0072] 22 Memory

[0073] 23 Light Bar

[0074] H Main radiation direction

[0075] L1, L2 length

[0076] α1, α2 angle

[0077] Pitch Angle

Claims

1. A device for detecting objects and states in an area (8) in front of a vehicle, the device comprising a headlamp (1) for emitting light into the area (8) in front of the vehicle according to a predetermined light distribution (12, 12'), a camera (2) for recording an image (13) of the area (8) in front of the vehicle, an image evaluation device (3) for evaluating the image (13) recorded by the camera (2), It is characterized in that The image analysis and evaluation device (3) comprises: a smoothing process (14), by means of which the acquired image (13) is smoothed as an original image to form an intermediate image (15), a normalization process (18), by means of which the original image (13) and the intermediate image (15) are normalized by mathematical operations to form a target image (19), so that the target image (19) has normalized light intensity values, An image analysis and evaluation process (20) is used to analyze and evaluate the light intensity values ​​of the target image (19) in order to detect structures (16, 16') and / or objects in the area (8) in front of the vehicle.

2. The device according to claim 1, It is characterized in that The smoothing process (14), the standardization process (18) and the image analysis and evaluation process (20) are constructed based on software.

3. The device according to claim 1 or 2, It is characterized in that The smoothing process (14) includes a Gaussian smoothing filter, by means of which the pixel values ​​of the original image (13) are weighted according to the distance between the pixel values ​​and the central pixel in accordance with a two-dimensional discrete Gaussian function.

4. The device according to any one of claims 1 to 3, It is characterized in that The normalization process (18) is designed so that the quotient of the original image (13) and the intermediate image (15) is calculated pixel by pixel, and each pixel of the calculated quotient forms the target image (19).

5. The device according to any one of claims 1 to 4, It is characterized in that The Gaussian smoothing formula has a σ of at least 8.

6. The device according to any one of claims 1 to 5, It is characterized in that The target image (19) has a greater contrast in a near region (17) of the area in front of the vehicle (8) than in a near region (17) of the original image (13) and / or the target image has a greater contrast in a far region (21) of the area in front of the vehicle than in a far region (21) of the original image, so that objects and / or lane characteristics located in the near region (17) and / or in the far region (21) can be identified by analyzing and evaluating the target image (19).

7. The device according to any one of claims 1 to 6, It is characterized in that The characteristic features of the roadway can be detected as potholes and / or a roadway surface and / or an object in the form of snow, rain or moisture.

8. The device according to any one of claims 1 to 7, It is characterized in that The line projection comprises a plurality of illuminated strips (23) extending in the direction of travel and / or in the main radiation direction (H), the angles of the illuminated strips relative to one another varying as a function of the pitch angle (φ) of the vehicle (9) relative to the horizontal plane.

9. The device according to any one of claims 1 to 8, It is characterized in that The headlight (1) is designed as a high-resolution headlight.

10. A method for object and state recognition in an area (8) in front of a vehicle, in, A headlamp (1) generates a light distribution (12, 12') in an area (8) in front of a vehicle, wherein a camera (2) records an image (13) of the area (8) in front of the vehicle, wherein the image (13) recorded by the camera (2) is evaluated, characterized in that: - smoothing the acquired image (13) as a raw image into an intermediate image (15) by means of mathematical operations, - calculating a standardized target image (19) from the original image (13) and the intermediate image (15) by means of mathematical operations, - evaluating the target image (19) with regard to brightness differences in the target image (19) in order to detect structures (16, 16') or objects in the target image (19).

11. The method according to claim 10, It is characterized in that The original image (13) is transformed into an intermediate image (15) by means of Gaussian filtering.

12. The method according to claim 10 or 11, It is characterized in that The target image (19) is calculated by formulaically dividing the brightness values ​​of the pixels of the original image (13) by the brightness values ​​of the pixels of the intermediate image (15), and only the pixel values ​​of the target image (19) are analyzed and evaluated.

13. The method according to any one of claims 10 to 12, It is characterized in that The target image (19) is checked for the presence of area-by-area or pixel-by-pixel differences in light intensity, so that the characteristic structure of the roadway can be inferred therefrom.

14. The method according to any one of claims 10 to 13, It is characterized in that The target image (19) is compared with a reference image or with a reference value, thereby identifying structural particles in a certain field of view.

Images