Method and apparatus for detecting dirt on a lidar window

Abstract

The invention relates to a method and a device for detecting dirt on a window of a laser radar, wherein - a laser beam is emitted by an emitter of the laser radar into a detection area, and - light present in the detection area is detected by a receiver of the laser radar, characterized in that - an intensity image of a gray-scale image as a laser reflection intensity is generated from the light reflected and detected as a result of the emission of the laser beam, - a background light image of a gray-scale image as background light is generated from the light detected without emission of the laser beam, - common features of the intensity image and the background light image are analyzed, and - a conclusion is drawn that dirt is present on the window when a preset number of common features is below.

Description

Technical Field

[0001] The present invention relates to a method for detecting dirt on the window of a lidar according to the preamble of claim 1.

[0002] The present invention also relates to an apparatus for detecting dirt on the window of a lidar according to the preamble of claim 7. Background Technology

[0003] Identifying dirt on the viewport of a LiDAR system is challenging, especially for autonomous vehicles (e.g., Level 3 or above). Dirt can degrade sensor performance, thus limiting the safety and availability of the systems in question. LiDAR is an active sensor that includes a transmitter (e.g., one or more laser diodes) and a receiver (e.g., one or more avalanche photodiodes, particularly single-photon avalanche diodes).

[0004] A method for identifying dirt in a lidar system is known from DE 10 2017 222 618 A1, which includes the following steps:

[0005] - Electromagnetic radiation is directionally emitted into the surrounding environment of the lidar system via the transmitting unit.

[0006] The electromagnetic radiation emitted by the transmitting unit is transmitted to the surrounding environment through the exit window, which separates the lidar system from the surrounding environment at least in the direction of emission of the transmitting unit.

[0007] - A portion of the electromagnetic radiation emitted by the transmitting unit and scattered back to the lidar system from the surface of the exit window is detected by a dirt sensor. This dirt sensor can be a single diode, a one-dimensional array detector, or a two-dimensional array surface detector used to detect the electromagnetic radiation.

[0008] / or the dirt sensor is integrated into the receiving unit;

[0009] - The dirt in the exit window is determined by evaluating the detection results of the dirt sensor. Summary of the Invention

[0010] The purpose of this invention is to provide an improved method and apparatus for detecting dirt on the window of a lidar, compared to existing technologies.

[0011] According to the invention, this objective is achieved by a method having the features specified in claim 1 and an apparatus having the features specified in claim 7.

[0012] Advantageous designs of the present invention are the subject of the dependent claims.

[0013] According to the present invention, a method for detecting dirt on the viewing window (e.g., windshield) of a lidar is proposed, wherein,

[0014] - The laser beam is emitted into the detection area by the transmitter of the lidar, and

[0015] - The light present in the detection area is detected by the receiver of the lidar.

[0016] According to the provisions of the present invention,

[0017] - An intensity image is generated from the light that is reflected and detected as a result of laser beam emission, serving as a grayscale image of the laser reflection intensity.

[0018] - A background light image generated from the light detected without emitting a laser beam, serving as the background light in a grayscale image.

[0019] -Analyze the common features of intensity images and background light images, and

[0020] - When the number of common features is below the preset amount, it is concluded that there is dirt on the window.

[0021] Due to active illumination from inside the sensor, the intensity of the reflected light, relative to the background light, shows different sensitivity to dirt on the viewing window, thus allowing for the assessment of dirt.

[0022] In one implementation, the laser beam is emitted in pulses.

[0023] In one implementation, the background light image is determined shortly before the laser beam is emitted. When using this method in a moving motor vehicle, it is particularly advantageous to draw the background light image and the intensity image in a close temporal order, since approximately the same scene is drawn in the two grayscale images.

[0024] In one implementation, features of buildings and / or vehicles and / or windows are determined from intensity images and background light images.

[0025] In one implementation, an edge detection algorithm is used to identify edges in intensity images and background light images.

[0026] In one implementation, edge distance and / or edge location are determined as features based on the identified edges.

[0027] According to one aspect of the present invention, an apparatus for detecting dirt on the window of a lidar is provided, the apparatus comprising a data processing unit connected to the lidar and configured to perform the method described above.

[0028] Furthermore, a motor vehicle incorporating such a device is proposed, particularly, for example, a Level 3 or higher autonomous driving vehicle.

[0029] The application of the above method or device in motor vehicles is also proposed. It can also be used in other autonomous platforms that use LiDAR for navigation (such as trucks, buses, or robots). Attached Figure Description

[0030] Embodiments of the present invention will now be explained in more detail with reference to the accompanying drawings.

[0031] In the attached diagram:

[0032] Figure 1 This diagram illustrates a grayscale image of the intensity of reflected laser radiation recorded by a lidar when the viewport is clean.

[0033] Figure 2 This diagram illustrates a grayscale image of the background light recorded by a lidar when the viewport is clean.

[0034] Figure 3 This diagram illustrates a grayscale image of the intensity of reflected laser radiation recorded by a lidar system when there are water droplets in the viewport.

[0035] Figure 4 This diagram illustrates a grayscale image of the background light recorded by a lidar with water droplets on the viewport.

[0036] In all the accompanying drawings, corresponding parts are labeled with the same reference numerals. Detailed Implementation

[0037] This invention relates to a method for detecting dirt on the viewing window (e.g., windshield) of a lidar sensor. A lidar is an active sensor having at least one transmitter (e.g., one or more laser diodes) and at least one receiver (e.g., one or more avalanche photodiodes, particularly single-photon avalanche diodes). In the method according to the invention, the transmitter emits a pulsed laser beam, and the receiver detects the reflection of the laser beam by objects within the detection area. In addition to distance information, supplementary information, such as the intensity of scene reflection and background light, is provided in a suitable receiver. This supplementary information reveals different sensitivities to dirt on the viewing window. Background light can be determined, for example, by the receiver recording a grayscale image when the transmitter is not emitting a laser beam (e.g., shortly before emitting the laser beam).

[0038] Figure 1 This is a schematic diagram of a grayscale image (hereinafter referred to as an intensity image) of the intensity of reflected laser radiation recorded by a lidar when the viewport is clean. Figure 2This is a schematic diagram of a grayscale image of the background light (hereinafter referred to as the background light image) recorded by a LiDAR with the aid of a clean viewing window. Building B, vehicle V, and window W can be visually identified in both images.

[0039] Therefore, for LiDAR that provides this additional information, image processing methods are used to compare appropriate features, such as edges, particularly the edges of road markings or windows in building walls, between the intensity image and the background light image. Edges can be extracted from the background light image and the intensity image using edge detection algorithms known from the field of image processing. The edge features of the two resulting edge images, such as edge distance and edge position, are then calculated and compared. This comparison yields a measure of the similarity between the intensity image and the background light image. If the number of common features exceeds a certain threshold, the images are interpreted as similar, and the conclusion is drawn that no dirt is present.

[0040] If as Figure 3 and Figure 4 If few or no common features are found, meaning the number of common features does not exceed a specific threshold, the image is interpreted as dissimilar, and the conclusion that viewport dirt exists is drawn.

[0041] Figure 3 This is a schematic diagram of a grayscale image (hereinafter referred to as an intensity image) of the intensity of reflected laser radiation recorded by a lidar when there are water droplets on the viewing window. Figure 4 This is a schematic diagram of a grayscale image of the background light (hereinafter referred to as the background light image) recorded by a lidar system when there are water droplets on the viewing window. Figure 4 Building B, vehicle V, and window W are clearly identifiable in the background light image. Figure 3 It is difficult or impossible to identify in the intensity image.

[0042] The threshold can be determined specifically for the sensor. This method can be used for the entire detection area of ​​a LiDAR or for a specific segment for localized dirt detection.

[0043] The proposed dirt detection method may have limited effectiveness if there are few or no structures / edges within the field of view of a lidar sensor, such as when recording a monochromatic wall or sky. However, this is an exception when used in road traffic. The minimum number of features (especially structures and / or edges) considered necessary for dirt detection in the sensor's field of view can be determined sensor-specifically.

[0044] List of reference numerals

[0045] Building B

[0046] V vehicle

[0047] W window

Claims

1. A method for detecting dirt on the window of a lidar, wherein... - The laser beam is emitted into the detection area by the transmitter of the lidar, and - The light present in the detection area is detected by the receiver of the lidar, characterized in that, - An intensity image is generated from the light that is reflected and detected as a result of laser beam emission, serving as a grayscale image of the laser reflection intensity. - A background light image generated from the light detected without emitting a laser beam, serving as the background light in a grayscale image. - Analyze the common features of the intensity image and the background light image, wherein features of buildings and / or vehicles and / or windows are determined in the intensity image and the background light image, and - When the number of common features is below a preset amount, it is concluded that dirt exists on the window.

2. The method according to claim 1, Its features are, The laser beam is emitted in pulses.

3. The method according to claim 1 or 2, Its features are, The background light image is determined shortly before the laser beam is emitted.

4. The method according to any one of the preceding claims, Its features are, Edges are identified in the intensity image and the background light image using an edge detection algorithm.

5. The method according to claim 4, Its features are, Based on the identified edges, determine the edge distance and / or edge location as features.

6. An apparatus for detecting dirt on a window of a lidar, the apparatus comprising a data processing unit connected to the lidar and configured to perform the method according to any one of claims 1 to 5.

7. A motor vehicle comprising the device according to claim 6.

8. The motor vehicle according to claim 7, wherein the motor vehicle is configured as an autonomous driving vehicle.

9. The application of the method according to any one of claims 1 to 5 or the apparatus according to claim 6 in a motor vehicle.

Images