Method and device for identifying contamination on a protective screen of a laser radar sensor

Abstract

The invention relates to a method and a device for identifying contamination (V) on a protective screen (1.1) of a laser radar sensor (1). According to the invention, a sector background noise is determined in a respective sector (S1 to Sn) of a detection area (S) of the laser radar sensor (1) and a detection area background noise is determined in the remaining detection area (S') or the entire detection area (S), and if the sector background noise is significantly lower than the detection area background noise, it is concluded that there is contamination (V) in the respective sector (S1 to Sn). Alternatively or additionally, the sector background noise is determined in the respective sector (S1 to Sn) at different sensitivities of a receiver of the laser radar sensor (1), and if the sector background noise determined at a higher sensitivity is not significantly higher than the sector background noise determined at a lower sensitivity, it is concluded that there is contamination (V) in the respective sector (S1 to Sn).

Description

Technical Field

[0001] This invention relates to a method for identifying contamination on the protective screen of a lidar sensor.

[0002] The present invention also relates to a device for identifying contamination on a protective screen of a lidar sensor and the application of such a device in vehicles and / or robots. Background Technology

[0003] DE 19948252A1 discloses a method for state recognition in an automated longitudinal and lateral control system for a motor vehicle, which operates based on lidar principles to identify sensor contamination. State recognition depends on two indicators based on signals received and transmitted by the sensors. The indicators are weighted using weighting factors and associated with a unique probability, from which a statement about sensor contamination is derived if a predetermined threshold is exceeded or fallen below for a predetermined duration. This duration is chosen to be longer at low vehicle speeds than at high vehicle speeds. Object stability, representing the failure rate of detection of target objects selected by the vehicle's longitudinal control, is used as an indicator, summing all objects detected during the measurement period.

[0004] Furthermore, DE102012112987B3 discloses a photoelectric sensor for detecting objects in a monitored area and determining their distance. This sensor has an analysis unit configured to determine the degree of visual ambiguity based on the angle of visual ambiguity in the direction of the sensor's emitted beam.

[0005] US2019 / 0107609A1 describes a lidar device with a foreign object detector that identifies obstructed states of the lidar device's transmission window by measuring the signal-to-noise ratio.

[0006] Furthermore, US 10317534B2 describes a lidar system with a field of view divided into multiple segments. A noise level is measured in each of these segments, and the sensitivity is adjusted based on the noise level determined in the respective segment. Summary of the Invention

[0007] The objective of this invention is to provide a method and apparatus for identifying contamination on the protective screen of a lidar sensor, which is an improvement over existing technologies, as well as the application of such apparatus.

[0008] According to the present invention, the task is solved by the method according to the present invention, the apparatus according to the present invention, and the application according to the present invention.

[0009] According to the present invention, in a method for identifying contamination on a protective screen of a lidar sensor, the detection area of ​​the lidar sensor is divided into multiple sectors, and the presence of contamination on the protective screen is determined sector by sector. For this purpose, sector background noise is determined in each sector at different sensitivities of the lidar sensor's receiver. If the sector background noise determined at a higher sensitivity is not significantly higher than the sector background noise determined at a lower sensitivity, contamination is inferred to exist in the corresponding sector. Optionally, additionally, sector background noise is determined in the corresponding sector, and detection area background noise is determined in the remaining detection area or the entire detection area. If the sector background noise is significantly lower than the detection area background noise, contamination is inferred to exist in the corresponding sector.

[0010] In this context, the terms “significantly lower than” and “significantly higher than” are understood to mean that the difference between the values ​​being compared is at least identifiable and / or higher than a predetermined threshold.

[0011] Contamination or dirt on the protective screen or front panel of a lidar sensor can degrade its performance and thus limit the safety and availability of systems that rely on lidar sensor data, such as automated, especially highly automated or autonomous vehicles and / or robots. Therefore, identifying contamination on lidar sensor protective screens poses a challenge for such systems, such as in Level 3+ autonomous vehicles.

[0012] A lidar sensor emits a laser pulse or beam and detects the reflection of that pulse or beam from an object within a detection area. If contamination is present on the protective screen, the power of both the received reflected light and the background light will decrease. Given the sensitivity of the lidar sensor's receiver, the background light defines the noise characteristics of the lidar system, including the lidar sensor. The presence of contamination will reduce both the quantity and intensity of this noise.

[0013] This method can reliably identify performance degradation of lidar sensors caused by contamination of the protective screen. Therefore, appropriate measures can be taken to ensure and / or increase the security and availability of systems using lidar sensor data.

[0014] If contamination is determined by comparing sector background noise at different sensitivities of the lidar sensor's receiver, spatial inhomogeneity has a relatively small impact because the sector is primarily compared to itself, and this comparison is performed at a scan frequency of 10 Hz over short time intervals, such as 100 ms. Therefore, effects from shadows or other sources of interference have a minimal impact on the results.

[0015] The choice of the threshold used for comparison depends, for example, on the desired level of safety in the sensor design and / or system design of the LiDAR sensor. That is, a lower threshold is chosen, for example, in a so-called Level 4 system, which is constructed with particular conservatism for autonomous vehicles or robots, than in a so-called Level 2 system where a driver or controller takes over the driving task. Alternatively or additionally, the choice of threshold may depend, for example, on the desired application area of ​​the system using the LiDAR sensor data.

[0016] In one possible implementation of the method, alternatively or additionally, if the background noise of a sector determined at higher sensitivity is not at least 10% higher than the background noise of a sector determined at lower sensitivity, then contamination is inferred in the corresponding sector.

[0017] In one possible implementation of this method, if the background noise of a sector is at least 10% lower than the background noise of the detection area, it is inferred that contamination exists in the corresponding sector.

[0018] In another possible implementation of this method, the sector background noise and the detection area background noise are determined based on the background light intensity detected during the signal transmission time between the emission and reception of the infrared laser pulse during at least one scan. This allows for a particularly simple and reliable determination of the sector background noise and the detection area background noise.

[0019] In another possible implementation of this method, the intensity of the background light is detected over the entire signal transmission time, and the effective value of the intensity is determined. This also allows for a particularly simple and reliable determination of sector background noise and detection area background noise.

[0020] In another possible implementation of this method, the background noise of the detection area is determined by determining the noise level of the background light in the remaining detection area or the entire detection area. This allows for a particularly simple and reliable determination of the background noise of the detection area.

[0021] In another possible implementation of this method, sector background noise is determined by determining the noise level of the background light during exactly one scan in the corresponding sector. This allows for particularly simple and reliable determination of sector background noise.

[0022] In another possible implementation of this method, sector background noise is determined by identifying the noise level of the background light during multiple or all scans performed in the corresponding sector. This also allows for a particularly simple and reliable determination of sector background noise.

[0023] In another possible implementation of this method, sector background noise is determined as follows: all power received in the corresponding sector by means of the receiver is integrated; reflected power received in the corresponding sector by means of the receiver based on reflections of multiple infrared laser pulses is integrated; and the total noise power describing the sector background noise in the corresponding sector is determined by the difference between the integrated received power and the integrated reflected power. This also allows for particularly simple and reliable determination of sector background noise and eliminates any possible dependence on the distance to the first received infrared laser pulse.

[0024] In another possible implementation of this method, the sensitivity of the receiver is set by shifting the operating point or by changing the receiver's internal amplification factor. Increasing the sensitivity of the lidar sensor also increases the background noise generated by the background light.

[0025] According to the present invention, an apparatus for identifying contamination on a protective screen of a lidar sensor is characterized in that: the detection area of ​​the lidar sensor is divided into multiple sectors, and a data processing unit is configured to determine, sector by sector, whether contamination exists on the protective screen in a corresponding sector. To this end, the data processing unit determines the sector background noise in the corresponding sector at different sensitivities of the lidar sensor's receiver, and if the sector background noise determined at a higher sensitivity is not significantly higher than the sector background noise determined at a lower sensitivity, it infers that contamination exists in the corresponding sector. Optionally, the data processing unit further determines the sector background noise in the corresponding sector and the detection area background noise in the remaining detection area or the entire detection area, and if the sector background noise is significantly lower than the detection area background noise, it infers that contamination exists in the corresponding sector.

[0026] The aforementioned device can reliably identify performance degradation of lidar sensors caused by contamination of the protective screen. Appropriate measures can then be taken to ensure and / or increase the security and availability of systems using lidar sensor data.

[0027] In one possible application of the aforementioned device in vehicles and / or robots, the at least one lidar sensor is configured for environmental detection. With the aid of this device, contamination that hinders environmental detection by the protective shield of the at least one lidar sensor can be identified simply and reliably.

[0028] In one possible implementation of this application, the automation, particularly highly automated or autonomous operation of the vehicle and / or robot, is performed based on data detected by the at least one lidar sensor, and automated operation is restricted and / or at least one measure is taken to eliminate the at least one contaminant when at least one contaminant is detected on the protective screen of the at least one lidar sensor. This can significantly improve the reliability and safety of vehicle and / or robot operation. Attached Figure Description

[0029] Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The drawings are as follows:

[0030] Figure 1 A schematic perspective view showing a lidar sensor and multiple objects; and

[0031] Figure 2 The diagram schematically illustrates the time curves of the transmitted and received signals from the lidar sensor. Detailed Implementation

[0032] In all the accompanying drawings, the corresponding parts are given the same reference numerals.

[0033] Figure 1 A perspective view showing lidar sensor 1 and multiple objects O1 to O3 is shown. Figure 2 The diagram shows the curves of infrared laser pulse P1 emitted by lidar sensor 1 as a function of time t, and the curves of reflected light P2 received by the receiver of lidar sensor 1 as a function of time t. Reflection P2 is generated by the reflection of the emitted laser pulse P1 onto at least one object O1 to O3.

[0034] The lidar sensor 1 is, for example, a component of a vehicle and / or a robot (not shown in detail) and is configured here for environmental detection. Automation, particularly highly automated or autonomous operation, of the vehicle and / or robot is performed based on data detected by the lidar sensor 1.

[0035] The lidar sensor 1 includes a laser assembly (not shown in detail) for generating infrared laser pulses P1, which are emitted onto the scene to be scanned. The laser assembly may include a single laser diode or an array of laser diodes.

[0036] The lidar sensor 1 also includes a receiver, not shown in detail, configured as a photodetector assembly, for detecting infrared received pulses, i.e., reflections P2, which are reflected back from the scene to be scanned. Here, the photodetector assembly may include a single receiving diode or an array of receiving diodes.

[0037] The laser pulse P1 and reflection P2 can be directed directly from the laser assembly to the scene or from the scene to the receiver, or directly from the scene to the receiver, or via a deflection device (not shown in detail). The deflection device may include a rotating mirror, a rotating prism, and / or an array of deflectable micromirrors.

[0038] The laser assembly and receiver are positioned behind at least one protective screen 1.1 (also known as a protective window or lidar window) and are therefore at least largely protected from mechanical damage and foreign object intrusion. A laser pulse P1 is transmitted from the laser assembly through the protective screen 1.1 to the scene. A reflected pulse P2 travels from the scene through the protective screen 1.1 to the receiver.

[0039] The scanning or detection area S of the lidar sensor 1 is divided into multiple sectors S1 to Sn. An infrared laser pulse P1 generated by the laser assembly passes through a protective screen 1.1 and is guided to the scene to be scanned, where it is reflected back by objects O1 to O3. The reflected infrared laser pulse, i.e., reflected P2, reaches the receiver via the protective screen 1.1 or via another protective screen (not shown).

[0040] If contamination V is present in the region of sector S1 on protective screen 1.1 or another protective screen (not shown), the power of the reflected P2 received in sector S1 and the power of the received background light L will both decrease. Therefore, the noise level of the background light L in sector S1 will also decrease. This noise level is hereinafter also referred to as background noise.

[0041] The background noise, along with the sensitivity of lidar sensor 1, determines the noise characteristics of lidar sensor 1. The sensitivity of lidar sensor 1 is here determined by, for example, the sensitivity of a receiver configured as a photodetector assembly. Figure 2 Background noise can be seen in the time-varying curves of the received reflected light P2 and the background light L.

[0042] according to Figure 2 The illustration shows that, in order to scan the scene, the laser component is positioned at a specific time point T. P A laser pulse P1 is generated. This laser pulse is deflected onto the scene to be scanned. In environmental detection using the lidar sensor 1, in particular, multiple lidar pulses P1 are used to scan individual sectors S1 to Sn of the detection area S.

[0043] Laser pulse P1 is reflected back to lidar sensor 1 by objects O1 to O3 in the scene as reflection P2. The signal transmission time T is related to the distance between the corresponding objects O1 to O3 and lidar sensor 1. L Subsequently, the corresponding reflection P2 reaches the receiver of lidar sensor 1 and is detected by it as a received pulse.

[0044] The signal transmission time T between the emission of laser pulse P1 and the reception of the associated reflection P2 L In the process, the receiver detects noise in the background light L. For example, by measuring noise over a predetermined time range, such as during signal transmission time T. L The intensity of the background light L is detected and the noise level of the background light L is determined by determining the effective value of the intensity detected within that time range.

[0045] The background noise in a sector S1 to Sn (hereinafter referred to as sector background noise) is determined by determining the noise level of the background light L in exactly one scan performed in the respective sector S1 to Sn. Alternatively, the sector background noise is determined by determining the noise level of the background light L in multiple or all scans performed in the respective sector S1 to Sn.

[0046] As an alternative, sector background noise is determined by integrating all power received by the receiver in the corresponding sectors S1 to Sn and integrating the reflected power received by the receiver in the corresponding sectors S1 to Sn based on reflections P2 of multiple infrared laser pulses P1. Then, the total noise power describing the sector background noise in the corresponding sectors S1 to Sn is determined from the difference between the integrated received power and the integrated reflected power, thereby eliminating any possible dependence on the distance to the first received reflection P2.

[0047] The inspection for contamination V on the protective screen 1.1 is performed sector by sector, particularly using a data processing unit (not shown in detail). This inspection checks for contamination V on the protective screen 1.1 in the corresponding sectors S1 to Sn.

[0048] The following description uses sector S1 as an example to illustrate this check. Other sectors S2 through Sn are checked in a similar manner.

[0049] During scanning across the entire detection area S, the sector background noise in sector S1 is determined, and additionally, the detection area background noise across the entire detection area S is determined, or the detection area background noise in the remaining detection area S' formed by subtracting sector S1 and consisting of sectors S2 to Sn is determined. The detection area background noise is also determined, for example, by determining the noise level of the background light L in the remaining detection area S' or the entire detection area S. If the sector background noise is significantly lower than the detection area background noise, it is inferred that contamination V of the protective screen 1.1 exists in sector S1.

[0050] Alternatively or additionally, the sector background noise in sector S1 is determined at multiple different time points with different receiver sensitivity settings. Here, the receiver sensitivity is set, for example, by shifting the operating point or by changing the receiver's internal amplification factor. Specifically, measurements are performed at different sensitivities within short time intervals of, for example, 100 ms, at a scan frequency of 10 Hz. If the sector background noise determined at a higher sensitivity is not significantly higher than that determined at a lower sensitivity, then contamination V of the guardrail 1.1 is inferred to be present in sector S1. The determination of contamination V in this embodiment has the advantage that spatial non-uniformity has only a small impact on the measurement results because sector S1 is primarily compared to itself, and this comparison is based on the scan frequency within short time intervals. Therefore, the effects of shadows or interference sources have a relatively small impact on the measurement results.

[0051] When at least one contaminant V is identified on the protective screen 1.1 of the lidar sensor 1, automated operation can be restricted and / or at least one measure can be taken to eliminate the at least one contaminant V when applied in vehicles and / or robots.

[0052] List of reference numerals

[0053] 1 radar sensor

[0054] 1.1 Protective Screen

[0055] L background light

[0056] Objects O1 to O3

[0057] P1 laser pulse

[0058] P2 reflection

[0059] S detection area

[0060] S1 to Sn sectors

[0061] S' remaining detection area

[0062] t time

[0063] T L Signal transmission time

[0064] T P Time point

[0065] V pollution

Claims

1. A method for identifying contamination on a protective screen (1.1) of a lidar sensor (1), characterized in that, The detection area of ​​the lidar sensor (1) is divided into multiple sectors, and it is determined sector by sector whether there is contamination on the protective screen (1.1) in the corresponding sector. For this purpose, the background noise of the sector is determined at different sensitivities of the receiver of the lidar sensor (1) in the corresponding sector. If the background noise of the sector determined at a higher sensitivity is not significantly higher than the background noise of the sector determined at a lower sensitivity, it is inferred that there is contamination in the corresponding sector.

2. The method according to claim 1, characterized in that, The background noise of a sector is determined in the corresponding sector and the background noise of the detection area is determined in the remaining detection area or the entire detection area. If the background noise of a sector is significantly lower than the background noise of the detection area, it is inferred that there is pollution in the corresponding sector.

3. The method according to claim 2, characterized in that, Based on the signal transmission time (T) between the emission of an infrared laser pulse (P1) and the reception of its reflection (P2) at least once, L The background light intensity detected in the test area is used to determine the background noise of the sector and the background noise of the test area.

4. The method according to claim 3, characterized in that, Throughout the signal transmission time (T) L The background light intensity is detected and the effective value of the intensity is calculated.

5. The method according to claim 3 or 4, characterized in that, The background noise of the detection area is determined by measuring the noise level of the background light in the remaining detection area or the entire detection area.

6. The method according to claim 3 or 4, characterized in that, Sector background noise can be determined by determining the noise level of the background light in exactly one scan performed in the corresponding sector, or by determining the noise level of the background light in multiple or all scans performed in the corresponding sector.

7. The method according to claim 3 or 4, characterized in that, The sector background noise is determined as follows: - Integrate all the power received by the receiver in the corresponding sector. - Integrate the reflected power received in the corresponding sector by means of the receiver based on the reflection (P2) of multiple infrared laser pulses (P1), and The total noise power describing the background noise in the corresponding sector is determined by the difference between the received power and the reflected power.

8. The method according to any one of claims 1 to 4, characterized in that, The sensitivity of the receiver is set by shifting the operating point or by changing the receiver's internal amplification factor.

9. A device for identifying contamination on a protective screen (1.1) of a lidar sensor (1), characterized in that, - The detection area of ​​the lidar sensor (1) is divided into multiple sectors, and - The data processing unit is configured to determine, sector by sector, whether there is contamination on the protective screen (1.1) in the corresponding sector. For this purpose, the data processing unit determines the sector background noise in the corresponding sector at different sensitivities of the receiver of the lidar sensor (1), and if the sector background noise determined at the higher sensitivity is not significantly higher than the sector background noise determined at the lower sensitivity, it is inferred that there is contamination in the corresponding sector.

10. The device according to claim 9, characterized in that, The data processing unit determines the sector background noise in the corresponding sector and the detection area background noise in the remaining detection area or the entire detection area. If the sector background noise is significantly lower than the detection area background noise, it infers that contamination exists in the corresponding sector.

11. The application of the device according to claim 9 or 10 in vehicles and / or robots, wherein, At least one lidar sensor (1) is set up for environmental detection.

12. The application according to claim 11, characterized in that, - Perform automated operation of the vehicle and / or robot based on data detected by the at least one lidar sensor (1) and - When at least one contamination is detected on the protective screen (1.1) of the at least one lidar sensor (1), restrict automated operation and / or take at least one measure to eliminate the at least one contamination.

13. The application according to claim 12, characterized in that, The automated operation is a highly automated or autonomous operation.

Images