Detection of false positive points in a point cloud from an active sensor

Abstract

Method for determining faulty sensor results of an active sensor in a situation in which a first object (1) is illuminated by the active sensor and a second object (2) which is at least partially obscured by the illumination of the active sensor by the first object (1) is illuminated, comprising the steps: - Determining a point cloud from reflections occurring at the first object (1) and the second object (2) of the signals emitted by the active sensor for illumination; - Determining a geometry of a shadow (3) in the part of the point cloud belonging to the reflections on the second object (2); - Determining a reference geometry based on the geometry of the shadow (3); and - Classifying points outside the reference geometry from the part of the point cloud belonging to the reflections at the first object (1) as virtual points (4).

Description

[0001] The invention relates to a method for determining faulty sensor results of an active sensor in a situation in which a first object is illuminated by the active sensor and a second object, at least partially obscured by the illumination of the active sensor by the first object, is also illuminated, as well as a system for a vehicle for determining faulty sensor results of an active sensor of the vehicle in such a situation.

[0002] In highly automated vehicles such as passenger cars or trucks, environmental monitoring is typically performed using cameras, radar, and lidar. A lidar system emits light pulses which are used for object detection. Contamination such as water droplets, dust, or salt, or scratches on the front cover, can cause unwanted refraction and / or focusing of the reflected laser power, leading to unintended crosstalk at the APD (Avalanche Photodiode) receiver. This creates induced, and therefore virtual, (false-positive) scan points. The associated negative effects are called blooming and crosstalk.

[0003] Blooming occurs primarily with highly reflective objects, such as warning beacons, guideposts, and road signs. These induced scan points can lead to distortions in the object output during subsequent object tracking, which could hinder the driving strategy of a vehicle's control system. For example, a significant widening in scenarios like construction zone crossings can suggest an incorrectly widened obstacle that the vehicle would not be able to pass in its lane. Crosstalk is the electrical interference of the voltage level in the APD receiver array. Due to this distortion of the point cloud, it is generally no longer possible to determine the actual dimensions of the object in the tracker.

[0004] To improve this behavior, these false positives must be reliably identified so that they are not taken into account in the further processing.

[0005] In this context, DE 10 2020 128 732 A1 relates to a method for detecting blooming candidates in a lidar measurement, wherein a distance-based histogram of points from a point cloud generated in the lidar measurement is created, in which clusters of points at the same distance to a lidar sensor performing the lidar measurement are identified, the intensities of the points in a cluster are evaluated, and then, if the cluster contains points whose intensities exceed a predetermined limit, those points in the cluster whose intensities do not exceed the predetermined limit, in particular those whose intensities fall below the predetermined limit by more than a predetermined threshold, are classified as blooming candidates.

[0006] From DE 10 2020 110 809 B3 a method for detecting blooming in a lidar measurement is known.

[0007] DE 10 2021 205 061 A1 discloses a method for detecting hidden objects in a 3D point cloud representing an environment.

[0008] US patent 2021 / 0033 711 A1 discloses a method for detecting optical crosstalk in a LIDAR system, including the following steps: -selective activation and deactivation of light sources within a light source array; -triggering a field of view (FOV) measurement in which at least one target area of ​​the FOV is illuminated by the light source arrangement and at least one non-targeted area of ​​the FOV is not illuminated by the light source arrangement; -a generation of electrical signals based on at least one reflected light beam received by a photodetector array, wherein the photodetector array comprises a targeted pixel group corresponding to the at least one target area of ​​the FOV and a non-targeted pixel group corresponding to the at least one untargeted area of ​​the FOV; and -Detection of optical crosstalk appearing at at least part of the non-targeted pixel group, based on electrical signals from the targeted pixel group and the non-targeted pixel group.

[0009] From US patent 2019 / 0391270A1, a method for improving observations of an environment with a LiDAR device in the presence of highly reflective surfaces is known. In response to the determination that a first point cloud includes an observation of an obscuring object that is highly reflective, i) the emission of a scanning light beam with a scanning intensity different from the initial intensity of an initial light beam is used to acquire the first point cloud, and ii) dynamic control of the LiDAR device is used to acquire a second point cloud that omits the obscuring object.

[0010] The object of the invention is to recognize these virtual points with increased confidence in order to mitigate them in an evaluation and to further determine the correct dimensions of an illuminated real object.

[0011] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0012] A first aspect of the invention relates to a method for determining erroneous sensor results of an active sensor in a situation in which a first object is illuminated by the active sensor and a second object, at least partially obscured by the illumination of the active sensor by the first object, is also illuminated, comprising the steps: - Determining a point cloud from reflections occurring at the first and second objects of the signals emitted by the active sensor for illumination; - Determining a geometry of a shadow in the part of the point cloud belonging to the reflections on the second object; - Determining a reference geometry based on the geometry of the shadow; and - Classifying points lying outside the reference geometry from the part of the point cloud belonging to the reflections at the first object as virtual points.

[0013] The active sensor is preferably a lidar system, but can also be a radar system or another type of active sensor. A key characteristic of an active sensor, compared to a passive sensor, is the emission of its own signals in order to detect and analyze their reflections in the environment. A camera, for example, is typically a passive sensor and captures light reflected from its surroundings without emitting any light itself.

[0014] Lidar systems typically aim for high-frequency, yet discrete behavior by emitting a raster pattern of light pulses, similar to a laser beam, into their surroundings. This rasterization is achieved, for example, by rotating the lidar system to capture a row of points and by vertically rotating it to capture a column. This creates a matrix of individual reflection points, each with a travel time value as additional information to determine the distance between the lidar system and the reflecting object in the environment. This allows for the advantageous creation of a three-dimensional image of the area around the lidar system.

[0015] This can lead to erroneous points, particularly due to influences such as those described earlier. Reflection points can be detected where none should exist, since there is no object in the vicinity that is at a distance from the lidar system corresponding to the time of the detected supposed reflection.

[0016] In contrast, this method utilizes the effect that such erroneous points, i.e., the virtual points, are not reflected by an obscuring first object. Therefore, the first object cannot be illuminated in precisely this direction of radiation from the lidar system to create a shadow on the second object behind it, which is thus partially obscured. The shadow on the second object can only occur where the signals from the active sensor are blocked by the first object in front of it.

[0017] Therefore, a point cloud is first generated that shows reflections from the first and second objects. Since the first object lies between the active sensor and the second object, it partially obscures the second object. A shadow on the second object is located in those emission directions of the active sensor where the first object reflects and the second object does not.

[0018] Virtual, and therefore false-positive, reflections are obtained where the shadow of the first object has already ended and the second object is producing reflections, specifically in an area where supposed reflections, presumably belonging to the first object, are also detected. Since reflections from the first and second objects naturally exclude each other in the same direction of radiation, the geometry of the shadow can be used to filter out supposed reflections from the first object as false if they lie outside the reference geometry determined based on the shadow geometry, and reflections from the second object are detected in the same direction of radiation from the active sensor.

[0019] The reference geometry, as a logical derivation of the shadow geometry, allows for corrections and adjustments for the operation of a system to execute the procedure. While in the simplest case the geometry of the shadow on the second object can be used unchanged as the reference geometry, a customized shadow geometry on the second object can be determined, particularly through size adjustments and other geometric modifications for registration, especially by superimposing relevant edges.

[0020] This means the virtual points are known and can be removed from the sensor result. This error avoidance makes it advantageous to determine real object dimensions more reliably, especially the dimensions of the first object (as defined by this nomenclature) located in front of a second object.

[0021] According to an advantageous embodiment, a reduced point cloud is generated from the point cloud by removing virtual points from the point cloud, and the reduced point cloud is used for a vehicle application.

[0022] The reduced point cloud corresponds to a corrected point cloud, allowing for a more accurate and reliable measurement of the dimensions of the first object in particular. Object dimensions are often crucial for vehicle applications, such as an automatic driving control system, which is typically designed to avoid obstacles and generally plan trajectories for the vehicle.

[0023] According to another advantageous embodiment, the reference geometry is the same as the geometry of the shadow.

[0024] According to a further advantageous embodiment, the reference geometry is generated by shifting and / or resizing the geometry of the shadow, wherein the resizing depends on a distance of the first object and / or the second object to the active sensor.

[0025] According to a further advantageous embodiment, an area is determined in which the virtual points overlap with the part of the point cloud belonging to the reflections on the second object, and a predetermined error correction measure is initiated if the area exceeds a predetermined limit condition.

[0026] The boundary conditions can be defined in relation to an absolute area, or relatively, for example as a percentage of the surface area of ​​the first object.

[0027] According to a further advantageous embodiment, the fault correction measure comprises a reduction in the power of the emitted signal from the active sensor. Preferably, the emitted signal power is reduced only in the area of ​​the first object in order to maintain the detection range in the surrounding area. "In the area of ​​the first object" means that the signals directed at the first object are preferably reduced, specifically those directed at the edge region. The edge region is defined by the outer contour of the first object and an area of ​​predetermined width adjoining the outer contour on both sides of the outer contour, or at least by an area of ​​predetermined width facing the center of the object.

[0028] Reducing the power of the active sensor system can reduce the occurrence of blooming effects, especially when using a lidar system, particularly if the first object is highly reflective.

[0029] According to a further advantageous embodiment, the fault correction measure includes activating a cleaning device for the active sensor.

[0030] The cleaning device of the active sensor can remove any dirt or moisture from the active sensor using water and / or air and / or a rubber lip.

[0031] According to a further advantageous embodiment, the power of the emitted signal of the active sensor is adjusted depending on the distance of the first object and / or the second object to the active sensor.

[0032] According to another advantageous embodiment, the power is only adjusted if the distance of the first object and / or the second object to the active sensor is below a predetermined limit value.

[0033] The further reflective objects are from the active sensor, the less need there is for adjustment, since blooming effects are only particularly detrimental for driving maneuvers with a small distance to objects (parking, maneuvering, swerving).

[0034] Another aspect of the invention relates to a system for a vehicle for detecting erroneous sensor results of an active sensor of the vehicle in a situation in which a first object is illuminated by the active sensor and a second object, at least partially obscured by the illumination of the active sensor by the first object, is also illuminated, comprising a computing unit and the active sensor, wherein the computing unit is configured to determine a point cloud from reflections of the signals emitted by the active sensor for illumination occurring at the first object and at the second object, to determine a geometry of a shadow in the part of the point cloud belonging to the reflections at the second object, to determine a reference geometry based on the geometry of the shadow and to classify points lying outside the reference geometry from the part of the point cloud belonging to the reflections at the first object as virtual points.

[0035] Once the virtual points are classified and thus identified, they can be discarded in further processing, for example for use by a driver assistance system.

[0036] Advantages and preferred further developments of the proposed system result from an analogous and substantive transfer of the above statements made in connection with the proposed procedure.

[0037] Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.

[0038] They show: Fig. 1: An exemplary situation with a first object that partially obscures a second object. Fig. 2: A first part of a procedure for determining faulty sensor results from an active sensor in the situation of Fig. 1 according to an embodiment of the invention. Fig. 3: A second part of the procedure according to the part of the Fig. 2.

[0039] The representations in the figures are schematic and not to scale.

[0040] Fig. Figure 1 shows an example situation with a first object 1 and a second object 2. The first object 1 is a warning beacon, while the second object 2 is a vehicle behind it. The first object 1 obscures parts of the second object 2. In the Fig. Figure 1 illustrates a situation as it can be detected from the perspective of an active sensor. If a Liedar system is used as the active sensor, emitting light pulses in a variety of directions, reflections of the emitted light will occur at the first object 1 and the second object 2. These reflections can then be detected by the active sensor itself. By determining the travel time of the emitted light signals, a distance in each direction can be calculated. This allows the generation of a point cloud of reflections with corresponding distance information. To identify erroneous sensor results in this situation, the geometry of a shadow 3 in the part of the point cloud corresponding to the reflections at the second object 2 is also determined (see Figure 1). Fig. 2.

[0041] Fig. Figure 2 shows the reflections from the second object 2 as part of the point cloud. While the point cloud is also formed from reflections from the first object 1, for illustrative purposes only the portion of the point cloud resulting from reflections from the second object 2 is shown. However, the second object 2 is partially obscured by the first object 1, which is already reflecting signals from the active sensor and thus preventing them from reaching the second object 2. This results in a partial shadowing of the second object 2. The geometry of the shadow 3 is determined. From this shadow geometry 3, a reference geometry can be determined by resizing it and matching it by superimposing relevant edges.

[0042] Fig. Figure 3 shows further steps for carrying out the procedure for determining faulty sensor results from the active sensor in the situation of Fig. 1. After the reference geometry as described below Fig. As described in section 2, points lying outside the reference geometry from the part of the point cloud belonging to the reflections at the first object 1 can be classified as virtual points 4. In the Fig.For the sake of simplicity, only two of the numerous virtual points 4 are designated as such. While further reflections exist as points outside the reference geometry, which is based on the geometry of the shadow 3, these are actual reflections from the first object 1. Whereas the actual reflections from the first object 1 outside the reference geometry have no counterpart at the second object 2, this is precisely the case with the virtual points 4. These are characterized by the fact that reflections are also received at the second object 2 outside the geometry of the shadow 3. Since the reflected power of the reflections at the second object 2 in this area is higher than that of the virtual points 4, the actual width of the first object 1 in front of it can be discerned in the shadow.Thus, in the present example, the points of the point cloud outside the reference geometry, which are initially assigned to the first object 1 but are actually virtual points 4, are located to the left and right of the reference geometry, and not above or below it, since reflections from the second object 2 are only obtained to the left and right of the reference geometry. Therefore, the phenomenon is exploited that the first object 1 can be detected both through its reflections and through its shadow on the second object 2, which also provides information about the first object 1.

[0043] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description. Reference symbol list 1 first object 2 second object 3 Shadows 4 points of a point cloud

Claims

[1] Method for determining faulty sensor results of an active sensor in a situation in which a first object (1) is illuminated by the active sensor and a second object (2) which is at least partially obscured by the illumination of the active sensor by the first object (1), comprising the steps: - Determining a point cloud from reflections occurring at the first object (1) and the second object (2) of the signals emitted by the active sensor for illumination; - Determining a geometry of a shadow (3) in the part of the point cloud belonging to the reflections on the second object (2); - Determining a reference geometry based on the geometry of the shadow (3); and - Classifying points outside the reference geometry from the part of the point cloud belonging to the reflections at the first object (1) as virtual points (4). [2] Method according to claim 1, wherein a reduced point cloud is generated from the point cloud by removing virtual points (4) from the point cloud, wherein the reduced point cloud is used for a vehicle application. [3] Method according to one of claims 1 to 2, wherein the reference geometry is the same as the geometry of the shadow (3). [4] Method according to one of claims 1 to 2, wherein the reference geometry is generated by displacement and / or size adjustment of the geometry of the shadow (3), wherein the size adjustment depends on a distance of the first object (1) and / or the second object (2) to the active sensor. [5] Method according to one of the preceding claims, wherein an area is determined in which the virtual points (4) overlap with the part of the point cloud belonging to the reflections at the second object (2), wherein in the event that the area exceeds a predetermined limit condition, a predetermined error measure is initiated. [6] Method according to claim 5, wherein the fault correction measure comprises a reduction in the power of the emitted signal of the active sensor. [7] Method according to one of claims 5 to 6, wherein the fault correction measure comprises activating a cleaning device of the active sensor. [8] Method according to one of the preceding claims, wherein the power of the emitted signal of the active sensor is adapted as a function of the distance of the first object (1) and / or the second object (2) to the active sensor. [9] Method according to claim 8, wherein the power is adjusted only if the distance of the first object (1) and / or the second object (2) to the active sensor is below a predetermined limit value. [10] System for a vehicle for detecting erroneous sensor results of an active sensor of the vehicle in a situation in which a first object (1) is illuminated by the active sensor and a second object (2) is illuminated which is at least partially obscured by the illumination of the active sensor by the first object (1), comprising a computing unit and the active sensor, wherein the computing unit is configured to determine a point cloud from reflections of the signals emitted by the active sensor for illumination occurring at the first object (1) and at the second object (2), to determine a geometry of a shadow (3) in the part of the point cloud belonging to the reflections at the second object (2), to determine a reference geometry based on the geometry of the shadow (3) and to classify points lying outside the reference geometry from the part of the point cloud belonging to the reflections at the first object (1) as virtual points (4).

Images