Method and apparatus for identifying halos in lidar measurements

Abstract

The invention relates to a method for detecting haloes in laser radar measurements. According to the invention, the distance to a laser radar reflection point (R) is determined in active and passive measurements. In addition, in the active measurement, a first distance value is determined based on the signal transit time of a laser pulse, and in the passive measurement, a second distance value is determined based on triangulation from two-dimensional intensity measurements performed from different measurement positions, wherein a halo is then inferred when the second distance value exceeds the first distance value by a predetermined degree. In this case, a passive measurement based on a two-dimensional intensity measurement means detecting the surroundings by means of at least one laser radar, wherein the at least one laser radar only detects light already present in the surroundings without actively emitting laser light. The passive measurement is based on two two-dimensional intensity measurements, wherein, for example, a first intensity measurement is performed at a first time t1 by means of a laser radar (1) located at a first position, and a second intensity measurement is performed by means of the same laser radar (1) at a second time t2 after the first measurement in time and at a second position different from the first position. Between these two times t1, t2, the relative position of the laser radar (1) to the laser radar reflection point (R) changes due to the movement of the platform. In an alternative embodiment, the two intensity measurements are performed by a first lidar and a second lidar placed at a different location.

Description

Technical Field

[0001] The present invention relates to a method for identifying halos in lidar measurements.

[0002] The invention also relates to a device for detecting halos in lidar measurements using at least one lidar. Background Art

[0003] DE 10 2005 003 970 A1 discloses a method for determining the functional capability of a sensor device on a motor vehicle, wherein an area measured by the sensor device is divided into a plurality of sub-areas and sensor signals from a certain surrounding area that are assigned to the sub-areas are evaluated to determine the functional capability of the sensor device. In this case, sensor signals measured successively for a plurality of sub-areas when driving through a certain surrounding area are evaluated. The sub-areas are detection ranges of a plurality of lidar sensors or a plurality of angular sectors of a lidar sensor.

[0004] Furthermore, DE 10 2018 003 593 A1 discloses an operating method for a vehicle assistance system, wherein the vehicle is driven in autonomous driving by means of the assistance system and the assistance system comprises an environmental sensor system having a plurality of detection units arranged in and / or on the vehicle. The vehicle surroundings and objects located therein are detected by means of the detection units in autonomous driving of the vehicle, wherein the functions of the individual detection units are continuously monitored by means of a monitoring module and, in the event of a failure of a detection unit, only the assistance functions associated with the failed detection unit are deactivated by means of a planning module connected to the monitoring module. The detection units comprise sensors based on lidar.

[0005] US 2019 / 0391270 A1 describes a reflection system for improving environmental observation by means of a laser radar in the presence of highly reflective surfaces. The reflection system includes a plurality of processors and a memory communicating with the processors. In addition, the reflection system includes a scanning module with instructions, which, when executed by the processor, causes the processor to control the emission of scanning light having an initial intensity different from the initial intensity of the initial light used to measure the first point cloud in response to an observation that the first point cloud contains a strongly reflective occluding object, and dynamically controls the laser radar to measure a second point cloud free of the occluding object. In addition, there is an output module with instructions, which, when executed by the processor, causes the processor to generate a merged point cloud from the first point cloud and the second point cloud, which improves the observation of the surrounding environment by means of the laser radar by reducing the interference caused by the occluding object. Summary of the invention

[0006] The object of the invention is to specify a novel method and a novel device for detecting haloes in lidar measurements.

[0007] This object is achieved according to the invention by a method having the features of claim 1 and a device having the features of claim 7 .

[0008] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0009] In a method for detecting haloes in lidar measurements, according to the invention, the distance to a lidar reflection point is determined in an active measurement and in a passive measurement, wherein in the active measurement a first distance value is determined based on the signal transit time / signal transmission / reception time of a laser pulse, and in the passive measurement a second distance value is determined based on triangulation from two-dimensional intensity measurements performed from different measurement positions. A halo is then inferred when the second distance value exceeds the first distance value by a predetermined degree.

[0010] In this case, passive measurement based on two-dimensional intensity measurement means detecting the surroundings by means of at least one laser radar, wherein the at least one laser radar only detects light present in the surroundings without actively emitting laser light.

[0011] Furthermore, halo refers here to glare or crosstalk in lidar measurements. For example, a halo occurs when a laser pulse emitted by a lidar is reflected by a highly reflective target such as a traffic sign or a headlight reflector. In this case, a large amount of the emitted energy is reflected back to the lidar compared to a target with low reflectivity. At this time, the returning laser beam is usually not optimally focused. The reasons for this are various, the most common of which are that the reflection from the target is usually not optimally mirrored, atmospheric particles deflect the laser beam, or dirt on the lid of the lidar causes light scattering. This may cause the reflected light to reach multiple receiver units of the lidar that are arranged close to each other in space or the reflected light to be transferred to adjacent pixels. Therefore, the distance measurement is triggered depending on the sensitivity of the detector. The halo effect is usually stronger at a shorter distance from the lidar, because the energy reflected by the target decreases rapidly as the distance the light must travel increases.

[0012] LiDARs play an important role in driver assistance systems and other autonomous platforms such as robots, as they allow for an accurate 3D representation of the LiDAR environment. However, in the presence of haloing, this can lead to erroneous results when measuring the distance between the LiDAR and objects measured in its surroundings. In particular, the halo effect can cause false positive LiDAR measurements, making an accurate 3D representation of the surrounding environment difficult.

[0013] The method makes it possible to reliably detect haloes in lidar measurements in a simple manner, so that erroneous results in such distance measurements can be avoided or at least reliably detected. This results in safer operation of applications such as vehicles and robots that are driven or driven automatically, in particular highly automatically or autonomously.

[0014] In one possible embodiment of the method, the passive measurement is based on two two-dimensional intensity measurements, wherein a first intensity measurement is performed by means of a first lidar and a second intensity measurement is performed by means of a second lidar which is arranged at a different position than the first lidar. This allows a simple and reliable implementation of the passive measurement and thus a very reliable identification of the halo.

[0015] In another possible design of the method, the two two-dimensional intensity measurements are performed simultaneously or sequentially. In particular, when the intensity measurements are performed simultaneously, the passive measurement of the distance can be performed very quickly.

[0016] In another possible embodiment of the method, the passive measurement is based on two two-dimensional intensity measurements, wherein a first intensity measurement is performed by means of a lidar at a first position and a second intensity measurement is performed by means of the same lidar temporally subsequent to the first measurement and at a second position different from the first. This allows a simple and reliable implementation of the passive measurement and the consequently very reliable identification of halos, wherein only one lidar is required to perform the two-dimensional intensity measurement, which results in low hardware usage and investment costs.

[0017] In another possible embodiment of the method, the passive measurement is performed by evaluating the two-dimensional intensity images acquired in the two two-dimensional intensity measurements by means of a stereo method, which reliably determines the distance to the lidar reflection point and thus to objects in the lidar surroundings.

[0018] In another possible embodiment of the method, a semi-global matching algorithm is used as the stereo method, by means of which the distance to pixels in the two-dimensional intensity image and thus to objects in the lidar environment can be determined very reliably and particularly accurately.

[0019] The device for identifying halos in lidar measurements comprises at least one lidar and is characterized according to the invention by a processing unit which is designed to determine the distance of the at least one lidar from a lidar reflection point in active measurements and in passive measurements, in which a first distance value is determined based on the signal transit time of a laser pulse and in which a second distance value is determined based on triangulation from two-dimensional intensity measurements performed at different measuring positions, and then to infer a halo when the second distance value exceeds the first distance value by a predetermined degree.

[0020] The device makes it possible to reliably detect haloes in lidar measurements in a simple manner, so that erroneous results in distance measurements performed with lidar can be avoided or at least reliably detected. This allows for reliable operation of applications such as vehicles and robots that travel or drive automatically, in particular highly automatically or autonomously. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The embodiments of the present invention are explained in detail below with reference to the drawings, wherein:

[0022] Figure 1 The arrangement of the laser radar and the surrounding environment monitored by the laser radar are schematically shown.

[0023] Figure 2 The arrangement of the laser radar at different times and the surrounding environment monitored by the laser radar are schematically shown.

[0024] Figure 3 Schematically showing the Figure 2 The lidar image collected by the lidar at the first moment,

[0025] Figure 4 Schematically showing the Figure 2 The lidar image collected by the lidar at the second moment,

[0026] Figure 5 Schematically shows the arrangement of two laser radars and the surrounding environment monitored by these laser radars,

[0027] Figure 6 Schematically showing the Figure 5 The lidar image collected by the first lidar,

[0028] Figure 7 Schematically showing the Figure 5 A lidar image captured by a second lidar. DETAILED DESCRIPTION

[0029] Corresponding parts are provided with the same reference symbols in all the figures.

[0030] Figure 1 The arrangement of the laser radar 1 and the surrounding environment monitored by the laser radar 1 are shown.

[0031] In the environment of the laser radar 1 there are two objects O1 , O2 which are detected within a detection range E by the laser radar 1 .

[0032] The laser radar 1 is arranged, for example, on a vehicle that is driven autonomously, in particular highly autonomously or autonomously. Alternatively, the laser radar 1 may also be arranged on a robot.

[0033] The first object O1 is a highly reflective object O1, for example a traffic sign such as a road sign placed above the road FB. The second object O2 is located on the road and has a selective reflectivity, for example a smaller or greater reflectivity than the first object O1.

[0034] The distance to objects O1, O2 in its environment is determined by means of a laser radar 1, in that a laser pulse is emitted and the duration of the reflected laser pulse before it reaches a receiver of the laser radar 1 is determined. The laser radar 1 may include a plurality of lasers and / or a plurality of receivers in order to increase the sampling rate and the spatial resolution of the laser radar 1. The measurement, also referred to as a scan, performed by means of the laser radar 1 may be implemented in such a way that the complete scan can be presented in the form of a two-dimensional measurement grid, also referred to as a laser radar image.

[0035] In the environment of the laser radar 1 shown, the first object O1 generates halo points P1-Pn at the same distance above and below the object O1 during the laser measurement due to its strong reflectivity, so that the so-called halo phenomenon occurs. If the detected halo points P1-Pn are not like this, there is the risk that in the further data processing, such as sensor fusion, it is assumed that there is an obstacle, such as a traffic jam end, and the driver assistance system may trigger an undesired brake.

[0036] Figure 2 The arrangement of the laser radar 1 and the surroundings monitored by the laser radar 1 at different times t1 and t2 are shown. Figure 3 Show that based on Figure 2 The laser radar image B1 collected by the laser radar 1 at the first moment t1, Figure 4 Show that based on Figure 2 The lidar image B2 is acquired by the lidar 1 at a second time t2 following the first time t1. The lidar images B1, B2 are respectively two-dimensional measurement grids or two-dimensional intensity images, whose axes depict the values ​​of the vertical angle α and the horizontal angle β, so that the vertical angle α and the horizontal angle β form the image coordinates.

[0037] The laser radar 1 is mounted on a movable platform, such as an autonomous, in particular highly autonomous or self-driving or moving vehicle or robot.

[0038] As already described, the distance to objects O1, O2 in its surroundings is determined with the aid of the lidar 1 by emitting laser pulses and measuring the time that elapses before the reflected laser pulses reach the receiver of the lidar 1. In this case, reflections occur at lidar reflection points R belonging to the respective object O1, O2, for example so-called landmarks.

[0039] The lidar 1 is generally regarded as an active sensor, since it must actively emit energy in order to perform a transit time measurement (also called a time-of-flight measurement) according to the above description. If the receiver of the lidar 1 is sensitive enough, it can also be used to measure the intensity of ambient light at a given wavelength of the lidar 1, which is scattered back to the lidar 1 without active illumination. As a result, a highly dynamic grayscale gradient image of the scene can be generated in a passive two-dimensional intensity measurement with the aid of the lidar 1. Since the intensity of the passive reflected light is significantly lower, no halo effect occurs in such a passive measurement. Such a passive measurement can be performed immediately before or after the active measurement, so that the captured scene hardly changes between the two measurements. The active measurement provides an accurate three-dimensional illustration of the environment surrounding the lidar 1, while the passive measurement allows a high degree of detail in the two-dimensional appearance image of the objects O1, O2. The two measurement principles can therefore complement each other.

[0040] The shown laser radar 1 mounted on a movable platform is designed not only for performing active measurements of the distance to the laser radar reflection point R, but also for performing passive measurements of the intensity. The passive measurement can be performed immediately before or after the active measurement.

[0041] In order to determine the halo in the lidar measurement, the distance to the lidar reflection point R is determined in active measurement and passive measurement based on the data collected with the aid of the lidar 1. The first distance value is determined in the active measurement based on the signal transit time of the laser pulse from the lidar 1 to the lidar reflection point R and back to the lidar 1.

[0042] Next, a second distance value is determined in a passive measurement based on triangulation from two-dimensional intensity measurements performed at different measurement positions.

[0043] Subsequently, a halo is inferred if the second distance value exceeds the first distance value by a predetermined degree, in particular is significantly greater than the first distance value.

[0044] Here, the passive measurement is based on two two-dimensional intensity measurements, where a first intensity measurement is performed with the lidar 1 at a first position at a first time t1, and a second intensity measurement is performed with the same lidar 1 at a second time t2 temporally subsequent to the first measurement and at a second position different from the first position. Between these two times t1, t2, the relative position of the lidar 1 with respect to the lidar reflection point R changes due to the platform movement.

[0045] The movement of the lidar 1 between two measurements is known here, for example, by evaluation of an inertial measurement unit, which is also arranged on the movable platform and calibrated relative to the lidar 1 or to the same reference system.

[0046] By observing the representative position of, for example, a landmark in the lidar images B1, B2 from different viewing angles, a three-dimensional reconstruction of the observed scene can be performed. Because the lidar 1 moves, the representative position and thus the associated lidar reflection point R or the pixel measured in the two-dimensional intensity image representing it may appear at other positions in the lidar images B1, B2 acquired from other environmental positions. This effect is generally referred to as motion parallax. If the movement of the lidar 1 between two times t1, t2 is known and the position of the same lidar reflection point R is found in the two lidar images B1, B2, the three-dimensional position and thus the distance to the lidar reflection point R can be reconstructed by simple triangulation.

[0047] This passive measurement is performed, for example, by evaluating two two-dimensional intensity measurements by means of a stereoscopic method, such as a semi-global matching algorithm.

[0048] A possible embodiment of a method for identifying halos in lidar measurements is described below.

[0049] First, a well-known stereo matching algorithm such as a semi-global matching algorithm is used to determine the angular displacement between corresponding pixels in the passive lidar images B1, B2 acquired from two different perspective angles. For example, the lidar 1 is aimed at the lidar reflection point R or the pixel representing it at a vertical angle α of 10° and a horizontal angle β of 5° at time t1. At time t2, the lidar 1 is aimed at the lidar reflection point R or the pixel representing it at a vertical angle α of 10° and a horizontal angle β of 20°.

[0050] Since the three-dimensional movement of the lidar 1 between the measurements of the two lidar images B1, B2 is known, the information about the corresponding position angles from the first and second measurements can be used to triangulate the three-dimensional coordinates of the measured pixel position, i.e. the lidar reflection point R.

[0051] Comparing the active measurement at any pixel location with the passive measurement that has been derived by the triangulation method now allows to infer whether there is a halo. If the passive measurement derived with the structure from motion algorithm shows significantly larger distances than the active measurement, a halo can be inferred as a plausible explanation.

[0052] Figure 5 The arrangement of two laser radars 1 and 2 and the surrounding environment monitored by the laser radars 1 and 2 are shown. Figure 6 Show that based on Figure 5 A laser radar image B1 collected by a laser radar 1, Figure 7A lidar image B2 is shown which was acquired at the same time with the aid of another lidar 2. Here, the lidar images B1, B2 are each a two-dimensional measurement grid or a two-dimensional intensity image, whose axes depict the values ​​of the vertical angle α and the horizontal angle β, so that the vertical angle α and the horizontal angle β form the image coordinates.

[0053] The two laser radars 1 and 2 are mounted on a movable platform, such as an autonomous, in particular highly autonomous or self-driving or moving vehicle or robot. The laser radars 1 and 2 are synchronized in time so that they are designed to detect the same spatial angle at the same time.

[0054] The two laser radars 1 , 2 are designed not only to perform active measurements of the distance to the laser radar reflection point R, but also to perform passive measurements of the intensity. The passive measurements can be performed immediately before and after the active measurements.

[0055] In order to determine the halo in the lidar measurement, in this embodiment, the distance to the lidar reflection point R is determined in active measurement and passive measurement based on the data collected with the help of lidar 1 and 2. The method is that the first distance value is determined in the active measurement based on the signal transit time of the laser pulse from lidar 1 and / or lidar 2 to the lidar reflection point R and back to lidar 1 and / or lidar 2.

[0056] Next, a second distance value is determined in a passive measurement based on triangulation from two-dimensional intensity measurements performed from different measurement positions.

[0057] Subsequently, a halo is inferred if the second distance value exceeds the first distance value by a predetermined degree, in particular is significantly greater than the first distance value.

[0058] The extrinsic parameters of the laser radars 1 , 2 , ie their position and / or orientation, are known. To this end, the laser radars 1 , 2 are calibrated relative to one another or to a common reference system.

[0059] Different from the previous Figures 2 to 4 In the described embodiment, this allows a passive measurement based on two two-dimensional intensity measurements, wherein a first intensity measurement is performed with the aid of a first lidar 1 and a second intensity measurement is performed with the aid of a second lidar 2 placed at a position different from the first lidar 1 .

[0060] By simultaneously detecting the scene from different viewing angles with the aid of laser radars 1 and 2, it is possible to observe the representative positions of, for example, landmarks in the laser radar images B1 and B2 from different viewing angles and thus perform a three-dimensional reconstruction of the observed scene. Because the positions of the laser radars 1 and 2 are different, the representative positions and thus the corresponding laser radar reflection points R or pixels representing them can appear at other positions in the laser radar images B1 and B2 taken from other environmental positions. Because the relative positions of the laser radars 1 and 2 and their external parameters are known, when the position of the same laser radar reflection point R or the pixel representing it is found in the two laser radar images B1 and B2, the three-dimensional position and thus the distance to the laser radar reflection point R can be reconstructed by simple triangulation.

[0061] For example, the passive measurement is performed by evaluating the two intensity measurements by means of a stereo method, such as a semi-global matching algorithm.

[0062] One possible embodiment of a method for identifying halos in lidar measurements is described below.

[0063] First, a well-known stereo matching algorithm such as a semi-global matching algorithm is used to determine the angular displacement between corresponding pixels in the passive lidar images B1 and B2 acquired from two different perspective angles. For example, lidar 1 aims at lidar reflection point R or a pixel representing it at a 10° vertical angle α and a 5° horizontal angle β. Lidar 1 aims at lidar reflection point R or a pixel representing it at the same time at a 10° vertical angle α and a 20° horizontal angle β.

[0064] Since the transformation between the coordinate systems of the two lidars 1 , 2 is known, the information about the corresponding position angles in the passive intensity measurement performed with the aid of lidars 1 , 2 can be used to triangulate the three-dimensional coordinates of the measured pixel position, i.e. the lidar reflection point R.

[0065] Now, by comparing the active measurement at any pixel location with the passive measurement derived by the triangulation, it is possible to infer whether there is a halo. If the passive measurement indicates a significantly greater distance than the active measurement, a halo can be inferred as a plausible explanation for this.

[0066] Reference numerals list

[0067] 1 LiDAR

[0068] 2 LiDAR

[0069] B1 LiDAR image

[0070] B2 LiDAR image

[0071] E Detection range

[0072] FB Highway

[0073] O1 Object

[0074] O2 Object

[0075] P1~Pn Halo Point

[0076] R Lidar reflection point

[0077] Time t1

[0078] Time t2

[0079] α Angle

[0080] β angle

Claims

1. A method for identifying halo in lidar measurements, characterized in that: - Determine the distance to the lidar reflection point (R) in both active and passive measurements, - in the active measurement a first distance value is determined based on the signal transit time of the laser pulses, - determining a second distance value in the passive measurement based on triangulation from two-dimensional intensity measurements performed from different measurement positions, and - A halo is then inferred when the second distance value exceeds the first distance value by a predetermined degree.

2. The method according to claim 1, characterized in that: - This passive measurement is based on two two-dimensional intensity measurements, - performing a first intensity measurement by means of a first laser radar (1), and - performing a second intensity measurement with the aid of a second laser radar (2) arranged at a different location than the first laser radar (1).

3. The method according to claim 2, characterized in that: The two two-dimensional intensity measurements are performed simultaneously or sequentially in time.

4. The method according to claim 1, characterized in that: - This passive measurement is based on two two-dimensional intensity measurements, - a first intensity measurement is performed by means of a first lidar (1) located at a first position, and A second intensity measurement is performed by means of the first lidar (1) temporally subsequent to the first intensity measurement and at a second position different from the first position.

5. The method according to any one of claims 2 to 4, characterized in that: The passive measurement is performed by evaluating the two-dimensional intensity images acquired in the two two-dimensional intensity measurements by means of stereoscopic methods.

6. The method according to claim 5, characterized in that: A semi-global matching algorithm is used as the stereo vision method.

7. A device for detecting halos in lidar measurements using at least one lidar (1, 2), characterized in that A processing unit is provided, which is designed to: - determining the distance of the at least one laser radar (1, 2) from the laser radar reflection point (R) in active and passive measurements, - in the active measurement a first distance value is determined based on the signal transit time of the laser pulses, - determining a second distance value in the passive measurement based on triangulation from two-dimensional intensity measurements performed at different measurement positions, and - A halo is then inferred when the second distance value exceeds the first distance value by a predetermined degree.

Images