Method and apparatus for detecting blooming in LiDAR measurements

Abstract

The present invention relates to a method for detecting blooming in lidar measurements. According to the present invention, the distance to a lidar reflection point (R) is calculated in active and passive measurements. Additionally, a first distance value is calculated in an active measurement based on the signal transmission duration of a laser pulse, and a second distance value is calculated in a passive measurement based on triangulation of two-dimensional intensity measurements performed at different measurement locations, wherein blooming is inferred when the second distance value exceeds the first distance value by a predefined amount. Furthermore, the present invention relates to an apparatus for detecting blooming in lidar measurements using at least one lidar (1, 2).

Description

Technology Field

[0001] The present invention relates to a method for detecting blooming in lidar measurements.

[0002] The present invention also relates to an apparatus for detecting blooming in lidar measurements using at least one lidar. Background Technology

[0003] DE 10 2005 003 970 A1 discloses a method for determining the operability of a sensor array in an automobile, wherein the area detected by the sensor array is divided into various sub-regions and sensor signals assigned to sub-regions in a specific surrounding area are evaluated to determine the operability of the sensor array. In this case, sensor signals detected for different sub-regions are evaluated in turn as passing through a specific surrounding area. The sub-regions are detection areas of various lidar sensors or various angle sectors of lidar sensors.

[0004] Additionally, DE 10 2018 003 593 A1 discloses a method for operating a vehicle's assistance system, wherein the vehicle moves in an autonomous driving mode using an assistance system, and the assistance system includes a surround sensor having a plurality of detection devices positioned inside and / or outside the vehicle. When the vehicle is in an autonomous driving mode, objects around the vehicle and inside the vehicle are detected by the detection devices, and the functions of individual detection devices are continuously monitored by a monitoring module. If a detection device fails, only the auxiliary function connected to the failed detection device is disabled by a planning module connected to the monitoring module. The detection devices include LiDAR-based sensors.

[0005] US 2019 / 0391270 A1 describes a reflectance system for improving environmental monitoring using LiDAR in the presence of a highly reflective surface. The reflectance system includes a multiprocessor which is a memory that communicates with the processor. Additionally, the reflectance system includes a scanning module having a command that, when executed by the processor, causes the processor to control the emission of a scanning beam with a scanning intensity distinct from the initial intensity of the initial beam used to detect the first point cloud and to dynamically control the LiDAR to detect a second point cloud that bypasses the concealed object in response to a determination that the first point cloud contains observation of a highly reflective concealed object. Additionally, the output module is provided with a command that, when executed by the processor, causes the processor to generate a composite point cloud in the first point cloud and the second point cloud, which improves environmental monitoring using LiDAR by reducing interference caused by the concealed object. means of solving the problem

[0006] The object of the present invention is to provide a novel method and a novel apparatus for detecting blooming in lidar measurements.

[0007] These objectives are achieved according to the present invention by a method having the features specified in claim 1 and an apparatus having the features specified in claim 7.

[0008] A preferred embodiment of the present invention is the subject of a dependent claim.

[0009] In a method for detecting blooming in lidar measurement, according to the present invention, the distance to the lidar reflection point is calculated in active measurement and passive measurement, wherein a first distance value is calculated based on the signal transmission duration of a laser pulse in active measurement and a second distance value is calculated based on triangulation of a two-dimensional intensity measurement performed at a different measurement location in passive measurement. Then, if the second distance value exceeds the first distance value by a predefined amount, it is inferred to be blooming.

[0010] At this time, passive measurement based on two-dimensional intensity measurement means ambient environment detection using at least one lidar, in which at least one lidar without active emission of laser radiation detects only optical radiation present in the surrounding environment.

[0011] Here, blooming refers to overexposure or crosstalk in LiDAR measurements. For example, blooming occurs when a laser pulse emitted from a LiDAR is reflected by a highly reflective object, such as a traffic sign or a headlight reflector. In this case, a larger amount of emitted energy is returned to the LiDAR compared to a less reflective object. The returned light is typically poorly focused. There are various reasons for this; frequently, reflections from the object are not optimal, atmospheric particles deflect the laser beam, or dust on the LiDAR cover scatters the light. This can cause reflected light to strike multiple receiver cells within the LiDAR that are spatially close to each other, or to be transmitted to adjacent pixels. Consequently, distance measurements are triggered depending on the detector's sensitivity. The blooming effect is generally stronger at shorter distances from the LiDAR because the amount of energy reflected from the object decreases rapidly as the distance the light must travel increases.

[0012] Lidar plays an important role in other automation platforms, such as driver assistance systems and robots, because it can accurately represent its surroundings in three dimensions. However, if blooming occurs, incorrect results may be obtained when measuring the distance between the lidar and objects detected in its vicinity. In particular, the blooming effect can lead to false positive lidar measurements, making it more difficult to accurately represent the surroundings in three dimensions.

[0013] Using this method, blooming in LiDAR measurements can be reliably detected in a simple manner, thereby preventing erroneous results in such distance measurements or at least enabling stable recognition. As a result, safe operation is possible, for example, in automation, particularly in highly autonomous driving, autonomous vehicles, or robots.

[0014] In one possible embodiment of the method, manual measurement is based on two two-dimensional intensity measurements, wherein the first intensity measurement is performed with a first lidar and the second intensity measurement is performed with a second lidar positioned at a different location from the first lidar. This allows manual measurement to be performed easily and reliably, and consequently enables the detection of blooming in particular reliably.

[0015] In another possible embodiment of the method, two two-dimensional intensity measurements are performed simultaneously or sequentially. In particular, when intensity measurements are performed simultaneously, manual measurement of distance can be performed very quickly.

[0016] In another possible embodiment of the method, manual measurement is based on two two-dimensional intensity measurements, wherein the first intensity measurement is performed with a lidar at a first position and the second intensity measurement is performed using the same lidar at a second position other than the first position after the first measurement in time. This allows manual measurement to be performed easily and reliably, and consequently enables particularly reliable blooming detection, and since only one lidar is required to perform the two-dimensional intensity measurement, hardware usage and cost are particularly reduced.

[0017] In another possible embodiment of the method, manual measurement is performed by evaluating two-dimensional intensity images recorded from two two-dimensional intensity measurements using a stereoscopic method. This stereoscopic method reliably calculates the distance to the lidar reflection point, and thus the distance to objects around the lidar.

[0018] In another possible embodiment of the method, a semi-global matching algorithm is used as a stereoscopic method in which the distance to a pixel of a two-dimensional intensity image, and thus the distance to an object around the lidar, is very reliable and particularly accurate.

[0019] A blooming detection device in a lidar measurement includes at least one lidar and, according to the present invention, calculates the distance of at least one lidar from a lidar reflection point in active and passive measurements, calculates a first distance value in active measurement based on the signal transmission duration of a laser pulse, calculates a second distance value in passive measurement based on triangulation of two-dimensional intensity measurements performed at different measurement locations, and is characterized by a processing unit designed to infer blooming if the second distance value exceeds the first distance value by a predefined amount.

[0020] Using this device enables reliable detection of blooming in LiDAR measurements in a simple manner, thereby preventing erroneous results in distance measurements performed using LiDAR or at least ensuring stable detection. As a result, safe operation is possible, for example, in automation, particularly in highly autonomous driving, autonomous vehicles, or robots. Brief explanation of the drawing

[0021] Exemplary embodiments of the present invention are described in more detail below with reference to the drawings. The drawing is as follows. Figure 1 schematically illustrates the arrangement of lidar and the surrounding environment monitored by lidar. Figure 2 schematically illustrates the placement of lidar at different points in time and the surrounding environment monitored by lidar. FIG. 3 schematically illustrates a lidar image captured by the lidar according to FIG. 2 at a first point in time. FIG. 4 schematically illustrates a lidar image captured by the lidar according to FIG. 2 at a second time point. Figure 5 schematically illustrates an array of two lidars and the surrounding environment monitored by the lidars. FIG. 6 schematically illustrates a lidar image captured by the first lidar according to FIG. 5. FIG. 7 schematically illustrates a lidar image captured by a second lidar according to FIG. 5. For mutually matching parts, the same reference symbol is attached in all drawings. Specific details for implementing the invention

[0022] Figure 1 illustrates an array of lidar (1) and the surrounding environment monitored by lidar (1).

[0023] Around the lidar (1), there are two objects (O1, O2) detected by the lidar (1) within the detection area (E).

[0024] Lidar (1) is deployed, for example, in automated driving vehicles, particularly in highly autonomous or autonomous driving vehicles. Lidar (1) can also be deployed in robots.

[0025] The first object (O1) is a highly reflective object (O1), such as a traffic sign, such as a highway sign placed on a road (FB). The second object (O2) is located on the road and has an arbitrary reflectiveness, such as a reflectiveness lower or higher than that of the first object (O1).

[0026] Using the lidar (1), a laser pulse is emitted and the time until the reflected laser pulse reaches the receiver of the lidar (1) is recorded to calculate the distance to surrounding objects (O1, O2). In this case, the lidar (1) may include multiple lasers and / or multiple receivers to increase the measurement speed and spatial resolution of the lidar (1). The measurement performed by the lidar (1), also called a scan, may be performed in such a way that the complete scan can be interpreted as a two-dimensional measurement grid, also called a lidar image.

[0027] In the urban surrounding environment of the lidar (1), the first object (O1) creates blooming points (P1 to Pn) at equal distances above and below the object (O1) due to high reflectivity during laser measurement, resulting in so-called blooming artifacts. If these blooming points (P1 to Pn) are not detected as blooming points, there is a risk that additional data processing, such as sensor fusion, will determine that there is an obstacle, such as a traffic tail, and that the driver assistance system will perform unexpected braking.

[0028] FIG. 2 illustrates an array of lidar (1) and the surrounding environment monitored by lidar (1) at different time points (t1, t2). FIG. 3 illustrates a lidar image (B1) captured by lidar (1) according to FIG. 2 at a first time point (t1), and FIG. 4 illustrates a lidar image (B1) captured by lidar (1) according to FIG. 2 at a second time point (t2) following the first time point (t1). The lidar images (B1, B2) are two-dimensional intensity images in which a two-dimensional measurement grid and axes represent values ​​of vertical angle α and horizontal angle β, respectively, and vertical angle α and horizontal angle β form image coordinates.

[0029] Lidar (1) is deployed on a mobile platform, such as an automated vehicle or robot, particularly in a highly autonomous vehicle or autonomous vehicle or robot.

[0030] As previously explained, the distance to objects (O1, O2) in the surrounding environment is calculated by the lidar (1) by recording the time until a laser pulse is emitted and the reflected laser pulse reaches the receiver of the lidar (1). In this case, the reflection occurs at a lidar reflection point (R) belonging to the corresponding object (O1, O2), for example, a so-called landmark.

[0031] Lidar (1) is considered an active sensor because, in order to perform real-time measurements, also known as Time of Flight (ToF) measurements, it must generally emit energy actively as described above. If the receiver of Lidar (1) is sufficiently sensitive, it may also be used to measure the intensity of ambient light at a specified wavelength of Lidar (1) that is scattered back to Lidar (1) without active illumination. This allows Lidar (1) to be used in passive two-dimensional intensity measurements to generate highly dynamic grayscale images of the scene. Since the intensity of passive reflected light is significantly low, blooming effects do not occur in these passive measurements. Since these passive measurements can be performed immediately before or after active measurements, the recorded scene shows almost no change between the two measurements. Active measurements provide an accurate three-dimensional representation of the area around Lidar (1), whereas passive measurements allow for high detail in the two-dimensional representation of objects (O1, O2). Thus, the two measurement principles can complement each other.

[0032] The lidar (1) displayed and placed on a moving platform is designed to actively measure the distance to the lidar reflection point (R) and to passively measure the intensity. The passive measurement can be performed immediately before or after the active measurement.

[0033] In order to calculate blooming in the lidar measurement, the first distance value is calculated based on the duration of the signal transmission of the laser pulse from the lidar (1) to the lidar reflection point (R) and back to the lidar (1), and the distance to the lidar reflection point (R) in the active measurement and passive measurement is calculated based on the data detected by the lidar (1).

[0034] Next, the second distance value is calculated from manual measurements based on the triangulation of two-dimensional intensity measurements performed at various measurement locations.

[0035] Subsequently, if the second distance value exceeds the first distance value by a predefined amount, particularly if it is significantly larger than the first distance value, it is inferred to be blooming.

[0036] Manual measurement is based on two two-dimensional intensity measurements, the first intensity measurement is performed with the lidar (1) at the first position at the first time point (t1), and the second intensity measurement is performed with the same lidar (1) at the second time point (t2), which is after the first measurement in time, at a second position different from the first position. Between the two time points (t1, t2), the relative position of the lidar (1) with respect to the lidar reflection point (R) changes due to the movement of the platform.

[0037] The movement of the lidar (1) between two measurements is known, for example, through the evaluation of an inertial measurement unit that is placed on a moving platform and calibrated for the lidar (1) or a joint reference system.

[0038] If characteristic locations, such as landmarks, are observed in the lidar images (B1, B2) from a different perspective, a three-dimensional reconstruction of the observed scene can be performed. Due to the movement of the lidar (1), the characteristic location and the associated lidar reflection point (R), or the pixel captured in the two-dimensional intensity image representing it, may appear at a different location in the lidar images (B1, B2) captured at a different surrounding location. This effect is generally referred to as motion parallax. If the movement of the lidar (1) between two time points (t1, t2) is known and the location of a single identical lidar reflection point (R) is found in both lidar images (B1, B2), the three-dimensional location and thus the distance to the lidar reflection point (R) can be reconstructed by triangulation.

[0039] For example, manual measurement is performed by evaluating two two-dimensional intensity measurements using a stereoscopic method, such as a semi-global matching algorithm.

[0040] Possible exemplary embodiments of a method for detecting blooming in lidar measurements are described below.

[0041] First, a generally well-known stereo matching algorithm, such as a semi-global matching algorithm, is used to calculate the angular displacement between each pixel in the LiDAR images (B1, B2) recorded manually and from two different perspectives. For example, at time point t1, the LiDAR (1) views the LiDAR reflection point (R) or the pixel representing it at a vertical angle (α) of 10 degrees and a horizontal angle (β) of 5 degrees. At time point t2, the LiDAR (1) views the LiDAR reflection point (R) or the pixel representing it at a vertical angle (α) of 10 degrees and a horizontal angle (β) of 20 degrees.

[0042] Since the three-dimensional movement of the lidar (1) between the capture of the two lidar images (B1, B2) is known, the three-dimensional coordinates of the measured pixel position, i.e., the lidar reflection point (R), can be triangulated using information about the corresponding position angle in the first measurement and the second measurement.

[0043] By comparing the passive measurements derived from the described triangulation with the active measurements at every pixel location, one can draw a conclusion regarding whether blooming is present. If the distances derived from the passive measurements by the Structure From Motion algorithm are significantly greater than the active measurements, blooming can be inferred as a valid explanation.

[0044] FIG. 5 illustrates an array of two lidars (1, 2) and the surrounding environment monitored by the lidars (1, 2). FIG. 6 illustrates a lidar image (B1) captured by the lidar (1) according to FIG. 5, and FIG. 7 illustrates a lidar image (B2) captured at the same time by another lidar (2). The lidar images (B1, B2) are two-dimensional intensity images in which a two-dimensional measurement grid and axes represent the values ​​of a vertical angle α and a horizontal angle β, respectively, and as a result, the vertical angle α and the horizontal angle β form image coordinates.

[0045] Two lidars (1, 2) are deployed on a mobile platform, such as an automated vehicle or robot, particularly a highly autonomous or autonomous vehicle or autonomous mobile vehicle. The lidars (1, 2) are designed to be time-synchronized to simultaneously capture the same type of spatial angle.

[0046] Both lidars (1, 2) are designed to actively measure the distance to the lidar reflection point (R) and to passively measure the intensity. The passive measurement can be performed immediately before or after the active measurement.

[0047] In this exemplary embodiment, to detect blooming in the lidar measurement, the distance to the lidar reflection point (R) in the active measurement and the distance to the lidar reflection point (R) in the passive measurement are calculated using data recorded by the lidar (1, 2), while calculating a first distance value in the active measurement based on the signal transmission duration of a laser pulse from the lidar (1) and / or lidar (2) to the lidar reflection point (R) and back to the lidar (1) and / or lidar (2).

[0048] Next, the second distance value is calculated from manual measurement based on the triangulation of two-dimensional intensity measurements performed at different measurement locations.

[0049] Subsequently, if the second distance value exceeds the first distance value by a predefined amount, particularly if it is significantly larger than the first distance value, it is inferred to be blooming.

[0050] The external parameters of the lidar (1, 2), namely the position and / or alignment, are known. For this purpose, the lidar (1, 2) are calibrated relative to each other or in relation to a joint reference system.

[0051] In contrast to the exemplary embodiment described above with reference to FIGS. 2 through 4, this allows manual measurement to be based on two two-dimensional intensity measurements, wherein the first intensity measurement is performed with the first lidar (1) and the second intensity measurement is performed with the second lidar (2) positioned at a different location from the first lidar.

[0052] By simultaneously capturing scenes from different perspectives with lidars (1, 2), characteristic locations, such as landmarks, in lidar images (B1, B2) can be observed from different perspectives, thereby enabling the 3D reconstruction of the observed scenes. Because the positions of lidars (1, 2) are different, characteristic locations and associated lidar reflection points (R) or pixels representing them may appear at different locations in lidar images (B1, B2) recorded in different environmental locations. Since the relative positions of lidars (1, 2) and external parameters are known, if the location of a single identical lidar reflection point (R) or a pixel representing it is found in both lidar images (B1, B2), the distance from the lidar reflection point (R) can be reconstructed by 3D positioning and simple triangulation.

[0053] For example, manual measurement is performed by evaluating two intensity measurements using a stereoscopic method such as a semi-global matching algorithm.

[0054] Possible exemplary embodiments of a method for detecting blooming in lidar measurements are described below.

[0055] First, a generally well-known stereo matching algorithm, such as a semi-global matching algorithm, is used to calculate the angular displacement between each pixel in the LiDAR images (B1, B2) recorded manually and from two different perspectives. For example, the LiDAR (1) views the LiDAR reflection point (R) or the pixel representing it at a vertical angle (α) of 10 degrees and a horizontal angle (β) of 5 degrees. The LiDAR (1) simultaneously views the LiDAR reflection point (R) or the pixel representing it at a vertical angle (α) of 10 degrees and a horizontal angle (β) of 20 degrees.

[0056] Since the transformation between the coordinate systems of the two lidars (1, 2) is known, the three-dimensional coordinates of the measured pixel location, i.e., the lidar reflection point (R), can be triangulated using information about the corresponding position angle of the manual intensity measurement performed by the lidars (1, 2).

[0057] By comparing the manual measurements derived from the described triangulation with the active measurements at every pixel location, one can draw a conclusion regarding whether blooming is present. If the manual measurement results in a much greater distance than the active measurement, blooming can be a valid explanation. Explanation of the symbols

[0058] 1 Lida 2 LiDAR B1 LiDAR image B2 LiDAR image E detection area FB Road O1 object O2 object P1 ~ Pn Blooming Point R LiDAR reflection point time t1 time t2 Angle α β angle

Claims

Claim 1 A method for detecting blooming in lidar measurement, characterized by: - ​​calculating the distance based on the duration of transmission of a laser pulse signal by emitting laser light in lidar measurement, and the distance to the lidar reflection point (R) in passive measurement that detects only light radiation present in the surrounding environment at different measurement locations without emitting laser light; - calculating a first distance value in active measurement based on the duration of signal transmission of the laser pulse; - calculating a second distance value in passive measurement based on triangulation of a two-dimensional intensity measurement performed at different measurement locations; and - inferring blooming when the second distance value exceeds the first distance value by a predefined amount. Claim 2 A method according to claim 1, characterized in that the manual measurement is based on two two-dimensional intensity measurements, the first intensity measurement is performed with the first lidar (1), and the second intensity measurement is performed using the second lidar (2) placed at a different location from the first lidar (1). Claim 3 A method according to paragraph 2, characterized in that two manual strength measurements are performed simultaneously or sequentially in time. Claim 4 A method according to claim 1, characterized in that the manual measurement is based on a two-dimensional intensity measurement, the first intensity measurement is performed with a lidar (1) at a first position, and after the first measurement, a second intensity measurement is performed with the same lidar (1) at a second position different from the first position. Claim 5 A method characterized in that, in any one of paragraphs 2 through 4, manual measurement is performed by evaluating a two-dimensional intensity image recorded in a two-dimensional intensity measurement using a stereoscopic method. Claim 6 A method according to claim 5, characterized in that the stereoscopic method includes a semi-global matching algorithm. Claim 7 A device for detecting blooming in lidar measurements using at least one lidar (1, 2), comprising: an active measurement that calculates distance based on the duration of transmission of a laser pulse signal by emitting laser light in lidar measurements, and a passive measurement that detects only light radiation present in the surrounding environment at different measurement locations without emitting laser light, and calculates the distance from at least one lidar (1, 2) to a lidar reflection point (R); calculates a first distance value in the active measurement based on the duration of signal transmission of the laser pulse; calculates a second distance value in the passive measurement based on triangulation of two-dimensional intensity measurements performed at different measurement locations; and then, if the second distance value exceeds the first distance value by a predefined amount, it is inferred to be blooming.

Images