Method for identifying and classifying high road traffic marks

Abstract

The invention relates to a method for detecting and classifying elevated roadway markings (14) by means of a lidar system (36) of a vehicle (10), said lidar system (36) comprising a transmitting unit (38), a receiving unit (44), at least one laser radiation source (46) and at least one detector (48). The following method steps are carried out individually or in combination with one another: a) a comparison is carried out by means of the lidar system (36) between a map containing information about the roadway course (20) and higher roadway markings (14) that may be identified along the roadway course (20), b) determining normal roadway markings from at least one intensity-coded or background-coded 3D point cloud (28) by means of the lidar system (36) and comparing the normal roadway markings with higher roadway markings (14) that may be detected along the roadway course (20); c) determining context-sensitive information from the accompanying infrastructure or from the traffic planning correlation of the location of the elevated roadway markings (14) and / or d) analyzing the intensity-coded or background-light-coded 3D point cloud (28) for a regular pattern of the arrangement (60) of the elevated roadway markings (14).

Description

Technical Field

[0001] The present invention relates to a method for identifying and classifying raised roadway markings by means of a lidar system of a vehicle. The lidar system includes a transmitting unit, a receiving unit, at least one laser ray source, and at least one detector. Furthermore, the present invention also relates to the application of this method in a lidar system for reliable identification and crossability classification of raised roadway markings. Background Art

[0002] DE 10 2011 082 477 A1 relates to a method and a system for creating a digital image of the vehicle surroundings. The system describes a driving assistance system for autonomous driving. For this purpose, measurement data about the vehicle surroundings are acquired by a first sensor system based on the vehicle, and at least one optically detectable marking fixedly arranged in the vehicle surroundings is acquired by a video system based on the vehicle. Here, at least one optically detectable feature of at least one marking is identified from the acquired data of the video system, and the relative position and orientation of the marking with respect to the vehicle surroundings are determined.

[0003] EP 3 529 561 B1 relates to a system and a method for generating a digital road model from aerial images or satellite images and data detected by a vehicle. At least one vehicle trajectory for at least one road section is received in a database outside the vehicle. At least one image is received, which shows at least some parts of the at least one road section, wherein the image has a viewing angle corresponding to an image acquired substantially vertically downward from an elevated position. Then the at least one trajectory and the at least one image are superimposed such that the orientations of the roads in the at least one trajectory and the at least one image are consistent. Subsequently, the at least one image is analyzed in a corridor extending along and including the trajectory, and driving-related or position-related features of the road section are identified in the corridor.

[0004] DE 10 2019 106 213 A1 has a method for determining at least one position information of at least one object in a monitoring area by means of an optical detection device and an optical detection device regarding the subject matter. Here, a method for determining at least one position information of at least one object in a monitoring area by means of an optical detection device and the detection device are described. In this method, at least one optical emission signal having a main axis of emission signal propagation is emitted into the monitoring area. At least a part of at least one emission signal is reflected on an object that may be present in the monitoring area as a received signal, and at least that part of the received signal whose original signal propagation main axis extends parallel to the emission signal propagation main axis at least on the side facing the object changes.

[0005] Current lidar systems use SPAD detectors, and there may be disadvantages related to the dynamic range in SPAD detectors in the case of different intensities of reflected objects at different distances. Here, the lidar system has the task of, on the one hand, reliably identifying objects with a particularly low reflectivity (e.g., 5%) at a large distance of, for example, 200 m, and also reliably identifying objects with a particularly high reflectivity (e.g., 10,000%) at a small distance (e.g., a few meters). The resulting dynamic coefficient between the darkest and brightest identifiable objects can reach 2000 in the case of the same distance; in addition, the part of the signal intensity that increases or decreases with the square of the distance (producing an attenuation effect) comes into play. If the lidar sensor system is designed to identify weak-reflection objects at the greatest possible distance while following functional reliability, the possibility of optical saturation increases, accompanied by false detections caused by scattered light, also known as "crosstalk". This is a significant effect especially in the case of nearby highly reflective objects. Here, the challenge is to carry out the optical design within the framework of a lidar sensor system consisting of a transmitting and receiving objective lens, a laser diode, and detector pixels such that the dynamic range can be reliably detected. Summary of the Invention

[0006] According to the present invention, a method for identifying and classifying raised roadway markings by means of a lidar system of a vehicle is proposed, wherein the lidar system includes a transmitting unit, a receiving unit, at least one laser beam source, and at least one detector, and the method has the following method steps that can be implemented at least individually or in combination with each other:

[0007] a) Performing a comparison between a map containing information about the roadway alignment and raised roadway markings that may be identified along the roadway alignment by means of the lidar system,

[0008] b) Obtaining normal roadway markings from at least one intensity-encoded or background-light-encoded 3D point cloud by means of the lidar system and comparing them with raised roadway markings that may be identified along the roadway alignment,

[0009] c) Obtaining context-sensitive information from the accompanying infrastructure or from the traffic planning relevance of the installation location of the raised roadway markings and

[0010] d) Analyzing the intensity-encoded or background-light-encoded 3D point cloud for a regular pattern of the arrangement of the raised roadway markings.

[0011] Advantageously, a reliable distinction can be achieved between false detections caused by raised roadway markings and actually impassable objects by means of the method proposed according to the present invention.

[0012] In an advantageous expansion of the solution according to the present invention, according to method step a), map comparison is performed in a cloud-based database based on the geographical location of the vehicle and a map or the observation of the vehicle traveling ahead.

[0013] In an advantageous expansion of the method according to the present invention, according to method step d), an approximation of a resolvable curve or geometric plane is implemented, which is carried out as follows: the potentially recognizable raised lane markings are appropriately connected to each other and a continuous check of credibility is performed.

[0014] Within the framework of the method according to the present invention, advantageously according to method step d), the potentially recognizable raised lane markings can be regularly repeated along the resolvable curve or the geometric plane at the same mutual spacing respectively.

[0015] In the method according to the present invention, according to method step d), circles can be drawn around the potentially recognizable raised lane markings such that the intersection points of adjacent circles form horizontal, vertical, or mutually orthogonal lines. The layout pattern of the lane markings can be extrapolated based on these horizontal, vertical, or mutually orthogonal lines.

[0016] In an advantageous expansion of the method according to the present invention, according to method step d), geometric credibility verification is implemented to determine the geometric regularity of the appearance of the raised lane markings for distinguishing from individual objects located on the lane.

[0017] In an advantageous expansion of the method according to the present invention, according to method step d), targeted prediction is carried out such that with the help of a small number of visible raised lane markings, additional raised lane markings are targeted extrapolated from the intensity-encoded 3D point cloud as a continuation of the map geometric pattern.

[0018] In the method according to the present invention, a comparison is made between on the one hand the intensity-encoded 3D point cloud and on the other hand the background light-encoded 3D point cloud. The intensity-encoded 3D point cloud is a cloud composed of 3D points having brightness values corresponding to the received reflected laser pulse signals. The background light-encoded 3D point cloud is a cloud composed of 3D points having brightness values corresponding to the brightness of the received background noise signals.

[0019] Finally, the present invention relates to the application of this method in a lidar system for reliable identification and crossability classification of raised lane markings.

[0020] Advantages of the invention

[0021] With the solution in the form of the method according to the present invention, reliable identification and classifiability of crosswalk markings that protrude can be achieved. Therefore, an accurate distinction can be made between crosswalk markings that protrude and objects that are actually non-crossable, such as warning signs, traffic signs, or the like, which have a non-negligible structural height such that vehicles and passengers will necessarily be damaged. With the solution according to the present invention, there is a possibility of reliably identifying and classifying crosswalk markings that protrude by analyzing the geometric structure, placement model, quantity, spacing, and the decline of the optical crosstalk intensity (crosstalk strength) with respect to distance and angle as well as via other recurring patterns. What is decisive in the method according to the present invention is that crosswalk markings that protrude rarely emerge alone and out of context. This can be understood as that a crosswalk marking that protrudes is generally used in combination with other crosswalk markings that protrude, and is combined with the accompanying infrastructure or the traffic planning relevance at the placement location. Here, for example, there may be dangerous points, such as merging lanes, near airports, highway intersections, construction sites, or the like. With the solution according to the present invention, false detections caused by crosswalk markings that protrude and objects that are actually non-crossable, such as objects configured in the form of warning signs extending in the vertical direction, can be reliably distinguished. In addition, the quality of the point cloud after optical crosstalk filtering (crosstalk filtering) can be significantly improved by the method according to the present invention, and thus the performance of the lidar sensor system can be significantly improved. With the solution according to the present invention, the optical crosstalk that occurs, i.e., the optical crosstalk, is effectively compensated after it is identified.

[0022] As a result, with the method according to the present invention, it can be achieved that the lidar sensor of the lidar sensor system generates a crosstalk-free, that is, crosstalk-free, point cloud when crosswalk markings that protrude appear. Therefore, the basic source of the optical crosstalk (crosstalk) that occurs can be simply and reliably identified.

[0023] By using the method according to the present invention, false braking can be prevented in the automated driving function in addition; in addition, the situation of lack of braking before relevant objects can be prevented, thus improving traffic safety. In addition, by using the method according to the present invention, unwanted braking on irrelevant objects can be prevented, which is also extremely beneficial to traffic safety. The data integrity existing within the 3D point cloud can be supported by the method according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Embodiments of the present invention are explained in more detail according to the drawings and the following description.

[0025] The drawings show:

[0026] Figure 1: A vehicle equipped with a lidar system, the roadway alignment, and a 3D point cloud derived from the roadway alignment, in front of a raised roadway marking.

[0027] Figure 1.1 : Components of the lidar system.

[0028] Figure 2 : A schematic illustration of a raised roadway marking, the occurring crosstalk range (crosstalk area), and the identified crosstalk range, and the error range bounded by these ranges.

[0029] Figure 3 : An exemplary schematic illustration of a raised roadway marking, arranged in a regular pattern along the lanes of the roadway following a described curve.

[0030] Figure 4 : Narrowing of the roadway in a perspective view, the narrowing of the roadway being bounded on both sides by paired raised roadway markings, and the inference of the scene obtained by extrapolation, and

[0031] Figure 5 : A circle composed of several raised roadway markings, bounded by two linear patterns. Detailed Description of the Embodiment

[0032] In the following description of the embodiments of the present invention, the same or similar elements are identified by the same reference numerals, and in some cases, the repeated description of these elements is omitted. The drawings only schematically depict the subject matter of the present invention.

[0033] Figure 1 A vehicle 10 equipped with a lidar system 36 is shown. A roadway 12 extends in front of the vehicle 10, and the roadway has raised roadway markings 14 within the visible range of the lidar system 36. This raised roadway marking 14 has a structural height 16, which is typically a few centimeters. Above the raised roadway marking 14, a crosstalk range 18 extends, which is also referred to as a crosstalk area.

[0034] In addition, Figure 1 shows the field of view of the scene extending in front of the vehicle 10. The roadway alignment 20 is shown as a straight alignment 22 in this scene. Warning signs 26 are located on the roadway 12, and these warning signs extend vertically from the surface of the roadway 12. In addition, there are two raised roadway markings 14 spaced apart from each other beside the two warning signs 26.

[0035] Below the scene in front of the vehicle 10 on the roadway 12, an intensity - encoded or background - light - encoded 3D point cloud 28 is shown, which represents the scene collected by the lidar system 36 according to the roadway alignment 20. From according to Figure 1The display of the intensity-coded or background-light-coded 3D point cloud 28 shows that the warning signs 26 extending in the vertical direction are shown as objects 30 that are classified as not traversable. In contrast, the elevated roadway markings 14 that are spaced apart from one another in this scene are shown as objects 32 extending in the horizontal direction on the roadway 12 and are therefore distortedly reproduced objects 34. Despite the relatively small structural height 16 of the elevated roadway markings 14, these appear in the intensity-coded or background-light-coded 3D point cloud 28 as extended objects 32 and cannot be clearly classified as traversable due to their crosstalk properties or crosstalk.

[0036] According to Figure 1.1 The diagram schematically shows the installation of the laser radar system 36 in Figure 1 Components in the vehicle 10 shown. The lidar system 36 comprises, for example, a transmitting unit 38, which comprises at least one laser beam source 46. In addition, the lidar system 36 comprises, for example, a receiving unit 44, which contains a detector 48, to which the beam reflected by the object detected by the lidar system 36 is returned. The lidar system 36 described can also include other components that are not shown in detail here and are conventionally arranged in the front area of ​​the vehicle 10 for detecting the scene in front of the vehicle 10.

[0037] Figure 2 A schematic diagram of a raised lane marking 14 and the crosstalk range 18 originating from the lane marking is shown. Figure 2 It is known that the elevated lane marking 14 has a first retroreflective surface 40 and a second retroreflective surface 42 opposite to the first retroreflective surface. The retroreflective surfaces 40, 42 are detected by the laser radar system 36 as an intensity-coded or background-light-coded 3D point cloud 28. Above and below the elevated lane marking 14 extend crosstalk ranges (crosstalk) 18, which can be divided into error ranges 50 and identified crosstalk ranges 52. The identified crosstalk range 52 is a range identified as a safe crosstalk range 52. Starting from the crosstalk range 18, it is not possible to identify whether the elevated lane marking 14 can be correctly classified as traversable. Therefore, when such a detection is performed by the laser radar system 36, unreasonable misbraking or unreasonable lane change interruptions may occur, accompanied by the resulting disadvantages in driving experience, trajectory planning and accident safety.

[0038] refer to Figure 2Note that outside the error range of 50, crosstalk points 54 may be misclassified as object points 56 around the raised roadway marking 14, for example, with a probability of less than 5%. The object points misclassified as optical crosstalk (crosstalk) are identified with reference marker 56, although the object point 56 is part of the raised roadway marking 14. Figure 2 Visualize the critical performance requirements through the object point 56, that is, the object point 56 must be recognized by the lidar system 36. By implementing the method proposed according to the present invention in the lidar system 36 of the vehicle 10, reliable identification and passability classification of the raised roadway marking 14 can be achieved, enabling a safe distinction from objects 30 that are actually impassable, such as the warning signs 26 shown in the drawings. Reliable identification and reliable passability classification of the raised roadway marking 14 can be carried out, which can be implemented by analyzing the geometric structure, placement model, quantity, spacing, and crosstalk intensity with respect to distance and angle and with respect to the decline of other rules. The key point here is the situation: a raised roadway marking 14 rarely appears alone or without context, such as without other raised roadway markings 14, without accompanying infrastructure, or without traffic planning relevance at the placement location (such as near an airport, in a merging area, at a dangerous point, or similar locations).

[0039] The fundamental solution on which the method proposed according to the present invention is based is that the raised roadway markings 14 rarely appear alone, but these roadway markings appear in an arrangement 60 of a regular pattern 78, as shown in Figure 3 , 4 and 5.

[0040] Fundamentally, in order to detect and reliably classify the raised roadway markings 14, different information sources are used, either individually or in combination in the sense of information fusion.

[0041] According to method step a), a comparison is made between the map with information about the road alignment and the raised roadway markings 14 that may be recognized on the curve alignment. Usually, for this purpose, map comparison is carried out based on the geographical location of the vehicle 10 with the map or by observing the vehicles traveling ahead, for example, via a cloud-based database.

[0042] According to method step b), roadway markings are extracted from the intensity-encoded or background-light-encoded 3D point cloud 28 and compared with the raised roadway markings 14 that may be recognized on the curve alignment 24, as shown, for example, in Figure 3As shown. According to method step c), context-sensitive information regarding the traffic planning relevance of the placement of the accompanying infrastructure or infrastructure components can be used, for example, for hazardous points, to indicate near an airport or near a hospital, or as a reminder for lane merges, government buildings, tunnels, etc.

[0043] Within the framework of detecting and reliably classifying the raised carriageway markings 14, the intensity-encoded or background-light-encoded 3D point cloud 28 can also be used for the occurrence of a regular pattern 78 in the arrangement 60 of the raised carriageway markings 14.

[0044] For the analysis of, for example, Figure 1 the intensity-encoded or background-light-encoded 3D point cloud 28 shown, a search is made for the occurrence of a regular pattern 78 within which the raised carriageway markings 14 are in a defined position. For example, an approximately resolvable description of a curve 62 can be approximated as shown in Figure 3 or a geometric plane 65 can be shown (see Figure 3 ), within which the potentially recognizable raised carriageway markings 14 are appropriately connected to each other, and a continuous check of credibility is carried out. The intensity-encoded or background-light-encoded 3D point cloud 28 can also be studied for the repeatability of the regular occurrence of potentially recognizable raised carriageway markings 14, which have the same spacing 76 between each other, where the regular repetition of potentially recognizable raised carriageway markings 14 with the same spacing 76 between each other is reproduced along a curve path 24 or a straight path 22 according to the carriageway alignment 20.

[0045] Furthermore, for example, within the framework of analyzing the intensity-encoded or background-light-encoded 3D point cloud 28, a circle 70 can be hypothetically drawn around each of the potentially recognizable raised carriageway markings 14, which has a large enough radius such that the resulting intersections of these circles 70 produce straight lines not only horizontally but also vertically and orthogonally relative to each other, and the paths of these straight lines are parallel to each other.

[0046] Within the framework of the previously described method steps a), b), c), and d), a number of geometric credibility checks can be applied to the intensity-encoded 3D point cloud 28. The aim here is to prove that the occurrence of geometric regularity is a distinguishing feature, for example, relative to individual debris on the carriageway 12.

[0047] By means of pattern recognition algorithms and / or matching algorithms, an installation model of the raised carriageway markings 14 can be generated as a clear indication that they are artificial infrastructure measures created by humans. For example, a speed warning can be surrounded by a group of circular configurations of the raised carriageway markings 14, as shown, for example, in Figure 5as shown, such that for recognition it may only be necessary to perform the association of the intensity-encoded or background-light-encoded 3D point cloud 28 with the circular pattern (see Figure 5 at position 70 in).

[0048] The optical crosstalk 18 itself can also provide reliable information for recognizing the raised roadway markings 14. The intensity of false detections decreases non-linearly with the viewing angle in the lidar system 36 and with the distance, yet in an analytically describable manner. Here, not only can the currently existing information sources be used for classification decisions, but also predictions can be made purposefully, such that an extrapolation 68 is obtained, for example in combination with Figure 4 as shown. For analyzing the intensity-encoded or background-light-encoded 3D point cloud 28, an extrapolation 68 is obtained from a small number of visible raised roadway markings 14, such that the continuation of the geometric regularities once recognized can be inferred. In Figure 4 the case of, there is a regular spacing 76 between the individual raised roadway markings 14 and a gradual decrease in the spacing between these roadway markings, such that an extrapolation 68 of the further course of the roadway 12 as shown in Figure 4 is obtained.

[0049] There is the possibility of comparing the background-light-encoded 3D point cloud and the intensity-encoded 3D point cloud 28. Since the retroreflective surfaces 40, 42 reflect the incident light back in principle in the same direction in which the light enters, the retroreflective surfaces 40, 42 do not appear as strongly in the background-light-encoded point cloud 28 as in the intensity-encoded 3D point cloud 28, because the background light of the first-mentioned type of encoding is not measured together. The sunlight is not reflected to the lidar sensor but is reflected back to the sky.

[0050] From Figure 3 the illustration of the raised roadway markings 14 can be seen, which are arranged, for example, in a regular pattern 78 along the lanes 80 of the roadway 12. Figure 3 shows that the individual raised roadway markings 14 have a very small structural height 16, i.e., are objects that can be traversed. The raised roadway markings 14 are used as the boundaries of the lanes 80 of the multi-lane roadway 12 in the embodiment according to Figure 3 . The individual raised roadway markings 14 are arranged in an arrangement 60 that on the one hand adapts to the curved course 24 of the individual lanes 80 of the roadway 12 and on the other hand has a regular spacing 76 between each other. In the vertical direction, the individual raised roadway markings 14 are evenly spaced from each other and in this regard are used as the boundaries of the individual lanes 80 of the roadway 12. The arrangement 60 of the individual raised roadway markings 14 in the embodiment according to Figure 3The curve 62 that follows the parseable description in the illustration. The carriageway 12 has a plurality of lanes 80, which are separated from each other by a regular pattern 78 consisting of raised carriageway markings 14, and the carriageway is bounded on the right by a lane edge 64.

[0051] Contrary to the illustration according to Figure 3 In the illustration according to Figure 4 A geometric plane 65 is identified, which reproduces the straight-line trend 22. It can be seen from Figure 4 that the raised carriageway markings 14 arranged in pairs on the left and right in the geometric plane 65 are arranged at a uniform mutual spacing 76. From the regular pattern 78 according to Figure 4 an extrapolation 68 can be obtained from the geometric plane 65, which corresponds to continuing the identified regular pattern 78 of the arrangement 60 along the straight-line trend 22 of the carriageway 12. In this way, additional information about the section of the carriageway 12 located further in front of the vehicle 10, that is, the carriageway 12, can be obtained by means of the extrapolation 68 from a small number of visible raised carriageway markings 14.

[0052] Figure 5 The geometric plane 65 is shown, which is bounded on the left by a first linear pattern 72 and on the right by a second linear pattern 74. It can be seen from the schematic illustration according to Figure 5 that the individual raised carriageway markings 14 are arranged in a regular pattern 78 along the first linear pattern 72 and the second linear pattern 74, and this pattern has a uniform mutual spacing 76. A circle 70 is shown in the middle between, for example, the two linear patterns 72, 74, and this circle is composed of the individual raised carriageway markings 14 arranged in a regular pattern 78. Regarding the pattern of the circle 70, the individual raised carriageway markings 14 have a uniform mutual spacing 76.

[0053] The invention is not limited to the embodiments described herein and the aspects emphasized therein. On the contrary, within the scope specified by the claims, a large number of modifications can be made, which are within the framework of the actions of a professional person.

Claims

1. A method for identifying and classifying raised road markings (14) by means of a lidar system (36) of a vehicle (10), wherein, the lidar system (36) includes a transmitting unit (38), a receiving unit (44), at least one laser ray source (46) and at least one detector (48), and the method has the following method steps that can be implemented separately or in combination: a) Performing a comparison between a map containing information about the road alignment (20) and raised road markings (14) that may be recognized along the road alignment (20) by means of the lidar system (36); b) Obtaining normal road markings from at least one intensity - encoded or background - light - encoded 3D point cloud (28) by means of the lidar system (36) and comparing them with raised road markings (14) that may be recognized along the road alignment (20); c) Obtaining context - sensitive information from the accompanying infrastructure or from the traffic - planning relevance of the installation locations of the raised road markings (14) and / or d) Analyzing the intensity - encoded or background - light - encoded 3D point cloud (28) for a regular pattern of the arrangement (60) of the raised road markings (14).

2. The method according to claim 1, characterized in that, according to method step a), a map comparison is performed in a cloud - based database according to the geographical location of the vehicle (10) with the map or with the observation of the vehicle traveling ahead.

3. The method according to claims 1 and 2, characterized in that, according to method step d), an approximation of an analyzable curve (62) or a geometric plane (65) is performed in such a way that the raised road markings (14) that may be recognized are appropriately connected to each other and a continuous check of credibility is performed.

4. The method according to claims 1 to 3, characterized in that, according to method step d), the raised lane markings (14) that may be recognized are regularly repeated along the analyzable curve (62) or the geometric plane (65) at the same mutual spacing (76) respectively.

5. The method according to claims 1 to 4, characterized in that, according to method step d), circles (70) are drawn around the raised lane markings (14) that may be recognized in such a way that the intersection points of adjacent circles (70) form horizontal, vertical and mutually orthogonal straight lines (72, 74).

6. The method according to claims 1 to 5, characterized in that, a geometric credibility check is performed according to method step d) to determine the geometric regularity of the appearance of the raised road markings (14) and to distinguish them from single objects located on the road (12).

7. The method according to claims 1 to 6, characterized in that, according to method step d), a targeted prediction is performed: based on a small number of visible raised road markings, additional raised road markings (14) are targeted extrapolated from the intensity - encoded or background - light - encoded 3D point cloud (28) as a continuation of the recognized geometric regularity.

8. The method according to any one of claims 1 to 7, characterized in that, the intensity-encoded 3D point cloud (28) is compared with the background-light-encoded 3D point cloud (28).

9. The method according to any one of claims 1 to 8, characterized in that, by comparing the background-light-encoded point cloud and the intensity-encoded point cloud (28), crosstalk effects (crosstalk) (18) are minimized when describing the infrastructure.

10. Use of the method according to any one of claims 1 to 9, characterized in that, a lidar system (36) is used for reliable identification and traversability classification of raised roadway markings (14).

Images