Method and system for predicting road lane geometry

Abstract

The invention relates to a method (14) for predicting a road lane geometry. The invention provides a camera lane prediction comprising at least one camera predicted lane segment (6) based on a detection of lane markings by a camera (3). Furthermore, a front vehicle predicted lane prediction is provided comprising at least one front vehicle predicted lane segment (11) based on a trajectory (10) of at least one front vehicle (9). Then, the at least one camera predicted lane segment (6) and the at least one front vehicle predicted lane segment (11) are stitched to obtain a predicted road lane geometry (14). Furthermore, a system (2) for predicting a road lane geometry (14) and a vehicle (1) comprising the system (2) are provided.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for predicting a road lane geometry, to a system for predicting a road lane geometry and to a vehicle comprising a system for predicting a road lane geometry. BACKGROUND

[0002] For semi-autonomous and autonomous driving of a vehicle on a road, it is an important aspect to know the geometry of the road lane ahead. If there is a sharp bend ahead, for example, the speed of the own vehicle has to be reduced. Another example is that if there is a slower vehicle ahead in the same lane of the own vehicle, the own vehicle has to slow down, if the slower vehicle is in a different lane than the own vehicle, the own vehicle can maintain the existing speed.

[0003] Known methods and systems for predicting a road lane geometry use cameras and radar to detect road markings and to deduce the geometry of the road lane from the detected road markings. However, the range of road markings that can be covered by a camera is usually limited to a maximum of 100 meters, which can be too short for many requirements of semi-autonomous and autonomous driving. On the other hand, radar can only detect protruding road markings such as curbs, bollards or barriers, but not lane markings. Thus, radar can detect the geometry of the entire road ahead, but not the geometry of a single lane. SUMMARY

[0004] It is therefore an object of the present invention to provide an improved method and system for predicting a road lane geometry, in particular a method and system for predicting a road lane geometry with a larger coverage.

[0005] The object of the present invention is solved by the technical solutions described below.

[0006] According to a first aspect of the present invention, a method for predicting a road lane geometry is provided. The method is in particular a method for predicting a road lane geometry ahead of an own vehicle. In this context, the geometry refers to a two-dimensional representation of the road lane, in particular their path.

[0007] According to the method, a camera lane prediction is provided. The camera lane prediction is based on road markings detected by a camera. In this context, a "camera" refers to one or more cameras in the own vehicle, wherein the camera can work in the visible and / or near-infrared spectrum, i.e. the wavelength of the light is between 380 nanometers and 750 nanometers or 750 nanometers and 3 micrometers, respectively. The camera lane prediction comprises at least one camera-predicted lane segment.

[0008] Furthermore, a lane prediction of a forward driving vehicle is provided. The lane prediction of the forward driving vehicle is based on a trajectory of at least one forward driving vehicle, i.e. a trajectory of at least one vehicle driving in front of the ego vehicle. The lane prediction of the forward driving vehicle assumes that most other vehicles, in particular vehicles driving in front of the ego vehicle, move on a road lane, so that the trajectory of the forward driving vehicle indicates the road lane. The lane prediction of the forward driving vehicle comprises at least one predicted lane segment of the forward driving vehicle.

[0009] According to the method, the at least one camera predicted lane segment and the at least one forward driving vehicle predicted lane segment are stitched to obtain a predicted road lane geometry. Thereby, the range of the road lane geometry given by the camera lane prediction is extended by the road lane geometry given by the lane prediction of the forward driving vehicle. It is beneficial for semi-autonomous driving and autonomous driving to input a road lane geometry with an extended range.

[0010] According to an embodiment, the at least one camera predicted lane segment and the at least one forward driving vehicle predicted lane segment are each given by a left edge and a right edge. In this context, the left edge and the right edge refer to the edges of the lane segment in the driving direction of the vehicle on the lane. If several lanes are arranged next to each other, the left edge of a lane can coincide with the right edge of the adjacent lane. The left edge and the right edge are in particular given as a list of two-dimensional points. These two-dimensional points can be given in a coordinate system connected to the ego vehicle, wherein the Y direction can correspond to the driving direction of the ego vehicle and the X direction can correspond to a direction perpendicular to the driving direction. The arrangement of the list of two-dimensional points can be such that the first two-dimensional point is closest to the ego vehicle and the last two-dimensional point in the list is farthest away from the ego vehicle. By connecting the two-dimensional points in the list, the edge of the lane can be obtained.

[0011] According to an embodiment, the at least one camera predicted lane segment is obtained by image detection of road markings on the image provided by the at least one camera. The at least one camera can be a visible light or near-infrared light camera. The road markings can be lane markings, curbs, guide posts and / or guard rails. The road markings are identified by image detection, wherein the image detection can be based on artificial intelligence such as neural networks or decision trees.

[0012] According to an embodiment, the lane segment predicted by the at least one preceding vehicle is determined by tracking the at least one preceding vehicle. The preceding vehicle can be tracked by means of a camera, a laser radar and / or a radar. The output of the camera, laser radar and / or radar can then be analyzed by means of artificial intelligence, in particular in order to obtain position information of the at least one preceding vehicle, for example in the form of two-dimensional points. Furthermore, the at least one preceding vehicle can also be tracked on the basis of the camera output, laser radar output and / or radar output by means of artificial intelligence. Temporal position information of the at least one preceding vehicle is then stored in order to obtain a trajectory of the at least one preceding vehicle. In other words, a time sequence of positions of the at least one preceding vehicle can be obtained. The time sequence can be an ordered list of positions. If these positions are recorded in a coordinate system associated with the own vehicle, the fact that the coordinate system has moved must be taken into account when calculating the trajectory of the at least one preceding vehicle. A lane segment predicted by the at least one preceding vehicle is then generated around the trajectory of the at least one preceding vehicle with a lane width. If the at least one preceding vehicle is in a lane center, the generated lane segment predicted by the preceding vehicle corresponds to a lane segment of the actual road. That is, each position of the at least one preceding vehicle can correspond to a left edge and a right edge of the at least one preceding vehicle, two-dimensional points being established in each direction perpendicular to the driving direction of the at least one preceding vehicle at a distance of one-half lane width from the position of the at least one preceding vehicle. The lane width can have a predetermined value or can be inferred from the lane width of the lane segment predicted by the camera. The tracking of the at least one preceding vehicle can be carried out at a distance of at least 150 m, preferably 200 m, in particular preferably 300 m, from the own vehicle. The time sequence of positions can comprise positions at a distance of at least 2 s, preferably at least 3 s, in particular preferably at least 5 s, from the at least one preceding vehicle.

[0013] According to an embodiment, the lane segment predicted by the at least one camera and / or the lane segment predicted by the at least one preceding vehicle is smoothed. This can be achieved by polynomial, in particular cubic polynomial, fitting of the list of two-dimensional points, and then sampling the fitted data back into the list of two-dimensional points. Random errors in the lane segment are eliminated by the smoothing, and further processing is improved. For the lane segment predicted by the preceding vehicle, instead of smoothing the trajectory of the preceding vehicle, the smoothing can be used to determine the lane segment predicted by the preceding vehicle.

[0014] According to an embodiment, the lane segment predicted by at least one camera and / or the lane segment predicted by at least one vehicle driving ahead is extrapolated. The extrapolation can be performed over the entire lane segment, on the basis of the curvature of the lane segment at the end of the lane segment for which the extrapolation is performed, or on the basis of the entire lane segment, wherein a higher weight is assigned to the end point of the lane segment for which the extrapolation is performed and a lower weight is assigned to the end point of the lane segment opposite the extrapolation. Furthermore, a further smoothing of the lane segment can be performed, for example by means of Kalman smoothing. The extrapolation can be performed up to the point at which the adjacent lane segments touch one another and / or overlap one another. Here, adjacent lane segments are understood to mean lane segments which are adjacent to one another in the driving direction.

[0015] According to an embodiment, adjacent lane segments are spliced together if the overlap of the adjacent lane segments in the direction perpendicular to the driving direction of the lane is greater than 50%, preferably greater than 65%, and particularly preferably greater than 80%. Splicing is only performed if there is a clear overlap between the adjacent lane segments. If the overlap is small, it can be assumed that the lane segments do not belong to the same lane. This can occur, for example, in the case of a change of lane by a vehicle driving ahead, so that the lane segment predicted by the vehicle driving ahead does not correspond to the actual lane of the road.

[0016] According to an embodiment, a splicing quality measure is assigned to each splice of adjacent lane segments. The splicing quality measure can be based on the overlap of the adjacent lane segments in the direction perpendicular to the driving direction of the lane. The splicing quality measure can then be used, for example, in a Kalman filter for the description of the uncertainty of the predicted road lane geometry.

[0017] According to an embodiment, an additional road lane is added if the overlap of a lane segment predicted by a vehicle driving ahead with other lane segments in the direction perpendicular to the driving direction of the lane is less than 15%, preferably less than 5%, and particularly preferably 0%. In other words, the lane segment predicted by the vehicle driving ahead is outside the known lane and is added as an additional lane, since there is only a small overlap or almost no overlap with other lane segments. Such an additional lane can in particular correspond to an exit lane, a turning lane or an additional lane formed as a result of a widening of the road.

[0018] According to an embodiment, the method further comprises providing a prediction of a road boundary based on radar detections of the road boundary. Such road boundaries can be curbs, bollards and / or barriers detected by a radar system of the ego vehicle and can be analyzed by an artificial intelligence. The prediction of the road boundary can be smoothed, e.g. by fitting and resampling, as described above. Lane segments of a preceding vehicle prediction are rejected and not further used, e.g. not stitched with other lane segments, if they exceed the predicted road boundary by more than 25%, preferably more than 15%, and particularly preferably more than 5%. As an additional measure or alternative, the prediction of the road boundary can be used as an envelope for at least one camera predicted lane segment and / or at least one preceding vehicle predicted lane segment.

[0019] According to an embodiment, the method further comprises transmitting / broadcasting the predicted road lane geometry. The transmission can be done via a wired connection or a wireless connection. The transmission can also refer to copying the predicted road lane geometry to a different process in the same computing device or to sharing the predicted road lane geometry in a memory which can be accessed by different processes of the same computing device. The predicted road lane geometry is particularly transmitted to an advanced driver assistance system. The advanced driver assistance system can be or can comprise, for example, an automatic cruise control and / or an emergency brake assist. The advanced driver assistance system can also be a system for semi-autonomous or autonomous driving. The advanced driver assistance system greatly benefits from the extended reach of the predicted road lane geometry.

[0020] According to a second aspect of the present invention, a system for predicting a road lane geometry is provided. The system comprises at least one camera and a computing unit. The at least one camera can be a visible light camera and / or a near-infrared light camera. The computing unit is configured to generate a camera lane prediction based on road marking images taken by the camera and to generate a preceding vehicle lane prediction based on a trajectory of a preceding vehicle. Furthermore, the computing unit is configured to perform a stitching of lane segments of the camera lane prediction and lane segments of the preceding vehicle lane prediction. The thus generated road lane geometry prediction provides a greater reach compared to a camera lane prediction based on road marking images, which is beneficial, for example, for an advanced driver assistance system.

[0021] The system can particularly be configured to perform the method for predicting a road lane geometry as described above.

[0022] According to a further aspect of the present invention, a vehicle comprising a system for predicting a road lane geometry as described above is provided. The vehicle can particularly further comprise an advanced driver assistance system which benefits from the extended reach of the predicted road lane geometry provided by the system for predicting a road lane geometry. BRIEF DESCRIPTION OF DRAWINGS

[0023] These and other aspects of the present application will be explained in more detail in the following description, examples and with reference to the figures, in which

[0024] Figure 1 Fig. 1 shows an embodiment of a vehicle with a system for predicting road lane geometry;

[0025] Figure 2 Fig. 2 shows another embodiment of a vehicle with a system for predicting road lane geometry;

[0026] Figure 3a Fig. 3 shows an example of a method for predicting road lane geometry;

[0027] Figure 4 Fig. 4 shows another example of a method for predicting road lane geometry;

[0028] Figure 5a Fig. 5 shows yet another example of a method for predicting road lane geometry; and

[0029] Figure 6a Fig. 6 shows yet another example of a method for predicting road lane geometry.

[0030] It should be noted that the figures are purely schematic and not drawn to scale. In the figures, parts corresponding to described parts can have the same reference numerals. Examples, embodiments or features as options, whether or not they are indicated as non-limiting, are not to be understood as a limitation on the invention as claimed. DETAILED DESCRIPTION

[0031] Figure 1 Fig. 1 shows an embodiment of a vehicle 1 with a system 2 for predicting road lane geometry. The system 2 comprises a camera 3 and a computing unit 4, wherein the camera 3 is connected to the computing unit 4. In this embodiment, the connection between the camera 3 and the computing unit 4 is a wired connection, but it can also be a wireless connection.

[0032] The camera 3 is configured to take images of the area in front of the vehicle 1, in particular of road markings and of a vehicle driving in front. These images are transferred to the computing unit 4 and analyzed by the computing unit 4. The analysis can be performed by artificial intelligence, for example a neural network or a decision tree. As a result of the analysis, the road markings and the vehicle driving in front are recognized and their position is determined, for example as a two-dimensional point.

[0033] Based on the detected road markings, a camera lane prediction comprising at least one camera predicted lane segment is generated. Based on the trajectory of the preceding vehicle, a vehicle lane prediction comprising at least one preceding vehicle predicted lane segment is generated. Finally, the calculation unit 4 stitches the at least one camera predicted lane segment and the at least one preceding vehicle predicted lane segment. As a result, a predicted road lane geometry is obtained, which has a larger range of action than the camera lane prediction. This predicted road lane geometry can be used by the advanced driver assistance system of the vehicle 1 and benefits from the extended range of action.

[0034] Figure 2 Another embodiment of a vehicle 1 with a system 2 for predicting a road lane geometry is shown. The system also comprises a radar 5 connected to the calculation unit 4. By means of the radar, a detection of road boundaries and of a preceding vehicle is further possible. Such a detection can improve the results of the road markings and of the preceding vehicle detected by the camera 3 and detect additional road boundaries. The detected road boundaries are used by the calculation unit 4 for generating a road boundary prediction, which can improve the predicted road lane geometry.

[0035] Figures 3a to 3d An example of a method for predicting a road lane geometry is shown. Figure 3a In the middle, a vehicle 1 and three camera predicted lane segments 6.1 to 6.3 are shown. The camera predicted lane segments 6 are derived from road markings, such as lane markings, curbs, bollards and / or barriers detected in the images taken by the camera 3. The lane segments 6 are given by their left edges 7.1 to 7.3 or their right edges 8.1 to 8.3, respectively, wherein the left edge 7.2 coincides with the right edge 8.1 and the left edge 7.3 coincides with the right edge 8.2. Also shown is the driving direction D of the vehicle 1.

[0036] Furthermore, a preceding vehicle 9.1 and a trajectory 10.1 of the preceding vehicle 9.1 are shown. The trajectory 10.1 is derived by tracking the preceding vehicle 9.1 by means of the camera 3 and / or the radar 5 and storing the temporal position information of the preceding vehicle 9.1. Furthermore, the driving direction D' of the preceding vehicle 9.1 is shown.

[0037] As a next step, a preceding vehicle predicted lane segment 11.1 is generated from the trajectory 10.1 of the preceding vehicle 9.1. The preceding vehicle predicted lane segment 11.1 is also given by a left edge 12.1 and a right edge 13.1. The preceding vehicle predicted lane segment 11.1 is generated by setting a left edge 12.1 and a right edge 13.1 at a distance of one half lane width in any direction perpendicular to the lane 10.1 and the driving direction D' of the preceding vehicle 9.1.

[0038] The camera-predicted lane segment 6 and the front driving vehicle-predicted lane segment 11.1 are then extrapolated until they touch each other, as shown in Figure 3c In this example, the extrapolated front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6.2 match completely, i.e. the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6.2 overlap by 100% in a direction perpendicular to the driving direction D, D'.

[0039] Therefore, the camera-predicted lane segment 6.2 and the front driving vehicle-predicted lane segment 11.1 are stitched together in order to obtain the predicted road lane geometry 14 as shown in Figure 3d

[0040] Figure 4 An example is shown in Fig. 6, where the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segments 6.2 and 6.3 each overlap by about 50% in a direction perpendicular to the driving direction D, D'. This can be the result of a lane change of the front driving vehicle 9.1, for example. In this case, the front driving vehicle-predicted lane segment 11.1 is not stitched to the camera-predicted lane segment 6.2 or 6.3, but discarded.

[0041] An example is shown in Fig. 7, where the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6.2 overlap by 100% in a direction perpendicular to the driving direction D, D'. This can be the result of a lane change of the front driving vehicle 9.2, for example. In this case, the front driving vehicle-predicted lane segment 11.1 is stitched to the camera-predicted lane segment 6.2. Figure 5a An example is shown in Fig. 8, where the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6 do not overlap at all. This can indicate an exit lane or a turn lane, for example. Therefore, an additional road lane corresponding to the front driving vehicle-predicted lane segment 11.1 is added and incorporated into the road lane geometry 14. The start of the additional road lane is set to the point closest to the trajectory 10.1 of the ego vehicle 1, as this is the first confirmed position of the existence of the additional road lane. 5b An example is shown in Fig. 9, where the left and right road boundaries 15.1 and 15.2 are obtained by radar detection of road boundaries such as curbs, guide posts and barriers. A front driving vehicle predicted lane segment 11.2, such as the one predicted by the front driving vehicle 9.2, which exceeds the predicted road boundaries 15 by a predetermined portion, is also rejected from being used for generating the road lane geometry 14, as shown in Fig. 10.

[0042] Figure 6a An example is shown in Fig. 11, where the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6.2 overlap by 100% in a direction perpendicular to the driving direction D, D'. This can be the result of a lane change of the front driving vehicle 9.1, for example. In this case, the front driving vehicle-predicted lane segment 11.1 is stitched to the camera-predicted lane segment 6.2. 6b An example is shown in Fig. 12, where the front driving vehicle-predicted lane segment 11.1 and the camera-predicted lane segment 6 do not overlap at all. This can indicate an exit lane or a turn lane, for example. Therefore, an additional road lane corresponding to the front driving vehicle-predicted lane segment 11.1 is added and incorporated into the road lane geometry 14. The start of the additional road lane is set to the point closest to the trajectory 10.1 of the ego vehicle 1, as this is the first confirmed position of the existence of the additional road lane. Figure 6b An example is shown in Fig. 13, where the left and right road boundaries 15.1 and 15.2 are obtained by radar detection of road boundaries such as curbs, guide posts and barriers. A front driving vehicle predicted lane segment 11.2, such as the one predicted by the front driving vehicle 9.2, which exceeds the predicted road boundaries 15 by a predetermined portion, is also rejected from being used for generating the road lane geometry 14, as shown in Fig. 14.

[0043] ​Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from an study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other parts or steps, and the indefinite articles "a" and "an" do not exclude a plurality. In fact, the specific measures described in the dependent claims are intended to be combined with the measures of the independent claim in an arbitrary fashion. All the dependent claims mentioned in connection with the independent claims are to be understood as forming an integral part of the application and the independent claims. The reference signs in the claims should not be construed as limiting the scope of the claims.

Claims

1. Method (14) for predicting a road lane geometry, the method comprising: providing camera lane predictions comprising at least one camera predicted lane segment (6) based on detection of road markings by a camera (3); providing ahead driving vehicle lane predictions comprising at least one ahead driving vehicle predicted lane segment (11) based on a trajectory (10) of at least one ahead driving vehicle (9); and stitching together at least one camera predicted lane segment (6) and at least one ahead driving vehicle predicted lane segment (11) to obtain a predicted road lane geometry (14), wherein at least one camera predicted lane segment (6) and at least one ahead driving vehicle predicted lane segment (11) are given by their left edge (7; 12) and right edge (8; 13), respectively, wherein at least one camera predicted lane segment (6) and / or at least one ahead driving vehicle predicted lane segment (11) are extrapolated until they reach an adjacent lane segment and / or overlap with an adjacent lane segment, wherein adjacent lane segments are stitched together if they overlap in a direction perpendicular to the lane driving direction by more than 50%, wherein for each stitch of adjacent lane segments a stitch quality measure is assigned based on the overlap of adjacent lane segments in a direction perpendicular to the lane driving direction, wherein an additional lane is added if an ahead driving vehicle predicted lane segment (11) overlaps with other lane segments in a direction perpendicular to the driving direction by less than 15%. The left edge (7; 12) and right edge (8; 13) are given as a list of two-dimensional points.

2. The method of claim 1, wherein, The at least one camera predicted lane segment (6) is obtained by image detection of road markings on an image provided by at least one camera (3), wherein the road markings are at least one of a group comprising lane markings, curbs, guide posts and barriers.

3. The method of claim 1, wherein, The at least one ahead driving vehicle predicted lane segment (11) is obtained by tracking at least one ahead driving vehicle (9), storing temporal position information of the at least one ahead driving vehicle (9) by a camera (3), lidar and / or radar (5) to obtain a trajectory (10) of the at least one ahead driving vehicle (9), and generating the at least one ahead driving vehicle predicted lane segment (11) around the trajectory (10) of the at least one ahead driving vehicle (9) over the lane width.

4. The method of any one of claims 1-3, wherein, The at least one camera predicted lane segment (6) and / or the at least one ahead driving vehicle predicted lane segment (11) are smoothed by polynomial fitting of the list of two-dimensional points and then sampling the fitted data back to the list of two-dimensional points.

5. The method of claim 2, wherein, Adjacent lane segments are stitched together if they overlap in a direction perpendicular to the lane driving direction by more than 65%.

6. The method of any one of claims 1-3, wherein, Adjacent lane segments are stitched together if they overlap in a direction perpendicular to the lane driving direction by more than 80%.

7. The method of claim 6, wherein, An additional lane is added if an ahead driving vehicle predicted lane segment (11) overlaps with other lane segments in a direction perpendicular to the driving direction by less than 5%.

8. The method of any one of claims 1-3, wherein, ​ 9. The method of claim 8, wherein, if a lane segment (11) of a front driving vehicle prediction overlaps with other lane segments in a direction perpendicular to the driving direction by 0%, an additional lane is added.

10. The method according to any one of claims 1 to 3, further providing a road boundary (15) prediction based on radar detection (5) of road boundaries and rejecting a front driving vehicle predicted lane segment (11) that exceeds 25% outside the predicted road boundary (15).

11. The method according to claim 10, rejecting a front driving vehicle predicted lane segment (11) that exceeds 15% outside the predicted road boundary (15).

12. The method according to claim 11, rejecting a front driving vehicle predicted lane segment (11) that exceeds 5% outside the predicted road boundary (15).

13. The method according to any one of claims 1 to 3, further comprising transmitting the predicted road lane geometry (14) to an advanced driver assistance system.

14. A system for predicting a road lane geometry (14), the system (2) comprising: at least one camera (3) and a computing unit (4) configured to generate a camera lane prediction based on road marking images taken by the camera (3); generate a front driving vehicle lane prediction based on a trajectory (10) of a front driving vehicle (9) and stitch lane segments of the camera lane prediction and the front driving vehicle lane prediction, wherein the at least one camera predicted lane segment (6) and the at least one front driving vehicle predicted lane segment (11) are given by their left edge (7; 12) and right edge (8; 13), respectively, wherein the at least one camera predicted lane segment (6) and / or the at least one front driving vehicle predicted lane segment (11) are extrapolated until they reach an adjacent lane segment and / or overlap with an adjacent lane segment, wherein adjacent lane segments are stitched together if they overlap in a direction perpendicular to the lane driving direction by more than 50%, wherein a stitching quality measure is assigned to each stitch of adjacent lane segments based on the overlap of adjacent lane segments in a direction perpendicular to the lane driving direction, wherein if a lane segment (11) of a front driving vehicle prediction overlaps with other lane segments in a direction perpendicular to the driving direction by less than 15%, an additional lane is added.

15. A vehicle comprising a system (2) for predicting a road lane geometry (14) according to claim 14.

Images