Method for identifying lane markings for a vehicle - Patents.com

Abstract

The present disclosure provides a method (100) for identifying lane markings (2) of a first lane (1) for a vehicle (10), comprising: providing (102) measurement data from monitoring the surroundings of the vehicle (10); feeding (104) the measurement data to at least one machine learning model; evaluating (106) the path of lane markings (2) using at least one machine learning model; assessing (108) the width (3) of the lane markings (2) using at least one machine learning model; The method (100) includes: Additionally, the present disclosure includes a method (200) for training at least one machine learning model for use in the method (100) described above.

Description

[Technical Field]

[0001] The present invention relates to a method for identifying lane markings for a vehicle. Furthermore, the present invention relates to a method for training at least one machine learning model. [Background technology]

[0002] Lane marking detection for a vehicle is an important aspect, particularly in the field of autonomous driving. Traditionally, lane marking detection has been pursued by applying a variety of technical solution approaches.

[0003] The first approach is based on classical gradient methods. In gradient methods, gradients are extracted from images of the road recorded by sensors and / or cameras on the vehicle. From these, the lane markings of the lane in which the vehicle is located are determined. Here, the inner and outer edges of the lane markings are determined and combined, from which the width of the lane markings is calculated.

[0004] The second approach uses a segment-based method based on deep learning scenarios. This approach also determines the outer and inner edges of lane markings by estimating the individual segment masks of the lane markings. However, the accuracy of this method depends substantially on the resolution of the segment masks used.

[0005] A further approach to determining lane markings of a lane is pursued by the so-called anchor-based or anchorless approach. Here, the goal is to model the center of the detected lane marking using a direct, or immediate, representation of the line of the detected lane marking. This approach allows for greater accuracy in determining or locating the exact position of the lane marking. However, this approach does not allow for direct inferences to be drawn regarding the inner and outer edges of the detected lane marking. Summary of the Invention

[0006] According to a first aspect, the present disclosure relates to a method for identifying lane markings of a first lane for a vehicle, the method including the steps of: In the first step, monitoring of the vehicle's surroundings provides measurement data.

[0007] In a second step, the measurement data is fed to at least one machine learning model. In the third step, the path of the lane markings is evaluated using at least one machine learning model.

[0008] In the fourth step, the width of the lane markings is assessed using at least one machine learning model. Thus, the present invention provides the advantage that not only are the centers of the detected lane markings of the vehicle's lane modeled, but also the widths of the detected lane markings of the vehicle's lane are always detected or estimated, by adding or solving a corresponding regression problem.

[0009] The solution approach pursued by the present invention achieves the advantage of a higher accuracy in determining lane markings than known approaches, since the centers of the lane markings for which the lane is being detected are still identified.

[0010] A further advantage is that all lane markings can be represented or identified by the approach of the present invention. A further advantage of the solution according to the invention is that the solution of the regression problem for the width of the lane markings to be detected is much easier to solve than the continuous estimation of the inner and outer edges of the lane markings, since the width of the lane markings to be determined can be determined relative to the center line or reference line of the lane, making the method according to the invention independent of the position or orientation of the lane markings in the recorded images.

[0011] One possible form of this method provides for the path to show the centerlines of the lane markings in a discrete or continuous manner, thereby realizing the advantage of greater accuracy in determining the lane markings.

[0012] One possible embodiment of the method provides that the width of the lane markings is determined by regression, which has the advantage that the width of the lane markings can be determined efficiently.

[0013] One possible embodiment of the method provides that for regression, the subregion of the lane marking closest to the vehicle is selected, which has the advantage that the width of the lane marking can be determined efficiently.

[0014] One possible form of this method contemplates that regression is performed at multiple locations along the lane marking and the determined widths are aggregated to arrive at a final result regarding the width of the lane marking, thereby further reducing errors in determining the width of the lane marking.

[0015] One possible embodiment of the method provides for the path to be represented in the form of distances to reference lines passing through the monitored area around the vehicle, thereby achieving the advantage that the width of the lane markings can be determined efficiently and with high accuracy.

[0016] One possible version of the method contemplates that image data and / or video data are selected as measurement data, as these are the most important measurement modalities for detecting lane markings.

[0017] One possible embodiment of the method contemplates that the step of assessing the width of the lane markings of the first lane includes the following steps: In the first step, a location along the path of the lane markings is selected.

[0018] In a second step, a search is made for two boundary points between the lane markings on the one hand and the lane surface on the other hand in a given search direction relative to the path of the lane markings. In the third step, the width of the lane marking is determined from the separation distance between the two boundary points.

[0019] One possible embodiment of the method contemplates that the determination of the width of the lane markings of the first lane takes into account camera calibration data and / or information about the road surface conditions of the first lane, which can further improve accuracy.

[0020] According to a second aspect, the present disclosure relates to a method for training at least one machine learning model for use in the above-described method, the method comprising the steps of:

[0021] The first step involves providing training examples of measurement data recorded from the viewpoint of the ego vehicle and indicating the presence of one or more lane markings. In a second step, a target path and a target width for at least one lane marking that separates the lane in which the host vehicle is currently traveling are provided.

[0022] In the third step, the training examples are fed to a machine learning model to be trained, which identifies the path and width of lane markings using the methods described above.

[0023] In a fourth step, an assessment of the deviation between this path and this width on the one hand and the target path and target width on the other hand is carried out using a predetermined cost function. In the fifth step, the parameters characterizing the behavior of the machine learning model are optimized in the hope that further processing of training examples will improve the evaluation by the cost function.

[0024] The width of the lane marking that separates the lane in which the ego vehicle is currently traveling is the most accurately identifiable target width for labeling, since this lane marking is closest to the ego vehicle. Therefore, it is advantageous to use only this target width. Naturally, the more labeled training examples available, the better. However, for example, when more distant lane markings are labeled with inaccurately identified target widths, these noisy labels can adversely affect training success.

[0025] In one possible embodiment of the method, at least one target path is provided for determining a path for at least one further lane marking that does not demarcate the lane in which the host vehicle is currently traveling, and the cost function also evaluates the deviation of this path from the target path, thereby realizing the advantage that the method can also be efficiently used or adapted to determine further lane markings that are not in the currently traveling lane.

[0026] One possible version of this method contemplates disregarding at least one lane marking that is evident from the measurement data by setting its contribution to the cost function to zero, thereby allowing this lane marking to be excluded from the measurement data of the training examples more efficiently than exclusion based on separation distance.

[0027] According to a third aspect, the present disclosure relates to a computer program comprising machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the one or more computers or computer instances to perform a method according to the present invention.

[0028] According to a fourth aspect, the present disclosure relates to a machine-readable data carrier and / or download product comprising a computer program. According to a fifth aspect, the present disclosure relates to one or more computers and / or computer instances comprising a computer program and / or comprising a machine-readable data carrier and / or a downloadable product.

[0029] Further means for improving the invention will be presented in more detail below in conjunction with the description of preferred exemplary embodiments of the invention with reference to the drawings. [Brief explanation of the drawings]

[0030] [Figure 1] 1 is a schematic flow diagram of a method for identifying lane markings of a first lane for a vehicle. [Figure 2] 2 is a schematic flow diagram of a method for training at least one machine learning model for use in the method according to FIG. 1; [Figure 3] 1 is an exemplary diagram of a method known from the prior art for determining the center of a lane marking for a vehicle lane; [Figure 4] 1 is an exemplary diagram of a method for determining the width of lane markings for a vehicle lane, in accordance with the present invention; [Figure 5] FIG. 1 is an exemplary diagram of providing a target position and width of lane markings for a vehicle lane. DETAILED DESCRIPTION OF THE INVENTION

[0031] FIG. 1 with reference to FIG. 3 shows a schematic flow diagram of a method 100 for identifying lane markings 2 of a first lane 1 for a vehicle 10, the method 100 comprising the following steps: In a first step 102, measurement data is provided from monitoring the surroundings of the vehicle 10.

[0032] In a second step 104, the measurement data is fed to at least one machine learning model. In a third step 106, the path of the lane markings is evaluated using at least one machine learning model.

[0033] In a fourth step 108, the width 3 of the lane markings 2 is assessed using at least one machine learning model. The evaluation of the width 3 of the lane markings 2 of the first lane 1 in the fourth step 108 is preferably carried out in the following steps.

[0034] In a first step 120, a location along the path of lane marking 2 of lane 1 is selected. In a second step 122, two boundary points (see line segment 9 in FIG. 4) between the lane marking 2 on the one hand and the lane surface on the other hand are searched for in a predetermined search direction relative to the path of the lane marking 2.

[0035] In a third step 124, the width 3 of the lane marking 2 of the lane 1, i.e. the first lane 1, is determined from the separation distance between the two boundary points. FIG. 2 shows a schematic flow diagram of a method 200 for training at least one machine learning model for use in the method 100 according to FIG. 1, which method 200 comprises the following steps:

[0036] The first step 202 involves providing training examples of measurement data recorded from the perspective of the ego vehicle 10 and indicating the presence of one or more lane markings. In a second step 204, a target path and a target width for at least one lane marking that separates the lane in which the host vehicle 10 is currently traveling are provided.

[0037] In a third step 206, the training examples are fed to a machine learning model to be trained, which identifies the path and width 3 of the lane markings 2 using the method according to any one of claims 1 to 9.

[0038] In a fourth step 208, an assessment of the deviation between this path and this width 3 on the one hand and the target path and target width on the other hand is made using a predetermined cost function. In a fifth step 210, the parameters characterizing the behavior of the machine learning model are optimized in the hope that further processing of training examples will improve the evaluation by the cost function.

[0039] 3 shows an exemplary diagram of an image recorded by a vehicle 10 for applying a method known from the prior art for determining the center of a lane marking 2 for a lane 1 of the vehicle 10 using an anchor-based approach. Between the start point 6 and the end point 7 of the reference line 5, for each point, the individual distances of the individual line sections or individual line segments to the center 4 of each of the lane markings 2 of the lane 1 are determined using regression.

[0040] 4 shows by way of example how the method 100 proposed herein for determining the width 3 of lane markings 2 for a lane 1 of a vehicle 10 can be implemented on the same recorded image. Briefly, in this method, starting from a reference line 5 extending between a start point 6 and an end point 7, the width 3 of each of the lane markings 2 of the lane 1 is determined in the horizontal direction according to the following steps of the method 100:

[0041] In a first step 102, measurement data, here in the form of recorded images, is provided from the monitoring of the surroundings of the vehicle 10. Typically, the measurement data may be available as image data and / or video data from corresponding sensors or other technical detection devices (not shown) of the vehicle 10.

[0042] In a second step 104, the measurement data is fed to at least one machine learning model, where multiple models may be used, for example it may be envisaged to use a first model for determining the path of the lane markings 2 of the lane 1 and a second model for determining the width 3 of the lane markings 2 of the lane 1.

[0043] In a third step 106, an evaluation of the path of the lane markings 2 is performed using at least one machine learning model. The path may be, for example, the centerline 4 or an edge. In particular, the path of the centerline 4 may be represented, for example, in the form of a number of discrete positions. Here, the path of the centerline 4 may also be represented in a continuous form. The path of the centerline 4 may preferably be represented in the form of the separation distances of the individual line segments 9 of the lane markings 2 to the reference line 5 through the monitored area around the vehicle 10, as shown in FIG. 4.

[0044] In a fourth step 108, the width 3 of the lane markings 2 is evaluated with at least one machine learning model. Preferably, the width 3 of the lane markings 2 of the lane 1 is determined by regression. Here, in particular so-called scalar regression can be used, since scalar regression can be performed faster and provides more accurate results, since the lane markings 2 appear larger at a given pixel resolution of the vehicle camera. For the determination of the width 3 of the lane markings 2 of the first lane 1, preferably camera calibration data and / or information about the road surface conditions of the first lane 1 may also be taken into account.

[0045] Here, the regression may preferably be performed at multiple positions along the lane marking 2, and the determined widths are aggregated to obtain a final result for the width 3 of the lane marking 2. Here, appropriate weighting can be performed depending on the distance to the vehicle 10 to take into account different degrees of accuracy.

[0046] The evaluation of the width 3 of the lane markings 2 of the first lane 1 in the fourth step 108 is preferably carried out in the following steps. In a first step 120, a location along the path of lane marking 2 of lane 1 is selected.

[0047] In a second step 122, two boundary points (see line segment 9 in FIG. 4) between the lane marking 2 on the one hand and the lane surface on the other hand are searched for in a predetermined search direction relative to the path of the lane marking 2.

[0048] In a third step 124, the width 3 of the lane marking 2 of the lane 1, i.e. the first lane 1, is determined from the separation distance between the two boundary points. FIG. 5 illustrates, by way of example, how the target positions and widths of lane markings 2 for lane 1 of vehicle 10 can be obtained. Here, FIG. 5 shows how the centers 4 (dashed lines) of lane markings and the inner edges 11 of lane markings 2 for lane 1 of ego vehicle 1 are correspondingly represented. These representations can be used to determine the target width of the lane markings that demarcate the lane in which ego vehicle 10 is currently traveling. This target width can be used to label training examples for a machine learning model.

Claims

1. A method (100) for identifying lane markings (2) of a first lane (1) for a vehicle (10), comprising: providing (102) measurement data from monitoring the surroundings of the vehicle (10); feeding (104) the measurement data to at least one machine learning model; evaluating (106) a path of the lane markings (2) using the at least one machine learning model; assessing (108) a width (3) of the lane markings (2) using the at least one machine learning model; A method (100) comprising:

2. The method (100) of claim 1, wherein the path indicates the centerlines (4) of the lane markings (2) in a discrete or continuous manner.

3. 3. The method (100) of claim 1 or 2, wherein the width (3) of the lane markings (2) is determined by regression.

4. 4. The method (100) of claim 3, wherein the sub-region of the lane markings (2) closest to the vehicle (10) is selected for the regression.

5. 5. The method (100) according to claim 3 or 4, wherein the regression is performed at multiple positions along the lane marking (2) and the determined widths are aggregated to obtain a final result for the width (3) of the lane marking (2).

6. 6. The method (100) according to any one of claims 1 to 5, wherein the path is indicated in the form of a distance to a reference line (5) passing through the monitored area around the vehicle (10).

7. The method (100) according to any one of claims 1 to 6, wherein image data and / or video data are selected as measurement data.

8. The step (108) of evaluating the width (3) of the lane markings (2) of the first lane (1) comprises: selecting (120) a location along the route of the lane marking (2); searching (122) for two boundary points between the lane marking (2) on the one hand and the lane surface on the other hand in a predetermined search direction relative to the path of the lane marking (2); determining (124) the width (3) of the lane marking (2) from the separation distance between the two boundary points; The method (100) of any one of claims 1 to 7, comprising:

9. 9. The method (100) according to claim 1, wherein camera calibration data and / or information about road surface conditions in the first lane (1) are taken into account when determining the width (3) of the lane markings (2) of the first lane (1).

10. A method (200) for training at least one machine learning model for use in the method of any one of claims 1 to 9, comprising: providing (202) training examples of measurement data recorded from the perspective of an ego-vehicle (10) that indicate the presence of one or more lane markings; providing (204) a target path and a target width for at least one lane marking that separates the lane in which the host vehicle (10) is currently traveling; a step (206) of providing training examples to the machine learning model to be trained, the machine learning model identifying the path and width (3) of the lane markings (2) using the method of any one of claims 1 to 9; - evaluating (208) the deviation between said path and said width (3) on the one hand and a target path and target width on the other hand using a predetermined cost function; optimizing (210) parameters characterizing the behavior of the machine learning model in anticipation of improving its performance according to the cost function upon further processing of training examples; A method (200) comprising:

11. determining a route for at least one further lane marking that does not demarcate the lane in which the host vehicle is currently traveling, and providing at least one target route; the cost function also assesses the deviation of the path from its target path; The method (200) of claim 10.

12. 12. The method according to claim 10 or 11, wherein at least one lane marking (2) evident from the measurement data remains not taken into account by setting its contribution to the cost function to zero.

13. A computer program comprising machine-readable instructions which, when executed on one or more computers and / or computer instances, cause said one or more computers or computer instances to perform the method of any one of claims 1 to 9 and / or the method of any one of claims 10 to 12.

14. A machine-readable data carrier and / or download product comprising a computer program according to claim 13.

15. One or more computers and / or computer instances comprising a computer program according to claim 13 and / or comprising a machine-readable data carrier and / or downloadable product according to claim 14.

Images