A method for identifying lane markings in the lane a vehicle is traveling in.

Abstract

The present disclosure provides a method (100) for identifying lane markings (2) of a first lane (1) for a vehicle (10), comprising: providing (102) measurement data from monitoring the surroundings of the vehicle (10); feeding (104) the measurement data to at least one machine learning model; evaluating (106) the path of lane markings (2) using at least one machine learning model; assessing (108) the width (3) of the lane markings (2) using at least one machine learning model; The method (100) includes: Additionally, the present disclosure includes a method (200) for training at least one machine learning model for use in the method (100) described above.

Description

Technical Field

[0001] The present invention relates to a method for identifying lane dividing lines for a vehicle. Furthermore, the present invention relates to a method for training at least one machine learning model.

Background Art

[0002] The detection of lane dividing lines of lanes related to vehicles is an important aspect, especially in the field of autonomous driving. Conventionally, the detection of lane dividing lines of lanes has been pursued by applying various technical solution approaches.

[0003] The first approach is based on a classical gradient method. In the gradient method, gradients are extracted from an image of a road recorded by a sensor and / or camera in a vehicle. From this, the lane dividing lines of the lane in which the vehicle is located are determined. Here, the inner edge and the outer edge of the lane dividing line are determined and combined, and the width of the lane dividing line is calculated therefrom.

[0004] The second approach uses a segment-based method based on a deep learning scenario. Similarly in this approach, the outer edge and the inner edge of the lane dividing line of the lane are determined by estimating the individual segment masking of the lane dividing line. However, the accuracy of this method substantially depends on the resolution of the segment masking used.

[0005] A further approach for determining the lane dividing lines of a lane is pursued by a so-called anchor-based or anchorless approach. Here, the aim is to model the center of the lane dividing line to be detected using a direct, that is, a direct representation of the line of the lane dividing line to be detected. This approach enables higher accuracy in the accurate determination or identification of the position of the lane dividing line. However, in this approach, direct inferences regarding the inner edge and the outer edge of the lane dividing line to be detected cannot be derived.

Summary of the Invention

[0006] According to a first aspect, the Disclosure relates to a method for identifying lane markings of a first lane relating to a vehicle, the method comprising the following steps: In the first step, measurement data is provided from monitoring the area around the vehicle.

[0007] In the second step, measurement data is supplied to at least one machine learning model. In the third step, the lane markings' paths are evaluated using at least one machine learning model.

[0008] In the fourth step, the width of the lane markings is evaluated using at least one machine learning model. Therefore, the present invention offers the advantage that not only is the center of the lane markings to be detected for a vehicle's lane modeled, but the width of the lane markings to be detected for a vehicle's lane is also always detected or estimated. This is done by adding or solving a corresponding regression problem.

[0009] The proposed solution approach of the present invention still identifies the center of the lane markings to be detected, thus achieving the advantage of higher accuracy in determining lane markings compared to known approaches.

[0010] A further advantage is that the approach of the present invention can represent or identify all lane markings. Furthermore, a further advantage of the solution according to the present invention is that the regression problem concerning the width of the lane markings to be detected can be solved much more easily than the continuous estimation of the inner and outer edges of the lane markings. This is because the width of the lane markings to be determined can be determined relative to the center line or reference line of the lane, thereby making the method according to the present invention independent of the position or orientation of the lane markings in the recorded image.

[0011] One possible form of this method is to have the path represent the centerlines of the lane markings in a discrete or continuous manner. This offers the advantage of greater accuracy in determining the lane markings.

[0012] One possible form of this method involves determining the width of lane markings through regression. This achieves the advantage of efficiently determining the width of lane markings.

[0013] One possible form of this method is to select the sub-region of the lane marking closest to the vehicle for regression. This achieves the advantage of efficiently determining the width of the lane marking.

[0014] One possible form of this method involves performing regressions at multiple locations along the lane markings, aggregating the determined widths, and obtaining a final result regarding the lane marking width. This further reduces the error in determining the lane marking width.

[0015] One possible form of this method is intended to show the path in the form of the distance from a reference line through a monitored area around the vehicle. This achieves the advantage of efficiently and accurately determining the width of the lane markings.

[0016] One possible form of this method is intended to involve selecting image data and / or video data as measurement data. These are the most important measurement modalities for detecting lane markings.

[0017] One possible form of this method is intended to include the following steps for evaluating the width of the lane markings in the first lane: In the first step, a position is selected along the path of the lane markings.

[0018] In the second step, a search is performed for two boundary points between the lane marking line (one of which is the lane marking line) and the lane surface (the other of which is the lane marking line) in a predetermined search direction relative to the path of the lane marking line. In the third step, the width of the lane markings is determined from the distance between the two boundary points.

[0019] One possible form of this method involves taking into account camera calibration data and / or information regarding the road surface conditions of the first lane when determining the width of the lane markings of the first lane. This can further improve accuracy.

[0020] According to a second aspect, the disclosure relates to a method for training at least one machine learning model for use in the method described above, the method comprising the following steps:

[0021] In the first step, training examples are provided of measurement data recorded from the vehicle's perspective that indicate the presence of one or more lane markings. In the second step, a target path and target width are provided for at least one lane marking that separates the lane in which the vehicle is currently traveling.

[0022] In the third step, training examples are provided to the machine learning model to be trained, and this machine learning model identifies the path and width of lane markings using the method described above.

[0023] In the fourth step, the deviation between this path and its width (as one) and the target path and its width (as the other) is evaluated using a predetermined cost function. In the fifth step, the parameters characterizing the behavior of the machine learning model are optimized, with the expectation that the evaluation using the cost function will improve in further processing of the training examples.

[0024] The width of the lane dividing line that demarcates the lane in which the host vehicle is currently traveling is the target width that can be most accurately identified with respect to labeling because this lane dividing line is the closest to the host vehicle. Therefore, it is advantageous to use only this target width. As a tendency, naturally, it is better that more labeled training examples are available. However, for example, when an inaccurately identified target width is labeled for a lane dividing line that is further away, these labels containing noise ("noisy label") may have an adverse effect on the success of training.

[0025] One possible form of this method contemplates that at least one target path is provided and a path is determined with respect to at least one additional lane dividing line that does not demarcate the lane in which the host vehicle is currently traveling, and a cost function also evaluates the deviation between this path and the target path. Thereby, the advantage is realized that this method can be efficiently used or diverted also for determining an additional lane dividing line that is not on the currently traveled lane.

[0026] One possible form of this method contemplates that at least one lane dividing line apparent from measurement data is placed in a state where its contribution to the cost function is set to zero and thus is not taken into consideration. Thereby, this lane dividing line can be excluded from the measurement data of the training examples more efficiently than by exclusion based on the separation distance.

[0027] According to a third aspect, the present disclosure relates to a computer program including machine-readable instructions that cause one or more computers or computer instances to execute the method according to the present invention when executed on one or more computers and / or computer instances.

[0028] According to a fourth aspect, the present disclosure relates to a machine-readable data carrier and / or a download product comprising a computer program. According to a fifth aspect, the disclosure relates to one or more computers and / or computer instances comprising computer programs and / or machine-readable data carriers and / or downloadable products.

[0029] Further means of improving the present invention will be described in more detail below with reference to the drawings, along with a description of preferred exemplary embodiments of the present invention. [Brief explanation of the drawing]

[0030] [Figure 1] This is a schematic flowchart of a method for identifying the lane markings of the first lane for a vehicle. [Figure 2] This is a schematic flowchart of the method for training at least one machine learning model for use in the method shown in Figure 1. [Figure 3] This is an illustrative diagram of a method known from the prior art for determining the center of lane markings in a vehicle lane. [Figure 4] This is an illustrative diagram of a method for determining the width of lane markings related to vehicle lanes according to the present invention. [Figure 5] This is an illustrative diagram providing target positions and target widths for lane markings related to vehicle lanes. [Modes for carrying out the invention]

[0031] Figure 1, with reference to Figure 3, shows a schematic flowchart of a method 100 for identifying the lane markings 2 of the first lane 1 with respect to a vehicle 10, and this method 100 includes the following steps. In the first step 102, measurement data is provided from monitoring the surroundings of the vehicle 10.

[0032] In the second step 104, measurement data is supplied to at least one machine learning model. In the third step, 106, the lane markings' paths are evaluated using at least one machine learning model.

[0033] In the fourth step, step 108, the width 3 of the lane markings 2 is evaluated using at least one machine learning model. The evaluation of the width 3 of the lane marking line 2 of the first lane 1 in the fourth step 108 is preferably carried out in the following steps.

[0034] In the first step 120, a position along the path of lane marking 2 in lane 1 is selected. In the second step 122, two boundary points (see line segment 9 in Figure 4) between the lane marking line 2 as one and the lane surface as the other are searched in a predetermined search direction with respect to the path of the lane marking line 2.

[0035] In the third step 124, the width 3 of lane 1, i.e., the lane marking line 2 of the first lane 1, is determined from the distance between the two boundary points. Figure 2 shows a schematic flowchart of Method 200 for training at least one machine learning model for use in Method 100 according to Figure 1, and Method 200 includes the following steps:

[0036] In the first step 202, a training example of measurement data recorded from the perspective of the vehicle 10, indicating the presence of one or more lane markings, is provided. In the second step 204, a target path and target width are provided for at least one lane marking that separates the lane in which the vehicle 10 is currently traveling.

[0037] In the third step 206, training examples are supplied to the machine learning model to be trained, and this machine learning model identifies the path and width 3 of the lane markings 2 using the method according to any one of claims 1 to 9.

[0038] In the fourth step 208, the deviation between this path and its width 3 (as one) and the target path and target width (as the other) is evaluated using a predetermined cost function. In the fifth step, step 210, the parameters characterizing the behavior of the machine learning model are optimized, anticipating that the evaluation using the cost function will improve in further processing of the training examples.

[0039] Figure 3 shows an illustrative image of an image recorded by (vehicle) 10 for applying a method known from the prior art to determine the center of the lane marking line 2 with respect to lane 1 of vehicle 10 using an anchor-based approach. Between the start point 6 and end point 7 of the reference line 5, the individual distances of individual line sections or individual line segments to each center 4 of the lane marking line 2 of lane 1 are determined horizontally for each point using regression.

[0040] Figure 4 shows an example of how the method 100 proposed herein for determining the width 3 of lane dividers 2 for a vehicle 10 lane 1 can be implemented on the same recorded image. Briefly, in this method, the width 3 of each lane divider 2 of lane 1 is determined horizontally, starting from a reference line 5 extending between a start point 6 and an end point 7, according to the following steps of method 100:

[0041] In the first step 102, measurement data is provided from monitoring the surroundings of the vehicle 10, here in the form of recorded images. Generally, the measurement data may be available as image data and / or video data from corresponding sensors or other technical detection devices (not shown) of the vehicle 10.

[0042] In the second step 104, the measurement data is supplied to at least one machine learning model. Here, multiple models may be used; for example, a first model may be used to determine the path of the lane markings 2 of lane 1, and a second model may be used to determine the width 3 of the lane markings 2 of lane 1.

[0043] In the third step 106, the path of the lane marking line 2 is evaluated using at least one machine learning model. The path may be, for example, the center line 4 or the edges. In particular, the path of the center line 4 may be represented, for example, in the form of a number of discrete locations. Here, the path of the center line 4 may be represented in a continuous form. Preferably, the path of the center line 4 may be represented in the form of the spacing between individual line segments 9 of the lane marking line 2 to the reference line 5 passing through a monitored area around the vehicle 10, as shown in Figure 4.

[0044] In the fourth step 108, the width 3 of the lane marking 2 is evaluated by at least one machine learning model. Preferably, the width 3 of the lane marking 2 for lane 1 is determined by regression. Here, so-called scalar regression can be used in particular. This is because scalar regression can be performed more quickly and provides more accurate results, as the lane marking 2 appears larger at a given pixel resolution of the vehicle camera. For determining the width 3 of the lane marking 2 for the first lane 1, camera calibration data and / or information on the road surface conditions of the first lane 1 may also be taken into consideration, preferably.

[0045] Here, the regression may preferably be performed at multiple locations along the lane marking line 2, and the determined widths are aggregated to obtain a final result regarding the width 3 of the lane marking line 2. Here, appropriate weighting can be applied according to the distance to the vehicle 10 in order to take into account different accuracies.

[0046] The evaluation of the width 3 of the lane marking line 2 of the first lane 1 in the fourth step 108 is preferably carried out in the following steps. In the first step 120, a position along the path of lane marking 2 in lane 1 is selected.

[0047] In the second step 122, two boundary points (see line segment 9 in Figure 4) between the lane marking line 2 as one and the lane surface as the other are searched in a predetermined search direction with respect to the path of the lane marking line 2.

[0048] In the third step 124, the width 3 of lane 1, i.e., the lane marking line 2 of the first lane 1, is determined from the distance between the two boundary points. Figure 5 illustrates an example of how to obtain the target position and target width of the lane divider 2 for vehicle 10 in lane 1. Here, Figure 5 shows how the center 4 (dashed line) of the lane divider 1 and the inner edge 11 of the lane divider 2 are represented. Using these representations, the target width of the lane divider separating the lane in which vehicle 10 is currently traveling can be determined. This target width can be used to label training examples for a machine learning model.

Claims

1. A method (100) for identifying the lane markings (2) of a first lane (1) in which a vehicle (10) is traveling, Step (102) provides measurement data which is acquired image data and / or video data by imaging the area around the vehicle (10) using a sensor mounted on the vehicle (10), The steps include (104) supplying the measurement data to at least one machine learning model, Step (106) of evaluating the path of the lane markings (2) using at least one machine learning model, Step (108) of evaluating the width (3) of the lane markings (2) using at least one machine learning model, A method (100) including the following.

2. The method according to claim 1 (100), wherein the path indicates the center line (4) of the lane division line (2) in a discrete or continuous form.

3. The method according to claim 1 (100), wherein the width (3) of the lane divider (2) is determined by regression.

4. The method according to claim 3 (100), wherein for the regression, a portion of the lane marking line (2) closest to the vehicle (10) is selected.

5. The method according to claim 3 (100), wherein the regression is performed at multiple locations along the lane marking line (2), the identified widths are aggregated to obtain a final result relating to the width (3) of the lane marking line (2).

6. The method according to claim 1 (100), wherein the path is expressed in the form of a distance from the vehicle (10) to a reference line (5) passing through the imaged area around the vehicle (10).

7. The method according to claim 1 (100), wherein image data and / or video data are selected as measurement data.

8. The step (108) of evaluating the width (3) of the lane divider (2) of the first lane (1) is, The steps include selecting the position of the lane marking line (2) along the aforementioned path (120), Step (122) of searching for two boundary points between the lane marking line (2) as one and the lane surface as the other, in a predetermined search direction with respect to the path of the lane marking line (2), The step (124) of determining the width (3) of the lane divider (2) from the distance between the two boundary points, The method according to claim 1 (100), including the method described in claim 1.

9. The method according to claim 1 (100), wherein when the width (3) of the lane marking line (2) of the first lane (1) is determined, camera calibration data and / or information regarding the road surface conditions of the first lane (1) are taken into consideration.

10. A method (200) for training at least one machine learning model for use in the method according to any one of claims 1 to 9, Step (202) provides a training example of measurement data indicating the presence of one or more lane markings, recorded from the viewpoint of the vehicle (10), Step (204) provides a target path and target width with respect to at least one lane divider that separates the lane in which the vehicle (10) is currently traveling, Step (206) is to supply training examples to the machine learning model to be trained, wherein the machine learning model identifies the path and width (3) of the lane markings (2) using the method according to any one of claims 1 to 9, A step (208) to evaluate the deviation between the path and width (3) as one and the target path and target width as the other, using a predetermined cost function, Step (210) of optimizing the parameters that characterize the behavior of the machine learning model, anticipating that the evaluation by the cost function will improve in further processing of the training examples, A method including (200).

11. With respect to at least one further lane marking that does not demarcate the lane (1) in which the vehicle (10) is currently traveling, at least one target path is provided and the path is determined. The cost function also evaluates the deviation between the path and the target path. The method according to claim 10 (200).

12. The method according to claim 10, wherein at least one lane marking (2) that is evident from the measurement data is left unconsidered by setting its contribution to the cost function to zero.

13. A computer program that, when executed on one or more computers and / or computer instances, includes machine-readable instructions causing one or more computers or computer instances to perform the method according to any one of claims 1 to 9.

14. A computer program that includes machine-readable instructions, when executed on one or more computers and / or computer instances, causing one or more computers or computer instances to perform the method according to claim 10.

15. A computer-readable recording medium for storing the computer program described in claim 13.

16. A computer-readable recording medium for storing the computer program described in claim 14.

17. One or more computers and / or computer instances that perform the method according to any one of claims 1 to 9.

18. One or more computers and / or computer instances that perform the method according to claim 10.

Images