Method for determining a lane marking of a lane for a vehicle
Patent Information
- Application Number
- EP2023750957
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-08-12
- Filing Date
- 2023-07-27
- Publication Date
- 2025-06-18
AI Technical Summary
Current methods for determining lane markings in autonomous driving, such as gradient-based, segment-based, and anchor-based approaches, face challenges in accurately detecting both the center and width of lane markings, with existing solutions often being position-dependent and lacking precision in edge detection.
A method involving machine learning models that provide measurement data from vehicle surroundings, where the center and width of lane markings are evaluated using regression problems, allowing for accurate representation and determination of lane markings independent of position, with embodiments including discrete or continuous course specification and regression at multiple points along the lane marking.
This approach enhances accuracy in lane marking detection by modeling the center and width, reducing errors and improving robustness across various positions and conditions, while allowing for efficient determination of lane markings using image and video data.
Smart Images
Figure 1.1
Abstract
Description
[0001] Description
[0002] Title:
[0003] Method for determining a lane marking of a lane for a vehicle
[0004] The present invention relates to a method for determining a lane marking for a vehicle. Furthermore, the present invention relates to a method for training at least one machine learning model.
[0005] State of the art
[0006] The detection of lane markings for a vehicle is an important aspect, especially in the field of autonomous driving. To date, the detection of lane markings has been pursued using various technical approaches.
[0007] A first approach is based on a classic gradient method. Gradients are extracted from an image of the roadway captured by sensors and / or cameras in a vehicle. From this, a lane marking is determined for the lane in which the vehicle is located. The inner and outer edges of the lane marking are determined and combined to calculate the width of the lane marking.
[0008] A second approach uses segment-based methods based on deep learning scenarios. This approach also determines the outer and inner edges of a lane marking using an estimation of individual segment masks of the lane marking. However, the accuracy of this method depends significantly on the resolution of the segment mask used. Another approach to determining the lane marking of a lane is pursued using so-called anchor-based or anchor-less approaches. The goal here is to use a direct or immediate representation of a line of the lane marking to be detected to model a center of the lane marking to be detected. This approach does enable greater accuracy in the exact determination of the position of the lane marking.However, this approach does not allow direct conclusions to be drawn about the inner and outer edges of the lane marking to be detected.
[0009] Disclosure of the invention
[0010] According to a first aspect, the disclosure relates to a method for determining a lane marking of a first lane for a vehicle, comprising the following steps:
[0011] The first step involves providing measurement data from monitoring the vehicle’s surroundings.
[0012] In a second step, the measurement data is fed into at least one machine learning model.
[0013] In a third step, the course of the lane marking is evaluated using at least one machine learning model.
[0014] In a fourth step, the width of the lane marking is evaluated using at least one machine learning model.
[0015] The present invention therefore offers the advantage that not only is the center of a lane marking to be detected in a vehicle's lane modeled, but also the width of the lane marking to be detected in the vehicle's lane is detected or estimated at any given time. This is achieved by adding or solving a corresponding regression problem. The approach pursued by the present invention achieves the advantage of higher accuracy in determining the lane marking than with known approaches, since the center of the lane marking to be detected in a lane is still determined.
[0016] A further advantage is that all lane markings can be represented or determined by the inventive approach.
[0017] A further advantage of the inventive solution is that solving a regression problem for the width of the lane marking to be detected is much easier than continuously estimating the inner and outer edges of a lane marking. This is because the width of the lane marking to be determined can be determined relative to a centerline or reference line of the lane, making the inventive method independent of the position or orientation of the lane marking in a captured image.
[0018] One possible embodiment of the method provides for the course to indicate a center line of the lane marking in a discrete or continuous form. This provides the advantage of greater accuracy in determining the lane marking.
[0019] One possible embodiment of the method involves determining the width of the lane markings using regression. This has the advantage of allowing the width of the lane markings to be determined efficiently.
[0020] One possible embodiment of the method involves selecting a portion of the lane marking closest to the vehicle for regression. This provides the advantage of efficiently determining the width of the lane marking.
[0021] One possible implementation of the method involves performing the regression at multiple locations along the lane marking, and aggregating the determined widths to produce a final result for the lane marking width. This further reduces the error in determining the lane marking width.
[0022] One possible embodiment of the method provides for the course to be specified in the form of distances to a reference line through the monitored area of the vehicle's surroundings. This has the advantage of allowing the width of the lane markings to be determined efficiently and with high accuracy.
[0023] One possible implementation of the method involves selecting image data and / or video data as measurement data. These are the most important measurement modalities for detecting lane markings.
[0024] A possible embodiment of the method provides that the step of evaluating the width of the lane marking of the first lane comprises the following steps:
[0025] The first step is to select a position along the lane marking.
[0026] In a second step, the search is carried out in a predetermined search direction relative to the course of the lane marking for two boundary points between the lane marking on the one hand and the lane surface on the other.
[0027] In a third step, the width of the lane marking is determined from a distance between the two boundary points.
[0028] One possible embodiment of the method provides that camera calibration data and / or information on the road surface of the first lane are taken into account when determining the width of the lane markings of the first lane. Accuracy can thereby be further increased. According to a second aspect, the disclosure relates to a method for training at least one machine learning model for use in the method described above, comprising the following steps:
[0029] In a first step, training examples of measurement data recorded from the perspective of an ego vehicle are provided, indicating the presence of one or more lane markings.
[0030] In a second step, a target course and a target width are provided for at least one lane marking that limits the lane currently being used by the ego vehicle
[0031] In a third step, the training examples are fed to the machine learning model to be trained, so that this machine learning model determines a course and a width of the lane marking with the previous one.
[0032] In a fourth step, a deviation between this course and this width on the one hand, and the target course or the target width on the other hand, is evaluated using a given cost function.
[0033] In a fifth step, parameters that characterize the behavior of the machine learning model are optimized with the aim of improving the evaluation by the cost function with further processing of training examples.
[0034] The width of a lane marking that borders the lane currently occupied by the ego vehicle is the most accurately determinable target width within the label because this lane marking is closest to the ego vehicle. Therefore, it is advantageous to use only this target width.
[0035] Of course, it is always better to have more labeled training examples available. However, if, for example, a more distant lane marking is labeled with an only imprecisely determined target width, these noisy labels (C "noisy labels") can have a negative impact on the success of the training. One possible embodiment of the method provides that for at least one other lane marking that does not border the lane currently being traveled by the ego vehicle, at least one target path is provided and a path is determined, and the cost function also evaluates the deviation of this path from its target path. This has the advantage that the method can be efficiently used or transferred to determine other lane markings that are not located in the currently traveled lane.
[0036] One possible implementation of the method involves ignoring at least one lane marking evident in the measurement data by setting its contribution to the cost function to zero. This allows this lane marking to be masked out more efficiently than removing it from the measurement data of the training example.
[0037] According to a third aspect, the disclosure relates to a computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer or computer instances to carry out the method according to the invention.
[0038] According to a fourth aspect, the disclosure relates to a machine-readable data carrier and / or download product comprising the computer program.
[0039] According to a fifth aspect, the disclosure relates to one or more computers and / or computer instances with the computer program and / or with the machine-readable data carrier and / or download product.
[0040] Further measures improving the invention are presented in more detail below together with the description of the preferred embodiments of the invention with reference to figures.
[0041] Examples of implementation It shows:
[0042] Figure 1 Schematic flow diagram of the method for determining a lane marking of a first lane for a vehicle;
[0043] Figure 2 Schematic flow diagram of the method for training at least one machine learning model for use in the method according to Figure 1;
[0044] Figure 3 Exemplary representation of a method known from the prior art for determining a center of a lane marking for a lane of a vehicle;
[0045] Figure 4 shows an exemplary representation of a method according to the invention for determining a width of a lane marking for a lane of a vehicle; and
[0046] Figure 5 Example of providing a target position and a target width of a lane marking for a lane of a vehicle.
[0047] Figure 1 with reference to Figure 3 shows a schematic flow diagram of the method 100 for determining a lane marking 2 of a first lane 1 for a vehicle 10 with the following steps:
[0048] In a first step 102, measurement data from monitoring the surroundings of the vehicle 10 are provided.
[0049] In a second step 104, the measurement data is fed to at least one machine learning model.
[0050] In a third step 106, the path of the lane marking is evaluated using the at least one machine learning model. In a fourth step 108, the width 3 of the lane marking 2 is evaluated using the at least one machine learning model.
[0051] The evaluation of the width 3 of the lane marking 2 of the first lane 1 in the fourth step 108 is preferably carried out with the following steps:
[0052] In a first step 120, a position along the course of the lane marking 2 of the lane 1 is selected.
[0053] In a second step 122, a search is carried out in a predetermined search direction relative to the course of the lane marking 2 for two boundary points - see line segment 9 in Figure 4 - between the lane marking 2 on the one hand and the lane surface on the other hand.
[0054] In a third step 124, the width 3 of the lane marking 2 of lane 1 or the first lane 1 is determined from a distance between the two boundary points.
[0055] Figure 2 shows a schematic flow diagram of the method 200 for training at least one machine learning model for use in the method 100 according to Figure 1 with the following steps:
[0056] In a first step 202, training examples of measurement data recorded from the perspective of an ego vehicle 10 and indicating the presence of one or more lane markings are provided.
[0057] In a second step 204, a target course and a target width are provided for at least one lane marking that delimits the lane currently traveled by the ego vehicle 10.
[0058] In a third step 206, the training examples are fed to the machine learning model to be trained, so that this machine learning model determines a course and a width 3 of the lane marking 2 using the method according to one of claims 1 to 9.
[0059] In a fourth step 208, a deviation between this course and this width 3 on the one hand, and the target course or the target width on the other hand, is evaluated using a predetermined cost function.
[0060] In a fifth step 210, parameters that characterize the behavior of the machine learning model are optimized with the aim of improving the evaluation by the cost function with further processing of training examples.
[0061] Figure 3 shows an exemplary representation of an image captured by an (ego) vehicle 10 for applying a method known from the prior art for determining a center of a lane marking 2 for a lane 1 of the vehicle 10 using an anchor-based approach. Between the starting point 6 and the end point 7 of a reference line 5, the individual distances of the individual line sections or the individual line segments 8 to the respective center 4 of the lane marking 2 of the lane 1 are determined for each point in the horizontal direction using regression.
[0062] Figure 4 shows, by way of example, how the method 100 proposed here for determining a width 3 of a lane marking 2 for a lane 1 of a vehicle 10 can be carried out on the same recorded image. In this method, described in simple terms, starting from the reference line 5, which runs between a starting point 6 and an end point 7, the respective width 3 of the lane marking 2 of the lane 1 is determined in the horizontal direction according to the following steps of the method 100 as follows:
[0063] In a first step 102, measurement data from monitoring the surroundings of the vehicle 10 is provided, here in the form of the recorded image. The measurement data can generally be made available as image data and / or video data from corresponding sensors or other technical detection devices (not shown) of the vehicle 10. In a second step 104, the measurement data are fed to at least one machine learning model. In this case, multiple models can also be used; for example, it can be provided to use a first model for determining the course of the lane marking 2 of the lane 1 and a second model for determining the width 3 of the lane marking 2 of the lane 1.
[0064] In a third step 106, the course of the lane marking 2 is evaluated using the at least one machine learning model. The course can be, for example, a center line 4, but also an edge. The course of the center line 4 can in particular be specified, for example, in the form of many discrete positions. The course of the center line 4 can also be specified in a continuous form. The course of the center line 4 can preferably - and as shown in Figure 4 - be specified in the form of distances of the individual line segments 9 of the lane marking 2 to the reference line 5 through the monitored area of the surroundings of the vehicle 10.
[0065] In a fourth step 108, a width 3 of the lane marking 2 is evaluated using the at least one machine learning model. Preferably, the width 3 of the lane marking 2 of the lane 1 is determined by regression. In particular, a so-called scalar regression can be used here, as this can be performed at a higher speed and delivers more precise results, since the lane marking 2 appears larger for a given pixel resolution of a vehicle camera. Camera calibration data and / or information on the road surface of the first lane 1 can preferably also be taken into account to determine the width 3 of the lane marking 2 of the first lane 1.
[0066] The regression can preferably be performed at several locations along the lane marking 2, with the determined widths being aggregated to a final result for the width 3 of the lane marking 2. A corresponding weighting can be applied with the distance to the vehicle 10 to account for a different degree of accuracy. The evaluation of the width 3 of the lane marking 2 of the first lane 1 in the fourth step 108 is preferably carried out with the following steps:
[0067] In a first step 120, a position along the course of the lane marking 2 of the lane 1 is selected.
[0068] In a second step 122, a search is carried out in a predetermined search direction relative to the course of the lane marking 2 for two boundary points - see line segment 9 in Figure 4 - between the lane marking 2 on the one hand and the lane surface on the other hand.
[0069] In a third step 124, the width 3 of the lane marking 2 of lane 1 or the first lane 1 is determined from a distance between the two boundary points.
[0070] Figure 5 shows an example of how a target position and a target width of a lane marking 2 for a lane 1 of a vehicle 10 can be obtained. Figure 5 shows how the center 4 of the lane marking of lane 1 of the ego vehicle 1—dashed line—and the inner edge 11 of the lane marking 2 are marked accordingly. Using these markings, a target width of a lane marking can be determined that delimits the lane currently being traveled by the ego vehicle 10. This target width can be used to label training examples for the machine learning model.
Claims
Claims 1. Method (100) for determining a lane marking (2) of a first lane (1) for a vehicle (10) comprising the following steps: • Providing (102) measurement data from monitoring the surroundings of the vehicle (10); • feeding (104) the measurement data to at least one machine learning model; • Evaluating (106) a course of the lane marking (2) with the at least one machine learning model; • Evaluating (108) a width (3) of the lane marking (2) with the at least one machine learning model.
2. Method (100) according to claim 1, wherein the course indicates a center line (4) of the lane marking (2) in discrete or continuous form.
3. Method (100) according to one of claims 1 to 2, wherein the width (3) of the lane marking (2) is determined by regression.
4. The method (100) according to claim 3, wherein a partial area of the lane marking (2) closest to the vehicle (10) is selected for the regression.
5. The method (100) according to any one of claims 3 to 4, wherein the regression is performed at a plurality of locations along the lane marking (2) and the determined widths are aggregated to a final result for the width (3) of the lane marking (2).
6. Method (100) according to one of claims 1 to 5, wherein the course is specified in the form of distances to a reference line (5) through the monitored area of the surroundings of the vehicle (10).
7. Method (100) according to one of claims 1 to 6, wherein image data and / or video data are selected as measurement data.
8. Method (100) according to one of the preceding claims, wherein the step (108) of evaluating the width (3) of the lane marking (2) of the first lane (1) comprises the following steps: • Selecting (120) a position along the course of the Lane marking (2); • Searching (122) in a predetermined search direction relative to the course of the lane marking (2) for two boundary points between the lane marking (2) on the one hand and the lane surface on the other hand; and • Determining (124) the width (3) of the lane marking (2) from a distance between the two boundary points.
9. Method (100) according to one of the preceding claims, wherein camera calibration data and / or information on the road condition of the first lane (1) are taken into account in determining the width (3) of the lane marking (2) of the first lane (1).
10. A method (200) for training at least one machine learning model for use in the method according to one of claims 1 to 9, comprising the following steps: • Providing (202) training examples of measurement data recorded from the perspective of an ego vehicle (10) and indicating the presence of one or more lane markings; • Providing (204) a target course and a target width for at least one lane marking that delimits the lane currently traveled by the ego vehicle (10); • Feeding (206) the training examples to the machine to be trained Learning model, so that this machine learning model determines a course and a width (3) of the lane marking (2) using the method according to one of claims 1 to 9; • Evaluating (208) a deviation between this course and this width (3) on the one hand, and the target course or the target width on the other hand, with a given cost function; and • Optimize (210) parameters that affect the behavior of the machine Learning model, with the aim that with further Processing training examples is expected to improve the evaluation by the cost function.
11. Method (200) according to claim 10, wherein • for at least one additional road marking other than the one Ego vehicle (10) currently travelled lane (1) is limited, at least one target course is provided and a course is determined; and • the cost function also the deviation of this course from its Target course evaluated.
12. Method according to one of claims 10 to 11, wherein at least one lane marking (2) apparent from the measurement data is disregarded by setting its contribution to the cost function to zero.
13. A computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer or computer instances to carry out the method according to any one of claims 1 to 9 and / or the method according to any one of claims 10 to 12.
14. Machine-readable data carrier and / or download product with the computer program according to claim 13.
15. One or more computers and / or computer instances with the computer program according to claim 13, and / or with the machine-readable data carrier and / or download product according to claim 14.