Method for determining lane marking of a lane for a vehicle
By analyzing the direction and width of lane marks using machine learning models, the problem of insufficient detection accuracy of lane marks in the field of autonomous driving is solved, and a higher accuracy of lane mark detection is achieved.
Patent Information
- Application Number
- CN202380059399.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-08-12
- Filing Date
- 2023-07-27
- Publication Date
- 2025-03-25
AI Technical Summary
The prior art has insufficient accuracy in detecting lane markings in the field of autonomous driving, especially in determining the inner and outer edges of lane markings.
By providing measurement data from the vehicle's surroundings, it is transmitted to at least one machine learning model to analyze and evaluate the direction and width of lane markers. This method not only models the center of lane markers, but also detects and estimates the width of lane markers through regression problems.
It achieves higher accuracy in lane marking, can effectively determine the center and width of lane marking, simplifying the estimation of inner and outer edges in traditional methods, and improves accuracy and reliability.
Smart Images
Figure CN119698643A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for determining lane markings of a lane for a vehicle. The invention also relates to a method for training at least one machine learning model. Background Art
[0002] Detecting lane markings of a traffic lane for a vehicle is an important aspect, particularly in the field of automated driving. To date, lane markings of a traffic lane have been detected by using various technical solutions.
[0003] The first approach is based on the classic gradient method. Here, the gradient is extracted from an image of the lane captured by a sensor and / or camera in the vehicle. From this, the lane marking of the lane in which the vehicle is located is determined. Here, the inner and outer edges of the lane marking are determined and joined together in order to calculate the width of the lane marking from this.
[0004] The second approach uses a segmentation-based approach, which is based on a deep learning scenario. In this approach, the inner and outer edges of the lane markings are also determined, specifically by estimating the individual segment markings of the lane markings. However, the accuracy of this approach clearly depends on the resolution of the segment markings used.
[0005] Another approach for ascertaining lane markings of a lane is a so-called anchor-based or anchor-free approach. The goal here is to use a direct or indirect representation of the line of the lane marking to be detected in order to model the center of the lane marking to be detected. Although this approach allows for a higher degree of accuracy when precisely ascertaining or determining the position of the lane marking, it is not possible to directly infer the inner and outer edges of the lane marking to be detected in this approach. Summary of the invention
[0006] According to a first aspect, the invention relates to a method for determining lane markings of a first lane for a vehicle, comprising the following steps:
[0007] In a first step, measurement data from monitoring of the vehicle surroundings are provided.
[0008] In a second step, the measurement data are transmitted to at least one machine learning model.
[0009] In a third step, the course of the lane marking is evaluated using the at least one machine learning model.
[0010] In a fourth step, an evaluation of the width of the lane marking is performed using the at least one machine learning model.
[0011] The invention thus offers the advantage that not only the center of the lane marking to be detected of the vehicle's lane is modeled, but also the width of the lane marking to be detected of the vehicle's lane is detected or estimated at each time point in addition. This is done by adding or solving a corresponding regression problem.
[0012] The solution provided by the present invention achieves the advantage of a higher accuracy in ascertaining the lane markings compared to known solutions, since the center of the lane marking to be detected is also determined.
[0013] A further advantage is that all lane markings can be represented or determined using the approach according to the invention.
[0014] Another advantage of the solution according to the invention is that the solution to the regression problem for the width of the lane marking to be detected can be solved more easily than the conventional estimation of the inner and outer edges of the lane marking. Since the width to be determined of the lane marking can be determined relative to the center line or reference line of the lane, the method according to the invention is independent of the position or location of the lane marking in the acquired image.
[0015] One possible embodiment of the method provides that the course describes the center line of the lane marking in discrete or continuous form. This can achieve the advantage of greater accuracy in the ascertainment of the lane marking.
[0016] One possible embodiment of the method provides that the width of the lane markings is determined by a regression analysis. This achieves the advantage that the width of the lane markings can be determined efficiently.
[0017] One possible embodiment of the method provides that the subregion of the lane marking closest to the vehicle is selected for the regression analysis. This achieves the advantage that the width of the lane marking can be efficiently ascertained.
[0018] One possible embodiment of the method provides that a regression analysis is performed at a plurality of locations along the lane marking and the ascertained widths are aggregated to form a final result for the lane marking width. This further reduces errors in ascertaining the lane marking width.
[0019] One possible embodiment of the method provides that the course is given in the form of a distance relative to a reference line that passes through the monitored area of the vehicle's surroundings. This achieves the advantage that the width of the lane marking can be determined efficiently and with high precision.
[0020] One possible embodiment of the method provides that image data and / or video data are selected as measurement data. These are the most important measurement modalities for detecting lane markings.
[0021] One possible embodiment of the method provides that the step of evaluating the width of the lane markings of the first lane comprises the following steps:
[0022] In a first step, a position is selected along the course of the lane marking.
[0023] In a second step, a search is performed in a predefined search direction relative to the course of the lane marking for two boundary points between the lane marking on the one hand and the lane surface on the other hand.
[0024] In a third step, the width of the lane marking is ascertained from the distance between two boundary points.
[0025] One possible embodiment of the method provides that camera calibration data and / or information about the road properties of the first lane are taken into account when determining the width of the lane markings of the first lane. This can further increase the accuracy.
[0026] According to a second aspect, the invention relates to a method for training at least one machine learning model for use in the above method, having the following steps:
[0027] In a first step, training examples of measurement data are provided which were acquired from the perspective of the host vehicle and which show the presence of one or more lane markings.
[0028] In a second step, a desired course and a desired width are provided at least for lane markings which delimit the lane in which the host vehicle is currently traveling.
[0029] In a third step, the training examples are transmitted to the machine learning model to be trained, so that the machine learning model determines the course and width of the lane markings using the specified method.
[0030] In a fourth step, the deviations between the course and width on the one hand and the desired course or desired width on the other hand are evaluated using a predefined cost function.
[0031] In the fifth step, the parameters characterizing the behavior of the machine learning model are optimized with the goal of expected improvement of the evaluation by the cost function during further processing of training examples.
[0032] The width of the lane marking that delimits the lane in which the vehicle is currently traveling is the desired width that can be determined most accurately within the framework of the labeling, because this lane marking is closest to the vehicle. It is therefore advantageous to use only this desired width. Of course, as the trend develops, more labeled training examples are available. However, if, for example, lane markings that are far away and have a desired width that can only be determined imprecisely are labeled, these noise labels ("noise labels") have a negative impact on the success of the training.
[0033] One possible embodiment of the method provides that at least one desired course is provided and ascertained for at least one other lane marking that does not delimit the lane in which the host vehicle is currently traveling, and the cost function also evaluates deviations of this course from its desired course. This achieves the advantage that the method can also be used or transferred efficiently for ascertaining other lane markings that are not located in the lane in which the host vehicle is currently traveling.
[0034] One possible embodiment of the method provides that at least one lane marking that emerges from the measurement data is not taken into account by setting its contribution to the cost function to 0. In this way, such lane markings can be concealed more effectively than by deletion from the measurement data of the training example.
[0035] According to a third aspect, the invention relates to a computer program comprising machine-readable instructions which, when these instructions are executed on one or more computers and / or computer instances, cause the computers or computer instances to carry out the method according to the invention.
[0036] According to a fourth aspect, the invention relates to a machine-readable data carrier and / or a download product with a computer program.
[0037] According to a fifth aspect, the invention relates to one or more computers and / or computer instances with a computer program and / or with a machine-readable data carrier and / or a download product.
[0038] Other measures for improving the present invention will be described in detail below in conjunction with the description of preferred embodiments of the present invention based on the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings show:
[0040] Figure 1 : A schematic flow chart of a method for determining a lane marking of a first lane for a vehicle;
[0041] Figure 2 :Used for training according to Figure 1 A schematic flowchart of a method for using at least one machine learning model in a method;
[0042] Figure 3 : An exemplary description of a method known from the prior art for determining the center of a lane marking for a lane for a vehicle;
[0043] Figure 4 : An exemplary description of the method according to the invention for ascertaining the width of a lane marking of a lane for a vehicle; and
[0044] Figure 5 : Provides an exemplary description of the desired position and desired width of lane markings for vehicle lanes. DETAILED DESCRIPTION
[0045] Reference Figure 3 , Figure 1 A schematic flow chart of a method 100 for determining lane markings 2 of a first lane 1 for a vehicle 10 is shown, having the following steps:
[0046] In a first step 102 , measurement data from monitoring the surroundings of vehicle 10 are provided.
[0047] In a second step 104 , the measurement data are transmitted to at least one machine learning model.
[0048] In a third step 106 , the course of the lane marking is evaluated using the at least one machine learning model.
[0049] In a fourth step 108 , width 3 of lane marking 2 is evaluated using the at least one machine learning model.
[0050] The evaluation of width 3 of lane marking 2 of first lane 1 in fourth step 108 preferably comprises the following steps:
[0051] In a first step 120 , a position is selected along the course of lane marking 2 of lane 1 .
[0052] In a second step 122, a search is performed in a predetermined search direction relative to the course of the lane marking 2 for two boundary points between the lane marking 2 on the one hand and the lane surface on the other hand, see Figure 4 Line segment 9 in .
[0053] In a third step 124 , width 3 of lane 1 or of lane marking 2 of first lane 1 is ascertained from the distance between the two boundary points.
[0054] Figure 2 shows the training method for application in accordance with Figure 1 A schematic flow chart of a method 200 for learning at least one machine learning model in the method 100 has the following steps:
[0055] In a first step 202 , a training example of measurement data is provided which was acquired from the perspective of host vehicle 10 and which shows the presence of one or more lane markings.
[0056] In a second step 204 , a desired course and a desired width are provided at least for lane markings which delimit the lane in which host vehicle 10 is currently traveling.
[0057] In a third step 206 , training examples are transmitted to the machine learning model to be trained, so that the machine learning model determines the course and width 3 of lane marking 2 using the method according to any one of claims 1 to 9 .
[0058] In a fourth step 208 , the deviation between the course and width 3 on the one hand and the desired course or desired width on the other hand is evaluated using a predefined cost function.
[0059] In a fifth step 210 , the parameters characterizing the behavior of the machine learning model are optimized with the goal of expected improvement of the evaluation by the cost function during further processing of the training examples.
[0060] Figure 3 An exemplary description of an image captured by a vehicle 10 is shown for use of a method known from the prior art for ascertaining the center of a lane marking 2 of a lane 1 of the vehicle 10 using an anchor point-based approach. Between a starting point 6 and an end point 7 of a reference line 5, the respective distance of the respective line segment or line segment 8 relative to the corresponding center 4 of the lane marking 2 of the lane 1 is ascertained for each point in the horizontal direction by regression.
[0061] Figure 4 The following is an example of how the method 100 presented here for ascertaining the width 3 of the lane markings 2 of the lane 1 of the vehicle 10 can be performed on the same captured image. In this method, in simple terms, starting from a reference line 5 extending between a starting point 6 and an end point 7, the corresponding width 3 of the lane markings 2 of the lane 1 is ascertained in the horizontal direction according to the following steps of the method 100 as follows:
[0062] In a first step 102, measurement data from monitoring the surroundings of the vehicle 10 are provided, here in the form of captured images. The measurement data can generally be provided as image data and / or video data of corresponding sensors of the vehicle 10 or other technical detection devices (not described here).
[0063] In a second step 104 , the measurement data are transmitted to at least one machine learning model. In this case, multiple models can also be used, for example, a first model can be provided for determining the course of lane markings 2 of lane 1 and a second model can be provided for determining width 3 of lane markings 2 of lane 1 .
[0064] In a third step 106, the course of lane marking 2 is evaluated using the at least one machine learning model. For example, the course can be the center line 4, but also an edge. The course of center line 4 can be given, for example, in the form of a number of discrete positions. The course of center line 4 can also be given in a continuous form. Figure 4As shown, the course of center line 4 can be specified through a monitored area of the surroundings of vehicle 10 , preferably in the form of distances of individual line segments 9 of lane marking 2 relative to reference line 5 .
[0065] In a fourth step 108, the width 3 of the lane marking 2 is evaluated using the at least one machine learning model. The width 3 of the lane marking 2 of the lane 1 is preferably determined by regression. In particular, a so-called scalar regression can be used here, because it can be performed at a higher speed and provides more accurate results, because the lane marking 2 appears larger for a given pixel resolution of the vehicle camera. In order to determine the width 3 of the lane marking 2 of the lane 1, camera calibration data and / or information about the road properties of the first lane 1 can preferably be taken into account.
[0066] A regression analysis may preferably be performed at multiple locations along lane marking 2 , wherein the ascertained widths are aggregated to form a final result for width 3 of lane marking 2 . The distance from vehicle 10 may be weighted accordingly in order to take into account different degrees of accuracy.
[0067] The evaluation of width 3 of lane marking 2 of first lane 1 in fourth step 108 preferably comprises the following steps:
[0068] In a first step 120 , a position is selected along the course of lane marking 2 of lane 1 .
[0069] In a second step 122, a search is performed in a predetermined search direction relative to the course of the lane marking 2 for two boundary points between the lane marking 2 on the one hand and the lane surface on the other hand, see Figure 4 Line segment 9 in .
[0070] In a third step 124 , width 3 of lane 1 or of lane marking 2 of first lane 1 is ascertained from the distance between the two boundary points.
[0071] Figure 5 An example is shown of how the desired position and desired width of lane markings 2 of lane 1 of vehicle 10 can be determined. Figure 5 It is shown how the center 4 (dashed line) of the lane markings 2 of the lane 1 of the ego vehicle 1 and the inner edge 11 of the lane markings 2 are marked accordingly. These markings can be used to determine the desired width of the lane markings that delimit the lane currently traveled by the ego vehicle 10. This desired width can be used to label training examples for a machine learning model.
Claims
1. A method (100) for determining a lane marking (2) of a first lane (1) for a vehicle (10), comprising the following steps: providing (102) measurement data from monitoring of the vehicle (10)'s surroundings; · transmitting (104) the measurement data to at least one machine learning model; Analyzing and evaluating (106) the direction of the lane marking (2) using the at least one machine learning model; Analyzing (108) the width (3) of the lane marking (2) using the at least one machine learning model.
2. The method (100) according to claim 1, wherein: The course of the center line (4) of the lane marking (2) is given in discrete or continuous form.
3. The method (100) according to claim 1 or 2, wherein: The width (3) of the lane marking (2) is determined by regression analysis.
4. The method (100) according to claim 3, wherein: A sub-region of the lane marking (2) closest to the vehicle (10) is selected for regression analysis.
5. The method (100) according to claim 3 or 4, wherein: The regression analysis is performed at a plurality of locations along the lane marking (2), and the determined widths are aggregated to form a final result for the width (3) of the lane marking (2).
6. The method (100) according to any one of claims 1 to 5, wherein: The course is given in the form of a distance relative to a reference line (5) which passes through a monitored area of the surroundings of the vehicle (10).
7. The method (100) according to any one of claims 1 to 6, wherein: Image data and / or video data are selected as measurement data.
8. The method (100) according to any one of the preceding claims, wherein: The step of evaluating (108) the width (3) of the lane marking (2) of the first lane (1) comprises the following steps: · selecting (120) a position along the direction of the lane marking (2); Searching (122) in a predetermined search direction relative to the course of the lane marking (2) two boundary points between the lane marking (2) on the one hand and the lane surface on the other hand; and The width (3) of the lane marking (2) is determined (124) from the distance between the two boundary points.
9. The method (100) according to any one of the preceding claims, wherein: Camera calibration data and / or information about the road properties of the first lane (1) are taken into account when determining the width (3) of the lane marking (2) of the first lane (1).
10. A method (200) for training at least one machine learning model for use in a method according to any one of claims 1 to 9, comprising the following steps: Providing (202) training examples of measurement data, the measurement data being obtained from the host vehicle (10) The view captures and displays the presence of one or more lane markings; providing (204) a desired course and a desired width for at least the boundary from the lane markings of the lane in which the vehicle (10) is currently traveling; Transmitting (206) the training examples to a machine learning model to be trained so that the machine learning model determines the course and width (3) of the lane marking (2) using the method according to any one of claims 1 to 9; evaluating (208) the deviation between the course and width (3) on the one hand and the desired course or desired width on the other hand using a predetermined cost function; and Optimizing (210) parameters characterizing the behavior of the machine learning model with the goal of expected improvement of the evaluation by the cost function upon further processing of training examples.
11. The method (200) of claim 10, wherein: providing at least one desired course and ascertaining a course for at least one further lane marking which does not delimit the lane (1) in which the vehicle (10) is currently travelling; and The cost function also evaluates the deviation of the direction from what it should be.
12. The method according to claim 10 or 11, wherein: At least one lane marking (2) that emerges from the measurement data is not taken into account by setting its contribution to the cost function to zero.
13. A computer program comprising machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computers or computer instances to implement the method according to any one of claims 1 to 9 and / or the method according to any one of claims 10 to 12.
14. Machine-readable data carrier and / or download product having a computer program according to claim 13.
15. One or more computers and / or computer instances having a computer program according to claim 13 and / or having a machine-readable data carrier and / or a download product according to claim 14.