A curved lane line detection method based on a bow height feature point

Abstract

The application relates to a curved lane line detection method based on bow height feature points, and belongs to the image processing technical field of deep learning. First, feature points are obtained by sampling lane lines in an image, and the starting point and the ending point in the feature points of each lane line are determined, then the bow height feature points in the feature points are solved according to the position information of the starting point and the ending point, the first bow height feature point is obtained by using a point distance criterion, and the first bow height feature point is taken as a main feature point, all the bow height feature points are obtained by recursively calculating between the main feature point and the starting point and between the main feature point and the ending point, and the bow height feature points are taken as key points. The key point information is taken as prior features and is input into a lane line detection network, and the key points are further fitted through curve feature extraction to obtain a predicted lane line shape. The application improves the curved lane line detection effect, optimizes the problem of insufficient bending degree in lane line detection for non-linear structure lane lines such as large curves, and makes the lane line detection in a curved environment more robust.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing based on deep learning, and particularly relates to a curved lane line detection method based on bow height feature points. BACKGROUND

[0002] With the rapid development of global automobile industry, it has become one of the largest industries in the world, and the lane line detection problem is a key driving factor for fully autonomous driving. The current lane line detection algorithm can clearly detect accurate lane lines under the condition of good road conditions, clear lane line markings and single straight lane lines. However, in actual road conditions, there are a large number of curved lanes and other non-straight shapes on the actual road, which greatly increases the difficulty of detection, making the robustness of the system worse, and the accuracy also decreases. Lane line detection systems often face many difficulties in practical applications, and with the evolution of urban transportation systems, road information has become increasingly complex, to some extent, increasing the challenge task currently faced by the vehicle assisted driving system. Therefore, in order to improve the effectiveness of the algorithm, the large curve lane line structure must be considered.

[0003] In the problem of lane line detection, how to effectively solve the problem of poor detection effect of curved lane lines in actual road conditions is one of the key points. At present, the main means to solve the large curve is to improve the parameter regression curve or the curve feature extraction scheme. In 2019, SAD(Self Attention Distillation) was proposed, which further characterizes learning by performing self-top-down and hierarchical attention distillation network in the network itself, while maintaining real-time efficiency and achieving high performance; In 2020, Wang B et al. proposed a three-branch network to regress the polynomial coefficients of each lane line, which decomposes lane detection into a main subtask and two auxiliary subtasks, namely polynomial regression, initialization classification and height regression; In 2021, Chuan Lv et al. proposed a new type of lane line key point detection scheme, which linearly samples the key points of the lane line at different intervals on the image vertical axis to solve the problem of data imbalance between lane lines, and then fixes the anchor points, and the model only needs to predict the horizontal coordinates of each lane line at the anchor points; In the same year, LSTR considers the road structure together with the camera pose, models the lane line shape, and then introduces the Transformer into the lane line detection task to obtain the global feature; In 2022, Shanghai Jiaotong University and Shangtang Technology jointly proposed a new type of deep lane detector based on Bezier curve, which can effectively simulate the geometric shape of the lane line and has natural robustness to harsh driving conditions.

[0004] In the current algorithm engineering of lane line detection, no good targeted solution is proposed for the problem of large curve except for parameter regression curve fitting, and the parameter-based method is sensitive to the predicted parameters, for example, the wrong prediction of high-order coefficients may lead to changes in lane line shape. Although the parameter-based method has fast reasoning speed, it is still difficult to achieve higher performance. Therefore, how to effectively solve the difficult problem of large curve lane line detection has become a problem to be solved in the current lane line detection technology. SUMMARY

[0005] The present application provides a curved lane line detection method based on the sagitta feature point, which uses the sagitta feature point (a point representing the curve feature) to guide the lane line shape prior information and fits the curve feature to solve the difficult problem of large curve lane line detection. Thus, the curved lane line detection effect is improved, the problem of insufficient curvature for large curve lane line detection and the like non-straight structure lane line detection is optimized, and the lane line detection scene is more robust when facing curved environments.

[0006] The technical solution of the present application is: a curved lane line detection method based on the sagitta feature point, which introduces the mathematical concept of the sagitta feature point into the key point-based lane line detection task. For the key points fitting the lane line shape, the sagitta feature point is used instead of the uniformly sampled points, so that the more curved part needs more key points to be fitted. The sagitta feature point is used as the main feature point to better fit the shape of the curved lane line. The specific steps are as follows:

[0007] Step 1: uniformly interval sampling is performed on the lane line data set to obtain a plurality of feature points;

[0008] Step 2: determining the starting point and the ending point of the feature points in each lane line according to the feature points obtained in Step 1;

[0009] Step 3: the feature point farthest from the line connecting the starting point and the ending point in Step 2 is taken as the first sagitta feature point, and the sagitta feature point is taken as the main feature point;

[0010] Step 4: recursively calculating between the main feature point and the starting point, and between the main feature point and the ending point according to the method of Step 3, and sequentially obtaining all the sagitta feature points between the starting point and the ending point, and taking all the sagitta feature points as the key points;

[0011] Step 5: inputting the key point information obtained in Step 4 as prior features into the lane line detection network, and further fitting the key points through curve feature extraction to obtain the predicted lane line shape.

[0012] The sampling refers to finding the points on the lane line uniformly in the vertical direction for each lane line data set, specifically:

[0013]

[0014]

[0015] wherein, represents the number of feature points of each lane line, represents the interval number of feature points; represents the width of the image in the lane line data set, represents the vertical distance from each feature point to the adjacent feature point.

[0016] The starting point refers to the feature point with the smallest y coordinate value in each lane line feature point; the ending point refers to the feature point with the largest y coordinate value in each lane line feature point; specifically, the starting point is denoted as , and the ending point is denoted as .

[0017] The first camber feature point is obtained by using the point distance criterion, and the distances d of other feature points between the starting point and the ending point to the line segment are calculated in turn, if the distance of the feature point to the line segment is the largest, the feature point is the first camber feature point, and it is taken as the main feature point; specifically:

[0018]

[0019]

[0020]

[0021]

[0022] wherein, x represents the horizontal coordinate of the feature point, y represents the vertical coordinate of the feature point, A, B, and C respectively represent three coefficients of the general equation of the straight line composed of the starting point and the ending point ; represents the distance of other feature points between the starting point and the ending point to the line segment .

[0023] The recursive calculation refers to the first camber feature point obtained is respectively combined with the starting point and the ending point again.Two new arch height feature points are solved as end points, and the obtained arch height feature points are respectively solved as end points to obtain new arch height feature points.

[0024] The prior feature is specifically represented as:

[0025]

[0026] In the formula, P is a sequence of all key points obtained by Step 4, and N represents the total number of key points. The predefined lane prior can help the network better locate the lane line.

[0027] The beneficial effects of the present application are:

[0028] The present application guides the lane line shape prior information by finding the arch height feature point (a point expressing the curve feature), that is, for the key points fitting the lane line shape, the arch height feature point is used to replace the uniformly sampled point, so that the more curved part needs more key points to be fitted, and the arch height feature point is used as the main feature point to better fit the shape of the curved lane line, so that the curved lane line detection effect is better, and the lane line detection is more robust to curved environments. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is the flowchart of the present application. Specific implementation method

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] Embodiment 1

[0032] In order to achieve the purpose that the more curved part of the lane line can be fitted with more key points, the embodiment of the present application is based on an anchor point-based lane line detection algorithm, and guides the lane line shape prior information by finding the arch height feature point (a point expressing the curve feature), that is, for the key points fitting the lane line shape, the arch height feature point is used to replace the uniformly sampled point, so that the more curved part needs more key points to be fitted, and the arch height feature point is used as the main feature point to better fit the shape of the curved lane line. The specific process is as shown in Figure 1 .

[0033] Step 1: Sample the lane line dataset at uniform intervals to obtain several feature points. The sampling refers to densely and uniformly finding points on the lane lines in the vertical direction for each lane line dataset. Specifically:

[0034]

[0035]

[0036] In the formula, This indicates the number of feature points for each lane line. Indicates the number of intervals between feature points; Indicates the width of the image. This represents the vertical distance from each feature point to its adjacent feature points. In this embodiment, If set to 720, then The original dataset image size was 1640*590, and its size was adjusted to 800*320. The result is 719. It is 320. = 9.

[0037] Step 2: Based on the feature points obtained in Step 1, determine the start and end points of each lane line sampling point. The start point is the sampling point with the smallest y-coordinate value among all lane line sampling points; the end point is the sampling point with the largest y-coordinate value among all lane line sampling points. Specifically, the start point is denoted as... The termination point is .

[0038] Step 3: Based on the position information of the starting and ending points obtained in Step 2, calculate the bow height feature points among the sampling points. Specifically, find the feature point with the longest line connecting the starting and ending points as the first bow height feature point and use it as the principal feature point. The bow height feature points refer to the main feature points obtained from the discrete sampling points using the point distance criterion, and the starting point is calculated sequentially. and the end point Other sampling points between line segments distance If sampling points Place The largest, then point The first feature point of the bow height is taken as the principal feature point; specifically:

[0039]

[0040]

[0041]

[0042]

[0043] wherein x represents the horizontal coordinate of the feature point, y represents the vertical coordinate of the current feature point, A, B, and C represent three coefficients of the general equation of the straight line formed by the start point and the end point ; represents the distance from the other feature point between the start point and the end point to the line segment .

[0044] Step4: According to the method of Step3, recursively calculate between the main feature point and the start point and between the main feature point and the end point, and sequentially obtain all the camber feature points between the start point and the end point, and take them as key points. The recursive calculation refers to that the obtained camber feature point is again solved with the start point and the end point as two end points to obtain two new camber feature points, and takes them as key points, that is, the camber feature points are obtained by using the point distance criterion in and two segments; continue to recursively calculate, that is, each camber feature point and its adjacent key point (camber feature point, start point or end point) are taken as two end points to solve a new camber feature point, until the number of key points reaches the required number of fitted curves. In this embodiment, the number of key points is 72.

[0045] Step5: The key point information obtained in Step4 is input into the lane line detection network as prior features, and the key points are further fitted by curve feature extraction to obtain the predicted lane line shape. The prior feature refers to that the lane line is thin and long, and has strong shape prior, so the pre-defined lane prior can help the network better locate the lane line. In ordinary object detection, the object is represented by a rectangular box. However, such a rectangular box is not suitable for representing the lane line. Therefore, the 2d key points obtained by Step4 are used as the lane representation with non-equidistant intervals. The lane prior information is specifically represented as:

[0046]

[0047] wherein P is the sequence of all key points obtained by Step4, and N represents the total number of key points. In this embodiment, N is 72.

[0048] ​The data set in the embodiment adopts the CULane data set, key points are selected by using the above method in generating the prior information function, and in the training process, the AdamW optimizer is used for parameter updating, wherein the weight attenuation factor is 0.6e-3. A total of 15 times of training is performed, the batch size is 24, and the total loss of model training is the weighted sum of each loss.

[0049] The application provides a curved lane line detection method based on a bow height feature point. The core of the method is to introduce the bow height feature point into the key point-based lane line detection task. For the key points for fitting the lane line shape, the bow height feature points are used to replace the uniformly sampled points, so that the more curved part needs more key points for fitting. The bow height feature points are used as main feature points to better fit the shape of the curved lane line. The curved lane line detection effect is improved, the problem of insufficient curvature in the curved lane line detection for non-linear structure lane lines such as large curves is optimized, and the lane line detection scene is more robust when facing the curved environment.

Claims

1. A method for detecting curved lane lines based on bow height feature points, characterized in that, Specifically, it includes the following steps: Suddenly: Step 1: Sample the lane line dataset at uniform intervals to obtain several feature points; Step 2: Determine the start and end points of each lane line feature point based on the feature points obtained in Step 1; Step 3: The feature point with the longest line connecting the starting point and the ending point in Step 2 is taken as the first bow height feature point and is also taken as the main feature point; Step 4: Following the method in Step 3, recursively calculate between the main feature point and the starting point, and between the main feature point and the ending point, to obtain all the bow height feature points between the starting point and the ending point, and use all the bow height feature points as key points. Step 5: Input the key point information obtained in Step 4 as prior features into the lane detection network, and further fit the key points through curve feature extraction to obtain the predicted lane shape. The recursive calculation mentioned in Step 4 refers to obtaining the first bow height feature point Then compare with the starting point and termination point Two new bow height feature points are obtained by using them as endpoints. The obtained bow height feature points are then used as endpoints with the adjacent feature points to obtain new bow height feature points. All bow height feature points are used as key points until the number of key points reaches the number required for fitting the curve. The prior features mentioned in Step 5 are specifically represented as follows: ； In the formula, P is the sequence of all keypoints obtained from Step 4, and N represents the total number of keypoints.

2. The method for detecting curved lane lines based on bow height feature points according to claim 1, characterized in that, The sampling mentioned in Step 1 refers to finding points on the lane lines uniformly in the vertical direction for each lane line dataset, specifically: ； ； In the formula, This indicates the number of feature points for each lane line. Indicates the number of intervals between feature points; This represents the width of the image in the lane line dataset. This represents the vertical distance from each feature point to its adjacent feature points.

3. The method for detecting curved lane lines based on bow height feature points according to claim 1, characterized in that, In Step 2, the starting point refers to the feature point with the smallest y-coordinate value among the feature points of each lane line; the ending point refers to the feature point with the largest y-coordinate value among the feature points of each lane line. Specifically, the starting point is denoted as... The termination point is .

4. The method for detecting curved lane lines based on bow height feature points according to claim 1, characterized in that, In Step 3, the first bow height feature point is obtained using the point distance criterion, and the starting points are calculated sequentially. and the end point Other feature points between line segments The distance d between feature points, if the feature points to line segment distance The largest feature point The first feature point of the bow height is taken as the principal feature point; specifically: ； ； ； ； In the formula, x represents the x-coordinate of the feature point, y represents the y-coordinate of the feature point, and A, B, and C represent the coordinates of the feature point from the starting point, respectively. and the end point The general form of the equation of the line formed The three coefficients; Indicates the starting point and the end point Other feature points between line segments The distance.

Images