Lane line detection method based on Anchor
By adding column prediction heads and data enhancement technology to the Anchor-based lane line detection algorithm, the fitting difficulty and blind spot problems caused by insufficient sampling points are solved, and the accuracy of lane line detection and the ability to handle occlusion are improved.
Patent Information
- Application Number
- CN202411402161.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-10
AI Technical Summary
The Anchor-based lane line detection algorithm uses row sampling when sampling, resulting in only half of the points on the left and right lanes, which increases the fitting difficulty and makes it difficult to predict lanes for blind spots in the field of view caused by occlusion.
By adding an additional column prediction head to the traditional Anchor-based lane line detection algorithm, combining row prediction and column prediction, we use data augmentation techniques such as rotation, scaling, offset and lane line completion to improve the model's lane line Identify capabilities, and use feature pyramids to fuse features at different scales, calculate foreground/background classification losses, sampling point offset losses and IOU losses, and perform model training and inference.
The recognition effect of lane lines on both sides and lane line detection effect is improved, the model's ability to handle occlusion situations is enhanced, and the lane line fitting ability is improved.
Smart Images
Figure CN119296064A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of lane line detection methods, and in particular to a lane line detection method based on Anchor. Background Art
[0002] Mainstream lane detection algorithms can be divided into three categories: segmentation-based lane detection algorithms, anchor-based lane detection algorithms, and parameter-based lane detection algorithms. Each of these three methods has its own advantages and disadvantages: the segmentation-based method has the best effect, but because its prediction needs to be accurate to the pixel point, the amount of calculation is large and the speed cannot reach real-time; the detection-based method has the second best effect, and its calculation amount is determined by the preset number of anchors, which is much smaller than the segmentation-based method and can achieve real-time prediction; the parameter-based method is the fastest, but the effect is slightly worse.
[0003] The Anchor-based lane detection algorithm uses row sampling when sampling, which results in that the more inclined left and right lanes can only sample half of the points of the lanes. Fewer sampling points brings greater fitting difficulty, which ultimately leads to poor results for the left and right lanes. At the same time, it is difficult to predict lanes in blind spots caused by occlusion.
[0004] To this end, the present invention provides a lane line detection method based on Anchor to solve the above problems. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention provides a lane line detection method based on Anchor, which solves the problem that the lane line detection algorithm based on Anchor uses row sampling when sampling, resulting in that the more inclined left and right lanes can only sample half of the points of the lanes. Fewer sampling points bring greater fitting difficulty, and finally lead to poor effect on the left and right lanes. At the same time, it is difficult to predict the lane in the blind spot caused by occlusion.
[0006] To achieve the above objectives, the present invention is implemented by the following technical solutions: a lane line detection method based on Anchor, comprising the following steps:
[0007] Step 1: Perform data enhancement and lane line completion on the original image;
[0008] Step 2: Perform model training. During model training, the data-augmented image is input into the model to obtain the prediction results of the row branch and column branch. The prediction value is respectively compared with the annotations sampled from the row and column to calculate the loss.
[0009] Step 3: Perform model inference. During model inference, the original image is input into the model to obtain the prediction results of the row branch and the column branch. The results of each branch are filtered using a threshold and an NMS operation is performed. The input of NMS is IOU and foreground / background probability. The lines retained by the two branches are merged, and the lines with similar starting points are retained with more sampling points.
[0010] Preferably, the specific method of performing data enhancement and lane line completion on the original image in step 1 is: randomly rotating, scaling and offsetting the original image, and filling the content-free area with black borders after the operation to ensure that the image size remains unchanged, and extending the lane line label to the bottom of the image.
[0011] Preferably, the model in step 2 and step 3 uses Resnet50 as the backbone network of the model, and uses a feature pyramid to fuse features of different scales, and finally uses two prediction heads to predict the row sampling results and the column sampling results respectively.
[0012] Preferably, the specific outputs of the two prediction heads are the foreground / background probability cls of the corresponding Anchor and the xy offset between the predicted sampling point and the Anchor sampling point.
[0013] Preferably, the loss calculated in step 2 is divided into three parts, namely, foreground / background classification loss, sampling point offset loss, and IOU loss between the predicted lane line and the marked lane line.
[0014] Preferably, the calculation formula of the IOU is:
[0015]
[0016] Among them, the value range of i is [0, number of sampling points], x p is the prediction point, x g For marking points.
[0017] Preferably, during model training in step 2, both the column sampling branch and the row sampling branch perform positive and negative sample division of the Anchor and loss calculation.
[0018] Beneficial Effects
[0019] The present invention provides a lane line detection method based on Anchor. Compared with the prior art, it has the following beneficial effects:
[0020] Based on the traditional Anchor-based lane line detection algorithm, the present invention adds an additional column prediction head to enhance the recognition effect of lane lines on both sides. At the same time, row prediction and column prediction are used to improve the lane line detection effect. The data enhancement method of rotation, scaling, offset and completion is used to prompt the model's associative ability for lane lines, improve the effect of occlusion situations, and improve the model's lane line fitting ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is the overall flow chart of the present invention;
[0022] Figure 2 It is an effect diagram after data enhancement of the original image of the present invention;
[0023] Figure 3 It is a row and column sampling case diagram of the present invention;
[0024] Figure 4 It is an IOU schematic diagram of the lane line of the present invention;
[0025] Figure 5 It is the overall framework diagram of the model of the present invention. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] Example:
[0028] See also Figure 1-5 , a lane line detection method based on Anchor, comprising the following steps:
[0029] Step 1: Perform data augmentation and lane line completion on the original image: Randomly rotate, scale, and offset the original image, and fill the content-free area with black borders to ensure that the image size remains unchanged, and extend the lane line labels to the bottom of the image. The enhanced image effect is as follows: Figure 2 shown.
[0030] Step 2: Perform model training. During model training, input the data-enhanced image into the model to obtain the prediction results of the row branch and column branch. The prediction value is compared with the annotations sampled from the row and column to calculate the loss. The calculated loss is divided into three parts: foreground / background classification loss, sampling point offset loss, and IOU loss between the predicted lane line and the labeled lane line. Both the column sampling branch and the row sampling branch divide the positive and negative samples of the anchor and calculate the loss. The row and column sampling examples are shown in the figure below. Figure 3 shown.
[0031] The calculation formula of IOU is:
[0032]
[0033] Among them, the value range of i is [0, number of sampling points], x p is the prediction point, x g The IOU diagram of the lane line is as follows: Figure 4 shown.
[0034] Step 3: Perform model inference. During model inference, the original image is input into the model to obtain the prediction results of the row branch and the column branch. The results of each branch are filtered using thresholds and NMS operations are performed. The input of NMS is IOU and foreground / background probability. The lines retained by the two branches are merged, and the lines with similar starting points are retained with more sampling points. NMS refers to non-maximum suppression, which in the present invention suppresses lane lines that are not maximum values and retains lane lines with maximum values.
[0035] The model in step 2 and step 3 uses Resnet50 as the backbone network of the model, and uses feature pyramid to fuse features of different scales. Finally, two prediction heads are used to predict the row sampling results and column sampling results respectively. The specific outputs of the two prediction heads are the foreground / background probability cls of the corresponding anchor and the xy offset between the predicted sampling point and the anchor sampling point. The overall framework of the model is as follows: Figure 5 shown.
[0036] Based on the traditional Anchor-based lane line detection algorithm, the present invention adds an additional column prediction head to enhance the recognition effect of lane lines on both sides. At the same time, row prediction and column prediction are used to improve the lane line detection effect. The data enhancement method of rotation, scaling, offset and completion is used to prompt the model's associative ability for lane lines, improve the effect of occlusion situations, and improve the model's lane line fitting ability.
[0037] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.
[0038] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0039] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A lane line detection method based on Anchor, characterized in that: The following steps are involved: Step 1: Perform data enhancement and lane line completion on the original image; Step 2: Perform model training. During model training, the data-augmented image is input into the model to obtain the prediction results of the row branch and column branch. The prediction value is respectively compared with the annotations sampled from the row and column to calculate the loss. Step 3: Perform model inference. During model inference, the original image is input into the model to obtain the prediction results of the row branch and the column branch. The results of each branch are filtered using a threshold and an NMS operation is performed. The input of NMS is IOU and foreground / background probability. The lines retained by the two branches are merged, and the lines with similar starting points are retained with more sampling points.
2. The lane line detection method based on Anchor according to claim 1, characterized in that: The specific method of performing data enhancement and lane line completion on the original image in the step 1 is: randomly rotating, scaling and offsetting the original image, and filling the content-free area with black borders after the operation to ensure that the image size remains unchanged, and extending the lane line label to the bottom of the image.
3. The lane line detection method based on Anchor according to claim 1, characterized in that: The model in step 2 and step 3 uses Resnet50 as the backbone network of the model, and uses a feature pyramid to fuse features of different scales. Finally, two prediction heads are used to predict the row sampling results and the column sampling results respectively.
4. The lane line detection method based on Anchor according to claim 3, characterized in that: The specific outputs of the two prediction heads are the foreground / background probability cls of the corresponding Anchor and the xy offset between the predicted sampling point and the Anchor sampling point.
5. The lane line detection method based on Anchor according to claim 1, characterized in that: The loss calculated in step 2 is divided into three parts: foreground / background classification loss, sampling point offset loss, and IOU loss between predicted lane lines and marked lane lines.
6. The lane line detection method based on Anchor according to claim 5, characterized in that: The calculation formula of the IOU is: Among them, the value range of i is [0, number of sampling points], x p is the prediction point, x g For marking points.
7. The lane line detection method based on Anchor according to claim 1, characterized in that: During model training in step 2, both the column sampling branch and the row sampling branch perform positive and negative sample division of the anchor and loss calculation.