Lane line detection method and system based on global context awareness and geometric constraint

Abstract

The invention discloses a lane line detection method and system based on global context perception and geometric constraint, and the method comprises the steps: obtaining a lane line image data set, carrying out the normalization processing, and extracting a multi-scale feature map of an image; enhancing global context information of the feature map through a mixed attention module, and generating an enhanced feature map; on the basis of the enhanced feature map, a candidate lane line starting proposal point coordinate is generated through a Transform decoder; according to the candidate proposal points, adopting dynamic convolution to generate prediction parameters of lane line instances; predicting line-by-line coordinates of a lane line through line-level offset mapping, and introducing geometric constraint loss to optimize offset prediction; and carrying out self-adaptive sliding window smoothing post-processing on the predicted lane line coordinates, and outputting a final lane line detection result. Through global context-aware dynamic convolution parameter generation and geometric constraint offset mapping optimization, the problems of proposal point leak detection and error accumulation are solved, and the lane line detection robustness and long-distance fitting precision of CondLaneNet in a complex scene are improved.

Description

Technical Field

[0001] This invention relates to a lane line detection method and system, and more particularly to a lane line detection method and system based on global context awareness and geometric constraints. Background Technology

[0002] In recent years, with the development of autonomous driving technology, lane detection, as one of the core tasks of environmental perception, has also made significant progress. Traditional lane detection techniques rely on manual feature extraction, followed by post-processing techniques such as Hough transform to fit lane lines. However, these methods cannot adapt to lane lines in different scenarios, resulting in poor adaptability of manual models in complex environments and an inability to guarantee sufficient robustness. Deep learning-based lane detection methods, as a current research focus, have achieved end-to-end automated detection processes. Methods based on semantic segmentation, anchor point detection, and keypoint detection, for example, can extract richer lane line information and significantly improve detection accuracy in complex scenarios.

[0003] The CondLaneNet framework is a top-down lane detection framework based on conditional convolution and line-by-line formulas. It achieves end-to-end instance differentiation by detecting the proposal points of lane lines and generating dynamic convolution kernel parameters, avoiding the post-clustering problem of traditional segmentation methods. Furthermore, it introduces a RecurrentInstance Module (RIM), which uses LSTM to dynamically generate multiple sets of convolution parameters, effectively handling bifurcation lines and dense line fields. Simultaneously, it uses conditional convolution to directly predict lane shape, avoiding the cumbersome post-processing steps such as clustering and curve fitting in traditional methods.

[0004] Despite CondLaneNet's excellent performance in complex lane detection tasks, several key issues remain. Firstly, the parameters of dynamic convolutions have limitations. CondLaneNet predicts lane shape through conditional convolutions, but the generation of its dynamic parameters is highly dependent on the initial detection quality of the proposal head. If proposal points are missed due to occlusion or noise, subsequent shape predictions may fail to correct for these issues. Secondly, offset mapping suffers from error accumulation. In row-level position prediction, a row-by-row offset map is used to refine lane coordinates, but local errors accumulate with the number of rows, leading to inaccurate long-distance fitting. Therefore, addressing these challenges is also a necessary consideration. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a lane detection method based on global context awareness and geometric constraints. By generating dynamic convolution parameters with global context awareness and optimizing offset mapping based on geometric constraints, the method solves the problems of missed proposal points and error accumulation, thereby improving the robustness of CondLaneNet in lane detection in complex scenes and the accuracy of long-distance fitting.

[0006] Technical solution: The lane line detection method based on global context awareness and geometric constraints described in this invention includes:

[0007] (1) Obtain the lane line image dataset, perform normalization processing, and extract multi-scale feature maps of the images;

[0008] (2) Enhance the global contextual information of the feature map by using a hybrid attention module to generate an enhanced feature map;

[0009] (3) Based on the enhanced feature map, the coordinates of the starting point of the candidate lane line are generated by the Transformer decoder;

[0010] (4) Based on the candidate proposal points, use dynamic convolution to generate prediction parameters for lane line instances;

[0011] (5) Predict the row-by-row coordinates of lane lines through row-level offset mapping and introduce geometric constraint loss to optimize offset prediction;

[0012] (6) Perform adaptive sliding window smoothing post-processing on the predicted lane line coordinates and output the final lane line detection results.

[0013] Furthermore, step (1) uses a ResNet network to extract multi-scale feature maps, which include low-level detail features, mid-level semantic features, and high-level global features.

[0014] Furthermore, the input for step (2) is the high-level global feature F3 in the multi-scale feature map, and the formula for generating the enhanced feature map is:

[0015]

[0016] Where F3 is the input feature map, Q,K,V are the Query, Key, and Value matrices generated by 1×1 convolution, and W... s Let d be a learnable channel attention weight matrix. k This is a scaling factor used for stable gradient calculation.

[0017] Furthermore, the coordinates of the proposed point in step (3) are calculated as follows:

[0018] Anchor features are iteratively updated using a 3-layer Transformer decoder, and the updated anchor features are output. A confidence score s is predicted for each anchor. i and coordinate offset Δp i ,Right now:

[0019] s i =σ(MLP(A) i )),

[0020]

[0021] Here, σ() is the Sigmoid function, which ultimately outputs the set of coordinates of candidate proposal points.

[0022] Furthermore, the prediction parameters for the lane line instances generated by the dynamic convolution in step (4) are generated as follows:

[0023] W dynamic =MLP(ROIAlign(F enhanced ,P proposal ))

[0024] Among them, P proposal The coordinates of the candidate proposal points output by the Transformer decoder; ROIAlign is used to extract local features, and MLP is used to map and generate dynamic convolution weights.

[0025] Furthermore, the geometric constraint loss in step (5) includes curvature consistency loss and lateral offset constraint loss, which are calculated as follows:

[0026] Curvature uniformity loss:

[0027]

[0028] Lateral offset constraint loss:

[0029]

[0030] Where, Δy i This represents the horizontal offset of the i-th row, and τ is the preset maximum allowed offset threshold.

[0031] Furthermore, the calculation method for the adaptive sliding window smoothing in step (6) is as follows:

[0032] W j ={p k |||p k -p i ||2<ò}

[0033] Where: p jp is the original prediction point. j ' represents the smoothed point; W j is a dynamic window whose size is adaptively adjusted by the local curvature; ò is the distance threshold used to control the smoothing range.

[0034] The lane detection system based on global context awareness and geometric constraints described in this invention includes:

[0035] Feature extraction module: used to acquire lane line image datasets, perform normalization processing, and use backbone network to extract multi-scale feature maps of the images;

[0036] Global Context Enhancement Module: Used to enhance the global context information of the feature map by using a hybrid attention module, generating an enhanced feature map;

[0037] Proposal point generation module: used to generate the coordinates of the starting proposal points of candidate lane lines based on the enhanced feature map using the Transformer decoder;

[0038] Dynamic convolution parameter generation module: used to generate prediction parameters for lane line instances using dynamic convolution based on candidate proposal points;

[0039] Geometric constraint optimization module: used to predict the row-by-row coordinates of lane lines through row-level offset mapping, and to optimize the offset prediction by introducing geometric constraint loss;

[0040] Adaptive smoothing module: This module performs adaptive sliding window smoothing on the predicted lane line coordinates and outputs the final lane line detection results.

[0041] A computer device includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of the lane line detection method based on global context awareness and geometric constraints.

[0042] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the lane line detection method based on global context awareness and geometric constraints.

[0043] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0044] (1) By adding a hybrid attention module to fuse global context and local detail features, the feature expression capability is significantly enhanced; the generation of dynamic convolution parameters depends on global context-aware features, avoiding subsequent prediction failures caused by missed detection of proposal points, improving the robustness of lane line detection and the stability of complex scenes; (2) By introducing a geometric constraint loss function, the long-distance fitting accuracy is improved and the long-distance detection error is significantly reduced. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method described in this invention.

[0046] Figure 2 This is a structural diagram of the Transformer encoder of the present invention.

[0047] Figure 3 This is a structural diagram of the hybrid attention module of the present invention. Detailed Implementation

[0048] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0049] This invention discloses a lane detection method based on global context awareness and geometric constraints, comprising:

[0050] (1) Obtain the lane line image dataset and perform normalization processing, then use the backbone network to extract multi-scale feature maps of the images;

[0051] Obtain the publicly available datasets CurveLanes, CULane, and TuSimple. Input the RGB image set and normalize it. Extract multi-scale feature maps of the input images through the backbone network. The multi-scale features include low-level detail features at a scale of 1 / 4, mid-level semantic features at a scale of 1 / 8, and high-level global features at a scale of 1 / 16. Then output the multi-scale feature map set {F1, F2, F3}.

[0052] (2) Apply the Hybrid Attention Module (HAB) to the feature map to enhance the global context information of the feature map and generate an enhanced feature map;

[0053] A Hybrid Attention (HAB) module is applied to the feature map to enhance global contextual information and generate an enhanced feature map. For the application of the HAB module, multi-head self-attention is first calculated, with high-level global features at a scale of 1 / 16 as input. Then, it is generated through 1×1 convolution. Calculate attention weights:

[0054]

[0055] Then output attention features

[0056] In the channel attention enhancement part, the SE module is applied to F3:

[0057]

[0058] GAP stands for Global Average Pooling. Channel-weighted output F se =W s ·F3.

[0059] In the feature fusion section, F attn and F se Element-wise multiplication is performed to calculate and the enhanced feature map is output:

[0060]

[0061] (3) Based on the enhanced feature map, the coordinates of the starting point of the candidate lane line are generated by the Transformer decoder;

[0062] Initialize 100 learnable positional codes, and iteratively update the Anchor features using a 3-layer Transformer decoder:

[0063] Query = Anchor, Key / Value = F enhanced ,

[0064] Output updated Anchor features Then, based on the prediction of the proposed points, predict the confidence score s for each Anchor. i and coordinate offset Δp i ,Right now

[0065] s i =σ(MLP(A) i )),

[0066]

[0067] Retain the top 20 high-confidence proposals Output the set of candidate proposal point coordinates P proposal .

[0068] (4) Based on the candidate proposal points, use dynamic convolution to generate prediction parameters for lane line instances;

[0069] Input augmented feature map F enhanced and the set of coordinates of candidate proposal points P proposal First, feature clipping is performed, using each proposal point p i A 32x32 area is cropped from the center, and fixed-size features are extracted using ROIAlign. The predicted parameters for lane line instances are generated using dynamic convolution, and R is used.i After flattening, input a 2-layer MLP:

[0070]

[0071] The superscript 5 indicates the number of dynamic convolution kernel parameters, and the final output is the set of dynamic convolution weights.

[0072] (5) Predict the row-by-row coordinates of lane lines through row-level offset mapping and introduce geometric constraint loss to optimize offset prediction;

[0073] The dynamic convolution weights obtained from the input and feature map F enhanced By predicting the row-by-row coordinates of lane lines through row-level offset mapping, the image height is divided into 80 rows with a 6-pixel interval. Dynamic convolution is applied to each row j to obtain:

[0074]

[0075] We introduce geometric constraint loss to optimize offset prediction. The classification loss uses Focal Loss to supervise the proposal point confidence, and the offset loss uses SmoothL1 Loss to supervise Δy. j The geometric constraint loss includes curvature consistency loss and lateral offset constraint loss, which are calculated as follows:

[0076] Curvature uniformity loss:

[0077]

[0078] Lateral offset constraint loss:

[0079]

[0080] Where, Δy i This represents the horizontal offset of the i-th row, where τ is the preset maximum allowable offset threshold.

[0081] Calculate the geometric constraint loss:

[0082] L geo =0.3L curve +0.2L lateral ,

[0083] Where τ = 5 pixels, is the maximum allowed offset.

[0084] (6) Perform adaptive sliding window smoothing post-processing on the predicted lane line coordinates and output the final lane line detection results;

[0085] First, input the original prediction point set. Perform curvature estimation and calculate the local curvature c at each point.j =||p j+1 -2p j +p j-1 2. Then adjust the dynamic window, c j The portion >1.0 is designated as a high curvature region, and the window size is adjusted to ò = 3; j The portion ≤1.0 is designated as a low curvature region, and the window size is adjusted to ò=5.

[0086] Perform smoothing calculations for each point p. j Select neighborhood W j ={p k |||p k -p j ||2<ò},output smooth point p j '=mean(W j Output the final set of lane line points.

[0087] This invention also discloses a lane detection system based on global context awareness and geometric constraints, comprising:

[0088] Feature extraction module: Acquires lane line image dataset, performs normalization processing, and uses backbone network to extract multi-scale feature maps of the image;

[0089] Global Context Enhancement Module: Used to enhance the global context information of the feature map by using a hybrid attention module, generating an enhanced feature map;

[0090] Proposal point generation module: used to generate the coordinates of the starting proposal points of candidate lane lines based on the enhanced feature map using the Transformer decoder;

[0091] Dynamic convolution parameter generation module: used to generate prediction parameters for lane line instances using dynamic convolution based on candidate proposal points;

[0092] Geometric constraint optimization module: used to predict the row-by-row coordinates of lane lines through row-level offset mapping, and to optimize the offset prediction by introducing geometric constraint loss;

[0093] Adaptive smoothing module: This module performs adaptive sliding window smoothing on the predicted lane line coordinates and outputs the final lane line detection results.

[0094] This invention addresses the issues of missed proposal points and error accumulation by generating globally context-aware dynamic convolution parameters and optimizing offset mapping based on geometric constraints, thereby improving CondLaneNet's robustness in lane detection and long-distance fitting accuracy in complex scenarios.

Claims

1. A lane detection method based on global context awareness and geometric constraints, characterized in that, include: (1) Obtain the lane line image dataset, perform normalization processing, and extract multi-scale feature maps of the images; (2) Enhance the global contextual information of the feature map by using a hybrid attention module to generate an enhanced feature map; (3) Based on the enhanced feature map, the coordinates of the starting point of the candidate lane line are generated by the Transformer decoder; (4) Based on the candidate proposal points, use dynamic convolution to generate prediction parameters for lane line instances; (5) Predict the row-by-row coordinates of lane lines through row-level offset mapping and introduce geometric constraint loss to optimize offset prediction; (6) Perform adaptive sliding window smoothing post-processing on the predicted lane line coordinates and output the final lane line detection results.

2. The lane detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, Step (1) uses a ResNet network to extract multi-scale feature maps, which include low-level detail features, mid-level semantic features, and high-level global features.

3. The lane detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, In step (2), the high-level global feature F3 in the multi-scale feature map is input, and the formula for generating the enhanced feature map is as follows: Where F3 is the input feature map, Q,K,V are the Query, Key, and Value matrices generated by 1×1 convolution, and W... s Let d be a learnable channel attention weight matrix. k This is a scaling factor used for stable gradient calculation.

4. The lane line detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, The coordinates of the proposed point in step (3) are calculated as follows: Anchor features are iteratively updated using a 3-layer Transformer decoder, and the updated anchor features are output. A confidence score s is predicted for each anchor. i and coordinate offset Δp i ,Right now: s i <σ(MLP(A) i )), Here, σ() is the Sigmoid function, which ultimately outputs the set of coordinates of candidate proposal points.

5. The lane detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, The prediction parameters for lane line instances are generated by dynamic convolution in step (4) as follows: W dynamic =MLP(ROIAlign(F enhanced ,P proposal )) Among them, P proposal The coordinates of the candidate proposal points output by the Transformer decoder; ROIAlign is used to extract local features, and MLP is used to map and generate dynamic convolution weights.

6. The lane line detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, The geometric constraint loss in step (5) includes curvature consistency loss and lateral offset constraint loss, which are calculated as follows: Curvature uniformity loss: Lateral offset constraint loss: Where, Δy i This represents the horizontal offset of the i-th row, and τ is the preset maximum allowed offset threshold.

7. The lane detection method based on global context awareness and geometric constraints according to claim 1, characterized in that, The calculation method for the adaptive sliding window smoothing in step (6) is as follows: Where: p j p is the original prediction point. j ' represents the smoothed point; W j is a dynamic window whose size is adaptively adjusted by the local curvature; ò is the distance threshold used to control the smoothing range.

8. A lane detection system based on global context awareness and geometric constraints, characterized in that, include: Feature extraction module: used to acquire lane line image datasets, perform normalization processing, and use backbone network to extract multi-scale feature maps of the images; Global Context Enhancement Module: Used to enhance the global context information of the feature map by using a hybrid attention module, generating an enhanced feature map; Proposal point generation module: used to generate the coordinates of the starting proposal points of candidate lane lines based on the enhanced feature map using the Transformer decoder; Dynamic convolution parameter generation module: used to generate prediction parameters for lane line instances using dynamic convolution based on candidate proposal points; Geometric constraint optimization module: used to predict the row-by-row coordinates of lane lines through row-level offset mapping, and to optimize the offset prediction by introducing geometric constraint loss; Adaptive smoothing module: This module performs adaptive sliding window smoothing on the predicted lane line coordinates and outputs the final lane line detection results.

9. A computer device, characterized in that, The method includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of a lane line detection method based on global context awareness and geometric constraints as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the lane line detection method based on global context awareness and geometric constraints as described in any one of claims 1-7.

Images