A lane line detection method based on edge perception attention and continuity enhancement

Abstract

The application discloses a lane line detection method based on edge perception attention and continuity enhancement. The method comprises the following steps: acquiring a lane line data set and performing preliminary processing; inputting into a backbone network to acquire multi-level features; using an edge perception attention module to analyze high-frequency parts and low-frequency parts of a feature map and output a feature map fused with key high-frequency features; using a continuity enhancement module to convert the feature map from a time domain to a frequency domain to acquire global semantic information; calculating the inclination angles of predicted lane lines and target lane lines through an angle loss function, and iteratively updating the positions of lane line anchor boxes to acquire a final lane line prediction sequence. The method combines the edge perception attention module and the continuity enhancement module, can accurately capture local semantic information and global semantic information, and improves the precision of lane line detection.

Description

Technical Field

[0001] This invention relates to a lane line detection method based on edge-aware attention and continuity enhancement, belonging to the field of computer vision technology. Background Technology

[0002] With economic development, traffic elements such as vehicles and roads continue to grow rapidly, and how to obtain more accurate lane lines to assist traffic control has become an urgent problem to be solved.

[0003] In urban road environments, there are various traffic participants, including motor vehicles, non-motor vehicles, and pedestrians, as well as complex scenarios such as inclement weather, low light levels, and obstacles. These factors pose significant challenges to traffic safety and efficiency. Traditional lane line target detection algorithms suffer from low accuracy when handling complex scenarios such as blurred lane lines and occluded lane targets. With the development of deep learning, the ability to process multi-scale and blurred lane lines has been improved by automatically learning features from large amounts of data. Furthermore, the detection accuracy for dynamic lane targets and irregular drivable areas has been enhanced, gradually replacing traditional methods and becoming the mainstream research direction.

[0004] Deep learning-based lane detection methods have made some progress in terms of accuracy and adaptability, but the following problems still exist. First, in low-visibility scenes, they typically rely on global semantics and local texture information to predict precise location and shape; existing deep learning models are prone to accuracy degradation in complex scene detection. Second, lane line features contain multi-scale edge information, which requires deep fusion and accurate capture by the model. However, existing deep learning models are prone to missing key features when dealing with lane line wear and similar road surface textures. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned problems in existing methods, and to disclose a lane line detection method based on edge-aware attention and continuity enhancement.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A lane detection method based on edge-aware attention and continuity enhancement is characterized by including the following steps:

[0008] S1: Obtain the lane line dataset, clean up the abnormal data, and then input the data into the backbone network to obtain a five-level feature map, denoted as Fi, where i represents the feature level, 3≤i≤5;

[0009] S2: The edge-aware attention module is used to enhance the features of the i=3 layer feature map. This module focuses on the extraction of local features. Wavelet transform is used to extract low-frequency and high-frequency components from the feature map. After extraction, three dimensional maps are generated. The attention mechanism is used to generate a weight map and multiply it with the dimensional map to finally output the key high-frequency feature weight map.

[0010] S3: The continuity enhancement module is used to enhance the features of the i=4 and i=5 layers. This module is designed to meet the long-distance continuity requirements of lane lines. The channel is divided into two parts. The feature map of one part is extracted with local trend information through depthwise separable convolution, and the feature map of the other part is extracted with global trend through Fourier transform. Finally, the feature map with global semantic information is output by residual connection with the original feature map.

[0011] S4: After the processed feature map is fused from top to bottom, the channels of Fi are concatenated, where i represents the feature level, 3≤i≤5;

[0012] S5: After the channels are stitched together, the feature map is passed through the detection head Ha, where a represents the feature level, 1≤a≤3;

[0013] S6: The prior box Pb is continuously updated iteratively, where b represents the number of optimizations, 1≤b≤3. Iterative training is performed using the angle loss function, and finally, lane line prediction maps are output through confidence filtering and NMS deduplication.

[0014] A further technical solution involves the edge-aware attention module first performing wavelet kernel convolution on the horizontal direction of the feature map to generate a low-frequency component X. l and high-frequency component X h Secondly, the low-frequency component X l and high-frequency component X h The system generates three dimensional maps (LH, HL, and HH) by multiplying the feature map with wavelet transform convolution in the vertical direction. Here, LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, and HH represents the corners and intersections of the lane line. Simultaneously, the feature map is input into another branch, where a weight map is obtained through depthwise separable convolution and attention mechanism. Then, the weight map is downsampled and adjusted to the same size as the three dimensional maps by channel transformation. Finally, the three dimensional maps are multiplied with the weight map and then channel compression is performed by a 1×1 convolution. During this process, important high-frequency features are learned through the convolution weights to output the feature map. Finally, the feature map is restored to its original size by bilinear interpolation.

[0015] A further technical solution involves the continuity enhancement module first performing channel alignment and normalization on the feature map, then dividing the channels into two parts. One part, feature map X1, is used to extract local information using depthwise separable convolution, outputting feature map X. a The other part of the feature map X2 is transformed from the spatial domain features to the frequency domain features through Fourier transform rfft2 to obtain X. f The high-frequency components represent local details of the lane lines, while the low-frequency components represent global details of the lane lines. Then X... f The modulated frequency domain feature map X is generated by multiplying the modulator with its generated complex weight W. fmod Frequency domain feature map X fmod The time-domain feature map X is obtained by inverse Fourier transform. b If the magnitude of the complex weight W is close to 1, the frequency component is retained; if it is close to 0, the frequency component is suppressed. Finally, the feature map X is... a and X b Channel concat fusion is performed, and the output feature map is residually concatenated with the original input feature map to obtain the final feature map X. cat .

[0016] A further technical solution is that the top-down feature map fusion involves adjusting the size of the feature map after continuity enhancement and fusing it with the feature map enhanced by the edge-aware attention module, so that the lower-level feature map can obtain high-level semantic information, thereby increasing the resolution of the feature mapping.

[0017] Furthermore, the core of the angle loss function is to optimize the model by comparing the tilt angle difference between the predicted lane line and the target lane line. Specifically, firstly, corresponding y-coordinates are generated for the predicted value and the target value. Then, the slope grad between adjacent sampling points of the two is calculated. The tangent value of the angle difference is obtained through the relationship between the slope difference and the product. Then, the tangent value is converted into the angle difference again, and the cosine value of the angle difference is used as the single-segment angle loss line_angle. Finally, the angle loss of all segments is averaged to obtain the angle loss of each lane line. Then, the average value in the batch dimension is taken as the final angle loss angeloss.

[0018] The beneficial effects of this invention are as follows:

[0019] This invention discloses a lane detection method based on edge-aware attention and continuity enhancement. First, this invention extracts lane edge and corner features through wavelet transform, and combines this with an attention mechanism to enhance key high-frequency information, effectively improving the accuracy of edge detail detection, especially in complex lighting and occlusion scenarios. Second, this invention designs a continuity enhancement module that fuses the local trends of depthwise separable convolution with the global semantics of Fourier transform, and then preserves the original features through residual connections, significantly enhancing the long-distance continuity modeling capability of lane lines and reducing detection errors caused by viewpoint changes or lane line breaks. A top-down feature fusion strategy is adopted, allowing feature maps at different levels to complement each other, preserving low-level details while incorporating high-level semantics, thus improving the richness of feature representation. The optimized angle loss function further improves the accuracy of shape prediction by directly optimizing the lane line tilt angle difference, especially in curved lane scenarios, effectively reducing angle deviation. The overall method, through multi-module collaboration, achieves high-precision and robust lane detection in complex traffic scenarios, providing a reliable environmental perception foundation for autonomous driving path planning and safety decisions. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings:

[0021] Figure 1 This is a flowchart illustrating the overall network process according to an embodiment of the present invention.

[0022] Figure 2 This is a diagram illustrating the overall network architecture of an embodiment of the present invention.

[0023] Figure 3 This is a structural diagram of the edge-aware attention module according to an embodiment of the present invention;

[0024] Figure 4 This is a structural diagram of the continuity enhancement module according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] This invention provides a lane detection method based on edge-aware attention and continuity enhancement, such as... Figure 1 As shown, it includes the following steps:

[0027] S1: Obtain the lane line dataset, clean up abnormal images, construct a ResNet18 backbone network, and then input the feature maps to obtain multi-level features, denoted as Fi, where i represents the feature level, 3≤i≤5; For example... Figure 2 As shown;

[0028] S2: Construct the edge-aware attention module, the structure of which is shown below. Figure 3 ;

[0029] S2-1: The edge-aware attention module performs a wavelet kernel convolution on the feature map in the horizontal direction to generate a low-frequency component X. l and high-frequency component X h The output dimension is Then, the low-frequency component X l and high-frequency component X h Multiplying the wavelet transform convolution in the vertical direction generates LH, HL, and HH dimension maps, with the output dimension being... LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, HH represents the corners and intersections of the lane line, n represents the batch number, c represents the number of channels, h represents the height of the image, and w represents the width of the image. The calculation formula is as follows:

[0030]

[0031] S2-2: The feature map is input into another branch, where a weight map is obtained through depthwise separable convolution and attention mechanisms. This weight map is then downsampled and adjusted to the same size as the three dimensional maps by channel transformation. The three dimensional maps are multiplied by the weight map, and then a 1×1 convolution is used for channel compression. During this process, important high-frequency features are learned through the convolution weights to output a feature map. Finally, bilinear interpolation is used to restore the feature map to its original size, with output dimensions n, c, h, and w. i,j These are the weights of the bilinear interpolation, and f_wavelet represents the feature map recovered by bilinear interpolation. The specific calculation formula is as follows:

[0032]

[0033] S3: Construct the continuity enhancement module, the structure of which is shown below. Figure 4 .

[0034] S3-1: First, perform channel alignment and normalization on the feature map, and then divide the channels into two equal parts.

[0035] S3-2: A portion of the feature map X1 is used to extract local information using depthwise separable convolution, resulting in X. a .

[0036] S3-3: Another part of the feature map X2 is the transformation of the spatial domain features of the feature map into the frequency domain features X. f The high-frequency components represent the local details of the lane lines, while the low-frequency components represent the global trend of the lane lines. Then X... f X is generated by multiplying a modulator with its generated complex weight W. fmod Frequency domain feature map X fmod The time-domain feature map X is obtained by inverse Fourier transform. b If the amplitude of W is close to 1, the frequency component is preserved; if it is close to 0, the frequency component is suppressed. Here, rfft2 represents the two-dimensional real Fourier transform, n represents the batch number, c represents the number of channels, and k... u Represents the vertical frequency, k v The horizontal frequency is represented by W1 and W2, which represent linear transformation layers. GlobalPool() represents global pooling, and sigmoid represents the activation function X. fmod , representing the result of multiplying the frequency matrix by its corresponding matrix weights. Its calculation formula is as follows:

[0037] X f [n,c,k u ,k v ] = rfft2(x2[n,c,:,:]),

[0038] W=sigmoid(W2·ReLU(W1·GlobalPool(x2))),

[0039] Xf mod [n,c,k u ,k v ]=Xf[n,c,k u ,k v ]×W[n,c,k u ,k v ].

[0040] S3-4: The modulated frequency domain feature matrix is ​​resynthesized into spatial domain features through inverse Fourier transform, and the feature map X is obtained by irfft2. a and the feature map X after local path enhancement b Perform concat concatenation and fusion, then perform residual concatenation between the output feature map and the original input feature map. Finally, normalize the output feature map again to obtain the final feature map X. cat irfft2 represents the two-dimensional inverse Fourier transform, outputting X. b This represents the matrix that represents the return from the frequency domain to the spatial domain. Its specific calculation formula is as follows:

[0041] X b[n,c,h,w]=irfft2(Xf mod [n,ck u ,k v ]),

[0042]

[0043] S4: Perform feature fusion from top to bottom on the feature maps of Fi and then stitch them together, where i represents the feature level, 3≤i≤5;

[0044] S5: Construct a detection head and determine the final output by iterating the position of the prior box during training.

[0045] S6: Construct an angle loss function, the core of which is to optimize the model by comparing the tilt angle difference between the predicted lane line and the target lane line. Specifically, first, generate corresponding y-coordinates for the predicted and target values; then calculate the slope grad between adjacent sampling points of the two, and obtain the tangent of the angle difference through the relationship between the slope difference and the product; where y i+1 -y i The difference in y-coordinates between adjacent sampling points, x i+1 -x i The x-coordinate difference between adjacent sampling points is denoted by ∈ = 2e-9 to avoid division by zero. The specific calculation formula is as follows:

[0046]

[0047] S6-1: The tangent value is then converted into an angle difference, and the cosine of this angle difference minus 1 is used as the single-segment angle loss line_angle, where k1 is the predicted slope and k2 is the target slope; arctan(tan(θ)) restores the angle difference θ, and 1-cos(θ) ensures that the larger the angle difference, the larger the loss value. The specific calculation formula is as follows:

[0048]

[0049] line-angle=1-cos(arctan(tan(θ))).

[0050] S6-2: Finally, the average angle loss of all segments is taken to obtain the angle loss of each lane line. Then, the average of these averages across the batch dimension is taken as the final angle loss (angeloss). Afterwards, confidence filtering and NMS deduplication are performed to output the lane line coordinates. Here, N is the number of lane lines, and 71 is the number of line segments. The specific calculation formula is as follows:

[0051]

[0052] To verify the effectiveness of the above examples, the performance of this invention was compared with other state-of-the-art methods on two datasets, CULane and TuSimple. The experimental results on the CULane dataset used eleven metrics: F1... 50 The ten metrics are: Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, Cross, and Night. Of these, only Cross is considered a valid F1 metric. 50 For the Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, and Night metrics, higher values ​​indicate better performance. Three metrics were selected for the TuSimple dataset: F1, Accuracy, and FP. Except for FP, higher values ​​for F1 and Accuracy generally indicate better performance.

[0053] As can be seen from the experimental results in Table 1, the proposed method outperforms existing methods in multiple metrics on the CULane dataset. The method proposed in this embodiment scores 81.20 on the CULane dataset, which is 1.07 points higher than the state-of-the-art CLRNet. Furthermore, it shows improvements in three challenging scenarios: Noline, Curve, and Night, demonstrating the effectiveness of the proposed method.

[0054] Table 1 shows the comparison results of detection accuracy on the CULane dataset.

[0055]

[0056] Table 2 shows the comparison results of detection accuracy on the TuSimple dataset.

[0057] Method Backbone F1(％) Acc(%) FP (%) SCNN VGG16 95.97 96.53 6.17 RESA ResNet34 96.93 96.82 3.63 PolyLaneNet EfficientNetB0 90.62 93.36 9.42 UFLDv2 ResNet34 88.02 95.86 18.91 CondLaneNet ResNet34 96.77 95.37 2.20 FOLOLane ERFNet 96.59 96.92 4.47 CANet ResNet34 97.77 96.76 1.92 CLRNet ResNet18 97.89 96.84 2.28 OURS ResNet18 98.13 96.89 1.78

[0058] As shown in Table 2, the experimental results of this embodiment outperform existing methods in F1 score on the TuSimple dataset. The method of this embodiment surpasses the current state-of-the-art level by 0.24% in F1 score, proving the effectiveness of the method of this embodiment.

Claims

1. A lane line detection method based on edge-aware attention and continuity enhancement, characterized in that... Includes the following steps: S1: Obtain the lane line dataset, clean up the abnormal data, and then input the data into the backbone network to obtain a five-level feature map, denoted as Fi, where i represents the feature level, 3≤i≤5; S2: The edge-aware attention module is used to enhance the features of the i=3 layer feature map. This module focuses on the extraction of local features. Wavelet transform is used to extract low-frequency and high-frequency components from the feature map. After extraction, three dimensional maps are generated. The attention mechanism is used to generate a weight map and multiply it with the dimensional map to finally output the key high-frequency feature weight map. S3: The continuity enhancement module is used to enhance the features of the i=4 and i=5 layers. This module is designed to meet the long-distance continuity requirements of lane lines. The channel is divided into two parts. The feature map of one part is extracted with local trend information through depthwise separable convolution, and the feature map of the other part is extracted with global trend through Fourier transform. Finally, the feature map with global semantic information is output by residual connection with the original feature map. S4: After the processed feature map is fused from top to bottom, the channels of Fi are concatenated, where i represents the feature level, 3≤i≤5; S5: After the channels are stitched together, the feature map is passed through the detection head Ha, where a represents the feature level, 1≤a≤3; S6: The prior box Pb is continuously updated iteratively, where b represents the number of optimizations, 1≤b≤3. Iterative training is performed using the angle loss function, and finally, lane line prediction maps are output through confidence filtering and NMS deduplication.

2. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The edge-aware attention module first performs wavelet kernel convolution on the horizontal direction of the feature map to generate a low-frequency component X. l and high-frequency component X h Secondly, the low-frequency component X l and high-frequency component X h The system generates three dimensional maps (LH, HL, and HH) by multiplying the feature map with wavelet transform convolution in the vertical direction. Here, LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, and HH represents the corners and intersections of the lane line. Simultaneously, the feature map is input into another branch, where a weight map is obtained through depthwise separable convolution and attention mechanism. Then, the weight map is downsampled and adjusted to the same size as the three dimensional maps by channel transformation. Finally, the three dimensional maps are multiplied with the weight map and then channel compression is performed by a 1×1 convolution. During this process, important high-frequency features are learned through the convolution weights to output the feature map. Finally, the feature map is restored to its original size by bilinear interpolation.

3. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The aforementioned continuity enhancement module first performs channel alignment and normalization on the feature map, then divides the channels into two equal parts. One part, feature map X1, is used to extract local information using depthwise separable convolution, and the output feature map is X. a The other part of the feature map X2 is transformed from the spatial domain features to the frequency domain features through Fourier transform rfft2 to obtain X. f The high-frequency components represent local details of the lane lines, while the low-frequency components represent global details of the lane lines. Then X... f The modulated frequency domain feature map X is generated by multiplying the modulator with its generated complex weight W. fmod Frequency domain feature map X fmod The time-domain feature map X is obtained by inverse Fourier transform. b If the magnitude of the complex weight W is close to 1, the frequency component is retained; if it is close to 0, the frequency component is suppressed. Finally, the feature map X is... a and X b Channel concat fusion is performed, and the output feature map is residually concatenated with the original input feature map to obtain the final feature map X. cat .

4. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The top-down feature map fusion involves adjusting the size of the feature map after continuity enhancement and fusing it with the feature map enhanced by the edge-aware attention module. This allows the lower-level feature map to acquire higher-level semantic information, increasing the resolution of the feature mapping.

5. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The aforementioned angle loss function optimizes the model by comparing the tilt angle difference between the predicted lane line and the target lane line. Specifically, it first generates corresponding y-coordinates for the predicted and target values, then calculates the slope grad between adjacent sampling points, obtains the tangent of the angle difference through the relationship between the slope difference and the product, converts the tangent to the angle difference, and uses 1 minus the cosine of the angle difference as the single-segment angle loss line_angle. Finally, it averages the angle losses of all segments to obtain the angle loss of each lane line, and then averages the average across the batch dimension as the final angle loss angeloss.

Images