Lane line detection method based on edge perception attention and continuity enhancement

Abstract

The invention discloses a lane line detection method based on edge perception attention and continuity enhancement. The method comprises the following steps: acquiring a lane line data set and performing primary processing; inputting into a backbone network to obtain multi-level features; analyzing a high-frequency part and a low-frequency part of the feature map by using an edge perception attention module, and outputting a feature map fused with key high-frequency features; converting the feature map from a time domain to a frequency domain by using a continuity enhancement module to obtain global semantic information; and calculating the inclination angles of the predicted lane line and the target lane line through an angle loss function, and iteratively updating the position of the lane line anchor frame to obtain a final lane line prediction sequence. According to the method, an edge perception attention and continuity enhancement module is combined, local semantic information and global semantic information can be accurately captured, and the lane line detection precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a lane line detection method based on edge perception attention and continuity enhancement, and belongs to the technical field of computer vision. BACKGROUND

[0002] With the development of economy, the traffic elements such as vehicles and roads continue to grow rapidly, and how to obtain higher accuracy of lane lines to assist traffic control has become a problem to be solved.

[0003] In the urban road environment, not only various types of traffic participants such as motor vehicles, non-motor vehicles and pedestrians exist, but also various complex scenes such as bad weather, low light and obstacles. These factors bring great challenges to traffic safety and traffic efficiency. The traditional lane line target detection algorithm has low precision when dealing with complex scenes such as unclear lane lines and occluded lane targets. With the development of deep learning, the processing ability of multi-scale and fuzzy lane lines is improved by automatically learning a large number of data features, and the detection accuracy of dynamic lane targets and irregular drivable areas is also improved, gradually replacing traditional methods and becoming the mainstream research direction.

[0004] The lane line detection method based on deep learning has made certain progress in precision and adaptability, but there are still the following problems. First, in low visibility scenes, it usually relies on global semantics and local texture information to predict accurate position and shape, and existing deep models are prone to precision decline in complex scene detection. Secondly, the features of lane lines contain multi-scale edge information, which need to be deeply fused and accurately captured by the model. However, the existing deep model is prone to missing key features when facing lane line wear and road texture similarity. SUMMARY

[0005] The present application aims to solve the above-mentioned problems in the prior art and discloses a lane line detection method based on edge perception attention and continuity enhancement.

[0006] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows:

[0007] A lane line detection method based on edge perception attention and continuity enhancement, characterized by comprising the following steps:

[0008] S1: Obtain a lane line data set, clean the abnormal data, and then input the data into a backbone network to obtain five levels of feature maps, denoted as Fi, where i represents the level of the feature, 3≤i≤5;

[0009] S2: Feature enhancement is performed on the i=3 layer feature map using an edge-aware attention module, which focuses on the extraction of local features, and wavelet transform is used to extract low-frequency components and high-frequency components from the feature map. After extraction, three-dimensional maps are generated, and an attention mechanism is used to generate a weight map which is multiplied with the dimensional map to finally output a key high-frequency feature weight map;

[0010] S3: Feature enhancement is performed on the i=4 and i=5 layer feature maps using a continuity enhancement module, which is designed for the requirement of long-distance continuity of lane lines. The channel is divided into two parts, one part of the feature map extracts local trend information through depth separable convolution, and the other part of the feature map extracts global trend through Fourier transform. Finally, a feature map with global semantic information is output through residual connection with the original feature map;

[0011] S4: After processing the feature map, feature fusion is performed from top to bottom, and then channel splicing is performed on Fi, where i represents the level of the feature, 3≤i≤5;

[0012] S5: After channel splicing, the feature map is passed through a detection head Ha, where a represents the level of the feature, 1≤a≤3;

[0013] S6: The prior box Pb is updated iteratively, where b represents the number of optimization times, 1≤b≤3. The iterative training is performed through an angle loss function, and finally the lane line prediction map is output through confidence screening and NMS de-duplication.

[0014] Further, the edge-aware attention module first uses wavelet kernel to perform convolution on the horizontal direction of the feature map to generate low-frequency components X l and high-frequency components X h Secondly, the low-frequency components X l and high-frequency components X h are multiplied with the wavelet transform convolution in the vertical direction to generate three-dimensional maps LH, HL and HH, where LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, and HH represents the corner and intersection of the lane line. At the same time, the feature map is input into another branch to obtain a weight map through depth separable convolution and attention mechanism. Then, the weight map is down-sampled by row, and the weight map is adjusted to the same size as the three-dimensional map through channel change. Finally, the three-dimensional map and the weight map are multiplied, and then a 1x1 convolution is performed for channel compression. In this process, the important high-frequency feature output feature map is learned through convolution weight, and finally the feature map is restored to the original size through bilinear interpolation.

[0015] Further technical solutions are that the continuity enhancement module is to first align and normalize the features of the feature map, then divide the channels into two parts, one part of the feature map X1 uses depth separable convolution to extract local information, and the output feature map is X a ; the other part of the feature map X2 converts the spatial domain features of the feature map to frequency domain features through Fourier transform rfft2 to obtain X f , wherein the high-frequency component is the local detail of the lane line, and the low-frequency part is the global detail of the lane line, then X f will be multiplied by the complex weight W generated by a modulator to generate a modulated frequency domain feature map X fmod , the frequency domain feature map X fmod is inverse Fourier transformed into a time domain feature map X b ; if the amplitude of the complex weight W is close to 1, the frequency component is retained, and if it is close to 0, the frequency component is suppressed, finally the feature map X a and X b are spliced and fused, and the output feature map is connected with the original input feature to obtain the final feature map X cat .

[0016] Further technical solutions are that the feature map is fused from top to bottom, that is, the feature map after continuity enhancement is fused with the feature map after edge perception attention module enhancement to make the bottom feature map obtain high-level semantic information and increase the resolution of feature mapping.

[0017] Further, the angle loss function is to optimize the model by comparing the difference in the inclination angle of the predicted lane line and the target lane line, specifically, first generate the corresponding y coordinates for the predicted value and the target value, then calculate the slope grad between adjacent sampling points, obtain the tangent value of the angle difference through the relationship between the slope difference and the product, then convert the tangent value to the angle difference, and use 1 minus the cosine value of the angle difference as the single segment angle loss line_angle, finally take the average of the angle losses of all segments to obtain the angle loss of each lane line, and take the average in the batch dimension as the final angle loss angeloss.

[0018] The beneficial effects of the present application are that:

[0019] The application discloses a lane line detection method based on edge perception attention and continuity enhancement. First, the application extracts lane line edges and corner point features through wavelet transform, combines attention mechanism to strengthen key high-frequency information, effectively improves edge detail detection accuracy, and performs better in complex light and shielding scenes. In addition, the application designs a continuity enhancement module, fuses the local trend of depth separable convolution and the global semantics of Fourier transform, and then retains the original features through residual connection, significantly enhances the lane line long-distance continuity modeling capability, and reduces the detection error caused by perspective change or lane line fracture. A top-down feature fusion strategy is adopted to complement the advantages of different level feature maps, retain the bottom details, and integrate high-level semantics to improve the feature expression richness. The optimized angle loss function directly optimizes the lane line inclination angle difference to further improve the shape prediction accuracy, especially in curved lane line scenes, effectively reduces the angle deviation. The overall method cooperates with multiple modules to realize high-precision and strong-robust lane line detection in complex traffic scenes, and provides a reliable environment perception basis for automatic driving path planning and safety decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the application, the application will be further described in detail below with reference to the drawings:

[0021] Figure 1 The network overall flowchart of the embodiments of the application is shown in the figure.

[0022] Figure 2 The network overall architecture diagram of the embodiments of the application is shown in the figure.

[0023] Figure 3 The edge perception attention module structure diagram of the embodiments of the application is shown in the figure.

[0024] Figure 4 The continuity enhancement module structure diagram of the embodiments of the application is shown in the figure. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the examples of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0026] The application provides a lane line detection method based on edge perception attention and continuity enhancement, as shown in the figure, comprising the following steps: Figure 1

[0027] ​S1: Obtain the lane line dataset, clean up abnormal images, construct a ResNet18 backbone network, and then input the feature maps to obtain multi-level features, denoted as Fi, where i represents the feature level, 3≤i≤5; For example... Figure 2 As shown;

[0028] S2: Construct the edge-aware attention module, the structure of which is shown below. Figure 3 ;

[0029] S2-1: The edge-aware attention module performs a wavelet kernel convolution on the feature map in the horizontal direction to generate a low-frequency component X. l and high-frequency component X h The output dimension is Then, the low-frequency component X l and high-frequency component X h Multiplying the wavelet transform convolution in the vertical direction generates LH, HL, and HH dimension maps, with the output dimension being... LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, HH represents the corners and intersections of the lane line, n represents the batch number, c represents the number of channels, h represents the height of the image, and w represents the width of the image. The calculation formula is as follows:

[0030]

[0031] S2-2: The feature map is input into another branch, where a weight map is obtained through depthwise separable convolution and attention mechanisms. This weight map is then downsampled and adjusted to the same size as the three dimensional maps by channel transformation. The three dimensional maps are multiplied by the weight map, and then a 1×1 convolution is used for channel compression. During this process, important high-frequency features are learned through the convolution weights to output a feature map. Finally, bilinear interpolation is used to restore the feature map to its original size, with output dimensions n, c, h, and w. i,j These are the weights of the bilinear interpolation, and f_wavelet represents the feature map recovered by bilinear interpolation. The specific calculation formula is as follows:

[0032]

[0033] S3: Construct the continuity enhancement module, the structure of which is shown below. Figure 4 .

[0034] S3-1: First, perform channel alignment and normalization on the feature map, and then divide the channels into two equal parts.

[0035] S3-2: A portion of the feature map X1 is used to extract local information using depthwise separable convolution, resulting in X. a .

[0036] S3-3: Another part of the feature map X2 is the transformation of the spatial domain features of the feature map into the frequency domain features X. f The high-frequency components represent the local details of the lane lines, while the low-frequency components represent the global trend of the lane lines. Then X... f X is generated by multiplying a modulator with its generated complex weight W. fmod Frequency domain feature map X fmod The time-domain feature map X is obtained by inverse Fourier transform. b If the amplitude of W is close to 1, the frequency component is preserved; if it is close to 0, the frequency component is suppressed. Here, rfft2 represents the two-dimensional real Fourier transform, n represents the batch number, c represents the number of channels, and k... u Represents the vertical frequency, k v The horizontal frequency is represented by W1 and W2, which represent linear transformation layers. GlobalPool() represents global pooling, and sigmoid represents the activation function X. fmod , representing the result of multiplying the frequency matrix by its corresponding matrix weights. Its calculation formula is as follows:

[0037] X f [n,c,k u ,k v ] = rfft2(x2[n,c,:,:]),

[0038] W=sigmoid(W2·ReLU(W1·GlobalPool(x2))),

[0039] Xf mod [n,c,k u ,k v ]=Xf[n,c,k u ,k v ]×W[n,c,k u ,k v ].

[0040] S3-4: The modulated frequency domain feature matrix is ​​resynthesized into spatial domain features through inverse Fourier transform, and the feature map X is obtained by irfft2. a and feature map X enhanced by local path b Perform concat concatenation and fusion, then perform residual concatenation between the output feature map and the original input feature map. Finally, normalize the output feature map again to obtain the final feature map X. cat irfft2 represents the two-dimensional inverse Fourier transform, outputting X. b This represents the matrix that represents the return from the frequency domain to the spatial domain. Its specific calculation formula is as follows:

[0041] X b[n,c,h,w]=irfft2(Xf mod [n,ck u ,k v ]),

[0042]

[0043] S4: Perform feature fusion from top to bottom on the feature maps of Fi and then stitch them together, where i represents the feature level, 3≤i≤5;

[0044] S5: Construct a detection head and determine the final output by iterating the position of the prior box during training.

[0045] S6: Construct an angle loss function, the core of which is to optimize the model by comparing the tilt angle difference between the predicted lane line and the target lane line. Specifically, first, generate corresponding y-coordinates for the predicted and target values; then calculate the slope grad between adjacent sampling points of the two, and obtain the tangent of the angle difference through the relationship between the slope difference and the product; where y i+1 -y i The difference in y-coordinates between adjacent sampling points, x i+1 -x i The x-coordinate difference between adjacent sampling points is denoted by ∈ = 2e-9 to avoid division by zero. The specific calculation formula is as follows:

[0046]

[0047] S6-1: The tangent value is then converted into an angle difference, and the cosine of this angle difference minus 1 is used as the single-segment angle loss line_angle, where k1 is the predicted slope and k2 is the target slope; arctan(tan(θ)) restores the angle difference θ, and 1-cos(θ) ensures that the larger the angle difference, the larger the loss value. The specific calculation formula is as follows:

[0048]

[0049] line-angle=1-cos(arctan(tan(θ))).

[0050] S6-2: Finally, the average angle loss of all segments is taken to obtain the angle loss of each lane line. Then, the average of these averages across the batch dimension is taken as the final angle loss (angeloss). Afterwards, confidence filtering and NMS deduplication are performed to output the lane line coordinates. Here, N is the number of lane lines, and 71 is the number of line segments. The specific calculation formula is as follows:

[0051]

[0052] To verify the effectiveness of the above examples, the performance of this invention was compared with other state-of-the-art methods on two datasets, CULane and TuSimple. The experimental results on the CULane dataset used eleven metrics: F1... 50 The ten metrics are: Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, Cross, and Night. Of these, only Cross is considered a valid F1 metric. 50 For the Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, and Night metrics, higher values ​​indicate better performance. Three metrics were selected for the TuSimple dataset: F1, Accuracy, and FP. Except for FP, higher values ​​for F1 and Accuracy generally indicate better performance.

[0053] As can be seen from the experimental results in Table 1, the proposed method outperforms existing methods in multiple metrics on the CULane dataset. The method proposed in this embodiment scores 81.20 on the CULane dataset, which is 1.07 points higher than the state-of-the-art CLRNet. Furthermore, it shows improvements in three challenging scenarios: Noline, Curve, and Night, demonstrating the effectiveness of the proposed method.

[0054] Table 1 shows the comparison results of detection accuracy on the CULane dataset.

[0055]

[0056] Table 2 shows the comparison results of detection accuracy on the TuSimple dataset.

[0057] Method Backbone F1(％) Acc(%) FP(%) SCNN VGG16 95.97 96.53 6.17 RESA ResNet34 96.93 96.82 3.63 PolyLaneNet EfficientNetB0 90.62 93.36 9.42 UFLDv2 ResNet34 88.02 95.86 18.91 CondLaneNet ResNet34 96.77 95.37 2.20 FOLOLane ERFNet 96.59 96.92 4.47 CANet ResNet34 97.77 96.76 1.92 CLRNet ResNet18 97.89 96.84 2.28 OURS ResNet18 98.13 96.89 1.78

[0058] As shown in Table 2, the experimental results of this embodiment outperform existing methods in F1 score on the TuSimple dataset. The method of this embodiment surpasses the current state-of-the-art level by 0.24% in F1 score, proving the effectiveness of the method of this embodiment.

Claims

1. A lane line detection method based on edge-aware attention and continuity enhancement, characterized in that... Includes the following steps: S1: Obtain the lane line dataset, clean up the abnormal data, and then input the data into the backbone network to obtain a five-level feature map, denoted as Fi, where i represents the feature level, 3≤i≤5; S2: The edge-aware attention module is used to enhance the features of the i=3 layer feature map. This module focuses on the extraction of local features. Wavelet transform is used to extract low-frequency and high-frequency components from the feature map. After extraction, three dimensional maps are generated. The attention mechanism is used to generate a weight map and multiply it with the dimensional map to finally output the key high-frequency feature weight map. S3: The continuity enhancement module is used to enhance the features of the i=4 and i=5 layers. This module is designed to meet the long-distance continuity requirements of lane lines. The channel is divided into two parts. The feature map of one part is extracted with local trend information through depthwise separable convolution, and the feature map of the other part is extracted with global trend through Fourier transform. Finally, the feature map with global semantic information is output by residual connection with the original feature map. S4: After the processed feature map is fused from top to bottom, the channels of Fi are concatenated, where i represents the feature level, 3≤i≤5; S5: After the channels are stitched together, the feature map is passed through the detection head Ha, where a represents the feature level, 1≤a≤3; S6: The prior box Pb is continuously updated iteratively, where b represents the number of optimizations, 1≤b≤3. Iterative training is performed using the angle loss function, and finally, lane line prediction maps are output through confidence filtering and NMS deduplication.

2. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The edge-aware attention module first performs wavelet kernel convolution on the horizontal direction of the feature map to generate a low-frequency component X. l and high-frequency component X h Secondly, the low-frequency component X l and high-frequency component X h The system generates three dimensional maps (LH, HL, and HH) by multiplying the feature map with wavelet transform convolution in the vertical direction. Here, LH represents the vertical edge of the lane line, HL represents the horizontal edge of the lane line, and HH represents the corners and intersections of the lane line. Simultaneously, the feature map is input into another branch, where a weight map is obtained through depthwise separable convolution and attention mechanism. Then, the weight map is downsampled and adjusted to the same size as the three dimensional maps by channel transformation. Finally, the three dimensional maps are multiplied with the weight map and then channel compression is performed by a 1×1 convolution. During this process, important high-frequency features are learned through the convolution weights to output the feature map. Finally, the feature map is restored to its original size by bilinear interpolation.

3. The lane detection method based on an edge-aware attention module and continuity enhancement as described in claim 1, characterized in that, The aforementioned continuity enhancement module first performs channel alignment and normalization on the feature map, then divides the channels into two equal parts. One part, feature map X1, is used to extract local information using depthwise separable convolution, and the output feature map is X. a The other part of the feature map X2 is transformed from the spatial domain features to the frequency domain features through Fourier transform rfft2 to obtain X. f The high-frequency components represent local details of the lane lines, while the low-frequency components represent global details of the lane lines. Then X... f The modulated frequency domain feature map X is generated by multiplying the modulator with its generated complex weight W. fmod Frequency domain feature map X fmod The time-domain feature map X is obtained by inverse Fourier transform. b If the magnitude of the complex weight W is close to 1, the frequency component is retained; if it is close to 0, the frequency component is suppressed. Finally, the feature map X is... a and X b Channel concat fusion is performed, and the output feature map is residually concatenated with the original input feature map to obtain the final feature map X. cat .

4. The lane detection method based on edge-aware attention and continuity enhancement as described in claim 1, characterized in that, The top-down feature map fusion involves adjusting the size of the feature map after continuity enhancement and fusing it with the feature map enhanced by the edge-aware attention module. This allows the lower-level feature map to acquire higher-level semantic information, increasing the resolution of the feature mapping.

5. The lane line detection method based on an edge-aware attention module and continuity enhancement as described in claim 1, characterized in that, The aforementioned angle loss function optimizes the model by comparing the tilt angle difference between the predicted lane line and the target lane line. Specifically, it first generates corresponding y-coordinates for the predicted and target values, then calculates the slope grad between adjacent sampling points, obtains the tangent of the angle difference through the relationship between the slope difference and the product, converts the tangent to the angle difference, and uses 1 minus the cosine of the angle difference as the single-segment angle loss line_angle. Finally, it averages the angle losses of all segments to obtain the angle loss of each lane line, and then averages the average across the batch dimension as the final angle loss angeloss.

Images