An Enhanced Single-Level Lane Detection Method for Autonomous Driving

Abstract

This invention relates to the field of deep learning object detection technology, specifically to an enhanced single-level lane detection method for autonomous driving, comprising the following steps: First, general features are extracted through a pre-trained backbone network. Then, the receptive field information of the corresponding original image on the feature map is enhanced by a Dilated Encoder module. Based on the set line-anchor position information, the position of the corresponding line-anchor on the feature map is found and its features are extracted. These extracted features are then fused with global lane feature information through a feature fusion module that introduces a self-attention mechanism. Finally, the lane detection result is output by three fully connected prediction branches. During the model training phase, the line-anchor is fine-tuned using the gradient information backpropagated from the loss. This invention proposes a new network structure, adopting a single-level network structure design. By improving the network structure and feature fusion module, and combining a multi-head self-attention mechanism, the method can better utilize global lane information, significantly improving the accuracy of lane detection. The lane lines are modeled as line-anchors, and a learnable line-anchor strategy is adopted. Simultaneously, the improved objective function enables faster network convergence.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based target detection technology, and more specifically to an enhanced single-level lane detection method for autonomous driving. Background Technology

[0002] Deep learning technology has made a series of advancements in the field of autonomous driving. Lane detection, as a crucial subsystem in autonomous driving systems, is essential for achieving autonomous driving with accurate and fast lane detection.

[0003] Currently, lane detection methods can be mainly categorized into segmentation-based methods, line-anchor-based methods, row classification-based methods, and parameter-based methods. In segmentation-based methods, lane lines are predicted pixel-by-pixel, determining whether an image pixel belongs to the lane line or background information. Line-anchor-based prediction methods employ an anchor mechanism, but unlike the rectangular anchors used in traditional object detection, lane detection uses line anchors to fit the lane line shape. Row classification-based methods predict the positional information of lane lines within each row. Parameter-based methods directly model the parameters of the lane lines.

[0004] However, existing detection methods suffer from low detection accuracy and consume a lot of computing resources, making it difficult to meet the detection needs in complex scenarios. Summary of the Invention

[0005] To enhance detection capabilities in complex scenes, this invention employs a single-level network structure design. By improving the network structure and feature fusion module, and incorporating a multi-head self-attention mechanism, the method can better utilize global lane line information. Simultaneously, the objective function is improved, enabling the network to converge faster.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An enhanced single-level lane detection method for autonomous driving includes the following steps:

[0008] First, common features are extracted through a pre-trained backbone network. Then, the receptive field information of the corresponding original image on the feature map is enhanced by the Dilated Encoder module (residual connection construction dilation module). Based on the set line-anchor position information, the position on the corresponding feature map is found and the features are extracted. These extracted features are better integrated with global lane line feature information by introducing a feature fusion module with a self-attention mechanism. Finally, the lane line detection results are output by three fully connected prediction branches.

[0009] Furthermore, more specifically, it includes the following steps:

[0010] Step 0: Initialization of the detection method: including network initialization and image preprocessing;

[0011] Step 1: Input is image data. Feature extraction is performed on the preprocessed image to obtain the feature map after 32x downsampling.

[0012] Step 2: The features extracted by the backbone network are further encoded by the Dilated Encoder to extract target information from different receptive fields. The Dilated Encoder module consists of four cascaded dilated convolutions with dilation rates of 2, 4, 6, and 8 times, respectively, so that the cascaded network can also obtain feature information from receptive fields of different scales.

[0013] Step 3: Based on the pre-set line-anchor information on the original image, obtain its position information on the 32x downsampled feature map and extract the corresponding features, that is, the features of each line-anchor. Then, through the anchor feature aggregation module based on the attention mechanism, fully learn the global lane line information.

[0014] Step 4: Predict using a linear layer to output the lane line's angle, length, and starting point coordinates;

[0015] Step 5: Calculate the loss: Calculate the loss between the predicted target and the actual value, including the loss of the lane line starting point coordinates, the lane line length, and the angle.

[0016] Furthermore, in Step 0, the backbone network uses pre-trained ResNet weights.

[0017] Furthermore, the image preprocessing in Step 0 includes image scaling, random horizontal flipping, and random affine transformation.

[0018] Furthermore, in Step 1, ResNet50 is used as the backbone network.

[0019] Furthermore, the calculation formula in Step 5 is as follows:

[0020] L all =λ1L start_xy +λ2L length +λ3L score +λ3L angle (1-1)

[0021] Where λ1L start_xy and L lengthThe losses are calculated based on the x and y coordinates of the lane starting point, and the lane length loss, respectively. The smooth L1 algorithm is used to calculate the losses. score The lane line confidence loss is calculated using Focal loss, L. angle The angle loss proposed in this invention is calculated using the formula shown in (1-2).

[0022]

[0023] N proposal This represents the number of points that make up a complete lane line, and α and β represent the angles between the predicted and actual line segments, respectively.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. This invention proposes a new network structure, which adopts a single-level network structure design. By improving the network structure and feature fusion module and combining a multi-head self-attention mechanism, the method can better utilize global lane line information, which can significantly improve the accuracy of lane line detection. At the same time, the improved objective function enables the network to converge faster.

[0026] 2. This invention improves the loss function by adding an angle regression loss, enabling the method to achieve rapid convergence within 5000 iterations. Furthermore, under the same number of training rounds, this method exhibits better convergence performance.

[0027] 3. This method introduces a learnable line-anchor strategy, which continuously fine-tunes the shape of the line-anchors during training, achieving better detection accuracy with fewer line-anchors. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0029] Figure 1 This is a flowchart of the algorithm training process of the present invention.

[0030] Figure 2 This is a network structure diagram of the single-level lane line detection method of the present invention.

[0031] Figure 3 The results are shown as the baseline network and the detection results using this algorithm.

[0032] Figure 4 This is a comparison chart of the loss curves. Detailed Implementation

[0033] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0034] The lane detection method proposed in this invention is a line-anchor based method. The network part adopts an end-to-end single-stage detection architecture, and the overall structure diagram is shown below. Figure 2 As shown. The detection system accepts RGB image data. First, it extracts general features through a pre-trained backbone network. The backbone network adopts a ResNet series structure. Then, the Dilated Encoder module enhances the receptive field information of the original image on the feature map. Based on the set line-anchor position information, it finds the position on the corresponding feature map and extracts features. These extracted features are then fused into a feature fusion module with a self-attention mechanism to better integrate global lane line feature information. Finally, the lane line detection result is output by three fully connected prediction branches. The specific detection method is as follows: Figure 1 As shown, it includes the following steps:

[0035] Step 0: Initialization of the detection method. This includes network initialization and image preprocessing. The backbone network uses pre-trained ResNet weights, and the image preprocessing part includes image scaling, random horizontal flipping, and random affine transformation.

[0036] Step 1: The input is image data. In this paper, ResNet50 is used as the backbone network to extract features from the preprocessed image, resulting in a feature map after 32x downsampling.

[0037] Step 2: The features extracted by the backbone network are further encoded by the Dilated Encoder to extract target information from different receptive fields. The Dilated Encoder module consists of four cascaded dilated convolutions with dilation rates of 2, 4, 6, and 8, respectively, enabling the cascaded network to obtain feature information from receptive fields of different scales.

[0038] Step 3: Based on the pre-set line-anchor information on the original image, obtain its position information on the 32x downsampled feature map and extract the corresponding features, that is, the features of each line-anchor. Then, through the anchor feature aggregation module based on the attention mechanism, fully learn the global lane line information.

[0039] Step 4: Predict using a linear layer to output the lane line's angle, length, and starting point coordinates.

[0040] Step 5: Calculate the losses. The losses are calculated by comparing the predicted target with the actual value, including the lane line starting point coordinate loss, lane line length loss, and angle loss. The calculation formula is shown in (1-1).

[0041] L all =λ1L start_xy +λ2L length +λ3L score +λ3L angle (1-1)

[0042] Where λ1L start_xy and L length The losses are calculated based on the x and y coordinates of the lane starting point and the lane length, respectively, using a smooth L1 algorithm. score Focal loss is used to calculate the lane line confidence loss. angle The angle loss proposed in this paper is calculated using the formula shown in (1-2).

[0043]

[0044] N proposal This represents the number of points that make up a complete lane line, and α and β represent the angles between the predicted and actual line segments, respectively.

[0045] Example: Lane line detection experiment

[0046] 1) Comparison of test results

[0047] This experiment uses the Tusimple lane detection dataset for both training and testing. Tusimple, proposed in 2017, is a lane detection dataset that includes complex weather scenarios and various traffic conditions. The training set consists of 3626 video clips and 3626 annotated frames, while the test set includes 2782 video clips. Each clip contains 20 frames of image data, with each frame measuring 1280*720 pixels.

[0048] like Figure 3 As shown, the left column represents the original image, the middle column represents the detection results of the baseline network, and the right column represents the detection results of our proposed method. The comparison reveals that in the first image, our method can detect all four lane lines, with better detection at the bend in the second lane line. In the third image, the third lane line is discontinuous and blurry, but our method still effectively detects it. Test data on Tusimple and actual detection results strongly demonstrate the superior detection performance of our proposed method.

[0049] like Figure 4 As shown, the left figure is the loss curve of our proposed method, and the right figure is the loss curve of the baseline method. By improving the loss function and adding angular regression loss, our proposed method can achieve fast convergence in 5000 iterations. Moreover, with the same number of training epochs, our proposed method achieves better convergence.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An enhanced single-level lane detection method for autonomous driving, characterized in that: Includes the following steps: Step 0: Initialization of the detection method: including network initialization and image preprocessing; Step 1: Input is image data. Feature extraction is performed on the preprocessed image to obtain the feature map after 32x downsampling. Step 2: The features extracted by the backbone network are further encoded by the Dilated Encoder to extract target information from different receptive fields. The Dilated Encoder module consists of four cascaded dilated convolutions with dilation rates of 2, 4, 6, and 8 times, respectively, so that the cascaded network can also obtain feature information from receptive fields of different scales. Step 3: Based on the pre-set line-anchor information on the original image, obtain its position information on the 32x downsampled feature map and extract the corresponding features, that is, the features of each line-anchor. Then, through the anchor feature aggregation module based on the attention mechanism, fully learn the global lane line information. Step 4: Predict using a linear layer to output the lane line's angle, length, and starting point coordinates; Step 5: Calculate the loss: By calculating the loss between the predicted target and the actual value, the losses include lane line start point coordinate loss, lane line length loss, and angle loss; the calculation formula is: Where L start_xy and L length The losses are calculated based on the x and y coordinates of the lane starting point, and the lane length loss, respectively. The smooth L1 algorithm is used to calculate the losses. score The lane line confidence loss is calculated using Focal loss, L. angle The angle loss is calculated using the formula shown in (1-2). N proposal This represents the number of points that make up a complete lane line, and α and β represent the angles between the predicted and actual lane segments, respectively.

2. The enhanced single-level lane detection method for autonomous driving as described in claim 1, characterized in that: In Step 0, the backbone network uses pre-trained ResNet weights.

3. The enhanced single-level lane detection method for autonomous driving as described in claim 1, characterized in that: The image preprocessing in Step 0 includes image scaling, random horizontal flipping, and random affine transformation.

4. The enhanced single-level lane detection method for autonomous driving as described in claim 1, characterized in that: In Step 1, ResNet50 is used as the backbone network.

Images