A deformable lane line detection method based on spatial prior guidance

Abstract

The present application relates to the technical field of computer vision and automatic driving, and discloses a deformable fine lane line detection method based on spatial prior guidance, which receives a video stream signal for preprocessing; extracts features by using a backbone network; after a multi-axis gate attention unit MGAC is integrated into a top transverse convolution layer based on an FPN, high-level features are processed by the MGAC and then multi-scale fusion is performed in the FPN; low-level feature maps output by the FPN are processed by using an SPG to form a spatial prior guidance map; the spatial prior guidance map is used to predict coordinate offsets of sampling points in a deformable lane line perception aggregation module; offsets are used to perform irregular dynamic sampling on middle and high-level features output by a feature pyramid; and the final lane line detection result is output by feeding into a prediction head. Compared with the prior art, the present application has higher robustness in a complex scene, greatly improves detection accuracy, and fully meets the stringent real-time requirements of an automatic driving system.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and autonomous driving technology, specifically to a deformable and refined lane line detection method based on spatial prior guidance. Background Technology

[0002] Current similar techniques for lane detection are anchor-based methods, such as LineCNN and LaneATT. These methods pre-set a large number of linear anchors and regress the offset between the sampling points and the pre-defined anchors. They then use NMS to select the lane with the highest confidence. CLRNet, on the other hand, reduces the number of pre-set linear anchors and proposes learnable anchors to further improve detection accuracy.

[0003] However, in real-world road environments, lane markings have a distinctly elongated structure and are highly susceptible to external environmental interference. For example, in complex scenarios such as nighttime driving, glare from strong lights, traffic congestion, or shadows, the visual characteristics of lane markings often become blurred, discontinuous, or even completely invisible.

[0004] The main problems are as follows: First, the receptive field is limited, and conventional convolutions are unable to effectively capture long-distance lane line dependencies that span the entire image; second, the sampling method is rigid, and traditional convolutional kernels use fixed grid sampling, which cannot adaptively fit the non-rigid curved geometry of lane lines, resulting in insufficient feature extraction; finally, there is insufficient utilization of low-level information, and existing algorithms rely too much on high-level semantic features, often ignoring the rich spatial details contained in low-level feature maps. This makes it difficult for the model to use residual texture clues for inference and completion when faced with local occlusion, ultimately leading to missed detections or localization errors. Summary of the Invention

[0005] Purpose of the invention: To address the problems existing in the prior art, the present invention provides [the solution / solution].

[0006] Technical solution: This invention provides a deformable and refined lane line detection method based on spatial prior guidance, comprising the following steps:

[0007] Step 1: Receive the video stream signal, extract the video stream signal into a public dataset of images, and preprocess the images;

[0008] Step 2: Extract features from the dataset using the backbone network and output feature maps at three different scales;

[0009] Step 3: Construct a multi-scale fusion module. Based on FPN, after integrating the multi-axis gated attention unit MGAC into the top-level horizontal convolutional layer, the high-level features are processed by MGAC and then fused at multiple scales in FPN to generate feature maps of three different scales. , , ;

[0010] Step 4: Use the Spatial Prior Generator (SPG) to process the low-level feature map output by the FPN, generate a heat map in an unsupervised manner, and use the CAM mechanism to generate the required heat map. The input feature map is fed into the hm heat map branch and the shape branch for processing, generating a class activation heat map representing position information and a shape map representing geometric edges, respectively. The two are then spliced ​​together to form a spatial prior guide map.

[0011] Step 5: Enter the Deformable Lane Line Perception Aggregation Module (DLAM), and use the spatial prior guidance map generated in Step 4 to predict the coordinate offset of the sampling point; use the offset to perform irregular dynamic sampling of the mid-to-high-level features output by the feature pyramid.

[0012] Step 6: The feature map passed through the deformable lane line perception aggregation module is fed into the prediction head, and the final lane line detection result is output.

[0013] Furthermore, in step 2, the backbone network uses ResNet18 or DLA34 to generate features at three scales: .

[0014] Furthermore, the multi-axis gated attention unit (MGAC) employs multi-directional depthwise convolution to simulate contextual receptive fields in different directions, thus processing the input feature map. The image is segmented by channel number, processed through multi-directional convolution, and then stitched together. Simultaneously, the feature maps are input into an attention unit. Oriented separable convolution enhances the response to elongated structures in the image. Finally, the resulting feature maps are summed to output the final feature map. In FPN, feature maps After upsampling and feature map The fusion yields a new feature map Feature map After upsampling and feature map The fusion yields a new feature map ,in , , These are the three scale features output by the backbone network.

[0015] Furthermore, the generated feature maps are fed into the Deformable Lane Awareness Aggregation Module (DLAM) and the Spatial Prior Generator (SPG), respectively. and The data is fed into the deformable lane line perception aggregation module (DLAM) for processing. Then it is sent to the Spatial Prior Generator (SPG) for processing.

[0016] Furthermore, the spatial prior generator SPG includes an hm heatmap branch and a shape branch:

[0017] In the heatmap branch, the prior information of the location, i.e. the existence region, is first extracted through the convolutional layer. Then, the feature map of the existence region is fed into the liner layer to calculate the weight value, and the weight is converted into the form of convolutional kernel. Each class corresponds to a set of (D,1,1) convolutional kernels. Finally, the heatmap is obtained through convolution operation and activation function.

[0018] The shape branch is obtained by performing two convolutional layers and ReLU activation operations to obtain the heatmap of the shape branch, which is the prior geometric information of the lane line. Then, the two heatmaps are stitched together to obtain the final guide map GM.

[0019] Furthermore, the deformable lane line perception aggregation module DLAM uses the guidance map GM and feature map... , As input, the DGA module uses deformable sampling and multi-head attention mechanisms to enable the model to focus on important regions and extract global contextual information;

[0020] The DGA module uses the guide map GM generated by SPG as the basis for prediction and guidance. It generates offsets through an offset generation network, normalizes the offsets and adds them to the reference points to obtain the final sampling positions. Then, it samples the feature map and maps the features to the corresponding q, k, v vectors. It outputs the refined features through a multi-head attention mechanism and finally obtains the refined features through flattening.

[0021] First, combine R0 and feature map In the input feature enhancement part, R0 is the initial anchor parameter, and parameters are uniformly selected on R0. The prior points of each lane line are used to obtain RefindR0; the feature enhancement part uses convolution and fully connected layers to align the region of interest to obtain new features of interest, which are then combined with the feature map. The feature maps processed by feature mapping are used to calculate the similarity between them through a scaling dot product attention mechanism to obtain an attention weight map. This weight map is then multiplied with the features refined by the DGA module to obtain the global features. The global features are then added to the new features of interest to obtain the final enhanced features of interest.

[0022] RefindR0 and feature map The input feature enhancement part undergoes the same operation;

[0023] The final output is the enhanced feature of interest. and .

[0024] Furthermore, the enhanced features of interest and A unified loss is calculated. Line IOU is used for regression of the offset of the predicted lane line horizontal sampling points, L1 loss is used for regression of the anchor line starting point coordinates, anchor line angle, and anchor line length, and Focal Loss is used for classification. The total loss function is as follows:

[0025] ;

[0026] in, It is the focus loss between prediction and label. It is the L1 loss of the regression of starting coordinates, angle, and lane length. It predicts the Line IoU loss between the lane and the actual ground conditions. , , These represent the weight values ​​of the corresponding loss functions.

[0027] Beneficial effects:

[0028] Firstly, the technical solution of this invention discloses a method for explicitly guiding high-level features to perform deformable sampling using a class-activated heatmap generated from low-level features. This method can fully exploit the spatial prior information contained in low-level features, enhance the complementary advantages of multi-scale features, and effectively compensate for the loss of spatial details caused by multiple downsampling of high-level features. Furthermore, the class-activated heatmap, as a clear spatial constraint, can guide deformable sampling points to accurately fit the topology of lane lines, prevent the divergence of sampling offsets, and effectively filter interference from complex background noise such as road shadows and reflections. Through this explicit guidance mechanism, features of different scales are integrated, preserving rich semantic information while compensating for the loss of accuracy in spatial positioning, thereby achieving precise positioning of lane lines.

[0029] Secondly, lane lines are long-distance structures spanning the entire image, and a small receptive field can only see local segments. To address this, a multi-axis gated attention unit is proposed. The specific construction of this multi-axis gated attention unit, particularly its technique of segmenting feature channels and performing parallel horizontal, vertical, and diagonal large-kernel convolutions to capture dependencies in slender structures, is described. Through large-kernel strip convolutions and multi-directional depth convolutions, the receptive field of different directions is simulated, which is beneficial for capturing the horizontal, vertical, and diagonal structures of lane lines, improving the model's contextual awareness and ability to capture long-distance dependencies, thereby enhancing the model's detection performance.

[0030] Thirdly, the spatial prior generator of this invention features a dual-branch architecture, which simultaneously includes a location heatmap branch and a geometry branch, and synthesizes a guidance map in an unsupervised manner. This guidance map does not directly participate in classification but serves as guiding information for subsequent dynamic sampling. Utilizing rich lane line spatial prior knowledge (such as continuity and approximate parallelism) from the low-level feature map, a heatmap with spatial distribution awareness and structural prior is generated to assist the model in achieving accurate localization in occluded or blurred areas.

[0031] Fourth, considering that lane lines are sparsely distributed and occupy only a small area of ​​the image, traditional convolutional sampling is fixed and difficult to align with non-rigid structures, resulting in incomplete capture of lane line details. Therefore, a deformable lane line perception aggregation module is proposed. In this invention, the deformable aggregation module uses a guided graph to predict offsets and combines a multi-head attention mechanism to perform global context enhancement on the sampled features. The offsets are used to perform irregular dynamic sampling of the mid- and high-level features output by the feature pyramid. This process allows the network to focus on the high-probability areas indicated by the heatmap, rather than blindly extracting features on a fixed grid. This enables the network to adaptively fit the curvature of the lane lines and, combined with the multi-head attention mechanism, complete the layer-by-layer refinement of features. Attached Figure Description

[0032] Figure 1 This is a flowchart of the deformable and refined lane line detection method based on spatial prior guidance of the present invention.

[0033] Figure 2 This is a block diagram of the SPDRNet model of the present invention;

[0034] Figure 3 This is a structural block diagram of the MGAC module of the present invention;

[0035] Figure 4 This is a structural block diagram of the spatial prior generator SPG of the present invention;

[0036] Figure 5 This is a structural block diagram of the deformable lane line sensing aggregation module (DLAM) of the present invention;

[0037] Figure 6 This is a schematic diagram illustrating the visualization results of different models of the present invention in different complex scenarios. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0039] This invention provides a spatial prior-guided deformable refinement for accurate lane detection method, SPDRNet (Spatial Prior-Guided Deformable Refinement for Accurate Lane Detection). Its core lies in constructing an end-to-end deep neural network, enhancing feature extraction through a multi-axis attention mechanism, and utilizing spatial prior information from low-level features to guide dynamic deformation sampling of high-level features. The workflow is as follows: Figure 1 As shown.

[0040] Step 1: Download the video stream signal received from the camera, extract images from the video stream to create a publicly available dataset, and preprocess the images. Preprocessing steps include: randomly cropping images by randomly selecting a cropping region from each input image to enhance the randomness of the data; randomly rotating images to simulate rotation transformations and increase the model's insensitivity to rotation; randomly adding Gaussian blur to a portion of the images in the image set to simulate image blurring; and randomly horizontally flipping images with a 50% probability to further enhance the model's insensitivity to horizontal flipping.

[0041] Step 2: Use ResNet or DLA networks to construct the backbone network to extract features, outputting three feature maps at different scales so that subsequent networks can perform multi-scale fusion of the feature maps.

[0042] Step 3: Construct a multi-scale fusion module to fuse features extracted from the backbone network at different scales. Based on the traditional FPN, an improvement is made. Considering the significant spatial dimensional differences of lane lines (length much greater than width), the traditional FPN structure is weak in capturing slender targets, has poor context awareness, and is poor at capturing long-distance dependencies. Therefore, a multi-axis gated attention cell (MGAC) is integrated into the top-level horizontal convolutional layer. High-level features are processed by MGAC before multi-scale fusion in the FPN. Through large-kernel strip convolution and multi-directional depth convolution, the receptive field of different directions is simulated, which is beneficial for capturing the horizontal, vertical, and diagonal structures of lane lines, improving the model's context awareness and ability to capture long-distance dependencies, thereby improving the model's detection performance.

[0043] Step 4: The Spatial Prior Generator (SPG) processes the low-level feature map output by the FPN, generating heatmaps in an unsupervised manner. It utilizes the CAM (Class Activation Mapping) mechanism to generate the required heatmaps. The input feature map is fed into the hm (heat map) branch and the shape branch for processing, generating a class activation heatmap representing location information and a shape map representing geometric edges, respectively. These two heatmaps are concatenated to form a spatial prior guide map. This guide map does not directly participate in classification but serves as guiding information for subsequent dynamic sampling.

[0044] Step 5: The Deformable Lane-aware Aggregation Module (DLAM) network uses the spatial prior guidance map generated in Step 4 to predict the coordinate offset of the sampling points. Subsequently, this offset is used to perform irregular dynamic sampling of the mid- and high-level features output from the feature pyramid. This process allows the network to focus on high-probability areas indicated by the heatmap, rather than blindly extracting features on a fixed grid. This enables the network to adaptively fit the curvature of the lane lines and, combined with a multi-head attention mechanism, refine the features layer by layer. Furthermore, the system selects several lane line prior points, extracts features from regions of interest, and calculates their attention relationship with the overall image features to further supplement local details. The features processed in the above way are finally fed into the classification and regression heads to calculate the specific parameters of the lane lines.

[0045] Step 6: The feature map from the deformable lane line perception aggregation module is fed into the prediction head. The classification branch predicts whether a pixel belongs to a lane line, while the regression branch predicts the specific geometric parameters of the lane line. Finally, combining the classification confidence and geometric parameters, a loss function is calculated for training, or the final lane line detection result is output after non-maximum suppression (NMS) during the inference stage.

[0046] The specific implementation method is as follows:

[0047] Step 1: Download the publicly available datasets CULane and TuSimple. Since the authors of TuSimple did not provide segmentation images, you need to generate them from the JSON annotations. Run the command `python tools / generate_seg_tusimple.py –root tusimple`, which will generate a `seg_label` folder under the `tusimple` folder. This completes the dataset preparation.

[0048] Step 2: Construct a deformable, refined lane line detection network based on spatial prior guidance, such as... Figure 2 As shown in the diagram. The backbone network uses ResNet18 or DLA34. The input image is preprocessed to reduce its size to 320×800. The preprocessed image is then fed into the backbone network for further processing, generating feature maps at three different scales.

[0049]

[0050] Step 3: Construct a multi-scale fusion module. Integrate the multi-axis gated attention unit (MGAC) into the top-level lateral convolutional layer, i.e., place it... In front of the layer. Feature map It will first be processed through the MGAC module, the module structure of which is as follows: Figure 3 As shown, Representation of feature map Thus, input, out represents the generated new feature map. This module employs multi-directional depthwise convolution to simulate receptive fields in different directions, which is beneficial for capturing the lateral, vertical, and diagonal structures of lane lines. The input feature map is segmented according to the number of channels, processed through multi-directional convolution, and then concatenated. Simultaneously, the feature map is input into the attention unit, where directional separable convolution enhances the response to elongated structures (such as lane lines) in the image. Finally, the resulting feature maps are summed and output. This process can be represented by the following formula:

[0051]

[0052]

[0053] In the formula, split means segmenting the feature map by channel, DW means depthwise separable convolution, the subscript k means the size of the convolution kernel is 11, Concat means concatenating by channel, Normal means normalization layer, MLP means multilayer perceptron, and means element-wise multiplication.

[0054] The feature map obtained at this time After upsampling and feature map The fusion yields a new feature map Similarly, feature maps This is how it is calculated, therefore the calculation formula can be uniformly expressed as follows: The process can be represented by the following formula:

[0055]

[0056] In the formula, Upsample means upsampling the image by a factor of 2, i.e., doubling its size. This step generates three feature maps at different scales:

[0057]

[0058] The generated feature maps will be fed into the Deformable Lane Awareness Aggregation (DLAM) module and the Spatial Prior Generator (SPG) respectively. and Send to DLAM processing, Then it will be sent to SPG for processing.

[0059] Step 4, the final result obtained after step 3 SPG (Spatial Projection Generation) is used to generate a heatmap with spatial distribution perception and structural priors, which serves as the basis for offset prediction in the deformable lane line perception aggregation module. The structure diagram of SPG is shown below. Figure 4 As shown, the input feature map is fed into the heat map branch (hm) and the shape branch for processing. In the heat map branch, prior location information, i.e., the existence region, is first extracted through convolutional layers. This can be expressed by the formula:

[0060]

[0061] Where B represents the batch quantity and D represents the number of channels as 256. The input feature map represents the low-level L2 feature map, and ReLU represents the activation function. Then the feature map... The weight values ​​are fed into the Liner layer for calculation. The weights are then converted into convolutional kernels, with each class corresponding to a set of (D,1,1) convolutional kernels. These can be understood as the attention weights for that class across all channels, and can be expressed by the formula:

[0062]

[0063] in Represents the identity matrix. Represents tensor product, , `classes` represents a class with a value of 1. Finally, after convolution and activation functions, the final heatmap is obtained, expressed as follows:

[0064]

[0065] in, This represents the Sigmoid activation function, and * represents channel-wise convolution. For shape branches, two convolutional layers and ReLU activation operations are used to obtain the heatmap of the shape branches, i.e., the prior geometric information of the lane lines. Then, the two heatmaps are stitched together to obtain the final guidance map. The process can be represented by the following formula:

[0066]

[0067]

[0068] in Concat indicates concatenation by channel. .

[0069] Step 5: The guide map GM obtained after step 4, and the feature map obtained in step 3. and It will then be processed in the Deformable Lane Aspect Recognition Aggregation (DLAM) module. The structure diagram of DLAM is as follows: Figure 5 As shown.

[0070] DLAM utilizes deformable sampling and multi-head attention mechanisms via DGA to enable the model to focus on important regions and extract global contextual information. Using the heatmap GM generated by SPG as the basis for prediction and guidance, an offset network generates offsets, which are then normalized and added to the reference point to obtain the final sampling position. The feature map is then sampled. This can be expressed by the formula:

[0071]

[0072] Where offset is the offset generation network, tanh is the hyperbolic tangent function, and dk is the scaling factor. Representing different levels of features, i=0 or 1, RefGrid indicates generating a uniformly distributed standard sampling grid, and GridSample indicates obtaining the feature map dynamically based on deformation points using the grid_sample function. Then, the features are mapped to the corresponding q, k, v vectors.

[0073]

[0074] Then, the refined features are output through a multi-head attention mechanism:

[0075]

[0076] Finally, the features are refined through flattening. (i=0,1).

[0077] At this point, the DGA operations are complete, and the extracted ROI features are enhanced. R0 and L0 are then used together as input to the feature enhancement section, such as... Figure 2 and Figure 5 As shown, R0 is the initial anchor line parameter. This represents 192 sampling points for the preset anchor line, and the three parameters are the coordinates (x, y) and angle of the lane line, respectively. First, uniformly select points on R0. The process involves extracting lane line priors, which is known as ROIAlign extraction. This leads to RefindR0, a process consistent with the CLRNet algorithm. Then, convolutional and fully connected layers are used to align the regions of interest (ROIs) to obtain new features of interest. (i=0,1), and combine it with the feature map processed by feature mapping. The similarity between the two features is calculated using a scaled dot product attention mechanism to obtain an attention weight map. This map is then multiplied with the refined features to obtain the global features. Finally, this global feature is added to the new features of interest to obtain the final enhanced features of interest. (i=0,1). Right now Figure 2 DLAM output and The output parameters represent the following meanings: Indicates the coordinates of the starting point of the anchor line. Indicates the angle of the anchor line. Indicates the length of the anchor line. This indicates the offset of the horizontal sampling point of the anchor line. This represents the probability of lane lines and background.

[0078]

[0079] from Figure 5 See, counting from top to bottom, the first red arrow represents the feature map, which is L0 or L1, and the second red arrow represents R0 or RefindR0. Figure 2 As can be seen from this, because it involves layer-by-layer refinement, L0 and R0 are first input, and P0 is output after processing. When it reaches the second layer, the ROI extracted by the first layer ROIAlign (as mentioned in step 5, Np lane line prior points are uniformly selected on R0) is concatenated, which is RefindR0. RefindR0 and L1 are input, processed, and then P1 is output.

[0080] Step 6, the final result obtained after step 5. The loss for these results is calculated uniformly. Line IOU is used for regression when regressing the offset of the predicted lane line horizontal sampling points. L1 loss is used for regression of the anchor line starting point coordinates, anchor line angle, and anchor line length. Focal Loss is used for classification loss. Therefore, the total loss function is as follows:

[0081]

[0082] It is the focus loss between prediction and label. It is the L1 loss of the regression of starting coordinates, angle, and lane length. It predicts the Line IoU loss between the lane and the actual ground conditions. These represent the weight values ​​of the corresponding loss functions.

[0083] Step 7: After completing the previous steps to build the network structure, prepare to set the network training parameters and use GPU for training. Set 15 training epochs for the CULane dataset and 70 training epochs for the TuSimple dataset. Set the learning rate to 0.0003, the batch size to 8, and use the ADMW optimizer with a cosine annealing scheduler. During training, the weight file for each epoch will be saved. After training, the best weight from the test results will be saved as best.pth.

[0084] This invention verifies the detection performance of the proposed method through experiments on the publicly available datasets CULane and TuSimple.

[0085] On the CULane dataset. Table 1 shows the experimental results of our proposed method SPDRNet and other lane detection methods on the CULane dataset. Compared with the baseline method CLRNet, which uses ResNet18 as its backbone network, SPDRNet (ResNet18) improves the F1@50 from 79.58% to 80.27%, demonstrating stronger overall detection capabilities. It also shows superior performance in more challenging sub-scenes such as congested, no-line, dimly lit, and shadowed scenes, improving performance by 0.79%, 0.41%, 1.12%, and 2.16%, respectively. Compared to the latest baseline improvement, CLRNetV2, SPDRNet still outperforms the latest baseline in the key metric F1@50. It achieves superior results in more challenging sub-scenes such as congested, no-line, dimly lit, and nighttime environments, with improvements of 1.04%, 0.93%, 1.52%, and 0.81%, respectively. Furthermore, using DLA34 as the backbone network, the F1@50 metric surpasses most current mainstream algorithms, achieving competitive results. Finally, SPDRNet (ResNet18) achieves an inference speed of 137.0 FPS, far exceeding CLRNet (112 FPS), while maintaining similar model complexity. Even the more accurate SPDRNet (DLA34) version achieves a speed of 102.7 FPS, enabling SPDRNet to provide more reliable detection results while meeting real-time requirements.

[0086] Table 1. Experimental results of different methods on CULane

[0087]

[0088] For the TUSimple dataset, Table 2 presents the results comparing the proposed method with mainstream algorithms. It can be observed that the performance differences between the different methods are very small. This is because the dataset's scenarios are too homogeneous, and performance has reached saturation. Complex network structures do not bring significant improvements. As shown in Table 2, the proposed method has achieved results comparable to state-of-the-art methods.

[0089] Table 2 Experimental results of different methods on TUSimple

[0090] method backbone network Acc(%) F1(%) FPR (%) FNR (%) SCNN VGG16 96.53 95.97 6.17 1.80 PolyLanenet EfficientNet 93.36 90.63 9.42 9.33 LaneATT ResNet18 96.71 95.57 3.56 3.01 CondLane ResNet101 96.54 97.24 2.01 3.50 CLRNet ResNet101 96.83 97.62 2.37 2.38 ADNet ResNet18 96.90 96.23 2.91 3.29 SPDRNet(ours) DLA34 96.63 97.66 2.17 2.50

[0091] From the experimental results and Figure 6The visualization results show that SPDRNet exhibits significant performance advantages in lane detection tasks in complex scenes, fully verifying the effectiveness and practicality of its structural design: (1) In occluded, no-line, brightly lit, and dark scenes, SPDRNet uses a spatial prior generator to generate class activation heatmaps, guiding the network to focus on potential lane areas. At the same time, through a deformable lane perception aggregation module, it dynamically samples to adapt to the lane structure, extracts rich global context information, and performs multi-level semantic feature fusion, thereby effectively compensating for the detection difficulties caused by local occlusion or feature loss, and enhancing the model's ability to model the global structure of lanes; (2) The introduced multi-axis gating attention unit expands the model's receptive field and improves the long-distance modeling ability of features at different scales, making the model perform well in normal lane and arrow lane scenes. The optimization strategy of this method improves the detection performance for most scenes, especially in complex scenes. (3) The computational cost of the SPDRNet (ResNet18) model in this method is 11.98 GFLOPs, which is on par with mainstream algorithms. While the more powerful SPDRNet (DLA34) has a computational cost of 18.46 GFLOPs, although it is higher, considering the significant accuracy gain it brings, this investment in computational cost is efficient and reasonable, and the inference speed still meets the real-time requirements. This shows that this method does not simply stack computational resources to exchange for accuracy, but achieves a performance leap through a better network structure design.

[0092] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A deformable and refined lane line detection method based on spatial prior guidance, characterized in that, Includes the following steps: Step 1: Receive the video stream signal, extract the video stream signal into a public dataset of images, and preprocess the images; Step 2: Extract features from the dataset using the backbone network and output feature maps at three different scales; Step 3: Construct a multi-scale fusion module. Based on FPN, after integrating the multi-axis gated attention unit MGAC into the top-level horizontal convolutional layer, the high-level features are processed by MGAC and then fused at multiple scales in FPN to generate feature maps of three different scales. , , ; Step 4: Use the Spatial Prior Generator (SPG) to process the low-level feature map output by the FPN, generate a heat map in an unsupervised manner, and use the CAM mechanism to generate the required heat map. The input feature map is fed into the hm heat map branch and the shape branch for processing, generating a class activation heat map representing position information and a shape map representing geometric edges, respectively. The two are then spliced ​​together to form a spatial prior guide map. Step 5: Enter the Deformable Lane Line Perception Aggregation Module (DLAM), and use the spatial prior guidance map generated in Step 4 to predict the coordinate offset of the sampling point; use the offset to perform irregular dynamic sampling of the mid-to-high-level features output by the feature pyramid. Step 6: The feature map passed through the deformable lane line perception aggregation module is fed into the prediction head, and the final lane line detection result is output.

2. The deformable refined lane line detection method based on spatial prior guidance according to claim 1, characterized in that, In step 2, the backbone network uses ResNet18 or DLA34 to generate features at three scales: .

3. The deformable refined lane line detection method based on spatial prior guidance according to claim 1, characterized in that, The multi-axis gated attention unit (MGAC) employs multi-directional deep convolution to simulate the receptive field of the context in different directions, and processes the input feature map. The image is segmented by channel number, processed through multi-directional convolution, and then stitched together. Simultaneously, the feature maps are input into an attention unit. Oriented separable convolution enhances the response to elongated structures in the image. Finally, the resulting feature maps are summed to output the final feature map. In FPN, feature maps After upsampling and feature map The fusion yields a new feature map Feature map After upsampling and feature map The fusion yields a new feature map ,in , , These are the three scale features output by the backbone network.

4. The deformable refined lane line detection method based on spatial prior guidance according to claim 3, characterized in that, The generated feature maps will be fed into the Deformable Lane Awareness Aggregation Module (DLAM) and the Spatial Prior Generator (SPG), respectively. and The data is fed into the deformable lane line perception aggregation module (DLAM) for processing. Then it is sent to the Spatial Prior Generator (SPG) for processing.

5. A deformable refined lane line detection method based on spatial prior guidance according to claim 1 or 4, characterized in that, The spatial prior generator SPG includes a heatmap branch (hm) and a shape branch (shape): In the heatmap branch, the prior information of the location, i.e. the existence region, is first extracted through the convolutional layer. Then, the feature map of the existence region is fed into the liner layer to calculate the weight value, and the weight is converted into the form of convolutional kernel. Each class corresponds to a set of (D,1,1) convolutional kernels. Finally, the heatmap is obtained through convolution operation and activation function. The shape branch is obtained by performing two convolutional layers and ReLU activation operations to obtain the heatmap of the shape branch, which is the prior geometric information of the lane line. Then, the two heatmaps are stitched together to obtain the final guide map GM.

6. The deformable refined lane line detection method based on spatial prior guidance according to claim 1, characterized in that, The deformable lane line perception aggregation module DLAM uses the guidance map GM and feature map. , As input, the DGA module uses deformable sampling and multi-head attention mechanisms to enable the model to focus on important regions and extract global contextual information; The DGA module uses the guide map GM generated by SPG as the basis for prediction and guidance. It generates offsets through an offset generation network, normalizes the offsets and adds them to the reference points to obtain the final sampling positions. Then, it samples the feature map and maps the features to the corresponding q, k, v vectors. It outputs the refined features through a multi-head attention mechanism and finally obtains the refined features through flattening. First, combine R0 and feature map In the input feature enhancement part, R0 is the initial anchor parameter, and parameters are uniformly selected on R0. The prior points of each lane line are used to obtain RefindR0; the feature enhancement part uses convolution and fully connected layers to align the region of interest to obtain new features of interest, which are then combined with the feature map. The feature maps processed by feature mapping are used to calculate the similarity between them through a scaling dot product attention mechanism to obtain an attention weight map. This weight map is then multiplied with the features refined by the DGA module to obtain the global features. The global features are then added to the new features of interest to obtain the final enhanced features of interest. RefindR0 and feature map The input feature enhancement part undergoes the same operation; The final output is the enhanced feature of interest. and .

7. The deformable refined lane line detection method based on spatial prior guidance according to claim 6, characterized in that, Enhanced features of interest and A unified loss is calculated. Line IOU is used for regression of the offset of the predicted lane line horizontal sampling points, L1 loss is used for regression of the anchor line starting point coordinates, anchor line angle, and anchor line length, and Focal Loss is used for classification. The total loss function is as follows: ； in, It is the focus loss between prediction and label. It is the L1 loss of the regression of starting coordinates, angle, and lane length. It predicts the Line IoU loss between the lane and the actual ground conditions. , , These represent the weight values ​​of the corresponding loss functions.

Images