Semantic alignment enhanced lane line detection method based on Mama architecture

Abstract

The invention discloses a semantic alignment enhanced lane line detection method based on a Mama framework. The method comprises the following steps: acquiring a lane line data set and inputting the lane line data set into a backbone network for multi-stage feature acquisition; flattening the feature map into a one-dimensional sequence by using a dynamic multi-scale adaptive screening module, stacking different levels of features, and highlighting lane line information through weights; then, through a Mama X branch, through global pooling and convolution, short-distance dependence is captured; then semantic completion is carried out on the output of the dynamic multi-scale adaptive screening module and the output of the Mama X branch through a semantic alignment fusion module, and alignment enhancement features are output; effective features are scanned through a gating module in the Mamba Z branch, and then the state is updated position by position through a scanning module to establish long-distance dependence; and inputting a feature sequence output by the Mama Z branch to a detection head, and outputting a lane line prediction map by continuously updating the position of a prior frame. According to the method, dynamic multi-scale adaptive screening, Mama architecture and semantic alignment fusion are combined, a long-distance dependency relationship can be established for the lane line, and the detection precision of the lane line is improved.

Description

Technical Field

[0001] This invention relates to a semantic alignment-enhanced lane line detection method based on the Mamba architecture, belonging to the field of computer vision technology. Background Technology

[0002] With the acceleration of urbanization and the surge in transportation demand, road network density continues to increase, and the number of motor vehicles is growing exponentially, making the need for refined and intelligent traffic management increasingly urgent. Lane lines, as the core guiding markers for road traffic order, are crucial for accurate detection. Their accuracy is not only a key basis for autonomous driving decisions and lane departure warnings, but also fundamental data support for traffic control work such as traffic flow regulation and violation identification. Therefore, achieving high-accuracy lane line detection in complex and ever-changing real-world road environments has become a core issue in promoting the implementation of intelligent transportation systems and ensuring road safety and efficiency.

[0003] Lane detection is a fundamental task in the field of autonomous driving, but it faces several technical bottlenecks in real-world scenarios. First, lane lines exhibit perspective distortion in images, appearing larger when closer and smaller when farther away, making it difficult for traditional single-scale features to capture effective information across different distances. Second, in lane detection, local features may become disconnected from global semantics, leading to increased false detection rates. Finally, because lane lines are continuous, elongated structures, traditional CNN networks have limited receptive fields, making it difficult to model the relationships between distant lane line segments. Mamba, as an emerging sequence modeling tool, achieves long-distance dependencies with linear complexity through selective scanning. However, its single-branch design struggles to balance the weights of global semantics and local details. Nevertheless, its computational complexity for long-distance dependency modeling is significantly lower than that of Transformer, making it suitable for real-time detection tasks like lane detection. Summary of the Invention

[0004] The purpose of this invention is to solve the above-mentioned problems in existing methods, and to disclose a semantic alignment-enhanced lane line detection method based on the Mamba architecture.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A semantic alignment-enhanced lane line detection method based on the Mamba architecture, characterized by the following steps: S1: Obtain the lane line dataset, input the images into the backbone network for feature extraction, and obtain a five-layer feature map after feature extraction, represented as follows. , where i represents the feature level, ; S2: The dynamic multi-scale adaptive filtering module optimizes the local features of the feature maps at layers i=3 and i=4. This module focuses on optimizing local features, flattening the two-dimensional feature maps into a one-dimensional sequence, and retaining feature information at different scales. The fine scale includes edge direction, and the coarse scale includes global direction. Then, the one-dimensional sequence converted from the feature map at layer i=4 is upsampled to ensure that the sequence lengths of the two layers are consistent. Then, it is stacked with the one-dimensional sequence converted from the feature map at layer i=3 to form a feature containing full-scale information. Then, attention weights are dynamically assigned to the features at different scales to highlight lane line related information. Finally, the output is the enhanced one-dimensional feature sequence. S3: The global semantics of lane lines are anchored and long-distance dependencies are established through a newly constructed Mamba network. The network is divided into two branches, X and Z. The X branch optimizes the feature map of i=5 layers, performs channel alignment through 1×1 convolution, extracts global semantics through global average pooling, and broadcasts it to all sequence positions. The Z branch scans effective features through a gating module, captures short distances through convolution, and then updates the state position by position through a scanning module to establish long-distance dependencies. S4: The semantic alignment fusion module performs semantic alignment between the output of the dynamic multi-scale adaptive filtering module and the output of the Mamba X branch. Then, the features are converted into sequence-first format, the dot product similarity is calculated position by position, and the alignment weight is generated by the activation function. After that, the weight is used to correct the local features and fuse them with the global semantic features by addition. Finally, the alignment enhancement features are output through MLP optimization and LayNorm normalization. S5: Input the feature sequence output from the Mamba Z branch into the detection head Ha, where a represents the layer of features. ; S6: The position Pb of the prior bounding box is continuously updated by iteratively fusing multi-scale enhanced features, where b represents the number of optimization iterations. The final output is a lane line prediction map.

[0006] A further technical solution involves the dynamic multi-scale adaptive filtering module first flattening the feature map into a sequence, then unifying it to the same sequence length. The length of the feature sequence at layer i=3 remains unchanged, while the length of the feature sequence at layer i=4 is converted to a uniform length through upsampling. Then, multi-scale feature stacking is performed, and the features of uniform length are concatenated along the channel dimension to fuse multi-scale information. Next, an attention mechanism is used to dynamically assign weights to each sequence position to highlight effective features. Then, a position-wise weight map is generated through convolution and activation functions. Finally, attention weights are used to weight the stacked features position-wise, ultimately outputting a one-dimensional feature sequence that fuses effective multi-scale information.

[0007] A further technical solution involves using the i=5 layer features as semantic features for the dual-branch input, transforming the feature map from a 2D feature map into a one-dimensional sequence. Then, it is divided into two input branches, X and Z, based on channels. The X branch focuses on global semantic enhancement. First, it performs dimensionality increase to accommodate higher semantic capacity, then global average pooling to aggregate sequence dimensions and extract more abstract global semantic information. Next, a broadcast mechanism diffuses the global semantic information to all sequence positions. Finally, 1×1 convolution is used for channel dimensionality reduction, and an activation function enhances non-linear expression. The Z branch focuses on long-distance association modeling. Similarly, it first expands the feature capacity through dimensionality increase, then inputs the channel sequence into a gating vector to filter high-level semantically relevant features, suppress irrelevant semantic information, and retain effective information. Then, 1D convolution is used to establish short-distance semantic associations, followed by selective scanning. The process involves initializing the state, updating the state vector position by position, extracting the convolutional features and gating values ​​for each sequence position, retaining current relevant features, forgetting irrelevant historical features, and finally collecting the states of all positions to form the scanned features.

[0008] A further technical solution involves the semantic alignment fusion module, which primarily addresses the semantic misalignment problem between the outputs of the dynamic multi-scale adaptive filtering module and the Mamba X branch module. First, the channel-first format is converted to a sequence-first format. Then, the similarity between local and global semantics at each sequence position is calculated. Finally, an activation function compresses the similarity to between 0 and 1, yielding position-by-position alignment weights. Finally, by aligning the weights Correct the local features in the dynamic multi-scale adaptive filtering module to output aligned features with strong semantic consistency and long-distance continuity.

[0009] A further technical solution is that the training steps of the trained lane detection network include: Obtain the lane line dataset; Input the data into the backbone network for feature extraction; Local features are optimized through a dynamic multi-scale adaptive filtering module; Global semantic feature extraction is performed using the X branch of an improved Mamba network. Features are enhanced through a semantic alignment fusion module; Establish long-distance dependencies through the Z-branch of an improved Mamba network; The detection head outputs a lane line prediction map; The beneficial effects of adopting the above technical solution are as follows: This invention provides a dynamic multi-scale adaptive filtering module that can effectively address the multi-scale differences in lane lines presented in images, adaptively allocate weights, and reduce redundant information interference; This invention designs a newly constructed Mamba network architecture, where the X branch focuses on extracting global semantic information and the Z branch focuses on long-distance feature association modeling, enabling accurate association of various parts of the lane lines; This invention develops a semantic alignment fusion module, dedicated to solving the misalignment problem between local features and global semantics. By correcting through alignment weights, it can ensure that the detected local details of the lane lines conform to the overall semantic logic, significantly improving the semantic consistency and accuracy of the detection results. The three modules used are integrated into the network, greatly improving the accuracy of lane line detection and demonstrating the advantages of the proposed technical solution. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating the overall network process according to an embodiment of the present invention. Figure 2 This is a diagram illustrating the overall network architecture of an embodiment of the present invention. Figure 3 This is a structural diagram of the dynamic multi-scale adaptive screening module according to an embodiment of the present invention; Figure 4 This is a diagram of the newly constructed Mamba network structure according to an embodiment of the present invention; Figure 5 This is a structural diagram of the semantic alignment and fusion module in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0012] This invention provides a semantic alignment-enhanced lane line detection method based on the Mamba architecture, such as... Figure 1 As shown, it includes the following steps: S1: Obtain the lane line dataset, construct a ResNet18 backbone network, input the images into the backbone network for feature extraction, and obtain a five-layer feature map after feature extraction, represented as... , where i represents the feature level, ,like Figure 2 As shown; S2: Construct a dynamic multi-scale adaptive screening module, the structure of which is shown below. Figure 3 ; S2-1: First, the feature maps are flattened into sequences, then unified to the same sequence length. The length of the feature sequences in layers i=3 remains unchanged, while the length of the feature sequences in layers i=4 is converted to a uniform length through upsampling. This indicates a predefined, known sequence length; interpolate means upsampling; and seq is an abbreviation for sequence. This represents the real number field, and the same applies below. The calculation formula is as follows: , S2-2: Next, multi-scale feature stacking is performed, concatenating features of uniform length along the channel dimension to achieve multi-scale information fusion. Here, Concat represents channel concatenation, and the calculation formula is as follows: , S2-3: Then, an attention mechanism is used to dynamically assign weights to each sequence position to highlight effective features. Next, convolution and activation functions are used to generate position-wise weight maps. Finally, attention weights are used to weight the stacked features position-wise. For element-wise multiplication, the formula is as follows: , S3: Construct a new Mamba network structure, the structure of which is shown below. Figure 4 ; S3-1: Conv5 layer features serve as higher-level semantic features for dual-branch input. First, the feature map is transformed from a 2D feature map into a one-dimensional sequence, and then split into two paths according to channels. S3-2: The X branch focuses on global semantic enhancement. First, it performs dimensionality increase to accommodate higher semantic capacity. Then, it uses global average pooling to aggregate sequence dimensions and extract more abstract global semantic information. Next, it uses a broadcast mechanism to diffuse this global semantic information to all sequence positions. Finally, it uses 1×1 convolution for channel dimensionality reduction and an activation function to enhance non-linear representation. Here, GAP stands for global average pooling, and Broadcast is the broadcasting mechanism. It is a sequence that has been increased in dimensionality through convolution, and ReLU is the activation function.

[0013]

[0014] S3-3: The Z branch focuses on long-distance correlation modeling. It also first expands the feature capacity through dimensionality increase operation and inputs the channel sequence into the gating vector to filter high-level semantically relevant features, suppress irrelevant semantic information, and retain effective information. The gating module is a two-branch parallel structure. One branch captures the target's global long-range dependence on multi-directional scattering information through the state space modeling module, while the other branch supplements local discrete details through the 1D convolution branch. After that, the two features are concatenated and fused with the linear layer, and the original information is retained by the residual connection.

[0015] S3-4: Then, selective scanning is performed. First, the state is initialized, then the state vector is updated position by position. The convolutional features and gating values ​​of each sequence position are extracted, the relevant current features are retained, and irrelevant historical features are forgotten. If the current position is a lane line, The state is primarily updated with the current feature; if the current position is the background, The state retains historical lane line features, then enhances the nonlinear representation through MLP, and finally collects the states at all locations to form post-scan features. This indicates that sequences with very short semantic associations are captured through 1D convolution. Indicates the gate value, This indicates the initial state. .

[0016] S4: Construct the semantic alignment and fusion module, the structure of which is shown below. Figure 5 ; S4-1: First, the channel-first format is converted to a sequence-first format. Then, the similarity between the local and global semantics of each sequence position is calculated. The larger the value, the higher the matching degree between the local details and the global semantics at that position, proving that the local edge does indeed belong to the global lane line. This is the output of the dynamic multi-scale adaptive screening module, where b is the sample index. Where c is the sequence position, and c is the number of channels. The calculation formula is as follows: .

[0017] S4-2: Then, the similarity is compressed to between 0 and 1 using an activation function to obtain the position-by-position alignment weights. Finally, the alignment weights are used... Correcting local features in the dynamic multi-scale adaptive screening module, if If the value is too small, it indicates a semantic misalignment, and the local feature is attenuated. Conversely, if the value is too large, it indicates a good match, and local details are preserved. The final output is an alignment feature with strong semantic consistency and long-range continuity, where Sigmoid represents the activation function and Permute represents the permutation. The calculation formula is as follows: .

[0018] S4-3: Finally, the aligned local features are deeply fused with the global semantic features, and the expressive power is enhanced through nonlinear transformation.

[0019] S5: Input the feature sequence output from the Mamba Z branch into the detection head Ha, where a represents the layer of features. ; S6: The position Pb of the prior bounding box is continuously updated by iteratively fusing multi-scale enhanced features, where b represents the number of optimization iterations. The final output is a lane line prediction map.

[0020] The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A semantic alignment-enhanced lane detection method based on the Mamba architecture, characterized in that... Includes the following steps: S1: Obtain the lane line dataset, input the images into the backbone network for feature extraction, and obtain a five-layer feature map after feature extraction, represented as follows. , where i represents the feature level, ; S2: The dynamic multi-scale adaptive filtering module optimizes the local features of the feature maps at layers i=3 and i=4. This module focuses on optimizing local features, flattening the two-dimensional feature maps into a one-dimensional sequence, and retaining feature information at different scales. The fine scale includes edge direction, and the coarse scale includes global direction. Then, the one-dimensional sequence converted from the feature map at layer i=4 is upsampled to ensure that the sequence lengths of the two layers are consistent. Then, it is stacked with the one-dimensional sequence converted from the feature map at layer i=3 to form a feature containing full-scale information. Then, attention weights are dynamically assigned to the features at different scales to highlight lane line related information. Finally, the output is the enhanced one-dimensional feature sequence. S3: The global semantics of lane lines are anchored and long-distance dependencies are established through a newly constructed Mamba network. The network is divided into two branches, X and Z. The X branch optimizes the feature map of i=5 layers, performs channel alignment through 1×1 convolution, extracts global semantics through global average pooling, and broadcasts it to all sequence positions. The Z branch scans effective features through a gating module, captures short distances through convolution, and then updates the state position by position through a scanning module to establish long-distance dependencies. S4: The semantic alignment fusion module performs semantic alignment between the output of the dynamic multi-scale adaptive filtering module and the output of the Mamba X branch. Then, the features are converted into sequence-first format, the dot product similarity is calculated position by position, and the alignment weight is generated by the activation function. After that, the weight is used to correct the local features and fuse them with the global semantic features by addition. Finally, the alignment enhancement features are output through MLP optimization and LayNorm normalization. S5: Input the feature sequence output from the Mamba Z branch into the detection head Ha, where a represents the layer of features. ; S6: The position Pb of the prior bounding box is continuously updated by iteratively fusing multi-scale enhanced features, where b represents the number of optimization iterations. The final output is a lane line prediction map.

2. The semantic alignment-enhanced lane detection method based on the Mamba architecture as described in claim 1, characterized in that, The dynamic multi-scale adaptive filtering module first flattens the feature map into a sequence, then unifies it to the same sequence length. The length of the feature sequence at layer i=3 remains unchanged, while the length of the feature sequence at layer i=4 is converted to a uniform length through upsampling. Then, multi-scale feature stacking is performed, and the features of uniform length are concatenated along the channel dimension to perform multi-scale information fusion. Next, an attention mechanism is used to dynamically assign weights to each sequence position to highlight effective features. Then, a position-wise weight map is generated through convolution and activation functions. Finally, attention weights are used to weight the stacked features position-wise, and the final output is a one-dimensional feature sequence that fuses effective multi-scale information.

3. The semantic alignment-enhanced lane detection method based on the Mamba architecture as described in claim 1, characterized in that, The Mamba dual-branch network structure uses the features of layer i=5 as semantic features as input to both branches, transforming the feature map from a 2D feature map into a one-dimensional sequence. Then, it is divided into two input branches, X and Z, based on channels. The X branch focuses on global semantic enhancement. First, it performs dimensionality increase to accommodate higher semantic capacity, then global average pooling to aggregate sequence dimensions and extract more abstract global semantic information. Next, a broadcast mechanism diffuses the global semantic information to all sequence positions. Finally, 1×1 convolutions are used for channel dimensionality reduction, and activation functions enhance non-linear expression. The Z branch focuses on long-distance association modeling. Similarly, it first expands the feature capacity through dimensionality increase, then inputs the channel sequence into a gating vector to filter high-level semantically relevant features, suppressing irrelevant semantic information and retaining effective information. Then, 1D convolutions are used to establish short-distance semantic associations, followed by selective scanning. The process involves initializing the state, updating the state vector position by position, extracting convolutional features and gating values ​​for each sequence position, retaining current relevant features, forgetting irrelevant historical features, and finally collecting the states of all positions to form the scanned features.

4. The semantic alignment-enhanced lane detection method based on the Mamba architecture as described in claim 1, characterized in that, The semantic alignment fusion module primarily addresses the semantic misalignment issue between the outputs of the dynamic multi-scale adaptive filtering module and the Mamba X branch module. First, it converts the channel-first format to a sequence-first format. Then, it calculates the similarity between local and global semantics at each sequence position. Finally, it uses an activation function to compress the similarity to between 0 and 1, obtaining the position-wise alignment weights. Finally, by aligning the weights Correct the local features in the dynamic multi-scale adaptive filtering module to output aligned features with strong semantic consistency and long-distance continuity.

5. The semantic alignment-enhanced lane detection method based on Mamba architecture as described in claim 1, characterized in that, The training steps for the trained lane detection network include: Obtain the lane line dataset; Input the data into the backbone network for feature extraction; Local features are optimized through a dynamic multi-scale adaptive filtering module; Global semantic feature extraction is performed using the X branch of an improved Mamba network. Features are enhanced through a semantic alignment fusion module; Establish long-distance dependencies through the Z-branch of an improved Mamba network; The detector outputs a lane line prediction map.

Images