A method, system, and apparatus for generating 3D lane lines based on selective state space.

Abstract

This invention belongs to the field of image processing technology, specifically relating to a method, system, and apparatus for generating 3D lane lines based on selective state space. It extracts features at different depth scales of an image through progressive upsampling and adaptive fusion, resulting in multi-scale feature maps used for sequential modeling of lane features. After capturing long-distance dependencies using a selective state space mechanism, a height feature map is generated through self-attention processing and deformable convolution. The projection plane of the inverse perspective mapping is adjusted in real time based on camera parameters to convert the 2D feature map into a 3D fused feature map, generating lane parameters for the 3D lane lines. This enhances the accuracy and anti-interference capability of 3D lane line generation, more effectively handling various changes in complex environments, and is particularly suitable for autonomous driving and assisted driving systems, demonstrating broad application prospects.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, system, and apparatus for generating three-dimensional lane lines based on selective state space. Background Technology

[0002] Through deep learning models, autonomous driving systems can effectively extract the geometric features of lane lines from images and perform reasoning and prediction in three-dimensional space, thus providing more accurate information for path planning and driving decisions. The three-dimensional lane line generation method not only overcomes the limitations of two-dimensional detection methods but also adapts to changes in complex environments, offering greater flexibility and adaptability.

[0003] Existing deep learning-based 3D lane line generation methods typically predict 3D coordinates directly from 2D image features, failing to model perspective transformations and camera geometric constraints, leading to numerous problems. For example, the depth prediction error for distant lane lines increases significantly. In monocular cameras, due to the lack of stereo vision cues, depth estimation relies on statistical priors, making it difficult to handle unfamiliar scenes. Furthermore, the dynamic changes in camera pitch angle during vehicle acceleration, deceleration, or driving over bumpy surfaces cause severe fluctuations in the 3D lane line prediction results. Summary of the Invention

[0004] This invention provides a method, system, and apparatus for generating three-dimensional lane lines based on a selective state space.

[0005] The technical solution of the present invention is as follows:

[0006] This invention provides a method for generating three-dimensional lane lines based on a selective state space, comprising:

[0007] S1: The acquired road image data is processed by depthwise separable convolution to obtain a low-dimensional initial feature map, a medium-dimensional initial feature map, and a high-dimensional initial feature map;

[0008] The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map; the mid-dimensional secondary feature map is then fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map.

[0009] High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps;

[0010] S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in the state space to generate a two-dimensional feature map.

[0011] The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected layer, and then fusing them to obtain the processed feature map.

[0012] S3: The two-dimensional feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map. Then, they are fused to obtain the height feature map.

[0013] The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view.

[0014] The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map;

[0015] S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0016] In step S2, the feature maps to be processed are processed by selective state space mechanism, self-attention mechanism, and fully connected layer respectively, and then fused to obtain the processed feature maps. Specifically:

[0017] The feature map to be processed is sequentially processed by the second convolution operation, the first activation function, the selective state space mechanism, and the standardization process to obtain the first processed feature map.

[0018] The feature map to be processed is sequentially processed by the self-attention mechanism, the second convolution operation, and the second activation function to obtain the second processed feature map;

[0019] The feature map to be processed is sequentially processed by a fully connected layer and then by the first activation function to obtain the third processed feature map;

[0020] The first, second, and third processed feature maps are fused together and then added to the feature map to be processed to obtain the processed feature map.

[0021] In step S1, the high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map. Specifically:

[0022] The high-dimensional initial feature map is subjected to the first convolution operation and the 2x upsampling operation in sequence to obtain the high-dimensional secondary feature map;

[0023] After fusing the high-dimensional secondary feature map with the mid-dimensional initial feature map, the first convolution operation and the 2x upsampling operation are performed sequentially to obtain the mid-dimensional secondary feature map.

[0024] In step S1, the high-dimensional secondary feature map, the mid-dimensional secondary feature map, and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically as follows:

[0025] The high-dimensional second-level feature map and the medium-dimensional second-level feature map are respectively subjected to the second convolution operation to obtain the corresponding high-dimensional third-level feature map and medium-dimensional third-level feature map;

[0026] The high-dimensional tertiary feature map, the mid-dimensional tertiary feature map, and the low-dimensional secondary feature map are concatenated to obtain a multi-scale feature map.

[0027] In step S3, the pixel coordinates of the two-dimensional feature map are combined with camera parameter values, and then subjected to inverse perspective mapping to obtain a bird's-eye view, based on the formula: ,accomplish;

[0028] In the formula, These are the pixel coordinates of the two-dimensional feature map. Let K be the inverse of the camera intrinsic parameter matrix. It refers to the three-dimensional spatial position.

[0029] The deformable convolution process, S3, specifically involves:

[0030] The two-dimensional feature map is convolved to obtain a set of offsets for each pixel position. Based on the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the adjusted sampling points are used for convolution calculation.

[0031] In step S1, the acquired road image data undergoes depthwise separable convolution processing to obtain a low-dimensional initial feature map, a mid-dimensional initial feature map, and a high-dimensional initial feature map, specifically:

[0032] The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map;

[0033] The low-dimensional initial feature map is processed by the first depthwise separable convolution to obtain the medium-dimensional initial feature map;

[0034] The mid-dimensional initial feature map is processed by a second depthwise separable convolution to obtain a high-dimensional initial feature map.

[0035] The S4 lane line parameters include: lane line existence status, lane line type, and lane line three-dimensional coordinate values.

[0036] The present invention also provides a three-dimensional lane line generation system based on selective state space, comprising:

[0037] Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature map, medium-dimensional initial feature map and high-dimensional initial feature map;

[0038] The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map; the mid-dimensional secondary feature map is then fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map.

[0039] High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps;

[0040] Two-dimensional feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in a state space to generate two-dimensional feature maps.

[0041] The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected layer, and then fusing them to obtain the processed feature map.

[0042] 3D fusion feature map generation module: The 2D feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map, and then fused to obtain the height feature map;

[0043] The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view.

[0044] The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map;

[0045] 3D lane line generation module: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0046] The present invention also provides a three-dimensional lane line generation device based on selective state space, comprising a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the three-dimensional lane line generation method based on selective state space.

[0047] Beneficial effects: This invention extracts features at different depth scales of images through progressive upsampling and adaptive fusion. The resulting multi-scale feature maps are used for sequential modeling of lane features. After capturing long-distance dependencies using a selective state-space mechanism, height feature maps are generated through self-attention and deformable convolution. The projection plane of the inverse perspective mapping is adjusted in real time by combining camera parameters to convert the two-dimensional feature map into a three-dimensional fused feature map, generating lane parameters for three-dimensional lane lines. This enhances the accuracy and anti-interference ability of three-dimensional lane line generation, and more effectively copes with various changes in complex environments. It is particularly suitable for autonomous driving and assisted driving systems, showing broad application prospects. Attached Figure Description

[0048] Figure 1The diagram shows the generation effect of the method of this application in road condition 1. The green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the road height. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0049] Figure 2 The diagram shows the generation effect of the method of this application in road condition 2. The green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the road height. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0050] Figure 3 The diagram shows the generation effect of the method of this application in road condition 3. The green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the road height. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0051] Figure 4 The diagram shows the generation effect of the method of this application in road condition 4. The green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the road height. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view. Detailed Implementation

[0052] The following examples are intended to illustrate the present invention, and not to further limit the invention.

[0053] This invention provides a method for generating three-dimensional lane lines based on selective state space, comprising:

[0054] S1: The acquired road image data is processed by depthwise separable convolution to obtain a low-dimensional initial feature map, a medium-dimensional initial feature map, and a high-dimensional initial feature map;

[0055] The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map; the mid-dimensional secondary feature map is then fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map.

[0056] High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps.

[0057] The actual operation is as follows:

[0058] First, depthwise separable convolution is used to extract multi-scale features, generating low-dimensional, mid-dimensional, and high-dimensional initial feature maps with resolutions of 1 / 8, 1 / 16, and 1 / 32. Specifically:

[0059] The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map;

[0060] The low-dimensional initial feature map is processed by the first depthwise separable convolution to obtain the medium-dimensional initial feature map;

[0061] The mid-dimensional initial feature map is processed by a second depthwise separable convolution to obtain a high-dimensional initial feature map.

[0062] The depthwise separable convolution process involves sequentially performing a 3×3 depthwise separable convolution, batch normalization, ReLU activation function processing, a 1×1 convolution, batch normalization, and ReLU activation function processing.

[0063] The first depthwise separable convolution process involves four depthwise separable convolution operations, while the second depthwise separable convolution process involves five depthwise separable convolution operations.

[0064] Next, the high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map, specifically:

[0065] The initial high-dimensional feature map with a resolution of 1 / 32 is subjected to a first convolution operation (such as 1×1) to further process the features and adjust the number of channels. Then, a 2x upsampling operation is performed to obtain a high-dimensional secondary feature map with a resolution of 1 / 16.

[0066] After fusing the high-dimensional secondary feature map with a resolution of 1 / 16 and the initial mid-dimensional feature map with a resolution of 1 / 16, a first convolution operation (such as 1×1) is performed to further process the features and adjust the number of channels. Then, a 2x upsampling operation is performed to obtain a mid-dimensional secondary feature map with a resolution of 1 / 8.

[0067] Then, the 1 / 8 resolution mid-dimensional secondary feature map is fused with the 1 / 8 resolution low-dimensional initial feature map to generate a 1 / 8 resolution low-dimensional secondary feature map.

[0068] This invention reduces the "chessboard effect" caused by upsampling through progressive fusion of upsampling, while preserving high-dimensional semantic information (such as lane line topology) and low-dimensional detail information (such as edge clarity), thus improving the quality of feature maps.

[0069] Finally, the high-dimensional secondary feature map, the mid-dimensional secondary feature map, and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically:

[0070] The high-dimensional secondary feature map with a resolution of 1 / 16 and the medium-dimensional secondary feature map with a resolution of 1 / 8 are subjected to a second convolution (e.g., 3×3) to obtain the corresponding high-dimensional tertiary feature map and medium-dimensional tertiary feature map. The high-dimensional tertiary feature map and the medium-dimensional tertiary feature map are adapted to the feature map size and number of channels of the low-dimensional secondary feature map with a resolution of 1 / 8.

[0071] The high-dimensional tertiary feature map, the mid-dimensional tertiary feature map, and the low-dimensional secondary feature map are concatenated to obtain a multi-scale feature map.

[0072] This invention performs 3×3 convolutions on a high-dimensional secondary feature map with a resolution of 1 / 16 and a mid-dimensional secondary feature map with a resolution of 1 / 8, adjusting their size and number of channels to match the 1 / 8 low-dimensional secondary feature map, before concatenating them. This solves the problem of mismatch between multi-scale feature resolution and number of channels, avoiding information loss. The concatenated features contain global semantics and local details, improving the robustness of lanes in occluded or long-distance scenarios and optimizing the geometric continuity of lane lines.

[0073] S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in the state space to generate a two-dimensional feature map.

[0074] The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected mechanism, respectively, and then fusing them to obtain the processed feature map.

[0075] The actual operation is as follows:

[0076] First, the multi-scale feature map (feature map with height and width of H×W) is flattened into a one-dimensional sequence, that is, the H×W feature map is flattened into a one-dimensional sequence with length H×W.

[0077] Then, the spatial location information of each element is recovered using position encoding (a vector of the same length as the sequence, which explicitly indicates the position of each feature vector by adding it to the original feature sequence), thereby obtaining the original image sequence.

[0078] This invention serializes features and combines them with position encoding to explicitly preserve spatial topological relationships, avoids the "floating point" phenomenon caused by pitch angle changes, and improves the ability to model the local continuity of lane lines (such as curved road sections).

[0079] Subsequently, the original image sequence undergoes a first state-space processing, a second state-space processing, and reshaping to generate a two-dimensional feature map.

[0080] Preferably, during the state-space processing, the feature maps to be processed are processed by a selective state-space mechanism, a self-attention mechanism, and a fully connected mechanism, respectively, and then fused to obtain the processed feature map, specifically as follows:

[0081] The feature map to be processed is sequentially processed by a second convolution (e.g., 3×3), a first activation function (e.g., SiLU), a selective state-space mechanism, and a standardization process to obtain the first processed feature map; this operation is used to capture the fine-grained geometry of lane lines.

[0082] The feature map to be processed is sequentially processed by a self-attention mechanism, a second convolution (e.g., 3×3), and a second activation function (e.g., ReLU) to obtain a second processed feature map; this operation is used to model the topological relationship between lane lines.

[0083] The feature map to be processed is sequentially processed by a fully connected layer and then by a first activation function (such as SiLU) to obtain a third processed feature map; this operation is used to supplement the global context.

[0084] The first, second, and third processed feature maps are fused together and then added to the feature map to be processed to obtain the processed feature map.

[0085] Thus, the feature map obtained from the first state space processing is subjected to a second state space processing. Then, the feature map obtained from the first state space processing and the feature map obtained from the second state space processing are reconciled and reshaped to generate a two-dimensional feature map.

[0086] The first state space processing is used to handle low-level geometric features (such as edges), and the second state space processing is used to handle high-level semantic features (such as lane topology). Gradient vanishing is avoided through residual connections.

[0087] This invention introduces a selective state-space mechanism to model long sequences of semantic features, accurately collecting relevant information and better focusing on important features in the sequence (lane line regions) while suppressing noisy features (such as false detections of occluded areas), rather than averaging all features. Especially when processing complex patterns, it can focus on the most contributing parts.

[0088] S3: The two-dimensional feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map. Then, they are fused to obtain the height feature map.

[0089] The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view.

[0090] The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map.

[0091] The actual operation is as follows:

[0092] First, the height information of the feature map is processed by two branches. One branch processes the 2D feature map using a self-attention mechanism to obtain a spatial distribution map describing the spatial details of the image. The other branch processes the 2D feature map using deformable convolution to obtain a height distribution map related to the height features. The spatial distribution map and the height distribution map are then fused to obtain a height feature map rich in information.

[0093] Preferably, in the process of obtaining the height distribution map, the deformable convolution processing specifically includes:

[0094] The two-dimensional feature map is convolved to obtain a set of offsets for each pixel position. Based on the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the adjusted sampling points are used for convolution calculation.

[0095] Furthermore, this can be achieved by two deformable convolutional processes that adapt to the object's shape, used to predict the height of each pixel. The feature map obtained from the first deformable convolutional process is then subjected to a second deformable convolutional process to predict a more refined offset, thereby capturing height features. In this way, the two deformable convolutions form a cascaded optimization, progressively adjusting the sampling position to make the features more focused on regions of height variation, automatically adapting to changes in the vehicle's pitch angle.

[0096] In addition, camera parameter embedding is introduced, which uses the camera's intrinsic and extrinsic parameters as conditions, transforms the camera parameters into a fixed-length vector through a fully connected process, and then processes this vector together with the two-dimensional feature map.

[0097] In this way, spatial information is injected into the two-dimensional feature map, and the image is correctly geometrically transformed according to the camera's perspective and shooting conditions, so that the influence of different perspectives on the spatial structure is taken into account during the reasoning process.

[0098] Next, the pixels of the 2D feature map are reverse-mapped into 3D space using inverse perspective mapping to further infer the depth information of each pixel. Preferably, the pixel coordinates of the 2D feature map are combined with camera parameter values, and after inverse perspective mapping, a bird's-eye view is obtained, based on the formula: ,accomplish;

[0099] In the formula, These are the pixel coordinates of the two-dimensional feature map. Let K be the inverse of the camera intrinsic parameter matrix. ,in f x , f y It's the focal length. cx , c y These are the coordinates of the principal point of the image. It refers to the three-dimensional spatial position.

[0100] By introducing camera parameters, this invention can be applied to changes in actual scenes. It adjusts the projection plane of the inverse perspective mapping in real time based on the height feature map and dynamically corrects projection errors. Compared with the traditional method that assumes the ground is flat, this invention has stronger applicability.

[0101] S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0102] Lane line parameters include: lane line presence (existence or non-existence), lane line type (solid line, dashed line, merged line, turning lane), and lane line 3D coordinates. Based on the 3D fusion feature map, lane line parameters can be predicted using deep neural networks. For example, convolutional neural networks can be used to extract local features, and then a regression network can be used to output the 3D lane line parameters. Through polynomial fitting or spline fitting methods, each point of the lane line can be accurately represented using a mathematical model.

[0103] This invention extracts features at different depth scales of an image through progressive upsampling and adaptive fusion. The resulting multi-scale feature maps are used for sequential modeling of lane features. After capturing long-distance dependencies using a selective state-space mechanism, height feature maps are generated through self-attention and deformable convolution. The projection plane of the inverse perspective mapping is adjusted in real time in conjunction with camera parameters to convert the two-dimensional feature map into a three-dimensional fused feature map, generating lane parameters for three-dimensional lane lines. This enhances the accuracy and anti-interference ability of three-dimensional lane line generation, and more effectively copes with various changes in complex environments. It is particularly suitable for autonomous driving and assisted driving systems, showing broad application prospects.

[0104] The present invention also provides a three-dimensional lane line generation system based on selective state space, comprising:

[0105] Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature map, medium-dimensional initial feature map and high-dimensional initial feature map;

[0106] The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map; the mid-dimensional secondary feature map is then fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map.

[0107] High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps;

[0108] Two-dimensional feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in a state space to generate two-dimensional feature maps.

[0109] The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected layer, and then fusing them to obtain the processed feature map.

[0110] 3D fusion feature map generation module: The 2D feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map, and then fused to obtain the height feature map;

[0111] The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view.

[0112] The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map;

[0113] 3D lane line generation module: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0114] The present invention also provides a three-dimensional lane line generation device based on selective state space, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the three-dimensional lane line generation method based on selective state space.

[0115] Experimental evaluation

[0116] To evaluate the generation effect of the three-dimensional lane line generation method of the present invention, images of different road conditions were collected and processed.

[0117] For road condition one, by Figure 1 It can be seen that the method of this application can accurately predict curvature changes when dealing with curved environments;

[0118] For road condition two, by Figure 2 It is evident that the method described in this application demonstrates excellent performance at both close and long distances when dealing with uphill / downhill and obstructed environments.

[0119] For road condition three, by Figure 3 It can be seen that the method of this application can accurately predict a large range of lanes when dealing with multi-lane environments;

[0120] Regarding road condition four, by Figure 4 It is evident that the method described in this application has high accuracy in processing rural road environments.

Claims

1. A method for generating three-dimensional lane lines based on selective state space, characterized in that, include: S1: The acquired road image data is processed by depthwise separable convolution to obtain a low-dimensional initial feature map, a medium-dimensional initial feature map, and a high-dimensional initial feature map; The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map. The mid-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps; S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in the state space to generate a two-dimensional feature map. The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected layer, and then fusing them to obtain the processed feature map. S3: The two-dimensional feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map. Then, they are fused to obtain the height feature map. The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view. The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map; S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

2. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, In step S2, the feature maps to be processed are processed by selective state space mechanism, self-attention mechanism, and fully connected layer respectively, and then fused to obtain the processed feature maps. Specifically: The feature map to be processed is sequentially processed by the second convolution operation, the first activation function, the selective state space mechanism, and the standardization process to obtain the first processed feature map. The feature map to be processed is sequentially processed by the self-attention mechanism, the second convolution operation, and the second activation function to obtain the second processed feature map; The feature map to be processed is sequentially processed by a fully connected layer and then by the first activation function to obtain the third processed feature map; The first, second, and third processed feature maps are fused together and then added to the feature map to be processed to obtain the processed feature map.

3. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, In step S1, the high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map. Specifically: The high-dimensional initial feature map is subjected to the first convolution operation and the 2x upsampling operation in sequence to obtain the high-dimensional secondary feature map; After fusing the high-dimensional secondary feature map with the mid-dimensional initial feature map, the first convolution operation and the 2x upsampling operation are performed sequentially to obtain the mid-dimensional secondary feature map.

4. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, In step S1, the high-dimensional secondary feature map, the mid-dimensional secondary feature map, and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically as follows: The high-dimensional second-level feature map and the medium-dimensional second-level feature map are respectively subjected to the second convolution operation to obtain the corresponding high-dimensional third-level feature map and medium-dimensional third-level feature map; The high-dimensional tertiary feature map, the mid-dimensional tertiary feature map, and the low-dimensional secondary feature map are concatenated to obtain a multi-scale feature map.

5. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, In step S3, the pixel coordinates of the two-dimensional feature map are combined with camera parameter values, and then subjected to inverse perspective mapping to obtain a bird's-eye view, based on the formula: To achieve; In the formula, These are the pixel coordinates of the two-dimensional feature map. Let K be the inverse of the camera intrinsic parameter matrix. It refers to the three-dimensional spatial position.

6. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, The deformable convolution process, S3, specifically involves: The two-dimensional feature map is convolved to obtain a set of offsets for each pixel position. Based on the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the adjusted sampling points are used for convolution calculation.

7. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, In step S1, the acquired road image data undergoes depthwise separable convolution processing to obtain a low-dimensional initial feature map, a mid-dimensional initial feature map, and a high-dimensional initial feature map, specifically: The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map; The low-dimensional initial feature map is processed by the first depthwise separable convolution to obtain the medium-dimensional initial feature map; The mid-dimensional initial feature map is processed by a second depthwise separable convolution to obtain a high-dimensional initial feature map.

8. The three-dimensional lane line generation method based on selective state space according to claim 1, characterized in that, The S4 lane line parameters include: lane line existence status, lane line type, and lane line three-dimensional coordinate values.

9. A three-dimensional lane line generation system based on selective state space, characterized in that, include: Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature map, medium-dimensional initial feature map and high-dimensional initial feature map; The high-dimensional initial feature map is upsampled step by step to obtain a high-dimensional secondary feature map, which is then fused with the mid-dimensional initial feature map to obtain a mid-dimensional secondary feature map. The mid-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; High-dimensional secondary feature maps, mid-dimensional secondary feature maps, and low-dimensional secondary feature maps are adaptively fused to obtain multi-scale feature maps; Two-dimensional feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is then processed and reshaped twice in a state space to generate two-dimensional feature maps. The state space processing involves processing the feature map to be processed through a selective state space mechanism, a self-attention mechanism, and a fully connected layer, and then fusing them to obtain the processed feature map. 3D fusion feature map generation module: The 2D feature maps are processed by self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map. Then, they are fused to obtain the height feature map. The pixel coordinates of the two-dimensional feature map are combined with the camera parameter values ​​and then subjected to inverse perspective mapping to obtain a bird's-eye view. The elevation feature map is fused with the bird's-eye view to obtain a 3D fused feature map; 3D lane line generation module: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

10. A three-dimensional lane line generation device based on selective state space, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the three-dimensional lane line generation method based on selective state space as described in any one of claims 1-8.

Images