Three-dimensional lane line generation method, system and device based on selective state space

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a three-dimensional lane line generation method, system and device based on a selective state space, features of different depth scales of an image are extracted through step-by-step up-sampling and self-adaptive fusion, and an obtained multi-scale feature map is used for carrying out sequence modeling on lane features to obtain a three-dimensional lane line. After a long-distance dependency relationship is captured by adopting a selective state space mechanism, a height feature map is generated through self-attention mechanism processing and deformable convolution processing, and a projection plane of inverse perspective mapping is adjusted in real time in combination with camera parameters, so that a two-dimensional feature map is converted into a three-dimensional fusion feature map, lane parameters of a three-dimensional lane line are generated, and the three-dimensional lane line is obtained. Therefore, the accuracy and the anti-interference capability of three-dimensional lane line generation are enhanced, various changes in a complex environment are more effectively dealt with, and the method is particularly suitable for automatic driving and auxiliary driving systems and shows a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method, system and device for generating three-dimensional lane lines based on selective state space. Background Art

[0002] Using deep learning models, autonomous driving systems can effectively extract lane geometry from images and perform inference and prediction in three-dimensional space, providing more accurate information for path planning and driving decisions. This 3D lane generation method not only overcomes the limitations of 2D detection methods but also adapts to changes in complex environments, providing greater flexibility and adaptability.

[0003] Existing deep learning-based 3D lane generation methods typically predict 3D coordinates directly from 2D image features without modeling perspective transformations and camera geometry constraints, leading to numerous issues. For example, depth prediction errors for distant lane lines increase significantly. In monocular cameras, due to the lack of stereo visual cues, depth estimation relies on statistical priors, making it difficult to adapt to new, unseen scenes. Furthermore, dynamic changes in the camera's pitch angle as the vehicle accelerates or decelerates or traverses bumpy roads can cause significant jitter in the 3D lane prediction results. Summary of the Invention

[0004] The present invention provides a method, system and device for generating three-dimensional lane lines based on selective state space.

[0005] The technical solutions of the present invention are as follows: The present invention provides a three-dimensional lane line generation method based on selective state space, comprising: S1: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; the medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map; S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a two-dimensional feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map; S3: The two-dimensional feature map is processed by the self-attention mechanism and the deformable convolution to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view map to obtain a three-dimensional fused feature map; S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0006] Said S2, the feature map to be processed is processed by the selective state space mechanism, the self-attention mechanism, and the full connection, and then fused to obtain the processed feature map, specifically: The feature map to be processed is sequentially subjected to a second convolution operation, a first activation function processing, a selective state space mechanism processing, and a normalization processing to obtain a first processed feature map; The feature map to be processed is processed by the self-attention mechanism, the second convolution operation, and the second activation function in sequence to obtain the second processed feature map; The feature map to be processed is processed by full connection processing and the first activation function processing in sequence to obtain a third processed feature map; The first processing feature map, the second processing feature map, and the third processing feature map are fused and then added to the feature map to be processed to obtain a processed feature map.

[0007] In S1, the high-dimensional initial feature map is upsampled step by step, and the obtained high-dimensional secondary feature map is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map, which is specifically: The high-dimensional initial feature map is sequentially subjected to the first convolution operation and the 2x upsampling operation to obtain a high-dimensional secondary feature map; After the high-dimensional secondary feature map is fused with the medium-dimensional initial feature map, the first convolution operation and the 2x upsampling operation are performed in sequence to obtain the medium-dimensional secondary feature map.

[0008] In S1, the high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically: The high-dimensional secondary feature map and the medium-dimensional secondary feature map are respectively subjected to the second convolution operation to obtain the corresponding high-dimensional third-level feature map and medium-dimensional third-level feature map; The high-dimensional three-level feature map, the medium-dimensional three-level feature map and the low-dimensional two-level feature map are spliced to obtain a multi-scale feature map.

[0009] In S3, the pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values, and the bird's-eye view is obtained by inverse perspective mapping, which is based on the formula: ,accomplish; Where, is the pixel coordinate value of the two-dimensional feature map, is the inverse matrix of the camera intrinsic parameter matrix K, is the three-dimensional space position.

[0010] The S3, deformable convolution processing, is specifically: The two-dimensional feature map is subjected to convolution operation to obtain a set of offsets for each pixel position. According to the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the convolution calculation is performed on the adjusted sampling points.

[0011] In S1, the acquired road image data is processed by depthwise separable convolution to obtain a low-dimensional initial feature map, a medium-dimensional initial feature map, and a high-dimensional initial feature map, specifically: The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map; The low-dimensional initial feature map is processed by the first depth-wise separable convolution to obtain a medium-dimensional initial feature map; The medium-dimensional initial feature map is processed by the second depth-wise separable convolution to obtain a high-dimensional initial feature map.

[0012] The lane line parameters in S4 include: lane line existence, lane line type, and lane line three-dimensional coordinate values.

[0013] The present invention also provides a three-dimensional lane line generation system based on selective state space, comprising: Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; the medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map; 2D feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a 2D feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map; 3D fusion feature map generation module: The 2D feature maps are processed by the self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view map to obtain a three-dimensional fused feature map; 3D lane line generation module: Based on the 3D fusion feature map, the lane line parameters are predicted and the 3D lane lines are generated.

[0014] The present invention also provides a three-dimensional lane line generation device based on selective state space, comprising a processor and a memory, wherein the processor implements the three-dimensional lane line generation method based on selective state space when executing a computer program stored in the memory.

[0015] Beneficial effects: The present invention extracts features of different depth scales of the image through step-by-step upsampling and adaptive fusion. The obtained multi-scale feature map is used to perform sequence modeling of lane features. After using the selective state space mechanism to capture long-distance dependencies, the height feature map is generated through self-attention mechanism processing and deformable convolution processing. The projection plane of the inverse perspective mapping is adjusted in real time in combination with the camera parameters to convert the two-dimensional feature map into a three-dimensional fused feature map, and the lane parameters of the three-dimensional lane line are generated, thereby enhancing the accuracy and anti-interference ability of the three-dimensional lane line generation, and more effectively coping with various changes in complex environments. It is particularly suitable for autonomous driving and assisted driving systems, and shows broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the generation effect of the method of the present application on road condition 1, where the green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the height of the road. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0017] Figure 2 This is a schematic diagram of the generation effect of the method of the present application on road condition 2, where the green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the height of the road. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0018] Figure 3 This is a schematic diagram of the generation effect of the method of this application on road condition three, where the green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the height of the road. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view.

[0019] Figure 4This is a schematic diagram of the generation effect of the method of the present application on road condition four, where the green line represents the generated three-dimensional lane line, the blue line represents the actual lane line, the x-axis represents the horizontal distance of the road, the y-axis represents the distance in front of the road, and the z-axis represents the height of the road. (a) is the front view, (b) is the three-dimensional spatial result of the three-dimensional lane line, and (c) is the top view. DETAILED DESCRIPTION

[0020] The following examples are intended to illustrate the present invention rather than to further limit the present invention.

[0021] The present invention provides a three-dimensional lane line generation method based on selective state space, comprising: S1: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; the medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map.

[0022] The actual operation is as follows: First, depthwise separable convolution is used to extract multi-scale features and generate low-dimensional, medium-dimensional, and high-dimensional initial feature maps with resolutions of 1 / 8, 1 / 16, and 1 / 32. Specifically: The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map; The low-dimensional initial feature map is processed by the first depth-wise separable convolution to obtain a medium-dimensional initial feature map; The medium-dimensional initial feature map is processed by the second depth-wise separable convolution to obtain a high-dimensional initial feature map.

[0023] Among them, the depth-wise separable convolution processing is to sequentially undergo 3×3 depth-wise separable convolution, batch normalization, ReLU activation function processing, 1×1 convolution, batch normalization, and ReLU activation function processing.

[0024] The first depth-wise separable convolution processing is performed by four depth-wise separable convolution processing operations, and the second depth-wise separable convolution processing is performed by five depth-wise separable convolution processing operations.

[0025] Next, the high-dimensional initial feature map is upsampled step by step, and the obtained high-dimensional secondary feature map is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map, which is specifically: The high-dimensional initial feature map with a resolution of 1 / 32 undergoes a first convolution (e.g., 1×1) operation to further process the features and adjust the number of channels, and then performs a 2x upsampling operation to obtain a high-dimensional secondary feature map with a resolution of 1 / 16. After the high-dimensional secondary feature map with 1 / 16 resolution is fused with the medium-dimensional initial feature map with 1 / 16 resolution, the first convolution (such as 1×1) operation is performed to further process the features and adjust the number of channels, and then a 2x upsampling operation is performed to obtain a medium-dimensional secondary feature map with 1 / 8 resolution.

[0026] Then, the medium-dimensional secondary feature map with 1 / 8 resolution is fused with the low-dimensional initial feature map with 1 / 8 resolution to generate a low-dimensional secondary feature map with 1 / 8 resolution.

[0027] Through the progressive fusion of step-by-step upsampling, the present invention can reduce the "chessboard effect" caused by upsampling, retain high-dimensional semantic information (such as lane line topology structure) and low-dimensional detail information (such as edge clarity), and improve the quality of feature maps.

[0028] Finally, the high-dimensional secondary feature map, the medium-dimensional secondary feature map, and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically: The high-dimensional secondary feature map with 1 / 16 resolution and the medium-dimensional secondary feature map with 1 / 8 resolution are respectively subjected to the second convolution (such as 3×3) operation to obtain the corresponding high-dimensional third-level feature map and medium-dimensional third-level feature map; the high-dimensional third-level feature map and the medium-dimensional third-level feature map are both adapted to the feature map size and number of channels of the low-dimensional secondary feature map with 1 / 8 resolution.

[0029] The high-dimensional three-level feature map, the medium-dimensional three-level feature map and the low-dimensional two-level feature map are spliced to obtain a multi-scale feature map.

[0030] This method performs 3×3 convolutions on the high-dimensional secondary feature map (1 / 16 resolution) and the medium-dimensional secondary feature map (1 / 8 resolution), adjusting their size and number of channels to match the low-dimensional secondary feature map (1 / 8 resolution). This method then concatenates the features. This solves the mismatch between multi-scale feature resolution and channel count, preventing information loss. The concatenated features incorporate both global semantics and local details, improving lane robustness in occluded and long-distance scenarios and optimizing the geometric continuity of lane lines.

[0031] S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a two-dimensional feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map.

[0032] The actual operation is as follows: First, the multi-scale feature map (feature map with height and width of H×W) is flattened into a one-dimensional sequence, that is, the H×W feature map is flattened into a one-dimensional sequence with a length of H×W.

[0033] Then, the spatial position information of each element is restored using position encoding (a vector with the same length as the sequence, which explicitly indicates the position of each feature vector by adding it to the original feature sequence) to obtain the original image sequence.

[0034] The present invention serializes features and combines them with position encoding to explicitly preserve spatial topological relationships, avoid the "floating point" phenomenon caused by changes in pitch angles, and improve the ability to model the local continuity of lane lines (such as curved sections).

[0035] Afterwards, the original image sequence undergoes the first state space processing, the second state space processing, and reshaping in sequence to generate a two-dimensional feature map.

[0036] Preferably, during the state space processing, the feature map to be processed is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism respectively, and then fused to obtain the processed feature map, specifically: The feature map to be processed is sequentially processed through a second convolution (e.g., 3×3), a first activation function (e.g., SiLU), a selective state space mechanism, and normalization to obtain a first processed feature map. This operation is used to capture the fine-grained geometric structure of the lane lines. The feature map to be processed is processed by the self-attention mechanism, the second convolution operation (e.g., 3×3), and the second activation function (e.g., ReLU) in sequence to obtain the second processed feature map. This operation is used to model the topological relationship between lane lines. The feature map to be processed is processed by full connection processing and the first activation function (such as SiLU) in sequence to obtain the third processed feature map; this operation is used to supplement the global context; The first processing feature map, the second processing feature map, and the third processing feature map are fused and then added to the feature map to be processed to obtain a processed feature map.

[0037] In this way, the feature map processed by the first state space is processed by the second state space, and then the feature map processed by the first state space and the feature map obtained by the second state space processing are residually connected and reshaped to generate a two-dimensional feature map.

[0038] Among them, the first state space processing is used to process low-level geometric features (such as edges), and the second state space processing is used to process high-level semantic features (such as lane topology), and the residual connection is used to avoid gradient disappearance.

[0039] By introducing a selective state space mechanism, this invention models long sequences of semantic features, accurately extracting relevant information, focusing on important features in the sequence (such as lane markings), and suppressing noisy features (such as false detection of occluded areas), rather than averaging all features. This allows the algorithm to focus on the most contributing features, especially when processing complex patterns.

[0040] S3: The two-dimensional feature map is processed by the self-attention mechanism and the deformable convolution to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view image to obtain a three-dimensional fused feature map.

[0041] The actual operation is as follows: First, the height information of the feature map is processed by two branches. One branch processes the 2D feature map through a self-attention mechanism to obtain a spatial distribution map that describes the spatial details of the image. The other branch processes the 2D feature map through a deformable convolution to obtain a height distribution map related to the height features. The spatial distribution map and the height distribution map are fused to obtain a height feature map with rich information.

[0042] Preferably, in the process of obtaining the height distribution map, the deformable convolution processing is specifically as follows: The two-dimensional feature map is subjected to convolution operation to obtain a set of offsets for each pixel position. According to the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the convolution calculation is performed on the adjusted sampling points.

[0043] Furthermore, a method can be implemented by performing two deformable convolutions that adapt to the object's shape to predict the height of each pixel. The feature map generated by the first deformable convolution is then used in a second deformable convolution to predict a finer offset, thereby capturing the height feature. In this way, the two deformable convolutions form a cascade optimization, gradually adjusting the sampling position to focus the features more closely on areas of height variation and automatically adapt to changes in the vehicle's pitch angle.

[0044] In addition, camera parameter embedding is introduced, which takes the internal and external parameters of the camera as conditions, converts the camera parameters into a fixed-length vector through a fully connected process, and then jointly processes the vector with the two-dimensional feature map.

[0045] In this way, spatial information is injected into the two-dimensional feature map, and the image is correctly geometrically transformed according to the camera's perspective and shooting conditions, so that the impact of different perspectives on the spatial structure is taken into account during the reasoning process.

[0046] Then, the pixels of the two-dimensional feature map are reversely mapped to the three-dimensional space through inverse perspective mapping to further infer the depth information of each pixel. Preferably, the pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values, and the bird's-eye view is obtained through inverse perspective mapping according to the formula: ,accomplish; Where, is the pixel coordinate value of the two-dimensional feature map, is the inverse matrix of the camera intrinsic parameter matrix K, ,in f x , f y is the focal length, cx , c y are the principal point coordinates of the image, is the three-dimensional space position.

[0047] After introducing camera parameters, the present invention can adapt to actual scene changes, adjust the projection plane of the inverse perspective mapping in real time according to the height feature map, and dynamically correct the projection error. Compared with the traditional method that assumes the ground is flat, the applicability of the present invention is stronger.

[0048] S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

[0049] Lane parameters include lane presence (presence or absence), lane type (solid, dashed, merging, turning lane), and 3D lane coordinates. Lane parameters can be predicted based on a 3D fusion feature map using deep neural networks. For example, a convolutional neural network extracts local features and then uses a regression network to output 3D lane parameters. Polynomial or spline fitting methods are used to accurately represent each lane point using a mathematical model.

[0050] The present invention extracts features of different depth scales of the image through step-by-step upsampling and adaptive fusion. The obtained multi-scale feature map is used to perform sequential modeling of lane features. After adopting the selective state space mechanism to capture long-distance dependencies, the height feature map is generated through self-attention mechanism processing and deformable convolution processing. The projection plane of the inverse perspective mapping is adjusted in real time in combination with the camera parameters to convert the two-dimensional feature map into a three-dimensional fused feature map, and the lane parameters of the three-dimensional lane line are generated, thereby enhancing the accuracy and anti-interference ability of the three-dimensional lane line generation, and more effectively coping with various changes in complex environments. It is particularly suitable for autonomous driving and assisted driving systems, and shows broad application prospects.

[0051] The present invention also provides a three-dimensional lane line generation system based on selective state space, comprising: Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; the medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain the low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map; 2D feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a 2D feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map; 3D fusion feature map generation module: The 2D feature maps are processed by the self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view map to obtain a three-dimensional fused feature map; 3D lane line generation module: Based on the 3D fusion feature map, the lane line parameters are predicted and the 3D lane lines are generated.

[0052] The present invention also provides a three-dimensional lane line generation device based on selective state space, comprising a processor and a memory, wherein the processor implements the three-dimensional lane line generation method based on selective state space when executing a computer program stored in the memory.

[0053] Experimental evaluation In order to evaluate the generation effect of the three-dimensional lane line generation method of the present invention, images of different road conditions were collected and processed.

[0054] Regarding road conditions, Figure 1 It can be seen that the method of the present application can accurately predict the curvature change when processing a curved environment; For road condition 2, Figure 2 It can be seen that the method of the present application has excellent effects at close and long distances when dealing with uphill and downhill and obstructed environments; For road condition three, Figure 3 It can be seen that the proposed method can accurately predict lanes in a large range when dealing with a multi-lane environment; For road condition 4, Figure 4It can be seen that the method of this application has high accuracy when dealing with rural road environments.

Claims

1. A three-dimensional lane line generation method based on selective state space, characterized in that: include: S1: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; The medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map; S2: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a two-dimensional feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map; S3: The two-dimensional feature map is processed by the self-attention mechanism and the deformable convolution to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view map to obtain a three-dimensional fused feature map; S4: Based on the 3D fusion feature map, predict lane line parameters and generate 3D lane lines.

2. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: Said S2, the feature map to be processed is processed by the selective state space mechanism, the self-attention mechanism, and the full connection, and then fused to obtain the processed feature map, specifically: The feature map to be processed is sequentially subjected to a second convolution operation, a first activation function processing, a selective state space mechanism processing, and a normalization processing to obtain a first processed feature map; The feature map to be processed is processed by the self-attention mechanism, the second convolution operation, and the second activation function in sequence to obtain the second processed feature map; The feature map to be processed is processed by full connection processing and the first activation function processing in sequence to obtain a third processed feature map; The first processing feature map, the second processing feature map, and the third processing feature map are fused and then added to the feature map to be processed to obtain a processed feature map.

3. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: In S1, the high-dimensional initial feature map is upsampled step by step, and the obtained high-dimensional secondary feature map is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map, which is specifically: The high-dimensional initial feature map is sequentially subjected to the first convolution operation and the 2x upsampling operation to obtain a high-dimensional secondary feature map; After the high-dimensional secondary feature map is fused with the medium-dimensional initial feature map, the first convolution operation and the 2x upsampling operation are performed in sequence to obtain the medium-dimensional secondary feature map.

4. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: In S1, the high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map, specifically: The high-dimensional secondary feature map and the medium-dimensional secondary feature map are respectively subjected to the second convolution operation to obtain the corresponding high-dimensional third-level feature map and medium-dimensional third-level feature map; The high-dimensional three-level feature map, the medium-dimensional three-level feature map and the low-dimensional two-level feature map are spliced to obtain a multi-scale feature map.

5. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: In S3, the pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values, and the bird's-eye view is obtained by inverse perspective mapping, which is based on the formula: ,accomplish; Where, is the pixel coordinate value of the two-dimensional feature map, is the inverse matrix of the camera intrinsic parameter matrix K, is the three-dimensional space position.

6. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: The S3, deformable convolution processing, is specifically: The two-dimensional feature map is subjected to convolution operation to obtain a set of offsets for each pixel position. According to the offsets, the sampling position of the convolution kernel is dynamically adjusted, and the convolution calculation is performed on the adjusted sampling points.

7. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: In S1, the acquired road image data is processed by depthwise separable convolution to obtain a low-dimensional initial feature map, a medium-dimensional initial feature map, and a high-dimensional initial feature map, specifically: The acquired road image data is processed by a first depthwise separable convolution to obtain a low-dimensional initial feature map; The low-dimensional initial feature map is processed by the first depth-wise separable convolution to obtain a medium-dimensional initial feature map; The medium-dimensional initial feature map is processed by the second depth-wise separable convolution to obtain a high-dimensional initial feature map.

8. The method for generating three-dimensional lane lines based on selective state space according to claim 1, characterized in that: The lane line parameters in S4 include: lane line existence, lane line type, and lane line three-dimensional coordinate values.

9. A three-dimensional lane line generation system based on selective state space, characterized in that: include: Multi-scale feature extraction module: The acquired road image data is processed by depthwise separable convolution to obtain low-dimensional initial feature maps, medium-dimensional initial feature maps, and high-dimensional initial feature maps; The high-dimensional initial feature map is upsampled step by step, and the high-dimensional secondary feature map obtained is fused with the medium-dimensional initial feature map to obtain the medium-dimensional secondary feature map; The medium-dimensional secondary feature map is fused with the low-dimensional initial feature map to obtain a low-dimensional secondary feature map; The high-dimensional secondary feature map, the medium-dimensional secondary feature map and the low-dimensional secondary feature map are adaptively fused to obtain a multi-scale feature map; 2D feature map generation module: After the multi-scale feature map is flattened into a one-dimensional sequence, position encoding is performed to obtain the original image sequence. The original image sequence is sequentially processed twice in state space and reshaped to generate a 2D feature map; The state space processing is to fuse the feature map to be processed after it is processed by the selective state space mechanism, the self-attention mechanism, and the full connection mechanism, to obtain a processed feature map; 3D fusion feature map generation module: The 2D feature maps are processed by the self-attention mechanism and deformable convolution respectively to obtain the corresponding spatial distribution map and height distribution map, which are then fused to obtain the height feature map; The pixel coordinate values of the two-dimensional feature map are combined with the camera parameter values and inverse perspective mapping to obtain a bird's-eye view. The height feature map is fused with the bird's-eye view map to obtain a three-dimensional fused feature map; 3D lane line generation module: Based on the 3D fusion feature map, the lane line parameters are predicted and the 3D lane lines are generated.

10. A three-dimensional lane line generation device based on selective state space, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the method for generating three-dimensional lane lines based on selective state space as described in any one of claims 1 to 8 is implemented.

Images