Road test point cloud 3D target detection method and system based on mixed state space model

Abstract

The invention relates to the technical field of computer vision and intelligent traffic systems, in particular to a road test point cloud 3D target detection method and system based on a mixed state space model. Comprising the steps of obtaining point cloud data collected by a roadside laser radar, converting the point cloud data into a dense aerial view BEV feature map through a cylindrical body feature encoder PFE, and inputting the BEV feature map into a backbone network for feature extraction; the method comprises the following steps of: performing channel dimension segmentation operation on an input feature by using a cross-stage state space group CSG with multiple levels, and dividing the input feature into a first feature component and a second feature component; splicing and fusing the processed first feature component and the processed second feature component to generate an output feature; and carrying out fusion and up-sampling, and outputting a detection result containing a target category and a 3D bounding box through a detection head. According to the invention, through collaborative design of column feature coding, a cross-stage state space group and a mixed state space block, balance between detection precision and real-time performance is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent transportation systems (ITS), specifically to a method and system for 3D target detection of roadside point clouds based on a hybrid state space model, particularly for feature extraction and target recognition technology for roadside perception scenarios. Background Technology

[0002] Roadside perception is a crucial component of intelligent transportation systems and vehicle-to-everything (V2X) tasks, extending the perception range of autonomous vehicles and improving traffic safety. However, existing roadside point cloud 3D object detection technologies face the following challenges:

[0003] (1) Limitations of existing architecture: Although traditional methods based on convolutional neural networks (CNN) have local perception capabilities, their receptive field is limited; while methods based on Transformer have high computational overhead and are difficult to meet the real-time and large-scene processing requirements of roadside perception.

[0004] (2) Deficiencies of State-Space Models (SSM / Mamba): Although state-space models, represented by Mamba, are efficient in long sequence modeling, they are not well-suited for direct application to bird's-eye view (BEV) features of roadside point clouds.

[0005] Local connectivity disruption: Mamba uses a recursive scanning method to flatten a 2D image into a 1D sequence, which causes physically adjacent grids to be too far apart in the sequence, disrupting local spatial dependencies.

[0006] Historical relationship obliteration: There are a large number of empty grids in the roadside point cloud scene. In recursive calculation, this invalid information can easily overwhelm the features of small targets, resulting in blurred foreground and background.

[0007] Therefore, there is an urgent need for a roadside point cloud detection method that can efficiently utilize the global context while preserving local geometric features and historical memory. Summary of the Invention

[0008] The purpose of this invention is to provide a 3D target detection method based on roadside point clouds using the PillarMamba framework. This method introduces Hybrid State Space Block (HSB) and Cross-Stage State Space Group (CSG), effectively solving the problems of local connectivity disruption and feature forgetting, while balancing computational efficiency and detection accuracy.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a 3D target detection method for road test point clouds based on a hybrid state-space model, comprising the following steps:

[0010] S1. Acquire point cloud data collected by roadside lidar, and convert the point cloud data into dense bird's-eye view (BEV) feature map through columnar feature encoder (PFE);

[0011] S2. Input the bird's-eye view BEV feature map into the backbone network for feature extraction; the backbone network contains multiple levels of cross-stage state space groups (CSGs) for extracting multi-scale feature representations;

[0012] The specific calculation formula for the cross-stage state space group is as follows:

[0013]

[0014]

[0015] in, As input features, and These are the feature components after segmentation. For dimensionality reduction convolution, This indicates the processing of mixed state space blocks. This indicates a splicing operation. This is an up-dimensional convolution.

[0016] S3. In the cross-stage state space group (CSG), the input features are segmented along the channel dimension into a first feature component and a second feature component; the first feature component is processed by a hybrid state space block (HSB) sequence, while the second feature component maintains cross-stage connectivity; the processed first feature component and the second feature component are concatenated and fused to generate the output features.

[0017] The hybrid state space block is configured to acquire the local-global context of the roadside point cloud, and its specific processing includes:

[0018] S3.1. The input features are processed using a 2D selective scanning module, and the global receptive field is obtained through a multi-directional scanning mechanism;

[0019] Furthermore, the 2D Selective Scan (SS2D) module employs a four-way cross-scan strategy to flatten the bird's-eye view features into a sequence for recursive state-space equation calculation, and then restores it to a two-dimensional feature map after the calculation is completed.

[0020] S3.2. Use local convolution operations to process features in order to aggregate local geometric features and maintain spatial neighborhood connectivity;

[0021] Furthermore, deep convolutional layers are used to replace the multilayer perceptron layers in the state-space model. Local feature extraction is performed on the layer-normalized features, and the features are added to the original features through residual connections.

[0022] S3.3. Use residual attention mechanism to weight the features in order to preserve historical feature information during the recursive scanning process.

[0023] Furthermore, the residual attention mechanism employs channel attention operations, applying channel weights to the features after upsampling convolution and performing element-wise operations with the original input features after depth convolution.

[0024] S4. The multi-scale features output by the backbone network are fused and upsampled, and the detection head outputs the detection results containing the target category and 3D bounding box.

[0025] Furthermore, the detection head adopts a center-based detection structure to predict the target's center point heatmap and bounding box regression parameters, including position coordinates, size, and yaw angle.

[0026] A 3D target detection system for road test point clouds based on a hybrid state-space model includes:

[0027] Memory, used to store computer programs;

[0028] A processor for executing the computer program to implement the steps of the method.

[0029] The core technical solution of this invention includes:

[0030] (1) Pillar feature encoding: convert the raw point cloud on the roadside into a dense bird's-eye view (BEV) feature map.

[0031] (2) Cross-stage state space group (CSG): As the basic unit of the backbone network, it reduces the computational burden and enhances the complementarity of inter-layer features through channel splitting and cross-stage fusion.

[0032] (3) Hybrid State Space Block (HSB): Based on the state space model, it integrates local convolution and residual attention. Local convolution is used to enhance neighborhood connections, and residual attention is used to preserve historical memory and prevent the loss of features of small targets.

[0033] This invention aims to solve the technical problems of poor adaptability of sparse methods for vehicle-side point cloud 3D target detection, local connection breaks and historical relationship forgetting in state space models, and excessive computational overhead in dense scenes. By using column feature encoding, cross-stage state space groups and hybrid state space blocks in a collaborative design, a balance between detection accuracy and real-time performance can be achieved. Attached Figure Description

[0034] Figure 1 This is a flowchart of the overall network architecture of the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of a cross-stage state space group (CSG);

[0036] Figure 3 This is a schematic diagram of the internal structure of the Hybrid State Space Block (HSB), illustrating the connection between SS2D, local convolution, and the attention mechanism. Detailed Implementation

[0037] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0038] like Figure 1 As shown, a 3D target detection method based on a hybrid state-space model for road test point clouds includes the following steps:

[0039] S1. Acquire and preprocess roadside point cloud data;

[0040] S11. Data Acquisition:

[0041] Raw point cloud data is collected using lidar installed on the roadside (such as traffic light poles or streetlight poles). Where N is the number of point clouds, and 4 represents the original features of each point. , representing the three-dimensional coordinates and reflection intensity, respectively.

[0042] S12. Feature Expansion:

[0043] Following the PointPillars method, the original point cloud is divided into vertical pillars. For each point within a pillar, its geometric center and offset are calculated, expanding the feature dimension from 4 to 9 dimensions. To enrich geometric information.

[0044] S13. Generate BEV features:

[0045] The augmented point cloud features are encoded using a linear layer, and then a pseudo-image feature map, namely a bird's-eye view (BEV) feature map, is generated through a max-pooling operation. .in The number of feature channels, and These are the grid dimensions of the BEV space in the x-axis and y-axis directions, respectively.

[0046] S2. Construct a CSG-based backbone network for feature extraction;

[0047] The backbone network is responsible for... Multi-scale features are extracted. This embodiment employs cross-stage state space groups (CSGs) to balance computational efficiency and network expressive power.

[0048] S21. Multi-stage processing: The backbone network is divided into multiple stages;

[0049] Phase 1: After processing by the first CSG module, the result is... To maintain resolution.

[0050] Downsampling phase: After processing by the downsampling layer and CSG module, the resolution is halved. And so on, generating sequentially. and .

[0051] S22.CSG internal calculation process: as follows Figure 2 As shown, for any input features The CSG processing logic is as follows:

[0052] First use Convolution pairs Perform dimensionality reduction, and then split it into two parts along the channel dimension: and As shown in formula (1):

[0053] (1)

[0054] The branch of the identity mapping remains unchanged. Feed into the stack Deep feature extraction is performed in a Mixed State Space Block (HSB).

[0055] The features after HSB processing and The parts are then assembled, and finally... The number of channels recovered from the convolution is calculated using the formula shown in formula (2):

[0056] (2)

[0057] This design significantly reduces the computational burden in dense point cloud environments through channel dimensionality reduction and segmentation, while retaining the ability to reuse features across layers.

[0058] S3. Specific implementation of Hybrid State Space Block (HSB);

[0059] HSB is the core component of this invention, designed to solve the problems of "local connectivity disruption" and "historical relationship forgetting" encountered by the standard Mamba model on roadside point clouds. Figure 3 As shown in the HSB module, the processing flow is as follows:

[0060] S31. Global Feature Modeling (SS2D):

[0061] The input features are first normalized by LayerNorm and then enter the 2D Selective Scan (SS2D) module. SS2D uses a cross-scan strategy to flatten the BEV feature map into a 1D sequence along four directions (upper left, lower right, etc.) and uses a recursive state space equation to capture long-range global dependencies, as shown in formula (3):

[0062] (3)

[0063] The output features are denoted as The output is shown in formula (4):

[0064] (4)

[0065] S32. Local connectivity repair:

[0066] To repair the spatial breaks caused by the scan, this embodiment replaces the MLP layer after the traditional Mamba block with a local depthwise convolution (DWConv, kernel size 3). This step forces the adjacent mesh features in the polymer physical space to be merged, as shown in Equation (5):

[0067] (5)

[0068] S33. Preservation of historical memory:

[0069] A residual attention mechanism is introduced. After convolutional upsampling of the features, channel weights are calculated using attention mechanisms (such as global average pooling and weight generation in SE Block). The final output combines the convolutional features of the original input with the attention-weighted features, as shown in formula (6):

[0070] (6)

[0071] This mechanism prevents small target features from being "overwhelmed" or forgotten in long recursive sequences containing a large number of empty grids.

[0072] S4. Multi-scale fusion and target detection;

[0073] S41. Feature Fusion:

[0074] Feature maps of different resolutions The features are concatenated along the channel dimension and then upsampled to the original feature map size to obtain the fused features. .

[0075] S42. Detection Output: [This will be...] Input detection head. This embodiment uses a center-based detection head.

[0076] S43. Result Generation: The detection head outputs a heatmap of the target's center point and regression parameters, ultimately generating a 3D bounding box. Includes the center coordinates of the target ,size and yaw angle .

[0077] The above description represents preferred embodiments of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Modifications and variations made by those skilled in the art based on the core ideas of the present invention should be within the scope of protection of the appended claims.

Claims

1. A 3D target detection method for road test point clouds based on a hybrid state-space model, characterized in that, Includes the following steps: S1. Acquire point cloud data collected by roadside lidar, and convert the point cloud data into dense bird's-eye view BEV feature map through columnar feature encoder PFE. S2. Input the bird's-eye view BEV feature map into the backbone network for feature extraction; the backbone network contains multiple levels of cross-stage state space groups (CSGs) for extracting multi-scale feature representations; S3. In the cross-stage state space group (CSG), the input features are segmented along the channel dimension into a first feature component and a second feature component. The first feature component is processed by the hybrid state space block (HSB) sequence, while the second feature component maintains cross-stage connectivity. The processed first feature component and the second feature component are then concatenated and fused to generate the output features. S4. The multi-scale features output by the backbone network are fused and upsampled, and the detection head outputs the detection results containing the target category and 3D bounding box.

2. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 1, characterized in that, The specific calculation formula for the cross-stage state space group described in S2 is as follows: ， ， in, As input features, and These are the feature components after segmentation. For dimensionality reduction convolution, This indicates the processing of mixed state space blocks. This indicates a splicing operation. This is an up-dimensional convolution.

3. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 1, characterized in that, The hybrid state space block described in S3 is configured to acquire the local-global context of the roadside point cloud, and its specific processing includes: S3.

1. The input features are processed using a 2D selective scanning module, and the global receptive field is obtained through a multi-directional scanning mechanism; S3.

2. Use local convolution operations to process features in order to aggregate local geometric features and maintain spatial neighborhood connectivity; S3.

3. Use residual attention mechanism to weight the features in order to preserve historical feature information during the recursive scanning process.

4. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 3, characterized in that, In step 3.2, a deep convolutional layer is used to replace the multilayer perceptron layer in the state space model. Local features are extracted from the features that have been normalized by the layers, and then added to the original features through residual connections.

5. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 3, characterized in that, In step S3.3, the residual attention mechanism employs channel attention operations, applying channel weights to the features after upsampling convolution and performing element-wise operations with the original input features after depth convolution.

6. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 3, characterized in that, The 2D selective scanning module described in S3 adopts a four-directional cross-scanning strategy to flatten the bird's-eye view features into a sequence for recursive state-space equation calculation, and restores it to a two-dimensional feature map after the calculation is completed.

7. The 3D target detection method for road test point clouds based on a hybrid state-space model according to claim 1, characterized in that, In S4, the detection head adopts a center-point-based detection structure to predict the center-point heatmap and bounding box regression parameters of the target. The regression parameters include position coordinates, size, and yaw angle.

8. A 3D target detection system for road test point clouds based on a hybrid state-space model, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the 3D target detection method for road test point clouds based on a hybrid state-space model as described in any one of claims 1 to 7.

Images