3D target detection method based on multi-coding feature fusion

Abstract

This invention discloses a 3D object detection method based on multi-encoding feature fusion, mainly addressing the problem of low detection accuracy in existing technologies due to high false positive and false negative rates in occluded scenes. The implementation scheme is as follows: A training sample set and a test sample set are established and the data is preprocessed; a 3D object detection network is constructed, including a voxel encoder, a multi-scale pillar encoder, a global feature learning module, and a center feature-guided prediction module, and its loss function is defined; the network is iteratively trained using the training sample set, the network loss value is calculated, the network gradient is calculated through backpropagation, and the weight parameters are updated until the maximum number of iterations is reached; the test sample set is input into the trained 3D object detection model to obtain the 3D object detection results. This invention reduces the number of false positives, false negatives, and candidate regions in occluded scenes, improves detection speed and 3D object detection accuracy, and can be applied to autonomous driving, robot navigation, and video surveillance.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and autonomous driving technology, and specifically relates to a three-dimensional target detection method that can be applied to autonomous driving, robot navigation and video surveillance. Background Technology

[0002] In the field of computer vision, object detection has always been an important research direction. Initially, object detection was mainly based on images, achieving this by recognizing and locating objects in two-dimensional images. However, with the increasing demand for scene understanding, image-based methods have gradually revealed some disadvantages, especially when dealing with complex scenarios such as autonomous driving. One of the disadvantages of image-based methods is their relatively low reliance on depth information. Because images are two-dimensional, it is difficult to directly obtain the three-dimensional shape and spatial information of objects. This may be less than ideal when dealing with tasks that require high depth perception, such as autonomous driving and outdoor navigation.

[0003] To overcome this disadvantage, researchers have gradually turned their attention to point cloud data. Point clouds, composed of a large number of three-dimensional points, are better able to reflect the geometric structure and spatial relationships of objects. Therefore, point cloud-based 3D target detection methods have emerged to more accurately capture the shape, position, and size of targets, providing more comprehensive information for understanding and perceiving the 3D environment.

[0004] Feature fusion plays a crucial role in point cloud 3D object detection. Due to the complexity of point cloud data, a single feature may not be sufficient to fully describe the target. Feature fusion improves the accuracy and robustness of object detection by integrating features from different sources and at different levels. By fusing information from different sensors or perspectives, the system can gain a more comprehensive understanding of the scene, thereby improving object detection performance.

[0005] In 2020, Kuang et al. proposed Voxel-FPN in Sensors. The entire framework is based on an encoder and a decoder. The encoder extracts multi-scale voxel information in a bottom-up manner, while the decoder fuses feature maps of different scales in a top-down manner to achieve accurate detection. However, when performing multi-scale feature fusion, this method introduces over-smoothing, which leads to the loss of target edges or local details. Furthermore, it suffers from false positives and false negatives when dealing with complex scenes with occlusion, resulting in a decrease in the accuracy of 3D target detection.

[0006] In 2019, Lang et al. proposed PointPillars at CVF. This method first projects the raw point cloud data onto a set of vertical pillars to form point pillar data. Then, a convolutional neural network is used to extract features from the point pillar data. Finally, a region proposal network and bounding box regression are applied to the extracted feature maps to detect and locate target objects. However, this method generates a large number of candidate boxes, leading to a decrease in detection speed. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and propose a three-dimensional target detection method and system based on multi-encoding feature fusion, so as to reduce false detections and missed detections in occluded scenes, improve the accuracy of three-dimensional target detection, and improve the detection speed.

[0008] To achieve the above objectives, the present invention provides a three-dimensional target detection method based on multi-coding feature fusion, comprising the following steps:

[0009] (1) Obtain labeled point cloud data of dimension N×K from the public dataset for T scenes, preprocess each point cloud data, and use the preprocessed T1 point cloud scene data as the training sample set E1, and use the remaining T2 preprocessed point cloud scene data to form the test sample set E2, where T≥500, T1>T2, T1+T2=T;

[0010] (2) Construct a 3D target detection network D based on multi-coding feature fusion:

[0011] (2a) Construct a feature fusion encoder H consisting of a voxel encoder based on three-dimensional sparse convolution and a multi-scale pillar encoder based on column extraction connected in parallel;

[0012] (2b) Select the existing Transformer-based global feature learning module M, which includes a standard encoder and decoder, and the prediction module N guided by the central feature;

[0013] (2c) The feature fusion encoder H, the global feature learning module M, and the prediction module N are cascaded in sequence to form a three-dimensional target detection network D;

[0014] (3) Define the loss function L of the 3D target detection network D:

[0015] L = L hea +L reg +L conf

[0016] Among them, L hea For heat map loss, L reg For regression loss, L conf For confidence loss;

[0017] (4) Iteratively train the 3D target detection network D:

[0018] (4a) Initialize the number of iterations to r, and the maximum number of iterations to R. Let the current 3D object detection network D be... r The weight parameter is θ r And let r = 1, D r =D;

[0019] (4b) Randomly select B training samples without replacement from the training sample set E1 as the current 3D object detection network D. r The input is propagated forward to obtain the 3D object detection model D after the current iteration. r ′:

[0020] (4b1) Use the training sample set E1 as the 3D object detection network D r The input is used to learn three-dimensional features for each training sample by using the voxel encoder and the strut encoder in the feature fusion encoder H, and the branches of the two three-dimensional feature learning are fused to obtain the multi-scale BEV feature map P.

[0021] (4b2) The encoder in the global feature learning module M is used to extract features from the multi-scale BEV feature map P to obtain the global features of P. Then, the decoder is used to perform channel-weighted decoding on the global features of P to obtain the BEV feature map.

[0022] (4c) Use the prediction module N guided by the center feature to find the center feature of the BEV feature map and use it to regress the properties of the 3D bounding box.

[0023] (4d) Calculate the heat map loss L respectively hea Regression loss L reg and confidence loss L conf Then, the network gradient is calculated and the weight parameters θ are updated through backpropagation. r The three-dimensional target detection model D for this iteration is obtained. r ′;

[0024] (4e) Determine whether r = R is true:

[0025] If so, then the trained 3D object detection network model D is obtained. * ,

[0026] Otherwise, let r = r + 1, D r =D r ′, return to step (4b);

[0027] (4) Input the test sample set E2 into the trained 3D object detection network model D. *Perform forward propagation to obtain the 3D object detection result for each test sample, including the predicted 3D object bounding box Y in each test sample. pre The angle value θ corresponding to the 3D target bounding box pre And target category.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] First, this invention employs a feature fusion encoder to perform preliminary feature extraction on the point cloud and learns multi-scale feature information of the BEV feature map in the columnar branch. Then, the BEV feature maps on the two branches are spliced ​​and fused. The attention mechanism of Transformer is used to learn the global features of the point cloud data, capture the long-range dependencies between points, and improve the accuracy of 3D target detection.

[0030] Second, the present invention uses a prediction module guided by central features, which can reduce the number of generated candidate regions and improve the speed of 3D target detection. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0032] Figure 2 This is a schematic diagram of the three-dimensional target detection network structure constructed in this invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0034] It should be noted that the step numbers in the specification and claims of this invention are only for the purpose of clearly describing the embodiments of this invention and facilitating understanding, and their order is not limited.

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] Reference Figure 1 The implementation steps of this example include the following:

[0037] Step 1: Obtain the training dataset and the test sample set.

[0038] 1.1) Obtain labeled point cloud data of dimension N×K from T scenes in a public dataset, and preprocess each point cloud data:

[0039] 1.1.1) The point cloud data is first denoised, and each denoised point cloud data is randomly horizontally flipped; 1.1.2) The point cloud data after random horizontal flipping is downsampled to obtain preprocessed point cloud data.

[0040] 1.2) Use the preprocessed T1 point cloud scene data as the training sample set E1, and the remaining T2 preprocessed point cloud scene data as the test sample set E2, where T≥500, T1>T2, T1+T2=T; where N=10000, K=4, T1=850, T2=150, T=1000;

[0041] The nuScenes dataset used in this example was created by the Motional team to support research in computer vision and autonomous driving. This massive dataset covers 1,000 driving scenes in the cities of Boston and Singapore, showcasing the diversity of these two traffic-intensive and challenging driving environments. Each scene is 20 seconds long and carefully selected to demonstrate a variety of driving maneuvers, traffic conditions, and unexpected behaviors. To support common computer vision tasks such as object detection and tracking, the dataset provides accurate 3D bounding box annotations at 2Hz, covering 23 object categories. Additionally, object-level attributes such as visibility, activity, and pose are annotated. The complete nuScenes dataset was released in March 2019 and includes approximately 1.4 million camera images, 390,000 LiDAR scans, 1.4 million radar scans, and 1.4 million object bounding boxes. The dataset contains 1.4 billion annotated points across 40,000 point clouds and 1,000 scenes, with 850 used for training and validation and 150 for testing. Each scene is approximately 20 seconds long, with the LiDAR frequency at 20 FPS.

[0042] Step 2, construct the feature fusion encoder H.

[0043] Reference Figure 2 The implementation of this step is as follows:

[0044] 2.1) Establish a voxel encoder based on three-dimensional sparse convolution: it includes eight sequentially connected three-dimensional sparse convolutional structures, each of which includes a sequentially connected submanifold three-dimensional sparse convolutional layer, a batch normalization layer, and a ReLU activation layer.

[0045] The submanifold 3D sparse convolutional layer comprises three convolutional layers, each using a 3×3×3 3D sparse convolutional kernel to perform different convolution operations, wherein:

[0046] The first convolutional layer performs a convolution operation on the input voxel features to keep the bottom dimension of the top view unchanged;

[0047] The second convolutional layer performs a convolution operation, which reduces the size by half in the top view and performs half downsampling in the height direction, resulting in a feature map of 240×400×64.

[0048] The third convolutional layer performs a convolution operation, which reduces the size by half again in the top view and performs half downsampling in the height direction, resulting in a feature map of 112×400×64.

[0049] The features of the second and third layers are concatenated to form a voxel feature with dimensions of 352×400×64×2. The concatenated voxel feature is then dimensionality-reduced in the height direction to transform the 64-dimensional feature into 128-dimensional features, resulting in a final BEV voxel feature map with dimensions of 352×400×128.

[0050] 2.2) An existing multi-scale pillar encoder based on pillar extraction is selected to convert the point cloud into a multi-scale BEV feature map. The steps are as follows:

[0051] 2.2.1) Along the z-axis, first discretize the point cloud P in 3D space into a uniformly spaced grid on the xy plane. Then create a cylindrical column of size 0.2 × 0.2 and a corresponding index vector. The coordinates of the points in the column are (x, y, z), and each point is represented by (r, x). c ,y c ,z c The vector is expanded as follows: (xp, yp), where r is the reflectivity, the subscript c represents the distance to the arithmetic mean of all points in the cylinder, and the subscript p represents the offset from the center coordinates x and y of the cylinder.

[0052] 2.2.2) By imposing restrictions on the number of non-empty pillars in each sample point cloud P and the number of points N in each pillar, a dense tensor (P, N, D) is created using the sparsity of the point cloud, where D is the dimension of the points. At the same time, a threshold is set to constrain the dense tensor. That is, once the dense tensor exceeds the threshold, it will be fitted by random sampling; otherwise, it will be fitted by applying zero.

[0053] 2.2.3) After obtaining the point cloud data represented in columnar form, the point dimension is expanded to C dimensions through multi-layer MLP processing. Then, the points in the columnar form are processed through max pooling to aggregate the channel information of each point in the columnar form. Finally, the dimensionality is reduced to obtain a BEV feature map with a size of 352×400.

[0054] 2.2.4) The obtained BEV feature map is subjected to three downsampling convolution processes to obtain three sets of features of different scales and dimensions, with dimensions of 64, 128 and 256 respectively. The large-scale feature of 256 is upsampled and the small-scale feature of 64 is deconvolved. The three features are then concatenated and fused to obtain a multi-scale BEV feature map.

[0055] 2.3) A feature fusion encoder H is formed by connecting a voxel encoder based on three-dimensional sparse convolution and a multi-scale pillar encoder based on pillar extraction in parallel.

[0056] Step 3: Select the existing Transformer-based global feature learning module M.

[0057] This example uses the Transformer-based global feature learning module M, which includes a standard encoder and decoder, where:

[0058] The encoder consists of three stacked feature encoding modules. Each module contains two linear layers, three dropout layers, and two normalization layers. The feature map is first linearly transformed by passing through the first linear layer, the first dropout layer, and the second linear layer in sequence; then it is normalized by passing through the first normalization layer, the second normalization layer, the second dropout layer, and the third dropout layer in sequence.

[0059] The decoder consists of 3 convolutional layers, 3 linear layers, 3 dropout layers, and 4 normalization layers. The feature map is first extracted through the 3 convolutional layers, then linearly transformed through the 3 linear layers, and finally normalized through the 3 dropout layers and 4 normalization layers.

[0060] Step 4: Select the existing prediction module N based on central feature guidance.

[0061] This example uses an existing prediction module N based on central feature guidance, including a central head, to obtain a predicted value Y. c The steps are as follows:

[0062] 4.1) Define the input image I∈R W×H×3 The image is input into the center head to generate a heatmap of K key points. Where the width is W, the height is H, R is the output stride, and C is the number of keypoint types;

[0063] 4.2) Extract peak points from the heatmap of each key point in sequence, and select points within 8 or more neighborhoods of these peak points as peak candidate points;

[0064] 4.3) Select the top n points from all candidate peak points to form the set of detected center points: in The coordinates of points within the set of detected center points;

[0065] 4.4) Select the set of detected center points Regression calculations are performed at any center point within the range to obtain the predicted value Y. c :

[0066]

[0067] Where δ is the offset prediction factor. They are The size predictor, c is the sample class.

[0068] Step 5: Construct the 3D target detection network D.

[0069] 5.1) The feature fusion encoder H, global feature learning module M, and prediction module N are cascaded in sequence to form a three-dimensional target detection network structure D;

[0070] 5.2) According to the heat map, the loss L hea Regression loss L reg Confidence loss L conf Define the loss function L for the 3D object detection network structure D:

[0071] L = L hea +L reg +L conf

[0072] in:

[0073]

[0074]

[0075]

[0076] Among them, Y c This is the predicted value, where N is the number of keypoints in the image, c is the sample category, α and β are adjustable factors, and Y is the predicted value. c For predicted values, For the corresponding target value, I is the confidence prediction value. t It corresponds to the ground truth confidence level, IoU. t It is the cross-union ratio between the t-th predicted box and the ground truth box.

[0077] Step 6: Iteratively train the 3D target detection network D.

[0078] 6.1) Initialize the number of iterations to r, and the maximum number of iterations to 100. Let the current 3D object detection network D be... r The weight parameter is θ r And let r = 1, D r =D;

[0079] 6.2) Randomly select two training samples without replacement from the training sample set E1 as the current 3D object detection network D. r The input is used to learn three-dimensional features for each training sample by using the voxel encoder and the strut encoder in the feature fusion encoder H, and the branches of the two three-dimensional feature learning are fused to obtain the multi-scale BEV feature map P.

[0080] 6.3) The encoder in the global feature learning module M is used to extract features from the multi-scale BEV feature map P to obtain the global features of P. Then, the decoder is used to perform channel-weighted decoding on the global features of P to obtain the BEV feature map.

[0081] 6.4) Use the prediction module N guided by the center feature to find the center feature of the BEV feature map and use it to regress the attributes of the 3D bounding box;

[0082] 6.5) Calculate the current 3D object detection network D respectively. r Heatmap loss L hea Regression loss L reg and confidence loss L conf The gradient of the network is then calculated through backpropagation, and the weight parameters θ are updated through the Adam optimizer. r The three-dimensional target detection model D for this iteration is obtained. r ′;

[0083] 6.6) Determine if r = 100 is true:

[0084] If so, then the trained 3D object detection network model D is obtained. * ,

[0085] Otherwise, let r = r + 1, D r =D r ', return to step 6.2).

[0086] Step 7: Obtain the 3D target detection results.

[0087] The test sample set E2 is used as the trained 3D object detection network model D based on multi-encoding feature fusion. * The input is forward-propagated to obtain the 3D object detection result for each test sample, including the predicted 3D object bounding box Y in each test sample. pre and the angle value θ corresponding to the 3D target bounding box.pre Categories of three-dimensional targets.

[0088] The effects of this invention can be further illustrated by the following simulation experiments:

[0089] I. Simulation conditions.

[0090] The simulation experiment was conducted on an Ubuntu 20.04 system. The model training used the PyTorch 1.9.0 deep learning framework, with one NVIDIA RTX 4090 graphics card and a total video memory of 64GB. During the training experiment, the batch size was set to 8, the epoch was set to 100, the initial learning rate was set to 0.01, and the general Adam optimizer was selected.

[0091] II. Simulation Content

[0092] Simulation 1 inputs the nuScenes test set into a pre-trained 3D object detection model to obtain 3D object detection results. Then, the performance is evaluated by comparing the mean accuracy (mAP), nuScenes detection score (NDS), and neural programming metric (PKL) with nine other publicly available 3D object detection methods. The results are shown in Table 1.

[0093] Sensor L indicates that only radar point cloud is used as input data, while L+C indicates that both radar point cloud and image are used as input data.

[0094] The nine existing three-dimensional target detection methods are as follows:

[0095] PointPillars, "PointPillars: Fast encoders for object detection from point clouds", was published in CVF in 2019.

[0096] PointPainting, "PointPainting: Sequential fusion for 3D object detection", was published in CVF in 2020.

[0097] CBGS, "Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection", was indexed by arXiv in 2019.

[0098] CVCNet, "Cross View Capture for Stereo Image Super-Resolution", was published in IEEE in 2021.

[0099] CenterPoint, "Center-based 3D Object Detection and Tracking", published at CVPR in 2021.

[0100] VoxelNext, "VoxelNext: Fully sparse voxelnet for 3D object detection and tracking", was published in CVF in 2023.

[0101] HVDetFusion, "HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework", was included on arXiv in 2023.

[0102] mmFUSION, "mmFUSION: Multimodal Fusion for 3D Objects Detection", was included in arXiV in 2023.

[0103] Table 1 Comparison results of the present invention on the nuScenes dataset.

[0104]

[0105] As shown in Table 1, under the mAP metric, this invention achieves a 7.7% performance improvement compared to the CenterPoint method with single-point cloud input, a 0.2% improvement compared to the fully sparse network VoxelNext method, and improvements of 12% and 0.5% respectively compared to the multi-view fusion 3D-CVF method and mmFUSION method. On the NDS detection benchmark, this invention also achieves considerable performance improvements. It improves by 1.2% compared to the CenterPoint method, and by 2.3% and 0.3% respectively compared to the multi-modal input HVDetFusion and mmFUSION methods. The improved performance in mAP and NDS metrics demonstrates the competitive performance of this invention in terms of detection accuracy. Under the PKL evaluation metric, compared to PointPainting, this invention improves performance by 0.17%. Although its performance decreases by 0.03% and 0.08% compared to CenterPoint and mmFUSION respectively, this invention still shows a significant improvement in accuracy compared to both, making it capable of handling 3D target detection tasks.

[0106] Simulation 2: The nuScenes test set is input into the trained 3D object detection model to obtain 3D object detection results. Then, the performance is evaluated using mAP, NDS metrics and the classic PointPillars method. The detection speed is evaluated using the detection time metric. The results are shown in Table 2.

[0107] Table 2 Comparison results between the present invention and PointPillars

[0108]

[0109] The PointPillars method, described in "PointPillars: Fast encoders for object detection from point clouds," was published in CVF in 2019.

[0110] As shown in Table 2, compared to PointPillars, the detection time of this invention is reduced by 69.8 seconds. This is because the present invention directly uses the center point of the target for detection, without needing to generate a large number of candidate boxes. Therefore, it can reduce...

[0111] Reduce the computational overhead required to generate and process a large number of candidate boxes, thereby improving computational efficiency.

[0112] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional target detection method based on multi-encoding feature fusion, characterized in that, Includes the following steps: (1) Obtain labeled point cloud data of dimension N×K from the public dataset for T scenes, preprocess each point cloud data, and use the preprocessed T1 point cloud scene data as the training sample set E1, and use the remaining T2 preprocessed point cloud scene data to form the test sample set E2, where T≥500, T1>T2, T1+T2=T; (2) Construct a 3D target detection network D based on multi-coding feature fusion: (2a) Construct a feature fusion encoder H consisting of a voxel encoder based on three-dimensional sparse convolution and a multi-scale pillar encoder based on column extraction connected in parallel; (2b) Select the existing Transformer-based global feature learning module M, which includes a standard encoder and decoder, and the prediction module N guided by the central feature; (2c) The feature fusion encoder H, the global feature learning module M, and the prediction module N are cascaded in sequence to form a three-dimensional target detection network D; (3) Define the loss function L of the 3D target detection network D: L=L hea +L reg +L conf Among them, L hea For heat map loss, L reg For regression loss, L conf For confidence loss; (4) Iteratively train the 3D target detection network D: (4a) Initialize the number of iterations to r, and the maximum number of iterations to R. Let the current 3D object detection network D be... r The weight parameter is θ r And let r = 1, D r =D; (4b) Randomly select B training samples without replacement from the training sample set E1 as the current 3D object detection network D. r The input is propagated forward to obtain the 3D object detection model D after the current iteration. r ′: (4b1) Use the training sample set E1 as the 3D object detection network D r The input is used to learn three-dimensional features for each training sample by using the voxel encoder and the strut encoder in the feature fusion encoder H, and the branches of the two three-dimensional feature learning are fused to obtain the multi-scale BEV feature map P. (4b2) The encoder in the global feature learning module M is used to extract features from the multi-scale BEV feature map P to obtain the global features of P. Then, the decoder is used to perform channel-weighted decoding on the global features of P to obtain the BEV feature map. (4c) Use the prediction module N guided by the center feature to find the center feature of the BEV feature map and use it to regress the properties of the 3D bounding box. (4d) Calculate the heat map loss L respectively hea Regression loss L reg and confidence loss L conf Then, the network gradient is calculated through backpropagation, and the weight parameters θ are updated through the Adam optimizer. r The three-dimensional target detection model D for this iteration is obtained. r ′; (4e) Determine whether r = R is true: If so, then the trained 3D object detection network model D is obtained. * , Otherwise, let r = r + 1, D r =D r ′, return to step (4b); (5) Input the test sample set E2 into the trained 3D object detection network model D. * Perform forward propagation to obtain the 3D object detection result for each test sample, including the predicted 3D object bounding box Y in each test sample. pre The angle value θ corresponding to the 3D target bounding box pre And target category.

2. The method according to claim 1, characterized in that, In step (1), each point cloud data is preprocessed by first denoising the point cloud data and then randomly flipping each point cloud data after denoising; then downsampling the point cloud data after random horizontal flipping to obtain T preprocessed point cloud data.

3. The method according to claim 1, characterized in that, The voxel coder based on three-dimensional sparse convolution in step (2a) includes eight sequentially connected three-dimensional sparse convolutional structures. Each three-dimensional sparse convolutional structure includes a sequentially connected submanifold three-dimensional sparse convolutional layer, a batch normalization layer, and a ReLU activation layer.

4. The method according to claim 3, characterized in that, The submanifold 3D sparse convolutional layer comprises three convolutional layers, each using a 3×3×3 3D sparse convolutional kernel to perform different convolution operations. The first convolutional layer performs a convolution operation on the input voxel features to keep the bottom dimension of the top view unchanged; The second convolutional layer performs a convolution operation, which reduces the size by half in the top view and performs half downsampling in the height direction, resulting in a feature map of 240×400×64. The third convolutional layer performs a convolution operation, which reduces the size by half again in the top view and performs half downsampling in the height direction, resulting in a feature map of 112×400×64. The features from the second and third layers are concatenated to form a voxel feature with dimensions of 352×400×64×2. The concatenated voxel feature is then dimensionality-reduced in the height direction, transforming the 64-dimensional feature into 128-dimensional features. The final BEV voxel feature map has dimensions of 352×400×128.

5. The method according to claim 1, characterized in that, The multi-scale pillar encoder based on pillar extraction in step (2a) first discretizes the point cloud in three-dimensional space into a uniformly spaced grid on the xy plane along the z-axis direction, and then creates a pillar with a plane size of 0.2×0.2 and a corresponding index vector. The coordinates of the points in the pillar are (x,y,z). Each point is expanded with a vector of (r,xc,yc,zc,xp,yp), where r is the reflectivity, the c subscript represents the distance to the arithmetic mean of all points in the pillar, and the p subscript represents the offset from the x and y centers of the pillar.

6. The method according to claim 1, characterized in that, The encoder and decoder included in the Transformer-based global feature learning module M in step (2b) have the following structure: The encoder consists of three stacked feature encoding modules. Each module contains two linear layers, three dropout layers, and two normalization layers. The feature map is first linearly transformed by passing through the first linear layer, the first dropout layer, and the second linear layer in sequence; then it is normalized by passing through the first normalization layer, the second normalization layer, the second dropout layer, and the third dropout layer in sequence. The decoder includes 3 convolutional layers, 3 linear layers, 3 dropout layers, and 4 normalization layers. The feature map is first extracted by passing through the 3 convolutional layers, then transformed by passing through the 3 linear layers, and finally normalized by passing through the 3 dropout layers and the 4 normalization layers.

7. The method according to claim 1, characterized in that, The prediction module N based on center features in step (2b) includes a center head implemented based on 2D convolution, which is used to generate a heatmap peak at the center location of any detected object to obtain a predicted value Y. c Its implementation includes the following: (2b1) Define the input image I∈R W×H×3 The image is input into the center head to generate a heatmap of K key points. Where the width is W, the height is H, R is the output stride, and C is the number of keypoint types; (2b2) ​​Extract peak points from each key point heatmap in sequence, and select points within 8 neighborhoods whose values ​​are greater than or equal to these peak points as peak candidate points; (2b3) Select the top n points from all peak candidate points to form the set of detected center points: in The coordinates of points within the set of detected center points; (2b4) Select the set of detected center points Regression calculations are performed at any center point within the range to obtain the predicted value Y. c : Where δ is the offset prediction factor. They are The size predictor, c is the sample class.

8. As described in claim 1, characterized in that, The heat map loss L in step (3) hea The calculation is as follows: Where Y c α is the predicted value, N is the number of keypoints in the image, c is the sample category, and α and β are adjustable factors.

9. As described in claim 1, characterized in that, The regression loss L in step (3) reg The calculation formula is as follows: Among them, Y c For predicted values, This is the corresponding target value.

10. As described in claim 1, characterized in that, The confidence loss L in step (3) conf The calculation is as follows: in, I is the confidence prediction value. t It corresponds to the ground truth confidence level, IoU. t It is the cross-union ratio between the t-th predicted box and the ground truth box.

Images