A three-dimensional target detection method based on multi-scale feature fusion

Abstract

The application discloses a three-dimensional target detection method based on multi-scale feature fusion, characterized by acquiring point cloud data of a target object to be detected; inputting the point cloud data into a target detection model, the target detection model comprising a feature coding module, a feature fusion module and a detection head; the feature coding module comprising a 3D Swin transformer module, a multi-scale feature fusion module and a dimension reduction module; the 3D Swin transformer module comprising a 3D partition, stage 1, stage 2, stage 3 and stage 4 connected in sequence; the detection head comprising an RPN network and a RoI Pooling layer; inputting the output of the target detection model into a full connection layer, the full connection layer being connected with two parallel classification branches and identification branches, and finally completing detection; the method has the advantages of enhancing feature extraction capability, improving the accuracy of three-dimensional target detection, making up for the problem of two-dimensional data loss, reducing the loss of point cloud feature and two-dimensional feature information, increasing the feature description of a target and improving the precision of three-dimensional target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional target detection, and particularly relates to a three-dimensional target detection method based on multi-scale feature fusion. BACKGROUND

[0002] Three-dimensional target detection is to analyze the spatial information of a target based on geometric analysis and mathematical statistics methods according to two-dimensional or three-dimensional sensor information, and to obtain the position, boundary, category, direction and other information of the target. Because the three-dimensional point cloud data generated by a laser radar scanner has a very high ability to restore the three-dimensional geometry, surface and scale information of a target, it plays an important role in obtaining scene semantic information and spatial information, but the three-dimensional point cloud data has the characteristics of disorder, non-structure and sparseness, and the three-dimensional feature of the point cloud is easy to be lost.

[0003] The existing three-dimensional target detection method of laser point cloud generally detects based on point cloud projection, projects the point cloud into two-dimensional views of different angles, and then uses a mature two-dimensional target detection network to realize target detection, but the projection operation inevitably loses the height information of the point cloud, resulting in a reduction in detection accuracy. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a three-dimensional target detection method based on multi-scale feature fusion, which not only enhances the three-dimensional feature extraction capability of the point cloud, but also improves the detection accuracy.

[0005] The technical scheme adopted by the present application to solve the above technical problem is: a three-dimensional target detection method based on multi-scale feature fusion, comprising the following steps:

[0006] ① scanning a target object to be detected by a laser radar scanner to obtain point cloud data of the target object to be detected;

[0007] ② inputting the point cloud data of the target object to be detected into a target detection model to obtain a candidate box of the target object to be detected;

[0008] ③ inputting the candidate box of the target object to be detected into a full connection layer, and connecting a classification branch and a recognition branch in parallel after the full connection layer, obtaining the category of the target object to be detected by the classification branch, and obtaining the coordinates, size and direction of the target object to be detected by the recognition branch, to complete the detection;

[0009] The target detection model comprises a feature coding module, a feature fusion module and a detection head; the feature coding module comprises a 3D Swin transformer module, a multi-scale feature fusion module and a dimension reduction module; the 3D Swin transformer module comprises a 3D partition, stage 1, stage 2, stage 3 and stage 4 connected in sequence; the detection head comprises an RPN network and a RoI Pooling layer, and the specific steps of obtaining the candidate frame of the target object to be detected are as follows:

[0010] 2-1 input the point cloud data of the target object to be detected into the 3D partition, and output and obtain 3D feature maps of different scale sizes from stage 1, stage 2, stage 3 and stage 4;

[0011] 2-2 input the 3D feature maps of different scale sizes into the multi-scale feature fusion module for feature fusion, and obtain the fused 3D feature maps;

[0012] 2-3 input the fused 3D feature maps into the dimension reduction module for dimension reduction operation, and obtain the 2D feature maps of the BEV perspective;

[0013] 2-4 input the 3D feature maps of different scale sizes and the 2D feature maps into the feature fusion module, and obtain the fused feature maps;

[0014] 2-5 input the fused feature maps into the RPN network, and obtain the ROI of the potential target object to be detected;

[0015] 2-6 input the ROI into the RoI Pooling layer to extract the features of the ROI, and obtain the candidate frame of the target object to be detected.

[0016] Compared with the prior art, the advantages of the present application are that the 3D Swin Transforme module can extract 3D feature maps of different scale sizes to provide accurate detailed features, construct semantic information of the target object to be detected into the feature maps, and enhance the three-dimensional feature extraction capability; the multi-scale feature fusion module is used to fuse the 3D feature maps of different scale sizes to obtain the fused 3D feature maps, improve the accuracy of three-dimensional target detection, and reduce the 3D feature maps to 2D feature maps through the dimension reduction module; the feature fusion module is used to fuse the 3D feature maps of different scale sizes and the 2D feature maps, which makes up for the problem of two-dimensional data loss, reduces the loss of point cloud feature and two-dimensional feature information, increases the feature description of the target object to be detected, and improves the accuracy of three-dimensional target detection. Verification on the KITTI dataset shows that for the car category, the 3D AP R40 of the present application is 93.20 under the simple level, and the 3D AP R40For 86.15, 3D AP under difficulty level R40 For 84.2, 3D AP under 3 difficulty levels R40 For 87.85, compared with the Voxel-RCNN method, it is increased by 0.82%, 0.86%, 1.34% and 1.01% respectively, and the 3D AP of the present application on the medium difficulty car R11 For 86.93%, compared with the Voxel-RCNN method, it is increased by 2.41%, so the present application can improve the accuracy of three-dimensional target detection.

[0017] Further, the specific operation process of step ②-1 is as follows:

[0018] The RGB channel number of the point cloud data of the target object to be detected obtained by scanning the target object to be detected by the laser radar scanner is 3, the block size in the 3D partition is set to 2x3x3, the input target object to be detected The point cloud data of the target object to be detected is divided by the 3D partition, and the vector is output as the input of stage 1, wherein D represents the spatial dimension of the input target object to be detected point cloud data, H represents the height dimension of the input target object to be detected point cloud data, and W represents the width dimension of the input target object to be detected point cloud data;

[0019] Stage 1 includes a linear embedding layer and two consecutive Point Swin Transformer Blocks, and outputs feature map and as the input of stage 2, wherein C represents the feature dimension of the 3D feature map;

[0020] Stage 2 includes a region merging layer and two consecutive Point Swin Transformer Blocks, and outputs 3D feature map and as the input of stage 3;

[0021] Stage 3 includes a region merging layer and six consecutive Point Swin Transformer Blocks, and outputs 3D feature map and as the input of stage 4;

[0022] Stage 4 includes a region merging layer and two consecutive Point Swin Transformer Blocks, and outputs 3D feature map;

[0023] Finally, the 3D feature map of 3D feature map, 3D feature map, 3D feature map and 3D feature map are obtained.

[0024] Further, the specific operation process of obtaining the fused 3D feature map in step ②-2 is as follows: the 3D feature map output by stage 1 is denoted as C1, the 3D feature map output by stage 2 is denoted as C2, the 3D feature map output by stage 3 is denoted as C3, and the 3D feature map output by stage 4 is denoted as C4. A 1x1 convolution operation is performed on C4 to obtain a feature map P4. The feature map P4 after upsampling is fused with the C3 after 1x1 convolution operation to obtain a feature map P3. The feature map P3 after upsampling is fused with the C2 after 1x1 convolution operation to obtain a feature map P2. The feature map P2 after upsampling is fused with the C1 after 1x1 convolution operation to obtain a feature map P1, which is taken as the fused 3D feature map, wherein the step size of the upsampling is 2.

[0025] Further, the specific operation process of obtaining the BEV perspective 2D feature map in step ②-3 is as follows: the data of the fused 3D feature map in the z-axis direction is compressed to obtain a BEV perspective 2D feature map with only x-axis direction data and y-axis direction data.

[0026] Further, the specific operation process of obtaining the fused feature map in step ②-4 is as follows: the 3D feature map output by stage 1 is fused with the 2D feature map by a concat function to obtain a feature map Z1. The 3D feature map output by stage 2 is fused with the feature map Z1 by a concat function to obtain a feature map Z2. The 3D feature map output by stage 3 is fused with the feature map Z2 by a concat function to obtain a feature map Z3. The 3D feature map output by stage 4 is fused with the feature map Z3 by a concat function to obtain a fused feature map.

[0027] Further, the loss function of the RPN network consists of a classification loss and a regression loss, wherein N fg represents the number of candidate boxes, represents the output of the classification branch of the jth candidate box, represents the output of the identification branch of the jth candidate box, represents the classification class of the jth candidate box, represents the identification target of the jth candidate box, L cls represents a Focal loss function, L reg represents a HuberLoss function, j represents the classification of the jth candidate box, j∈[0,b], b represents the total number of detection targets, represents an identification loss calculated only using the candidate box. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0029] Figure 2 This is a schematic diagram of the architecture of the 3D Swin transformer module in this invention;

[0030] Figure 3 This is a diagram showing a comparison of the visualization results of the present invention and the Voxel-Rcnn method. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0032] like Figure 1 As shown, a 3D target detection method based on multi-scale feature fusion includes the following steps:

[0033] ① The point cloud data of the target object to be detected is obtained by scanning the target object with a lidar scanner;

[0034] ② Input the point cloud data of the target object to be detected into the target detection model to obtain the candidate bounding box of the target object;

[0035] The object detection model includes a feature encoding module, a feature fusion module, and a detection head. The feature encoding module includes a 3D Swin transformer module, a multi-scale feature fusion module, and a dimensionality reduction module. The 3D Swin transformer module consists of sequentially connected 3D patch partitions, Stage 1, Stage 2, Stage 3, and Stage 4. The detection head includes an RPN network and an RoI Pooling layer. The specific steps for obtaining candidate bounding boxes for the target object are as follows:

[0036] ②-1 The point cloud data of the target object to be detected is input into the 3D partition, and 3D feature maps of different scales are output by stages 1, 2, 3, and 4; such as Figure 2 As shown;

[0037] The specific operation process of step ②-1 is as follows:

[0038] The target object is scanned using a LiDAR scanner to obtain point cloud data. The RGB channels of the point cloud data are set to 3. The block size in the 3D partition is set to 2×3×3. The 3D partition divides the input point cloud data of the target object and outputs... a vector and as the input of stage 1, wherein D represents the spatial dimension of the input point cloud data of the target object to be detected, H represents the height dimension of the input point cloud data of the target object to be detected, and W represents the width dimension of the input point cloud data of the target object to be detected;

[0039] Stage 1 includes a linear embedding layer and two consecutive Point SwinTransformer blocks, and outputs a 3D feature map of and as the input of stage 2, wherein C represents the feature dimension of the 3D feature map;

[0040] Stage 2 includes a patch merging layer and two consecutive Point SwinTransformer blocks, and outputs a 3D feature map of and as the input of stage 3; the patch merging layer is used to change the shape of the feature map;

[0041] Stage 3 includes a patch merging layer and six consecutive Point SwinTransformer blocks, and outputs a 3D feature map of and as the input of stage 4;

[0042] Stage 4 includes a patch merging layer and two consecutive Point SwinTransformer blocks, and outputs a 3D feature map of ;

[0043] Finally, a 3D feature map of , a 3D feature map of , a 3D feature map of and a 3D feature map of are obtained;

[0044] 2, the 3D feature maps of different scales are input into the multi-scale feature fusion module for feature fusion, and a fused 3D feature map is obtained;

[0045] The specific operation process of the fused 3D feature map obtained in step ②-2 is as follows: the 3D feature map output by stage 1 is denoted as C1, the 3D feature map output by stage 2 is denoted as C2, the 3D feature map output by stage 3 is denoted as C3, and the 3D feature map output by stage 4 is denoted as C4. A 1x1 convolution operation is performed on C4 to obtain a feature map P4. The up-sampled feature map P4 is fused with the 1x1 convolution operated C3 to obtain a feature map P3. The up-sampled feature map P3 is fused with the 1x1 convolution operated C2 to obtain a feature map P2. The up-sampled feature map P2 is fused with the 1x1 convolution operated C1 to obtain a feature map P1, which is taken as the fused 3D feature map, wherein the step length of the up-sampling is 2;

[0046] ②-3 input the fused 3D feature map into the dimension reduction module to perform dimension reduction operation to obtain a 2D feature map in the BEV perspective;

[0047] The specific operation process of obtaining the 2D feature map in the BEV perspective in step ②-3 is as follows: compress the data of the fused 3D feature map in the z-axis direction to obtain a 2D feature map in the BEV perspective with only x-axis direction data and y-axis direction data;

[0048] ②-4 input the 3D feature map and the 2D feature map of different scales into the feature fusion module to obtain a fused feature map;

[0049] The specific operation process of obtaining the fused feature map in step ②-4 is as follows: the 3D feature map output by stage 1 is fused with the 2D feature map by a concat function to obtain a feature map Z1. The 3D feature map output by stage 2 is fused with the feature map Z1 by a concat function to obtain a feature map Z2. The 3D feature map output by stage 3 is fused with the feature map Z2 by a concat function to obtain a feature map Z3. The 3D feature map output by stage 4 is fused with the feature map Z3 by a concat function to obtain a fused feature map;

[0050] The definition of the concat function is as follows: wherein Z concat represents the output channel, X i represents the input channel of the first feature map of the i-th layer, Y i represents the input channel of the second feature map of the i-th layer, K i represents the channel number of the i-th layer, K i+crepresents the number of channels of the i+c layer, i∈[1, c], c represents the total number of channels of the corresponding feature map; the first feature map is in turn the 3D feature map output by stage 1, the 3D feature map output by stage 2, the 3D feature map output by stage 3, and the 3D feature map output by stage 4, and the second feature map is in turn the 2D feature map, the feature map Z1, the feature map Z2, and the feature map Z3;

[0051] ②-5 input the fused feature map into the RPN network to obtain an ROI in which a potential to-be-detected target object exists;

[0052] The fused feature map extracts the features of the to-be-detected target object in the RPN network, sets an ROI, modifies the size and position of the ROI according to the features of the to-be-detected target object, and obtains the ROI in which the potential to-be-detected target object exists;

[0053] ②-6 input the ROI into the RoI Pooling layer to extract the features of the ROI, and obtain a candidate box of the to-be-detected target object;

[0054] The ROI is mapped to the corresponding region position on the feature map, and finally the ROIs of different sizes are changed into fixed 7×7 sizes, so that the ROIs are evenly divided into 7×7 regions, the maximum pixel value of each divided region is taken as the output of the region, and the candidate box of the to-be-detected target object is obtained;

[0055] ③ input the candidate box of the to-be-detected target object into the full connection layer, and after the full connection layer, the classification branch and the recognition branch connected in parallel, the class of the to-be-detected target object is obtained by the classification branch, and the coordinates, size and direction of the to-be-detected target object are obtained by the recognition branch, so that the detection is completed;

[0056] In this embodiment, the loss function of the RPN network is composed of a classification loss and a regression loss, wherein N fg represents the number of candidate boxes, represents the output of the classification branch of the i-th candidate box, represents the output of the recognition branch of the j-th candidate box, represents the classification class of the j-th candidate box, represents the recognition target of the j-th candidate box, L cls represents a Focal loss function, L reg represents a Huber Loss function, j represents the classification of the j-th candidate box, j∈[0, b], b represents the total number of to-be-detected targets, represents a recognition loss calculated only using the candidate boxes;

[0057] In order to verify the effectiveness of the present application, experiments are carried out on public data sets KITTI and nuScences, and the experimental results are compared with other existing methods. The experimental equipment is Ubuntu 8.04 operating system, NVIDIA GeForce RTX3090 GPU server, realized through python3.7, Pytorch 1.6.0 and CUDNN 8.1.0 framework, wherein the batch size is set to 4 during training of the target detection model, and the learning rate is set to 0.01.

[0058] Wherein during the training process, the loss function of the detection head is defined as Wherein N s represents the number of candidate boxes in the training stage, represents the confidence of the nth candidate box and the real box, n∈[0, d], d represents the total number of detected targets in the training stage, represents the output of the recognition branch of the nth candidate box, represents the recognition target of the nth candidate box, I(IOU n ≥θ reg ) represents that only the candidate box with classification confidence IOU greater than or equal to θ reg is used for recognition loss calculation, and the classification confidence IOU of the detection head is calculated using Entropy Loss;

[0059] Experiment 1: 3D target detection on KITTI dataset: KITTI dataset is the most widely used dataset in the field of three-dimensional target detection and segmentation. The dataset contains 7481 training samples, and in this experiment, the training samples are divided into training set and test set in the ratio of about 1:1, wherein the training set contains 3712 sample data, and the test set contains 3769 sample data. In this experiment, the three categories (including car (Car), pedestrian (Ped.) and cyclist (Cyc.)) of the target detection model trained on the test set are evaluated. For each category, AP is used to detect cars, pedestrians and cyclists under 40 recall thresholds (R40), and according to the size and occlusion degree of 3D objects, they are divided into three difficulty levels: easy (Easy), medium (Mod.) and difficult (Hard), and 3DAP of medium difficulty car objects under 11 recall thresholds (R11) is used as the evaluation metric of experimental results. In this experiment, the official evaluation index is used, the intersection over union (IOU) threshold of the car is set to 0.7, and the IOU thresholds of the pedestrian and the cyclist are both set to 0.5.

[0060] Table 1 Comparison results of the present application and existing methods on KITTI dataset

[0061] ​

[0062]

[0063] As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN.

[0064] As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN. Figure 3 As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN. Figure 3 As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN. Figure 3 As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN. Figure 3 As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN. Method As shown in Table 1, compared with the Voxel-RCNN method, the detection accuracy of the three difficulty levels of the vehicle category of the application is increased by 0.82%, 0.86% and 1.34% respectively, the detection accuracy of the three difficulty levels of the pedestrian and cyclist categories is increased, and the 3D AP R11 of the application on the medium difficulty vehicle is 2.41% higher than that of Voxel-RCNN.

[0065] In the ablation experiment on the KITTI dataset, the multi-scale feature fusion module (Multi-scale features) and the feature fusion module (Features fusion) are combined for experiment, and the performance of 3D AP R11 on the medium difficulty vehicle is used for evaluation, as shown in Table 2. The experiment is divided into (a), (b), (c), (d) and (e) five groups for training. Among them, (a) is the basic model, only linear transformation is performed on the point cloud data, multi-scale feature extraction is not used, and only 2D features are used to generate detection frame candidate regions. (b), (c), (d) increase one stage, two stages and three stages respectively on the basis of (a), the resolution of the feature map is halved and the channel number is doubled for each stage, multi-scale feature extraction is used, which has a larger receptive field and better target extraction ability for different sizes. (b), (c), (d) in 3D AP R11The upper is sequentially increased by 2.13%, 3.84% and 4.04%. (e) is added to the feature fusion module on the basis of (d), which is the complete network architecture proposed by the application, after multi-scale feature extraction, the BEV feature and the multi-scale feature data are fused to enhance the accuracy of the detection frame candidate area.

[0066] Table 2 Ablation experiment of the application

[0067]

[0068] Experiment two, 3D target detection on nuScences(mini) dataset: The nuScences dataset includes 1000 scenes, which contains images from 6 cameras. It is the latest popular benchmark for vision-based 3D object detection and BEV semantic segmentation. Since the nuScences dataset is too large, this experiment uses the nuScences mini dataset, which contains 10 scenes. The official evaluation indicators include mean average precision (mAP), mean average translation error (mATE), mean average scale error (mASE), mean average orientation error (mAOE), mean average velocity error (mAVE), mean average attribute error (mAAE) and NuScenes detection score (NDS). mAP is based on the matching of 2D center distance on the ground plane. NDS is a combination of other indicators that comprehensively judge the detection ability. The remaining indicators are used to calculate the result accuracy of the corresponding aspects (such as translation, scaling, direction, speed and attribute). ↑ indicates that the higher the value, the better, and ↓ indicates that the smaller the value, the better.

[0069] Table 3 Comparison results of the application and the prior art tested on nuScences(mini) dataset

[0070] mATE↓ mASE↓ mAOE↓ mAVE↓ mAAE↓ mAP↑ NDS↑ PP-MH S-MH 0.6339 0.5367 1.3782 1.3903 0.5340 0.1075 0.1833 CP-PP 0.7366 0.5380 1.0812 1.5363 0.5401 0.1001 0.1686 CP(0.1) 0.6130 0.5306 1.1604 0.9568 0.3921 0.1467 0.2241 CP(0.075) 0.6105 0.5215 1.1065 1.1405 0.3390 0.1487 0.2272 Invention 0.4943 0.5076 1.0510 0.5977 0.3237 0.2546 0.3350 ​ 0.5013 0.5081 1.0322 0.5939 0.3129 0.2637 0.3425

[0071] As shown in Table 3, the application has a decrease in the values of the five evaluation indicators of mATE, mASE, mAOE, mAVE and mAAE, and an increase in the values of mAP and NDS. Among them, the PP-MH method is a PointPillar method with multiple detection heads, the S-MH method is a SECOND method with multiple detection heads, the CP-PP method is a CenterPoint-PointPillar method, the CP(0.1) method is a CenterPoint-PointPillar method with a voxel grid size of 0.1, and the CP(0.075) method is a CenterPoint-PointPillar method with a voxel grid size of 0.075.

Claims

1. A three-dimensional object detection method based on multi-scale feature fusion, characterized in that: The method comprises the following steps: ① scanning a target object to be detected by a laser radar scanner to obtain point cloud data of the target object to be detected; ② inputting the point cloud data of the target object to be detected into a target detection model to obtain a candidate box of the target object to be detected; ③ inputting the candidate box of the target object to be detected into a full connection layer, and obtaining the category of the target object to be detected by a classification branch and the coordinates, size and direction of the target object to be detected by an identification branch after the full connection layer, so as to complete the detection; The target detection model comprises a feature coding module, a feature fusion module and a detection head; the feature coding module comprises a 3D Swin transformer module, a multi-scale feature fusion module and a dimension reduction module; the 3D Swin transformer module comprises a 3D partition, stage 1, stage 2, stage 3 and stage 4 connected in sequence; the detection head comprises an RPN network and a RoI Pooling layer, and the specific steps of obtaining the candidate box of the target object to be detected are as follows: ②-1 inputting the point cloud data of the target object to be detected into the 3D partition, and outputting and obtaining 3D feature maps of different sizes by stage 1, stage 2, stage 3 and stage 4; ②-2 inputting the 3D feature maps of different sizes into the multi-scale feature fusion module for feature fusion to obtain fused 3D feature maps; ②-3 inputting the fused 3D feature maps into the dimension reduction module for dimension reduction operation to obtain a 2D feature map in a BEV perspective; ②-4 inputting the 3D feature maps of different sizes and the 2D feature map into the feature fusion module to obtain fused feature maps; ②-5 inputting the fused feature maps into the RPN network to obtain ROIs in which potential target objects to be detected exist; ②-6 inputting the ROIs into the RoI Pooling layer to extract the features of the ROIs to obtain the candidate box of the target object to be detected. 2.The method of claim 1, wherein The specific operation process of step ②-1 is as follows: The RGB channel number of the point cloud data of the target object to be detected obtained by scanning the target object to be detected by a laser radar scanner is 3, the block size in 3D partition is set to 2*3*3, the input point cloud data of the target object to be detected is divided by 3D partition, and the output vector is taken as the input of stage 1, wherein D represents the spatial dimension of the input point cloud data of the target object to be detected, H represents the height dimension of the input point cloud data of the target object to be detected, and W represents the width dimension of the input point cloud data of the target object to be detected. Stage 1 includes one linear embedding layer and two consecutive Point Swin Transformer Blocks, outputting a 3D feature map of and serving as input to Stage 2, where C denotes the feature dimension of the 3D feature map. Stage 2 includes one region merging layer and two successive Point Swin Transformer Blocks, outputting a 3D feature map of 64 channels and serving as the input of Stage 3. Stage 3 includes one Point Swin Transformer Block, outputting a 3D feature map of 128 channels. Stage 3 includes one region merging layer and six consecutive Point Swin Transformer Blocks, outputting a 3D feature map of 64 channels and serving as the input of Stage 4. Stage 4 includes one region merging layer and six consecutive Point Swin Transformer Blocks, outputting a 3D feature map of 128 channels and serving as the input of Stage 5 Stage 4 includes one region merging layer and two successive Point Swin Transformer Blocks, outputting a 3D feature map of ​ a 3D feature map of a 3D feature map of a 3D feature map of a 3D feature map of a 3D feature map of 3.The method of claim 1, wherein The specific operation process of obtaining the fused 3D feature maps in step ②-2 is as follows: the 3D feature map output by stage 1 is denoted as C1, the 3D feature map output by stage 2 is denoted as C2, the 3D feature map output by stage 3 is denoted as C3, and the 3D feature map output by stage 4 is denoted as C4; a 1×1 convolution operation is performed on C4 to obtain a feature map P4; the feature map P4 after upsampling is fused with C3 after 1×1 convolution operation to obtain a feature map P3; the feature map P3 after upsampling is fused with C2 after 1×1 convolution operation to obtain a feature map P2; the feature map P2 after upsampling is fused with C1 after 1×1 convolution operation to obtain a feature map P1 as the fused 3D feature map, wherein the step length of the upsampling is 2. 4.The method of claim 1, wherein The specific operation process of obtaining the 2D feature map in the BEV perspective in step ②-3 is to compress the data of the fused 3D feature map in the z-axis direction to obtain a 2D feature map in the BEV perspective with only x-axis direction data and y-axis direction data.

5. The method of claim 1, wherein The specific operation process of the fused feature map obtained in the step ②-4 is as follows: the 3D feature map output by the stage 1 is fused with the 2D feature map through a concat function to obtain a feature map Z1, the 3D feature map output by the stage 2 is fused with the feature map Z1 through a concat function to obtain a feature map Z2, the 3D feature map output by the stage 3 is fused with the feature map Z2 through a concat function to obtain a feature map Z3, and the 3D feature map output by the stage 4 is fused with the feature map Z3 through a concat function to obtain a fused feature map. 6.The method of claim 1, wherein The loss function of the RPN network consists of a classification loss and a regression loss, wherein N fg represents the number of candidate boxes, P j a represents the output of the classification branch of the jth candidate box, represents the output of the identification branch of the jth candidate box, represents the classification category of the jth candidate box, represents the identification target of the jth candidate box, L cls represents a Focal loss function, L reg represents a HuberLoss function, j represents the classification of the jth candidate box, j∈[0,b],b represents the total number of detection targets, represents the identification loss calculated only using the candidate box.

Images