Mirror image completion point cloud 3D target cascade detection method for automatic driving

Abstract

The application discloses a mirror image completion point cloud 3D target cascade detection method for automatic driving, and the method comprises the following steps: performing a pretreatment operation on point cloud data, filtering out isolated points and out-of-bound points, performing a voxelization operation on the point cloud data, and obtaining a training data set; constructing a neural network; training the neural network through the training data set, adjusting the training parameters of the neural network through one-stage detection Loss, cascade detection Loss and mirror loss Loss, and obtaining a trained neural network; and performing target prediction on a prediction data set through the trained neural network, taking the class with the maximum prediction probability in one stage as a target signal prediction label of detection, and taking the prediction box result of cascade detection as the center position, size and rotation angle of the target.

Description

Technical Field

[0001] This invention belongs to the fields of autonomous driving and deep learning, and specifically relates to a mirror-complete point cloud 3D target cascade detection method for autonomous driving. Background Technology

[0002] With the rapid development of various computing hardware and the continuous progress of deep learning algorithms, the ability to process diverse information is becoming stronger and faster, making the real-time application of various sensors such as radar, lidar, and cameras feasible for autonomous driving tasks.

[0003] Object detection provides crucial information for autonomous driving's subsequent planning, decision-making, and actions by enabling real-time perception of the surrounding environment, making it a key technology for autonomous driving. Developing an accurate and fast object detection algorithm has become a hot topic in the entire research field.

[0004] LiDAR can acquire point cloud data, boasting extremely high detection accuracy and speed, making it the most suitable detector for autonomous driving. However, in the past, the massive amounts of data generated per second resulted in slow and inefficient point cloud data processing, hindering its deployment in real-world scenarios. With continuous advancements in hardware technology, point cloud 3D object detection has gradually become a viable solution. In recent years, numerous deep learning-based detection algorithms have been proposed, often exhibiting superior adaptability and accuracy compared to traditional algorithms. Currently, deep learning-based 3D point cloud object detection algorithms are mainly divided into three types: 1) projection and image-based detection algorithms; 2) point cloud-based detection algorithms; and 3) voxel-based detection algorithms.

[0005] Projection- and image-based detection algorithms primarily map point cloud data onto a two-dimensional plane, using image processing algorithms for feature extraction and target detection. Point cloud-based detection algorithms directly process and detect point cloud data. Voxel-based detection algorithms first distribute the point cloud into a uniformly sized space, then extract and detect features from the voxels. The voxel method preserves a significant portion of the 3D spatial information while avoiding the drawbacks of irregular point cloud data storage, giving it advantages in both accuracy and detection speed.

[0006] However, voxel-based 3D object detection algorithms still face several challenges, such as: 1) Voxels are stored using a sparse structure, resulting in a much larger data volume than images, leading to high computational overhead; 2) Voxelization directly causes the loss of spatial detail information, blurring the object's outline and reducing the algorithm's recognition ability; 3) Voxel space is completely uniformly distributed, and the number of voxels for small targets is much smaller than that for large targets, making small targets more likely to be missed during detection. Therefore, it is necessary to design an efficient real-time detection algorithm that is universally applicable to targets of various sizes. Summary of the Invention

[0007] In view of this, the main objective of the present invention is to provide a mirror-complete point cloud 3D target cascade detection method for autonomous driving.

[0008] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0009] This invention provides a mirror-complete point cloud 3D target cascade detection method for autonomous driving, the method comprising:

[0010] Preprocess the point cloud data to filter out isolated and out-of-bounds points, and perform voxelization on the point cloud data to obtain the training dataset.

[0011] Constructing a neural network;

[0012] The neural network is trained using a training dataset, and the training parameters of the neural network are adjusted by one-stage detection loss, cascade detection loss, and mirror loss to obtain a trained neural network.

[0013] The trained neural network is used to predict targets in the prediction dataset. The category with the highest prediction probability in the first stage is used as the predicted label of the detected target signal, and the prediction box results of the cascaded detection are used as the center position, size and rotation angle of the target.

[0014] In the above scheme, the construction of the neural network specifically refers to:

[0015] Construct a voxel feature encoding module and obtain voxel features and voxel numbers based on voxel data;

[0016] A 3D sparse convolution module is constructed, and voxel features at multiple levels are obtained based on the voxel features and voxel indices.

[0017] A bird's-eye view mapping module is constructed, and a 2D bird's-eye view is obtained based on the voxel features of the largest spatial level scale among the voxel features of the multiple level scales.

[0018] Construct a 2D convolutional module and determine a 2D feature image based on the 2D bird's-eye view;

[0019] A one-stage detection head is constructed, and a one-stage prediction result is obtained based on the 2D feature image;

[0020] A cascaded mirror completion detection head is constructed, and the detection results of the cascaded stages are obtained based on the voxel features of all multiple hierarchical scales.

[0021] In the above scheme, the construction of the cascaded mirror completion detection head and the acquisition of cascaded stage detection results based on voxel features at all multiple hierarchical scales are specifically as follows:

[0022] Construct a Voxel-RoI module, the output of which is a 1-dimensional feature vector F corresponding to the candidate region. RoI ;

[0023] Construct N1 3D detector heads, each consisting of a classification predictor composed of one MLP and a bounding box predictor composed of one MLP. All 3D detector heads have the same structure and number of trainable parameters. For each 3D detector head, set the positive sample loss calculation threshold FG_REG_THi. ， For i = 1, 2, 3, ..., N1, the requirement is that for any two adjacent thresholds, the previous threshold must be less than the next, i.e., FG_REG_THi < FG_REG_TH i+1 Each detector head outputs a predicted bounding box result. and classification results

[0024] In the above scheme, each 3D detection head outputs a predicted bounding box result. and classification results Specifically:

[0025] Step 401: Initialize the setting k=1, start the calculation from the first 3D detection head, obtain candidate region information, and execute step 402;

[0026] Step 402: If k=1, the candidate region information RoI_BL comes from the predicted bounding box results of the first-stage detection head. and classification results Otherwise, the candidate region information RoI_BL is derived from the predicted bounding box result of the previous 3D detector. and classification results

[0027] Step 403: Extract features from the current candidate region using the Voxel-RoI module to obtain the candidate region features FRo of the k-th 3D detector head. I_k If k < N1, then mirror completion is not used; otherwise, cascaded mirror completion is used.

[0028] Step 404: Through tensor F all Record the candidate region features of the current detection head and arrange them in ascending order according to the size of the 3D detection heads. all =[F RoI_1 F RoI_2 F RoI_2 ], F all Input the cross-attention feature mixing module to obtain the mixed candidate region features F_Mix RoI ;

[0029] Step 405: Mix the candidate region features F_MixRoI The predicted bounding box result of the current 3D detector is input into the k-th 3D detector. and classification results

[0030] Step 406: Execute k = k + 1 for k and iterate. If k < N1, then execute step 402; otherwise, execute step 407.

[0031] Step 407: If the neural network is in the training process, then use the output of the last 3D detection head. and As the final test frame result and classification results If the neural network is used for the testing process, then the average output of all 3D detectors is taken as the final bounding box result. and classification results

[0032] In the above scheme, the cascading mirror completion in step 403 specifically includes:

[0033] Step 501: Initialize the settings k=1, let P be the number of voxel feature layers to be extracted, and set the minimum mirror-symmetric score threshold TH. mirror ;

[0034] Step 502: Select all voxels in the current layer as set VSet0. Based on the current voxel's index [Vx, Vy, Vz], the lower limit of the detection range [RLx, RLy, RLz], and the current voxel's physical size [Lx, Ly, Lz], calculate the geometric center position [x] of all voxels. c y c , z c The specific calculation formula is: j c =(V j +0.5)*Lj+RLj, j=x,y,z;

[0035] Step 503: Based on the candidate region information RoI_BL, first filter the detection box scores, retaining only those with scores exceeding TH. mirror The detection box is then used to determine whether the geometric center of each voxel is within the detection box. Only voxels located within the detection box are retained to form a set BSet1.

[0036] Step 504: Based on the detection box position information in the candidate region information RoI_BL, calculate the mirror plane of each detection box, represented by four parameters: Ax + By + Cz + D = 0. The mirror plane is the XOZ plane of the detection box, where O is the center of the detection box, X is the normal forward movement direction of the target in the detection box, and Z is the direction perpendicular to the ground. The normal vector forming the mirror plane is obtained by predicting the box rotation angle θ and the normal vector before rotation. Determined, D passes through the center of the detection box [x] b y b , z b ]and Sure;

[0037] Step 505: Based on the center coordinates [x] of each voxel c y c , z c Determine the mirror voxel coordinates [x] of the bounding box and its mirror plane. c ′,y c ′,z c ['], and obtain the mirror voxel set VSet2;

[0038] Step 506: Merge all voxels VSet0 in the current layer and the mirror voxel set VSet2 to obtain the union result VSet3;

[0039] Step 507: For the neural network training phase, determine the number N(VSet1) of VSet1. Given a minimum percentage threshold γ, if N(VSet1) < γN(VSet0), then use VSet0 for candidate layer feature extraction; otherwise, use VSet3 for candidate layer feature extraction to obtain layer features F. layer_k k = 1, 2, 3, ..., P. Iterate by performing k = k + 1 on k. If k < P, proceed to step 402; otherwise, proceed to step 408.

[0040] Step 508: Transfer the features F from layer P layer_k The feature dimensions of k = 1, 2, 3, ..., P are concatenated to obtain the candidate region features F. RoI .

[0041] In the above scheme, the neural network is trained using a training dataset, and the training parameters of the neural network are adjusted using one-stage detection loss, cascade detection loss, and mirror loss to obtain a trained neural network.

[0042] Step 601: Set the prediction box indicator function based on the prediction box score.

[0043]

[0044] Step 602: For the first-stage detection head class probability, use the cross-entropy (CELoss) weight λ 1_1 The L1 loss function (L1Loss) is used for the center position, size, and rotation angle of the predicted bounding box, with weights λ.1_2 The L1 loss function (L1Loss) is used for the orientation of the predicted bounding box, with weights λ. 1_3 The sum of these three factors yields the result of the first-stage loss function calculation.

[0045]

[0046] Step 603: Apply binary cross-entropy (BCELoss) to the probabilities of all 3D detection head categories, with weights μ. 1_i , i = 1, 2, 3, ..., N1. L1Loss is used for the center position, size, and rotation angle of the predicted bounding box, with weights μ. 2_i , i = 1, 2, 3, ..., N1; Step 603: Sum all loss functions by weight to obtain the calculation result of the cascaded loss function;

[0047]

[0048] Step 604: For the voxel selection set VSet1 of the first layer, determine the true label loc of the center coordinates of the mirror voxels. c The voxel center coordinates of VSet2 are used as the predicted values. The Smooth L1 Loss function is applied to the true and predicted values ​​of the mirror coordinates, with a boundary threshold of δ and a loss weight of λ3.

[0049]

[0050] Step 605: Sum the results of the first-stage detection Loss, cascade detection Loss, and mirror Loss to obtain the final Loss, which is used to train the neural network.

[0051] Loss = Loss stage1 +Loss cas +Loss mirror .

[0052] Compared with existing technologies, this invention uses two detection heads to detect targets consecutively through cascaded detection. Then, it utilizes the mirror symmetry of the detected targets to complete the space within the candidate region. Finally, it uses a cross-attention mechanism to fuse the features of the candidate region, achieving higher detection performance while sacrificing some detection speed. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and, together with their descriptions, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0054] Figure 1 This is a schematic diagram of the cascaded detection head structure in an embodiment of the present invention;

[0055] Figure 2 This is a flowchart of the mirror completion algorithm in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] In this embodiment, the mirror-complete point cloud 3D target cascade detection algorithm for autonomous driving includes the following steps:

[0058] Step 1: Process the training dataset and preprocess the point cloud data in the training dataset to filter out isolated points and outliers. Finally, perform voxelization on the point cloud data to obtain the training dataset S. train ;

[0059] Step 1.1.1: Currently, the KITTI dataset is the most frequently used dataset in point cloud 3D object detection algorithms. Therefore, this invention will implement a specific solution based on the KITTI dataset. The Karlsruhe Institute of Technology in Germany and Toyota Research Institute of America jointly collected and labeled the KITTI dataset and made it publicly available. The vehicle is equipped with a Velodyne 64-line 3D LiDAR, two grayscale cameras, two color cameras, and a GPS system, capable of simultaneously recording data in many common formats such as 3D point clouds, 2D black and white and color images, and optical flow. It can be used in multiple research fields such as image segmentation, object detection, and object tracking. The official annotation labels include detection categories such as vans, cars, pickup trucks, pedestrians (standing / sitting), riders, and trams, along with truncation and occlusion levels. This invention uses only point cloud data samples collected by a 64-line LiDAR in the KITTI dataset for implementation, with a total of 7481 data points. Among them, 3769 samples are used as a validation dataset to detect vehicles (cars and vans), pedestrians (stations) and riders.

[0060] Step 1.1.2: Read in the training data, set the batch size to 8, and set the detection space range in the ego coordinate system. Limit the X direction to [0, 70.4) meters, the Y direction to [-40, 40) meters, and the Z direction to [-3, 1] meters. Set the maximum query radius for each point cloud to R. querry = 0.15 meters, minimum number threshold N th=5, isolating isolated points within the maximum query area that do not meet the minimum number threshold. The point cloud data feature dimension used is C0=5, including the three-dimensional spatial coordinates of the point cloud (XYZ) and signal reflection intensity. There is an additional feature dimension used to record the index of the point cloud in the batch; this dimension only serves to maintain the data calculation process and does not provide spatial information.

[0061] Step 1.2: Set the initial voxel size [g] x g y g z [0.05, 0.05, 0.1] meters, with the maximum number of point clouds in a single voxel set to K. max =5, the maximum number of voxels sampled in a single point cloud image is voxel_num. max =16000, thus obtaining the voxel relation tensor containing the sampled point cloud. Its corresponding voxel coordinate number is The latter is similar to the point cloud data storage structure; the first number is the sequence number of the current voxel data in the batch, and it does not provide spatial information. point This refers to the total number of sampled point cloud data, typically greater than 16,000. To meet computational requirements, we restrict this to 4 ≤ N. point ≤64000. Based on the given initial voxel size and the limited detection space range, the voxel number should be less than 1408, 1600 and 41 in three dimensions, respectively.

[0062] Step 2: Construct the various sub-modules of the neural network used in the mirror-complete point cloud 3D target cascade detection algorithm for autonomous driving:

[0063] Step 2.1: The voxel feature encoding module calculates the average value of VP based on the number of point clouds contained in each voxel to obtain the encoded voxel features.

[0064] Step 2.2: Construct a 3D sparse convolution module. Let M1 = 4, and the final output is the voxel feature xconv. i Let i = 1, 2, 3, 4. Each sparse convolutional layer has a stride of 2 in three dimensions, and all convolutional kernels are set to sizes of 3. The feature output dimensions for the four scales are set to [16, 32, 64, 64], respectively. The final spatial 3D uniform grid numbering for the four scales is [1408, 1600, 41], [704, 800, 21], [352, 400, 11], and [176, 200, 5]. The last sub-manifold convolutional layer has a stride of 2 in the Z direction and outputs a feature dimension of 128, resulting in the final 3D spatial features.

[0065] xconv1 = Slice1(xconv0)

[15]

[0066] xconv k =spconv k-1 (Slice k (xconv k-1 )), k = 2, 3, 4

[16]

[0067] Step 2.3: The bird's-eye view mapping module performs Z-direction feature stacking on the above 3D spatial features to obtain a 3D tensor as a new 2D bird's-eye view feature.

[0068] Step 2.4: In the 2D convolution module, set M2=2. The output dimensions of the convolutional layer modules at each scale are 128 and 256 respectively, and the output dimension of the deconvolution is 256. For the initial input X0 and the final output UP:

[0069] X i =block i (X i-1

[17]

[0070] UP i =deblock i (X i ), i = 1, 2

[18]

[0071] UP = [UP1, UP2]

[19]

[0072] Step 2.5: Construct a first-stage detection head, and set up anchor frames of a certain shape for cars, pedestrians and riders respectively. Each anchor has two orientations, 0° and 90°. The initial anchor offset angle is set to 45° and the orientation is set to 2, representing moving closer to and away from the origin of the ego coordinate system.

[0073] Step 2.6: Construct a cascaded mirror completion detection head, selecting the 160 Anchors with the highest classification probabilities as cascaded detection inputs. Initialize k = 1.

[0074] Step 2.6.1: Set the number of cascaded detection heads N1 = 2. Set the single-grid output feature dimension of the Voxel-RoI module to 32, with 6 grid points in each dimension, and retain a maximum of 16 voxels. Perform feature extraction on voxels at four spatial hierarchical scales, and set the neighborhood query radius to [0.2, 0.4, 0.8, 1.6] meters respectively.

[0075] Step 2.6.2: Set the minimum percentage threshold γ = 0.05 and the minimum mirror-symmetric score threshold TH. mirror=0.7. Set the positive sample threshold for each detector head separately.

[0076] FG_REG_TH1 = 0.5, FG_ERG_TH2 = 0.6. Determine if k < 2. If yes, extract features directly without mirror completion; otherwise, perform mirror completion on each candidate region before feature extraction. The hierarchical output feature for each candidate region is F. roi_i If i = 1, 2, 3, 4, then the final output for the k-th Anchor is:

[0077] F RoI_k =[F roi_1 F roi_2 F roi_3 F roi_4

[20]

[0078] Step 2.6.3: The input and output feature dimensions of the cross-attention module are 27648, the number of multi-heads is set to 4, and the initial settings are configured. For the output of the k-th detection head Anchor, use F all Update the storage as shown in formula

[21] , and then F all The input is fed into the cross-attention module to obtain the final candidate region features F_Mix. RoI_k .

[0079] F all =[F all F RoI_k ] [twenty one]

[0080] F_Mix RoI_k =CrossAttention(F all ) [twenty two]

[0081] Step 2.6.4: Mix F_Mix RoI_k The input is fed into the k-th detector head, which then outputs the prediction result. and Determine if k=2 is true. If not, set k=k+1 and proceed to step 2.6.2. If true, generate the final bounding box result based on the current training or testing state of the neural network. and classification results End the forward propagation calculation. If in training mode, use the output of the last 3D detector. and This serves as the final test frame result; if it is in test mode, the result is calculated according to the following formula:

[0082]

[0083]

[0084] Step 3: Calculate the first-stage detection loss, cascade segment detection loss, and mirror loss according to the set parameters, and then use the loss function to perform backpropagation calculations on the neural network model parameters.

[0085] Step 3.1: Set the maximum training period E of the neural network. max =80, batch size set to B n =8, the total number of training samples is set to S=3712, then the number of batches trained in each training cycle is...

[0086] Step 3.2: Set the weight values ​​for the loss function in the first-stage detection. The probability weight λ for the detected target class. 1_1 =1.0, spatial information weight λ of the prediction box 1_2 =2.0, the orientation information weight λ of the predicted bounding box 1_3 =2.0;

[0087]

[0088] Step 3.3: Set the weights of the loss function for cascaded detection. The probability weights μ for the detected target class. 1_1 =μ 1_2 =1.0, the weight μ of the predicted box 2_1 =μ 2_2 =1.0;

[0089]

[0090] Step 3.4: Set the weights of the loss function for mirror completion. The boundary threshold of the loss function is δ = 0.3, and the loss weight λ3 = 2.0.

[0091]

[0092] Step 3.5: Take the weighted sum of the Loss from the first stage, the Loss from the cascade stage, and the Loss from the mirror completion stage to obtain the final Loss;

[0093] Loss = Loss stage1 +Loss cas +Loss mirror

[29]

[0094] Step 3.6: Initialize the trainable parameters of all convolutional and linear connection layers in the neural network using the Xavier uniform distribution function. Initialize all batch normalized layers, setting their mean weights to 0 and their variance weights to 1.

[0095] Step 3.7: Select Adam as the optimizer for the neural network, set the initial learning rate to Ir0 = 0.01, select a cosine learning rate curve, and set the initial learning rate to 0.1 of the initial learning rate.

[0096] Step 3.8: Read in one batch of training data, the network performs forward propagation on the data, then calculates the loss function and performs back propagation, and the detection results must be output according to the training mode specifications.

[0097] Step 3.9: Determine if the current cumulative training steps (step) are divisible by the number of batches trained in a single cycle (k = 928). If yes, read the training data from the beginning according to the shuffled results; if no, read the next training data according to the shuffled results.

[0098] Step 3.10: Determine if the current cumulative training steps (step) are equal to B. n If ×k = 74240, end the training and proceed to step four; otherwise, continue training and return to step 3.8.

[0099] Step 4: Evaluate the performance of the invention:

[0100] Step 4.1: Determine the validation dataset and evaluation metrics;

[0101] This invention uses the validation set from the KITTI dataset to validate the performance of the trained model. Using the evaluation criteria provided by KITTI, the average precision (AP) and frame rate (FPS) of the detection algorithm are used to evaluate the detection performance and running speed of the algorithm, respectively. AP is a commonly used metric in object detection algorithms, directly reflecting the detection accuracy of a specific category; the higher the AP value, the higher the accuracy of the algorithm's detection results. FPS is the number of images processed per second, directly reflecting the running speed of the evaluation model. Faced with complex environments and unexpected situations, autonomous driving algorithms need a certain amount of time to react; the higher the FPS, the longer the reaction time.

[0102] The mathematical formula for calculating the AP index is as follows:

[0103]

[0104] Where P is precision, R is recall, TP represents positive samples predicted as positive, FP represents positive samples predicted as negative, and FN represents negative samples predicted as negative, the mathematical formulas for calculating precision and recall are as follows:

[0105]

[0106]

[0107] For each point cloud sample, the Precision and Recall of its detection result can be obtained. By plotting the Precision and Recall of multiple samples as x and y axes to form a Precision-Recall curve, the AP result can be calculated. To reduce computational complexity and implement discrete integration, the interval [0, 1] can be divided into L-1 equal parts using L equal points. The discrete numerical representation of AP is calculated using an approximate infinitesimal element approach; we set L = 11 to calculate the final result.

[0108] During training, prediction results need to be output according to the test mode. To demonstrate the practicality of this invention, three comparative experiments were set up. The Voxel-RCNN neural network model was trained and tested as Experiment Group A; the PV-RCNN neural network model was trained and tested as Experiment Group B; and the neural network model described in this invention was trained and tested as Experiment Group C. During training, all parameters and hyperparameters were set to the same values, and the programs were run on the same machine. Regarding the selection of detection targets, we only detected vehicles, pedestrians, and riders. Only predictions with a classification threshold higher than 0.7 were considered correct. According to the official KITTI classification, the difficulty of the detected objects was divided into easy, medium, and hard based on the degree of occlusion and truncation. The final results are shown in Table 1.

[0109] Table 1: Results of the comparative experiment

[0110]

[0111] Comparing experimental groups A and C, experimental group C showed significantly higher AP results for all three target categories compared to experimental group A. For car detection, the AP values ​​for the three difficulty levels increased by 2.35, 1.62, and 3.57, respectively; for pedestrian detection, the AP values ​​for the three difficulty levels increased by 2.68, 1.98, and 1.21, respectively; and for rider detection, the AP values ​​for the three difficulty levels increased by 6.44, 1.84, and 3.09, respectively. Comparing experimental groups B and C, experimental group C also showed significantly higher AP results for all three target categories compared to experimental group B. For car detection, the AP values ​​for the three difficulty levels increased by 2.88, 1.88, and 3.53, respectively; for pedestrian detection, the AP values ​​for the three difficulty levels increased by 4.69, 2.85, and 1.65, respectively; and for rider detection, the AP values ​​for the three difficulty levels increased by 6.59, 2.61, and 2.23, respectively.

[0112] Based on the test results above, Experimental Group C exhibits significantly higher detection performance than Experimental Groups A and B, demonstrating a substantial improvement in the overall detection capability of the algorithm. However, in terms of operating speed, Experimental Group C achieves only 25.7 frames per second (FPS), slightly lower than Experimental Group B, but still higher than the common 24 FPS, meeting the daily operating speed requirements for autonomous driving under normal conditions.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A mirror image completion point cloud 3D target cascade detection method for autonomous driving, characterized in that, The method comprises: The point cloud data is preprocessed to filter out isolated points and out-of-bound points, and voxelization is performed on the point cloud data to obtain a training data set; A neural network is constructed; The neural network is trained by the training data set, and the training parameters of the neural network are adjusted by one-stage detection loss, cascade detection loss and mirror loss, to obtain a trained neural network; The trained neural network is used to predict the target of a prediction data set, the class with the maximum prediction probability in the one-stage prediction is taken as the target signal prediction label, and the prediction box result of the cascade detection is taken as the center position, size and rotation angle of the target; The neural network is constructed, specifically: A voxel feature encoding module is constructed, and voxel features and voxel serial numbers are obtained according to voxel data; A 3D sparse convolution module is constructed, and voxel features of multiple hierarchical scales are obtained according to the voxel features and voxel serial numbers; An aerial view mapping module is constructed, and a 2D aerial view is obtained according to the voxel features of the maximum spatial hierarchical scale among the voxel features of the multiple hierarchical scales; A 2D convolution module is constructed, and a 2D feature image is determined according to the 2D aerial view; A one-stage detection head is constructed, and one-stage prediction results are obtained according to the 2D feature image; A cascade mirror completion detection head is constructed, and cascade stage detection results are obtained according to the voxel features of all the multiple hierarchical scales; The cascade mirror completion detection head is constructed, and the cascade stage detection results are obtained according to the voxel features of all the multiple hierarchical scales, specifically: Construct 1 Voxel-RoI module, the output is the 1-dimensional feature vector corresponding to the candidate region ; Construction 3D detection heads, each detection head consists of a classification predictor composed of an MLP and a bounding box predictor composed of an MLP, all 3D detection heads have the same structure and the number of trainable parameters, for each 3D detection head, set the positive sample loss calculation threshold , require that between two adjacent thresholds, the former is smaller than the latter, that is , each detection head outputs a prediction box result and a classification result ; Each 3D detection head outputs a prediction box result and a classification result Specifically: Step 401: initialize k=1, start calculation from the first 3D detection head, obtain candidate region information, and execute step 402; Step 402: If k = 1, candidate region information Prediction box result from a one-stage detection head and classification result ; otherwise, candidate region information Prediction box result from the previous 3D detection head and classification result ; Step 403: feature extraction is performed on the current candidate region by the Voxel-RoI module to obtain candidate region features of the first 3D detection head If , no mirror completion is used, otherwise, cascade mirror completion is used;​ Step 404: passing the tensor through Record the candidate region features of the current detection head, and arrange them in order from small to large according to the 3D detection head , the Input the cross attention feature mixing module to obtain the mixed candidate region features ; Step 405: Combine the features of the mixed candidate regions Enter the number Among multiple 3D detectors, the predicted bounding box result of the current 3D detector. and classification results ; Step 406: If performing , an iteration is performed, if , step 402 is performed, otherwise step 407 is performed; Step 407: if the neural network is a training process, taking the output of the last 3D detection head as the final bounding box result and as the final bounding box result and classification result ; if the neural network is a test process, taking the average of the outputs of all 3D detection heads as the final bounding box result and classification result ; In the step 403, the cascade mirror completion is specifically: Step 501: initialization, set k=1, set the number of feature layers to be extracted as , set the minimum mirror symmetry score threshold ; Step 502: select all voxels of the current layer as a set , according to the subscript of the current voxel , the lower limit of the detection range and the physical size of the current voxel Calculate the geometric center position of all voxels The specific calculation formula is: ; Step 503: determining whether the candidate region information meets the preset condition First, the detection frame score is screened, and only the detection frame with a score exceeding a preset threshold is retained. Then, it is determined whether the geometric center of each voxel is in the detection frame, and only the voxel in the detection frame is retained to constitute a set . Step 504: According to the detection frame position information in the candidate area information , the mirror plane of each detection frame is calculated, which is represented by four parameters as , the mirror plane is the XOZ plane of the detection frame, O is the center of the detection frame, X is the normal forward direction of the target in the detection frame, and Z is the direction perpendicular to the ground, wherein, The normal vector of the mirror plane is determined by the rotation angle of the prediction frame and the normal vector before rotation , which is determined by the center of the detection frame and ; Step 505: according to the center coordinates of each voxel and the mirror plane of the prediction box where the voxel is located, determine the mirror voxel coordinates of the voxel , and obtain a mirror voxel set ; Step 506: merge the current layer's set of voxels and the mirror voxel set to get the union result ; Step 507: For the neural network training phase, a determination is made as to the number of given a minimum percentage threshold , if , then using for candidate region layer feature extraction, otherwise using for candidate region layer feature extraction, obtaining layer features ; performing for iteration, if , then performing step 502, otherwise performing step 508;​​ Step 508: obtaining the candidate region feature by performing feature dimension splicing on the feature of the first layer and the feature of the second layer Layer characteristics Perform feature dimension splicing to obtain a candidate region feature .

2. The mirror completion point cloud 3D target cascade detection method for automatic driving according to claim 1, characterized in that, The neural network is trained by the training data set, and the training parameters of the neural network are adjusted by one-stage detection loss, cascade detection loss and mirror loss, to obtain a trained neural network; Step 601: setting a prediction box indicating function according to the prediction box score ； Step 602: use cross entropy CE loss weight for one-stage detection head class probability use L1 loss function L1Loss for the center position, size, and rotation angle of the prediction box, with weight use L1 loss function L1Loss for the orientation of the prediction box, with weight sum the three to obtain the calculation result of the one-stage loss function; ； Step 603: use binary cross-entropy BCELoss for all 3D detection head class probabilities, with weights ; use L1Loss for the center position, size, and rotation angle of the predicted box, with weights ; the weighted sum of all loss functions in step 603 obtains the calculation result of the cascaded loss function; ； Step 604: Selecting a set of voxels for the first layer , determining the real label of the mirror voxel center coordinate , taking the voxel center coordinate of as the predicted value , using the SmoothL1Loss loss function on the real value and the predicted value of the mirror coordinate, and the loss function boundary threshold is , the loss weight ; ； Step 605: sum the results of the one-stage detection loss, the cascade detection loss and the mirror loss as the final loss for training the trained neural network; 。

Images