Single-stage 3D point cloud target detection method and device, computer equipment, and medium
Through the combination of DBSCAN clustering and U-Net mask sampling module, the problem of long-distance point cloud recognition difficulties and high computational volume in 3D point cloud object detection is solved, and the detection performance and feature characterization are improved, especially in the KITTI data set performance on.
Patent Information
- Application Number
- CN202111271651.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-10-29
AI Technical Summary
The existing 3D point cloud object detection method has poor recognition effect on objects with long distances and sparse point distribution. Due to the lack of context information between the features obtained by spherical sampling, it leads to large amount of calculation and poor recognition performance, especially spatial position and rotation. Inadequate prediction performance of direction.
Using the data augmentation method based on DBSCAN clustering, point cloud data of different density regions is generated through clustering operations, and combined with the mask sampling module of the U-Net architecture and the feature fusion module of the compression-activation architecture, the point cloud target is improved. The effect of detection.
Improve the robustness of the model to point clouds, enhance the characterization of features, especially the recognition ability in long-distance point cloud distribution, improve detection performance, and reduce computational volume, and can obtain more on the KITTI dataset. Excellent results.
Smart Images

Figure CN114155524B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a single-stage 3D point cloud target detection method and device based on DBSCAN clustering data augmentation, computer equipment, and medium, and belongs to the field of computer software technology. Background Art
[0002] Object detection and recognition in outdoor point cloud scenes has been a research hotspot in recent years, among which object detection is the core part of the entire process. Its task is to correctly locate and identify the category of stereoscopic objects from irregularly distributed and sparse point clouds. With the development of deep learning, point cloud-based 3D object detection methods have made rapid progress. Inspired by the object detection methods in 2D images, the current mainstream point cloud 3D object detection methods focus on how to extract representative and regularly arranged features from point clouds through the backbone network, so that the convolution processing method applicable to 2D images can be directly applied to 3D point cloud features. The rich semantic features obtained by the backbone network are input into the head network to predict the spatial position and category of the object in the point cloud, and the loss related to the 3D label calculation is provided to provide supervision signals to the neural network to complete the training.
[0003] The above traditional solutions have the following defects:
[0004] 1. Existing methods have good detection and recognition effects for objects with close distance and dense point distribution, but poor recognition effects for objects with long distance and sparse point distribution.
[0005] 2. There are a large number of points in the point cloud. If all the corresponding features are extracted, it will take a lot of time and require a lot of calculations. In order to balance performance and required resources, the existing single-stage point cloud 3D object detection methods all require a sampling process. In the encoding process, the existing methods only select the features with the largest modulus as candidate features, and do not consider the rich contextual information between the features obtained by spherical sampling, and cannot obtain semantic features with stronger representation.
[0006] 3. The final detection of the existing method is based on high-level semantic features, which has good recognition performance. However, due to the large loss of low-level spatial features in the convolution process, the prediction performance of spatial position and rotation direction is poor. Summary of the invention
[0007] In view of the technical problems existing in the prior art, the purpose of the present invention is to provide a single-stage 3D point cloud target detection method and device, computer equipment, and medium based on DBSCAN clustering data augmentation, which includes a new data augmentation method, a new mask sampling method, and a new feature fusion module, thereby improving the effect of 3D point cloud target detection.
[0008] The technical solution of the present invention is:
[0009] A single-stage 3D point cloud target detection method, the steps of which include:
[0010] Training phase of the backbone network:
[0011] For the 3D point cloud in the training sample set, the data augmentation module sets different clustering radius parameters to perform clustering operations on the 3D point cloud according to the different distances between the target instance in the 3D point cloud and the sampling sensor for obtaining the 3D point cloud, and uses a stereo bounding box to enclose the same type of point cloud to obtain the minimum stereo bounding box and put it into the 3D stereo box set db_boxes; calculates the intersection and union ratio of each minimum stereo bounding box in the 3D stereo box set db_boxes and the real 3D stereo box gt_boxes of the corresponding target instance in the point cloud. If the intersection and union ratio is greater than the set threshold, the corresponding minimum stereo bounding box is saved in the saved_boxes set, and the points in the corresponding minimum stereo bounding box are removed from the 3D point cloud; then the points in the saved_boxes set are downsampled to obtain the point cloud data after data augmentation;
[0012] The backbone network samples the point cloud data after data augmentation based on the farthest distance in the sphere, extracts features from the sampled points, and inputs the extracted semantic features and spatial features into the mask sampling module of the backbone network;
[0013] The mask sampling module predicts the prediction score of each point according to the input spatial features and semantic features, and then completes point sampling from high to low according to the prediction scores and inputs them into the feature fusion module of the backbone network;
[0014] The feature fusion module decouples the features of the input sampling points to obtain semantic features and spatial information; convolution is performed on the decoupled semantic features to obtain compressed features of the semantic features and inputs them into the Sigmoid function to obtain a semantic attention map; convolution is performed on the decoupled spatial information to obtain compressed features of the spatial information and inputs them into the Sigmoid function to obtain a spatial attention map; the semantic attention map and the spatial attention map are added bit by bit to obtain a compressed attention map; the compressed attention map is multiplied point by point with the features of the input sampling points to obtain an activated feature map; the dimension size of the feature map of the input sampling points is adjusted so that it is consistent with the dimension of the activated feature map, and then bit by bit is added to obtain a fused feature map;
[0015] The regression prediction network of the backbone network predicts according to the fused feature map to obtain the position and category of the target instance in the point cloud; then the parameters of the backbone network are updated based on the prediction results and the set loss function;
[0016] Application phase:
[0017] The 3D point cloud to be processed is input into the trained backbone network; the backbone network samples the 3D point cloud data to be processed based on the farthest distance in the sphere, and extracts features from the sampled points, and the extracted semantic features and spatial features are processed in turn by the mask sampling module and the feature fusion module to obtain a fused feature map; then the regression prediction network performs prediction based on the fused feature map to obtain the position of the target in the 3D point cloud to be processed.
[0018] Optionally, the mask sampling module is a binary classification network consisting of a feature encoding module and a feature decoding module.
[0019] Optionally, the mask sampling module inputs the feature map obtained by dimensionally splicing the input spatial features and semantic features into the feature encoding module for downsampling, so as to obtain downsampled feature maps of different stages; the downsampled feature map of the final stage is input into the feature decoding module for upsampling, and the upsampled feature map of the obtained stage is spliced with the downsampled feature map of the same stage to perform upsampling in the next stage, and the prediction score of each point is predicted according to the splicing result of the upsampled feature map of the final stage and the downsampled feature map of the first stage, and then the point sampling is completed from high to low according to the prediction scores.
[0020] Optionally, the clustering operation is a DBSCAN clustering operation.
[0021] Optionally, the feature encoding module includes two 3x3 convolution units connected in sequence, wherein each 3x3 convolution unit is sequentially connected to a batch normalization processing unit, a linear correction unit, and a maximum pooling downsampling unit; the feature encoding module includes two 2x2 transposed convolution units and two 3x3 convolution units, the first 2x2 transposed convolution unit processes the input information of the feature encoding module and inputs the input information to the first 3x3 convolution unit, the information processed by the first 3x3 convolution unit is sequentially processed by the batch normalization processing unit and the linear correction unit and then input into the second 3x3 convolution unit, and the information processed by the second 3x3 convolution unit is sequentially processed by the batch normalization processing unit and the linear correction unit and then input into the second 2x2 transposed convolution unit.
[0022] Optionally, the sampling sensor is a lidar sensor.
[0023] Optionally, the regression prediction network is an anchor-free regression head network.
[0024] The present invention also provides a single-stage 3D point cloud target detection device, characterized in that the device comprises:
[0025] The data augmentation module is used to perform clustering operations on the 3D point clouds in the training sample set, for target instances in the 3D point clouds and the sampling sensors for obtaining the 3D point clouds at different distances, set different clustering radius parameters, enclose the same type of point clouds with a stereo bounding box to obtain a minimum stereo bounding box and put it into a 3D stereo box set db_boxes; calculate the intersection and union ratio of each minimum stereo bounding box in the 3D stereo box set db_boxes and the real 3D stereo box gt_boxes of the corresponding target instance in the point cloud, and if the intersection and union ratio is greater than a set threshold, save the corresponding minimum stereo bounding box in the saved_boxes set, and remove the points in the corresponding minimum stereo bounding box from the 3D point cloud; then downsample the points in the saved_boxes set to obtain point cloud data after data augmentation;
[0026] The feature extraction module is used to sample the point cloud data after data augmentation based on the farthest distance in the sphere, extract features from the sampled points, and input the extracted semantic features and spatial features into the mask sampling module;
[0027] The mask sampling module is used to predict the prediction score of each point based on the input spatial features and semantic features, and then complete the point sampling from high to low according to the prediction score and input it into the feature fusion module;
[0028] The feature fusion module is used to decouple the features of the input sampling points to obtain semantic features and spatial information; convolve the decoupled semantic features to obtain compressed features of the semantic features and input them into the Sigmoid function to obtain a semantic attention map; convolve the decoupled spatial information to obtain compressed features of the spatial information and input them into the Sigmoid function to obtain a spatial attention map; add the semantic attention map and the spatial attention map bit by bit to obtain a compressed attention map; multiply the compressed attention map and the features of the input sampling points point by point to obtain an activated feature map; adjust the dimension size of the feature map of the input sampling points so that it is consistent with the dimension of the activated feature map, and then add them bit by bit to obtain a fused feature map;
[0029] The regression prediction network is used to make predictions based on the fused feature map to obtain the location and category of the target in the point cloud.
[0030] The present invention also provides a computer device, characterized in that it includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the above method.
[0031] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0032] The advantages of the present invention are as follows:
[0033] In the 3D point cloud target detection task, the points are irregularly distributed in space, and the density of points in different areas varies greatly. For targets close to the sampling sensor, the point density is large and the spatial feature representation is good; for targets far away from the sampling sensor, the point density is low and the spatial feature representation is poor. The poor detection and recognition of these targets is the bottleneck of the current 3D point cloud target detection performance. The present invention proposes a new data augmentation module based on DBSCAN. This module obtains different density area categories by clustering point clouds. The corresponding minimum bounding box can be obtained from the point set of the same category. Then, the sparse distribution of distant point clouds is simulated by downsampling the point cloud within the minimum stereo bounding box, thereby improving the robustness of the model to point clouds.
[0034] A mask sampling module based on the U-Net architecture completes the sampling of points in the point cloud. The sampled points do not have the duplication in traditional methods, which can provide more context information for subsequent feature processing.
[0035] A feature fusion module based on compression-activation architecture can better fuse low-level position information and high-level semantic features. That is, this module can fuse spatial features and semantic features, better preserve spatial features that are conducive to spatial regression and semantic features that are conducive to recognition, and provide more representative features for the 3D point cloud object detection process of the head network.
[0036] Experiments show that the present invention can achieve better performance on the KITTI dataset, and the present invention can be seamlessly embedded in the existing 3D point cloud target detection method with point cloud as input. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 This is a network structure diagram.
[0038] Figure 2 This is the result of data augmentation and downsampling for DBSCAN clustering.
[0039] Figure 3 This is the Density-SA module structure diagram. DETAILED DESCRIPTION
[0040] The present invention is further described in detail below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention but not to limit the scope of the present invention.
[0041] This paper proposes a 3D point cloud target detection method (Density-Net) based on density perception. The overall structure is as follows Figure 2 As shown in the figure, Density-Net is a 3D target detection model based on Point-Net, which introduces three new modules:
[0042] (1) Data augmentation module based on DBSCAN clustering.
[0043] (2) Mask sampling module based on U-Net
[0044] (3) Feature fusion module based on compression-activation
[0045] Data augmentation module based on DBSCAN clustering:
[0046] The laser radar uses a rotating method to obtain the laser signals returned by different distance points, and then forms a 360-degree point cloud. The DBSCAN clustering data augmentation module sets different clustering radius parameters to cluster the point cloud according to the different distances between the target and the laser radar sensor, and then uses a stereo bounding box to enclose the same type of point cloud to obtain the minimum stereo bounding box, and downsamples the point set in the box to obtain the point cloud data after data augmentation. The specific operations are as follows:
[0047] Algorithm: DBSCAN Data Augmentation
[0048] enter:
[0049] DBSCAN clustering radius radius_scales = [s 1 , ..., s M1}
[0050] Cluster radius scaling factor s 1 s M1 Downsampling factor ds scales ={ds 1 , ..., ds M2}, ds∈[0,1], 0 means all points are deleted, 1 means all points are retained
[0051] Point cloud set P = {p 1 , ..., p N}
[0052] The real 3D box label of the object in the point cloud G = {g 1 , ..., g N}
[0053] Output:
[0054] The point cloud set C after data augmentation = {c 1 , ..., cN}
[0055] Training process:
[0056] (1) Only the foreground part (i.e., the driver’s perspective) is retained in the 360° point cloud, and the points in the point cloud are divided into several sectors according to the distance from the lidar, i.e., (0, 20 m], (20 m, 40 m], and (40 m+).
[0057] (2) Set the clustering radius radius_scales as the radius of DBSCAN clustering, set the minimum number of clustering points in DBSCAN to 5, and then perform DBSCAN clustering operations to obtain clustering information of points within the corresponding sector. Based on the points of the same category, a 3D stereo box is constructed to enclose all points of the same category, and then the minimum stereo bounding box can be obtained. If the center coordinates of this bounding box are within the current sector, the minimum stereo bounding box is retained, otherwise it is not retained. After processing the obtained clustering points as above, several 3D stereo box sets db_boxes can be obtained.
[0058] (3) The 3D stereo box set db_boxes obtained from step (1) is calculated one by one with the real 3D stereo box gt_boxes of the target instance in the point cloud. The Jaccard coefficient is greater than 0.01, and the 3D stereo box is retained in the saved_boxes set. The points in the 3D stereo box are removed from the point cloud set composed of the points collected by the sensor, and the points in the saved_boxes set are downsampled according to the downsampling factor ds. scales The downsampling is done sequentially, and the points that remain after downsampling are added to the point cloud.
[0059] (4) In this way, the point cloud C after data augmentation is obtained as the input of the model. The result after point cloud downsampling can be seen Figure 1 According to the size and position of the stereo box generated by DBSCAN clustering, it can be divided into overall downsampling and partial downsampling. Through this method, more point cloud distributions of distant instances can be simulated, which increases the samples that can participate in training and improves the robustness of the model.
[0060] MaskSample module based on U-Net:
[0061] In order to achieve a balance between performance and computational cost, the point cloud needs to be downsampled. The current mainstream method is to complete the sampling by constructing the farthest distance sampling within the sphere, but this method is not effective for sampling distant points because the distant points are sparsely distributed and the number is small. Many points will be discarded during the downsampling process. This treatment has little effect on target instances that are close to the sensor and contain more points, but it has a greater impact on target instances that are far away from the sensor and contain fewer points. It is possible that the points sampled in the end are not the points corresponding to the target instance. In the backbone network, the points obtained by this sampling method are subjected to feature extraction operations to obtain semantic features and spatial features. These features do not have good discriminatory power and will reduce the accuracy of the final head network prediction of the 3D stereo frame position regression and the recall rate of the target instance points.
[0062] In the current single-stage 3D point cloud object detector, point sampling is mostly based on the farthest distance sampling within the sphere. This sampling method first constructs a three-dimensional sphere with a known radius and performs farthest point sampling on the points contained in it. In scenes with dense points, many different points with good representation capabilities can be extracted; however, for scenes with sparse points, the number of points is far less than the number of point sampling required. At this time, the general processing method is repeated sampling, and the obtained points have poor representation capabilities. According to statistics, there are 12% duplications in the points obtained by repeated sampling, of which the points belonging to the target instance only account for 33.28% of the total sampling points.
[0063] The present invention constructs a mask sampling module based on U-Net, and its overall structure is as follows: Figure 1 As shown in the network structure in the upper middle part, the binary classification network consists of a feature encoding module and a feature decoding module. The output is the predicted score of whether each point is within the 3D real stereo frame, and then the point sampling is completed from high to low according to the predicted score (that is, the N points with the highest predicted scores are selected). The structure of the mask sampling module is as follows: The data processing flow of the mask sampling module: First, the spatial features and semantic features are spliced by dimension, and the new feature map is input into the encoder. The encoder consists of two 3x3 convolutions, each of which is followed by batch normalization (BN), linear rectification unit (ReLU), and 2x2 maximum pooling downsampling operations with a stride of 2. The encoder obtains the feature maps of downsampling at different stages. The decoder convolution layer consists of two 2x2 transposed convolutions and 3x3 convolution units. Each 3x3 convolution is followed by batch normalization (BN) and linear rectification unit (ReLU). The decoder obtains the upsampling feature maps at different stages.
[0064] The processing flow of the mask sampling module is as follows: the spatial features and semantic features are concatenated as the input of the 3x3 convolution in the encoder, and the output features are batch normalized (BN), rectified linear unit (ReLU), and 2x2 maximum pooling with a stride of 2. This operation is repeated twice to obtain the output of the encoder. The output feature map of the encoder is used as the input of the 2x2 transposed convolution in the decoder, and the output features are concatenated with the encoder features of the same stage (the feature maps of the same stage have the same width, height and dimension information), and then input into the 3x3 convolution to obtain the output features. This operation is repeated twice to obtain the output of the encoder. Finally, a 1x1 convolution is used to map the number of sampling points required, and then the first 1024 points are selected based on the prediction score of each candidate point.
[0065] Compression-activation feature fusion (Density-SA) module based on residual structure:
[0066] The collection extraction module in PointNet++ mainly includes three layers: sampling layer, collection layer and PointNet layer. Our goal is to obtain better high-level semantic features while preserving more accurate spatial features, so that more discriminative information can be provided during classification and position regression. Inspired by the compression-activation architecture, we designed a new feature fusion extraction module based on residual structure compression-activation, named Density-SA. The specific structure is as follows: Figure 3 As shown in the figure. First, the semantic features and spatial information in the initial features are decoupled, that is, the spatial position information of the point and the semantic information corresponding to the point are separated, and then the two are respectively input into the 1x1 convolution to obtain the compressed features of the output space and semantics. Each 1x1 convolution mentioned above is followed by batch normalization (BN) and linear rectification unit (ReLU) operations. Then the compressed features of space and semantics are input into the Sigmoid function to obtain the spatial attention map and the semantic attention map respectively, and the two are added bit by bit to obtain the final compressed attention map. The obtained compressed attention map is multiplied point by point with the initial feature to obtain the activated feature map, and then the initial feature is input into the 1x1 convolution, and the dimension size of the initial feature map is adjusted to make it consistent with the dimension of the activated feature map. Finally, the activated feature map is added bit by bit with the initial feature to obtain the final fused feature map.
[0067] The whole process of the present invention is as follows:
[0068] First, the laser reflection signal of the target instance is obtained through the lidar sensor to construct a point cloud data structure.
[0069] The input point cloud is processed by the DBSCAN clustering augmentation method to obtain point cloud data after data augmentation, which increases the point cloud data that can be used for training.
[0070] The point cloud is input into the backbone network, and the MaskSample module is used to complete the sampling of the point cloud. The sampled point cloud is extracted through the Density-SA module based on the residual structure to extract the fused spatial and semantic features and input into the anchor-free regression head network (3DSSD CVPR 2020).
[0071] The anchor-free frame regression head network predicts the position and category of the 3D stereo frame in the point cloud. The entire model supervises the training of the network by calculating the cross entropy loss of the target object category, the smooth L1 loss of the spatial position, the smooth L1 loss of the target object size, the binary cross entropy loss of the mask sampling, and the angle regression loss. The angle regression loss is as follows:
[0072] L angle =L c (d c ,t c )+D(d r ,t r )
[0073] First, 360° is evenly divided into 12 interval categories. We first predict the interval category corresponding to the angle, and then predict the distance from the center in this interval, that is, the residual. c and d r Represent the predicted angle category and the corresponding residual, t c and t r are their corresponding target values. c represents cross entropy loss (Cross Entropy), and D represents spatial position smooth L1 loss (Smooth L1).
[0074] We conducted extensive experiments to evaluate the effectiveness of Density-Net. Our model was trained on KITTI, a widely used 3D outdoor autonomous driving target detection dataset, and tested on the KITTI validation dataset. KITTI has 7481 training samples and 7518 test samples, of which the 7481 training samples are further divided into a training set (3712 samples) and a validation set (3769 samples). We mainly evaluated the performance of the model on the car class, following the official evaluation criteria of the KITTI dataset. The target is considered to be correctly detected if the spatial intersection-over-union ratio with the real 3D box is greater than 0.7 and the category is accurate.
[0075] Table 1 shows the experimental results of each module comparison
[0076]
[0077] Table 1 shows the effect comparison between each module of our model. The results prove that the DBSCAN clustering data augmentation proposed in the present invention can increase the sparse low-density area in the point cloud, simulate the distribution of distant target point clouds, and improve the model performance; the proposed mask sampling module completes the downsampling of non-repeated point clouds by sorting the values in the mask from high to low; the proposed Density-SA module can obtain optimized features with more discriminative power. The present invention can improve the performance of the baseline on the KITTI dataset, proving the effectiveness of the present invention.
[0078] Table 2 shows the comparison of the effects of the present invention and other mainstream methods on the KITTI test dataset. From the detection results of the present invention on the KITTI dataset, it can be found that the present invention can accurately detect and identify distant targets.
[0079] Table 2 shows the performance comparison of Density-Net and other methods on the KITTI dataset test set
[0080]
[0081]
[0082] Where '-' means that this category was not tested.
[0083] Although the specific embodiments of the present invention are disclosed for the purpose of illustration, the purpose is to help understand the content of the present invention and implement it accordingly, those skilled in the art will understand that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the best embodiment, and the scope of the present invention is subject to the scope defined in the claims.
Claims
1. A single-stage 3D point cloud object detection method, the steps of which are include: Training phase of the backbone network: For the 3D point cloud in the training sample set, the data augmentation module sets different clustering radius parameters to perform clustering operations on the 3D point cloud according to the different distances between the target instance in the 3D point cloud and the sampling sensor for obtaining the 3D point cloud, and uses a stereo bounding box to enclose the same type of point cloud to obtain the minimum stereo bounding box and put it into the 3D stereo box set db_boxes; calculates the intersection and union ratio of each minimum stereo bounding box in the 3D stereo box set db_boxes and the real 3D stereo box gt_boxes of the corresponding target instance in the point cloud. If the intersection and union ratio is greater than the set threshold, the corresponding minimum stereo bounding box is saved in the saved_boxes set, and the points in the corresponding minimum stereo bounding box are removed from the 3D point cloud; then the points in the saved_boxes set are downsampled to obtain the point cloud data after data augmentation; The backbone network samples the point cloud data after data augmentation based on the farthest distance in the sphere, extracts features from the sampled points, and inputs the extracted semantic features and spatial features into the mask sampling module of the backbone network; The mask sampling module predicts the prediction score of each point according to the input spatial features and semantic features, and then completes point sampling from high to low according to the prediction scores and inputs them into the feature fusion module of the backbone network; The feature fusion module decouples the features of the input sampling points to obtain semantic features and spatial information; convolution is performed on the decoupled semantic features to obtain compressed features of the semantic features and inputs them into the Sigmoid function to obtain a semantic attention map; convolution is performed on the decoupled spatial information to obtain compressed features of the spatial information and inputs them into the Sigmoid function to obtain a spatial attention map; the semantic attention map and the spatial attention map are added bit by bit to obtain a compressed attention map; the compressed attention map is multiplied point by point with the features of the input sampling points to obtain an activated feature map; the dimension size of the feature map of the input sampling points is adjusted so that it is consistent with the dimension of the activated feature map, and then bit by bit is added to obtain a fused feature map; The regression prediction network of the backbone network predicts according to the fused feature map to obtain the position and category of the target instance in the point cloud; then the parameters of the backbone network are updated based on the prediction results and the set loss function; Application phase: The 3D point cloud to be processed is input into the trained backbone network; the backbone network samples the 3D point cloud data to be processed based on the farthest distance in the sphere, and extracts features from the sampled points, and the extracted semantic features and spatial features are processed in turn by the mask sampling module and the feature fusion module to obtain a fused feature map; then the regression prediction network performs prediction based on the fused feature map to obtain the position of the target in the 3D point cloud to be processed.
2. The method according to claim 1, It is characterized in that The mask sampling module is a binary classification network composed of a feature encoding module and a feature decoding module.
3. The method according to claim 2, It is characterized in that The mask sampling module inputs the feature map obtained by splicing the input spatial features and semantic features according to the dimensions into the feature encoding module for downsampling, and obtains the downsampled feature maps of different stages; the downsampled feature map of the last stage is input into the feature decoding module for upsampling, and the upsampled feature map of the obtained stage is spliced with the downsampled feature map of the same stage to perform upsampling in the next stage, and the prediction score of each point is predicted according to the splicing result of the upsampled feature map of the last stage and the downsampled feature map of the first stage, and then the point sampling is completed from high to low according to the prediction score.
4. The method according to claim 2 or 3, It is characterized in that The feature encoding module includes two 3x3 convolution units connected in sequence, wherein each 3x3 convolution unit is sequentially connected to a batch normalization processing unit, a linear correction unit, and a maximum pooling downsampling unit; the feature encoding module includes two 2x2 transposed convolution units and two 3x3 convolution units, the first 2x2 transposed convolution unit processes the input information of the feature encoding module and then inputs it into the first 3x3 convolution unit, the information processed by the first 3x3 convolution unit is sequentially processed by the batch normalization processing unit and the linear correction unit and then input into the second 3x3 convolution unit, and the information processed by the second 3x3 convolution unit is sequentially processed by the batch normalization processing unit and the linear correction unit and then input into the second 2x2 transposed convolution unit.
5. The method according to claim 1, It is characterized in that The clustering operation is a DBSCAN clustering operation.
6. The method according to claim 1, It is characterized in that The sampling sensor is a laser radar sensor.
7. The method according to claim 1, It is characterized in that The regression prediction network is an anchor-free regression head network.
8. A single-stage 3D point cloud object detection device, It is characterized in that The device comprises: The data augmentation module is used to perform clustering operations on the 3D point clouds in the training sample set, for target instances in the 3D point clouds and the sampling sensors for obtaining the 3D point clouds at different distances, set different clustering radius parameters, enclose the same type of point clouds with a stereo bounding box to obtain a minimum stereo bounding box and put it into a 3D stereo box set db_boxes; calculate the intersection and union ratio of each minimum stereo bounding box in the 3D stereo box set db_boxes and the real 3D stereo box gt_boxes of the corresponding target instance in the point cloud, and if the intersection and union ratio is greater than a set threshold, save the corresponding minimum stereo bounding box in the saved_boxes set, and remove the points in the corresponding minimum stereo bounding box from the 3D point cloud; then downsample the points in the saved_boxes set to obtain point cloud data after data augmentation; The feature extraction module is used to sample the point cloud data after data augmentation based on the farthest distance in the sphere, extract features from the sampled points, and input the extracted semantic features and spatial features into the mask sampling module; The mask sampling module is used to predict the prediction score of each point based on the input spatial features and semantic features, and then complete the point sampling from high to low according to the prediction score and input it into the feature fusion module; The feature fusion module is used to decouple the features of the input sampling points to obtain semantic features and spatial information; convolve the decoupled semantic features to obtain compressed features of the semantic features and input them into the Sigmoid function to obtain a semantic attention map; convolve the decoupled spatial information to obtain compressed features of the spatial information and input them into the Sigmoid function to obtain a spatial attention map; add the semantic attention map and the spatial attention map bit by bit to obtain a compressed attention map; multiply the compressed attention map and the features of the input sampling points point by point to obtain an activated feature map; adjust the dimension size of the feature map of the input sampling points so that it is consistent with the dimension of the activated feature map, and then add them bit by bit to obtain a fused feature map; The regression prediction network is used to make predictions based on the fused feature map to obtain the location and category of the target in the point cloud.
9. A computer device, It is characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
