A 3D target detection method based on the search radius of the center point neighborhood guided by category

Abstract

This invention discloses a 3D target detection method based on the search radius of the neighborhood of the center point guided by the category, comprising the following steps: (1) The structure of the LiDAR point cloud is not changed. In the data preprocessing stage, the point clouds collected in different scenes are downsampled to a fixed value N as the input of the neural network; (2) A neural network structure is established. The neural network structure is trained using a known target detection dataset. The total loss function is set to supervise the output of the neural network, and the parameters of the trained neural network structure are obtained; (3) The point cloud collected by the LiDAR in the autonomous driving scenario is input into the trained neural network structure, and the category information of the neural network and the regression residual of the 3D prediction box are output. The 3D prediction box is calculated based on the classification result of the current scene and the regression residual to obtain the final prediction result. This invention provides a solution for 3D target detection in real-world autonomous driving scenarios.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving perception technology, and relates to a three-dimensional target detection method based on LiDAR point clouds, particularly a three-dimensional target detection method based on the search radius of the neighborhood of a category-guided center point. Background Technology

[0002] To comprehensively understand the driving environment, autonomous driving perception systems involve many fundamental vision tasks, such as object detection and tracking, lane detection, semantic and instance segmentation, etc. Among these perception tasks, 3D object detection is the most indispensable task in a vehicle perception system. 3D object detection aims to predict the position, size, orientation, and category of key objects in 3D space, including categories such as motor vehicles, pedestrians, and cyclists. Compared to 2D object detection, which only generates 2D bounding boxes on an image while ignoring the actual distance information between key objects and the current vehicle, 3D object detection focuses more on the localization and recognition of objects in the real-world 3D coordinate system. The geometric information predicted by 3D object detection in real-world coordinates can be directly used to measure the distance between the vehicle and key objects, further aiding in route planning and collision avoidance.

[0003] Based on the representation of LiDAR point clouds in 3D target detection methods and the corresponding feature extraction networks, they can be divided into four types: voxel-based, pillar-based, view-based, and point-based.

[0004] Voxels, analogous to pixels in a two-dimensional image, are a regular, ordered form of data representation in 3D space. Voxel-based methods first transform irregular point clouds into compact voxels based on coordinates, enabling efficient extraction of mesh features for 3D object detection via a 3D convolutional neural network. Then, the 3D feature map is reshaped along the height dimension into 2D features from a bird's-eye view perspective, reusing a region candidate network from the 2D object detection domain to generate the final 3D detection box. VoxelNet was the first network to utilize voxels for 3D object detection, introducing the concept of Voxel Feature Extraction (VFE), which captures 3D shape information from points within a single voxel, encoding the initial voxel feature vector. However, the computational efficiency of 3D convolutional neural networks leads to poor real-time performance. SECOND, building on this, proposed a more efficient backbone network that uses the 3D sparse convolution operator to extract features only from valid, non-empty voxels, significantly improving the network's inference speed. In addition to convolution operators, SST uses a self-attention mechanism network to extract features from voxels. The main motivation is that after voxels pass through a convolutional neural network, the feature map will have a reduced resolution compared to the input. The key target has very few features corresponding to the low-resolution feature map, resulting in false detections. The advantage of the attention mechanism is that the resolution remains unchanged during the feature extraction process, which can better improve the detection accuracy of the network. However, the attention mechanism has high computational complexity, resulting in poor real-time performance.

[0005] Similar to voxel representations, pillar-based representations ignore the division along the height dimension and divide the 3D space into pillars of fixed size. Intuitively, a pillar can be viewed as an undivided voxel of height; therefore, a pillar-based point cloud representation can be considered a multi-channel bird's-eye view pseudo-image. The most representative pillar-based feature extraction module was proposed by PointPillars. First, the point cloud is divided into multiple pillars according to coordinates, and the maximum number of points in each pillar is determined, padding with zeros if insufficient, and sampling if excessive. Each point within a pillar is encoded by a 9-dimensional vector consisting of its original position, reflection intensity, distance from the pillar center, and offset distance from the pillar centroid coordinates (the arithmetic mean of all points within the pillar). Pillar features are extracted using a miniature PointNet. The feature extraction backbone network consists of 2D convolutional operations, which significantly improves inference efficiency compared to the 3D convolutional operations of voxels, while also being easy to deploy, making it one of the commonly used perceptual algorithm frameworks in industry.

[0006] In projection-based representations, points in 3D space are projected onto a 2D plane under perspective transformation. Distance-view projection projects the point cloud onto a spherical surface with its origin located on the LiDAR sensor.

[0007] Given three-dimensional coordinates [x i ,y i ,z i The point is represented by spherical coordinates [φ]. i ,θ i ,d i The calculation is shown in formula (1–1).

[0008]

[0009] Range RCNN is a view-based representative network. First, the LiDAR point cloud is projected onto a range view, with each pixel encoded by a 5-dimensional vector (x, y, z, d, r), where (x, y, z) are the point coordinates, r is the point's reflection intensity, and d is the distance from the point to the LiDAR sensor. The feature extraction backbone network learns pixel features using 2D convolutional operations. To address the issues of object occlusion and scale variations in the range view, a range view -> point -> bird's-eye view module is implemented to transfer features from range view pixels to points in 3D space, and then from points in 3D space to the bird's-eye view feature map.

[0010] Point-based representations preserve the unstructured form of point clouds. However, for efficient parallel computation, downsampling to a fixed size is necessary. Point-based 3D object detection methods often employ random sampling and farthest-point sampling algorithms, repeatedly downsampling the point cloud from its original size to a smaller, fixed size of N points. In random sampling, points are randomly selected until N points are chosen. However, this sampling method has the drawback that points in denser regions of the point cloud are more easily sampled than those in sparse regions. The farthest-point sampling algorithm mitigates this bias by iteratively sampling points based on the farthest distance standard. Compared to randomly sampled point clouds, the farthest-point sampling algorithm produces a more representative point cloud, but it also increases computational cost. Each point in the generated subsampled point cloud is encoded using the same features. For 3D object detection tasks, foreground points within the target's 3D ground truth bounding box are more valuable. Therefore, variations have been developed that replace the farthest point sampling algorithm using Euclidean distance with semantic features generated by points in the neural network, or utilize semantic category information to supervise the downsampling process. The aim of these designs is to retain as many foreground points as possible during downsampling. However, when performing multi-class object detection in a network, to accommodate large, critical targets, the neighborhood search radius of the center point is usually set to a large, uniform value. This causes small targets to aggregate into a large amount of background information, leading to false detections. Achieving multi-class 3D object detection within the same network is also a fundamental requirement for perception systems in autonomous driving scenarios. Summary of the Invention

[0011] The purpose of this invention is to address the shortcomings of existing technologies by providing a 3D target detection method that uses category-guided center point neighborhood search radius. By utilizing the rich semantic features of points retained in the feature extraction backbone network within the neural network, the category information of the center point is inferred to control the generation of the center point neighborhood search radius, thereby achieving a balance in the accuracy of multi-category detection in the 3D target detection network and providing a solution for 3D target detection in real-world autonomous driving scenarios.

[0012] This invention proposes a 3D target detection method based on category-guided center point neighborhood search radius. Based on the representation of points, it preserves the original structure of the point cloud and retains the 3D geometric information of the point cloud to the greatest extent. Simultaneously, to address the computational time consumption during point cloud downsampling, this method utilizes the rich features of the point cloud in the neural network to infer the foreground / background classification results of points. Points with high confidence in the foreground are selected and retained for the next stage, accelerating the sampling process, avoiding extensive distance calculations, and ensuring that more effective foreground points are retained during downsampling, thus guaranteeing the algorithm's detection accuracy. To achieve a balance in accuracy across multiple target classes, the category-guided center point neighborhood search radius generation module sets different neighborhood search radii for different size classes, allowing small targets to focus more on their own point cloud features, eliminating interference from background points, and improving the detection accuracy of small targets. The center point feature aggregation module based on a channel-level attention mechanism considers not only different neighbor nodes but also the different contributions of different channels of node features to the center point, fully mining the geometric features of neighbor nodes to form more fine-grained instance-level features. The point-anchor-box mechanism balances the number of positive and negative samples and greatly alleviates the problem of unpredictable target orientation, thus improving the network's detection recall. This method meets real-time requirements while achieving the best detection accuracy among point-based algorithms of the same type.

[0013] The technical solution adopted in this invention has the following steps:

[0014] (1) In this invention, the lidar point cloud is represented by points, without changing the structure of the point cloud, thus preserving the original 3D geometric information of the point cloud to the greatest extent. In the data preprocessing stage, in order to enable parallel operation of the neural network, the point clouds collected from different scenes need to be downsampled to a fixed value N as the input of the neural network.

[0015] (2) Establish a neural network structure, train the neural network structure using the known dataset, set the total loss function to supervise the output of the neural network, and obtain the parameters of the trained neural network structure.

[0016] (3) Input the point cloud collected by the LiDAR in the autonomous driving scenario into the trained neural network structure, output the category information of the neural network and the regression residual of the 3D prediction box, calculate the 3D prediction box according to the classification result of the current scene and the regression residual, and obtain the final prediction result.

[0017] In step (1), since the present invention is based on the representation of points, there is no need to convert the data representation of the point cloud. However, in order to parallelize the neural network training process, the point clouds collected from different scenes need to be downsampled to the same value N as the input of the neural network.

[0018] In step (2), the neural network structure includes a point-based feature extraction backbone network, a center point generation module, a center point neighborhood search radius generation module, a center point feature aggregation module based on channel-level attention mechanism, and a detection head module based on point anchor box mechanism. The raw features of the point cloud are input into the point-based feature extraction backbone network. The center point generation module and the center point neighborhood search radius generation module run in parallel, providing the center point 3D coordinates and center point category information to the subsequent center point feature aggregation module based on channel-level attention mechanism. The detection head based on point anchor box mechanism is connected to the center point feature aggregation module and serves as the output of the neural network.

[0019] The point-based feature extraction backbone network consists of four downsampling-feature aggregation blocks connected end to end. The downsampling strategy in the first two blocks adopts the farthest point sampling algorithm, while the sampling strategy in the last two blocks is guided by semantic information. The operation of the feature aggregation blocks remains consistent.

[0020] The input point cloud consists of N points. First, after passing through the first downsampling module and the farthest point sampling algorithm, N1 points are retained, followed by feature aggregation. Specifically, a uniform neighborhood search radius is set for the N1 points, with the search range being the initial N points in the point cloud. A spherical search is used, and the search results are grouped, resulting in each of the retained N1 points having n1 neighbor nodes. For each group, feature aggregation is performed. First, the features of the neighbor nodes are input into a multilayer perceptron network for dimensionality enhancement. To retain obvious features, max pooling is performed on the point dimension to obtain the features of the current point. To obtain features at different levels, the neighborhood search process is performed twice, with different radii for each search. The different levels of features obtained from the two groupings are then input into the multilayer perceptron network for feature fusion. The N1 points and their features serve as the input for the second downsampling block, consistent with the previous block. The farthest point sampling algorithm is first applied to obtain the retained N2 points. After two neighborhood searches and grouping processes, feature aggregation is completed. The retained point features now possess rich semantic information. Therefore, this semantic information is used to supervise the downsampling process. Specifically, two multilayer perceptron layers are added to the encoding layer to further determine the semantic category of each point. The point cloud within the ground truth bounding box is labeled as foreground points, serving as the label information for supervising the training process. N2 points and their features are used as input to the third downsampling block. The N3 points with the highest confidence among the foreground points are retained, and feature aggregation is completed through two neighborhood search groupings. The N3 points and their features are used as input to the fourth downsampling block. The N4 points with the highest confidence among the foreground points are retained, and feature aggregation is completed through two neighborhood search groupings. A point-based feature extraction backbone network then performs feature extraction.

[0021] The center point generation module performs prediction by explicitly predicting the offset from the foreground point to the target's center. N4 points and features are used as input to this module, passing through two multilayer perceptron networks to obtain the offset output. The offset, plus the foreground point coordinates, yields the final predicted target center coordinates.

[0022] The center point neighborhood search radius generation module infers the center point category by predicting the category information of the foreground points. N4 points and features are used as input to this module, and the data is processed through two multilayer perceptron networks to obtain the category prediction information.

[0023] The input to the center point feature aggregation module based on the channel-level attention mechanism is the predicted coordinates of the center point, the category information of the center point, N3 points, and features. The first step is grouping. A spherical search is performed within the N3 points based on the coordinates of the N4 center points and the corresponding neighborhood search radius for each category to find neighboring nodes and obtain the grouping results. After grouping, for each group, the center point embedding vector feature is used as the Query vector, and the searched neighbor node features are used as the Key and Value vectors. First, the Query vector is transposed and multiplied by the Key vector to obtain an N×1 vector. Then, this is repeated along the second dimension to obtain an N×D vector, maintaining consistency with the Key dimension. A Hadamard product is then performed to obtain an N×D matrix, propagating spatial information to each channel to maintain channel differences. The N×D matrix is ​​then multiplied again by the Value vector to obtain the center point instance-level feature vector. To obtain features at different levels, the grouping process is performed twice, with different neighborhood search radii. The center point features obtained from the two groupings are then merged to obtain the final center point instance-level features.

[0024] The detection head based on the point-anchor-box mechanism comprises two branches: 3D predicted bounding box classification and residual regression. The point-anchor-box mechanism is essentially a compromise between anchor-box and no-anchor-box mechanisms. After obtaining the final center point instance-level features, anchor boxes are sparsely distributed at the center point's position (x, y, z) with the statistical mean of the target object size (l, w, h) for different categories, and eight orientations: 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees. Therefore, each center point has eight anchor boxes. The basis for determining positive and negative samples in the classification task is the intersection-union ratio (IUU) between the anchor boxes and the ground truth boxes.

[0025] The overall loss function includes the classification loss function of the 3D predicted bounding box, the regression loss function of the 3D predicted bounding box, the angle loss function, the foreground point classification loss function, the center point classification loss function, and the center point offset loss function. The calculation formula is as follows:

[0026] Loss = l cla +γ1l reg +l sem +γ2l center +l center-cla

[0027] Where Loss represents the total loss function, l cla The classification loss function for 3D predicted bounding boxes, l reg The regression loss function representing the 3D prediction bounding box, l sem Let l represent the foreground point classification loss function. center The offset loss function for the center point, l center-claLet γ1 and γ2 represent the classification loss function of the center point, respectively. reg l center The weights of the loss function.

[0028] In step (3), the classification confidence of the 3D prediction box is calculated based on the output of the neural network, and the parameters of the 3D prediction box are calculated based on the output of the 3D prediction box. The calculation formula is as follows:

[0029] x = x a +d a x p ,y=y a +d a y p ,z=z a +d a z p

[0030] l = l a l p ,w=w a w p h=h a h p ,θ=θ a +θ p

[0031]

[0032] Where x, y, z, l, w, h, θ represent the coordinates, size, and orientation of the final 3D prediction bounding box, respectively. a ,y a ,z a ,l a ,w a ,h a ,θ a These represent the coordinates, dimensions, and orientation of the 3D preset anchor frame, respectively. p ,y p ,z p ,l p ,w p ,h p ,θ p These represent the network output results.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] (1) This invention can effectively utilize the rich semantic information of foreground points in the neural network to predict the category information of foreground points. In the subsequent process of center point feature aggregation, different neighborhood search radii are set according to different category information. Compared with the unified setting scheme, it can provide a more suitable neighborhood search radius for small targets, filter out the interference of background points or other instance points, and improve the accuracy of small target object detection.

[0035] (2) The present invention designs a channel-level center point feature aggregation scheme, which fully exploits the 3D geometric features of point cloud by utilizing the channel-level attention mechanism.

[0036] (3) The present invention designs a detection head based on the point anchor frame mechanism, which can balance the number of positive and negative samples and solve the problem of orientation regression in the no-anchor frame mechanism.

[0037] (4) The present invention consists of a multilayer perceptron network and a single-layer attention network, without time-consuming operations such as 3D convolution, thus ensuring the real-time performance of the algorithm. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 This is a diagram of the overall neural network framework of the method of this invention;

[0040] Figure 3 It is a point-based feature extraction backbone network;

[0041] Figure 4 It is a center point feature aggregation module based on a channel-level attention mechanism;

[0042] Figure 5 It is the difference between point anchor frame mechanism, traditional anchor frame mechanism, and no anchor frame mechanism. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] like Figure 1 As shown in the flowchart, the embodiments and implementation process of the complete method of the present invention are as follows:

[0045] Using the KITTI 3D object detection dataset as a known dataset and 3D object detection as an example, this paper describes the methodological idea and specific implementation steps of 3D object detection guided by the center point neighborhood search radius.

[0046] The lidar point cloud data, labeled 3D ground truth boxes, and category information in this embodiment are all from the KITTI 3D object detection dataset. The labeled categories include Car, Pedestrian, and Cyclist.

[0047] Using the partitioning of the known KITTI 3D object detection dataset, a total of 7481 data points with ground truth annotations are provided, including 3712 training data points and 3769 validation data points. The training and validation sets are from different scene acquisition sequences, and there is no overlap between the data. Both the training and validation sets have category information annotations and 3D bounding box annotations for key targets. For the point cloud acquired by LiDAR in the training set, steps (1) to (2) are executed;

[0048] This invention includes the following steps:

[0049] (1) In order to parallelize network processing, the point clouds collected by lidar in different scenarios need to be collected to the same value N;

[0050] In step (1) of this example, the collected uniform values ​​are 16384 points.

[0051] (2) Establish a neural network, train the neural network using point cloud data and its ground truth box annotations in the KITTI 3D object detection dataset, set the total loss function to supervise the network structure, and obtain the parameters of the trained neural network.

[0052] In step (2), the neural network structure includes a point-based feature extraction backbone network, a center point generation module, a center point neighborhood search radius generation module, a center point feature aggregation module based on channel-level attention mechanism, and a detection head module based on point anchor box mechanism. The original 16384×4 features of the point cloud are input into the point-based feature extraction backbone network. The center point generation module and the center point neighborhood search radius generation module run in parallel, providing the center point 3D coordinates and center point category information for the subsequent center point feature aggregation module based on channel-level attention mechanism. The detection head based on point anchor box mechanism is connected to the center point feature aggregation module and serves as the output of the neural network.

[0053] The point-based feature extraction backbone network consists of four downsampling-feature aggregation blocks connected end to end. The downsampling strategy in the first two blocks adopts the farthest point sampling algorithm, while the sampling strategy in the last two blocks is guided by semantic information. The operation of the feature aggregation blocks remains consistent.

[0054] The input point cloud consists of 16384 points. First, it passes through a downsampling module. After the farthest point sampling algorithm, 4096 points are retained, and then feature aggregation is performed. A uniform neighborhood search radius is set for these 4096 points, covering the initial 16384 points. A spherical search is used, and the search results are grouped, resulting in each of the 4096 retained points having 8 neighbor nodes. For each group, feature aggregation is performed. First, the neighbor node features are input into a multilayer perceptron network for dimensionality enhancement. To retain prominent features, max pooling is performed on the point dimension to obtain the current point's features. To obtain features at different levels, the neighborhood search process is performed twice with different radii. The features obtained from the two neighbor search groupings are then input into the multilayer perceptron network for feature fusion. The 4096 points and their 64-dimensional features serve as the input for the second downsampling block, consistent with the first block. The farthest point sampling algorithm is first applied, resulting in 1024 retained points. After two neighborhood searches, feature aggregation is completed. At this point, the features already possess rich semantic information. Therefore, the semantic information is used to supervise the downsampling process. Specifically, two multilayer perceptron layers are added to the encoding layer to further determine the semantic category of each point. The point cloud within the ground truth bounding box is labeled as foreground points, serving as the label information for supervising the training process. The 1024 points and their features are used as input to the third downsampling block. The 512 points with the highest confidence among the foreground points are retained, and feature aggregation is completed through two neighborhood searches. These 512 points and their features are used as input to the fourth downsampling block. The 256 points with the highest confidence among the foreground points are retained, and feature aggregation is completed through two neighborhood searches. Specific hyperparameters are shown in Table 1.

[0055] Table 1. Specific parameter settings for the feature extraction backbone network.

[0056]

[0057] The center point generation module performs prediction by explicitly predicting the offset from the foreground point to the target's center. 256 points and 256-dimensional features are used as input to this module, passing through two multilayer perceptron networks to obtain the offset output. The offset, plus the foreground point coordinates, yields the final predicted target center coordinates.

[0058] The center point neighborhood search radius generation module infers the center point category by predicting the category information of the foreground points. 256 points and 256-dimensional features serve as input to this module, passing through two multilayer perceptron networks to obtain the category prediction information.

[0059] The input to the center point feature aggregation module based on the channel-level attention mechanism is the predicted coordinates of the center points and the category information of the center points. The search range is 512 points and 256-dimensional features. First, grouping is performed. Based on the coordinates of the 256 center points and the radius corresponding to the category, a spherical search is conducted within the range of 512 points to search for neighboring nodes, obtaining the grouping results. For example... Figure 4 As shown, after grouping, for each group, the 256-dimensional embedded vector feature of the center point is used as the Query vector, and the features of its searched neighbor nodes are used as the Key and Value vectors. First, the Query vector is transposed and multiplied by the Key vector to obtain a 256×1 vector. Then, it is repeated along the second dimension to obtain a 256×256 vector, maintaining the same dimension as the Key. A Hadamard product is then performed to obtain a 256×256 matrix, propagating spatial information to each channel to maintain channel differences. The 256×256 matrix is ​​then multiplied by the Value vector again to obtain the center point instance-level feature vector with a dimension of 256×256. To obtain features at different levels, the grouping process is performed twice, with a different neighborhood search radius each time. The specific hyperparameters are shown in Table 2. After each category is grouped twice, the features are merged, and the final center point instance-level feature size is 256×512.

[0060] Table 2 Hyperparameter settings for the center point feature aggregation module

[0061]

[0062] The detection head based on the anchor box mechanism comprises two branches: classification and residual regression. After obtaining the final center point instance-level features, anchor boxes for different target object categories are sparsely distributed at the center point's position (x, y, z), representing the statistical mean of the object size (l, w, h) and eight orientations: 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees. Therefore, each center point has eight anchor boxes. The basis for determining positive and negative samples in the classification task is the intersection-union ratio (IUGR) between the anchor boxes and the ground truth boxes. In the KITTI 3D object detection dataset, for the Car class, an IUGR greater than 0.6 is considered a positive sample, and less than 0.45 is considered a negative sample; for the Pedestrian and Cyclist classes, an IUGR greater than 0.5 is considered a positive sample, and less than 0.35 is considered a negative sample. The 256 center points predict the preset residuals and classification results of the anchor boxes based on their features.

[0063] In the KITTI 3D object detection dataset validation set, IA-SSD was used as the benchmark algorithm to test the accuracy improvement brought by different modules. The experimental results are shown in Table 3.

[0064] Table 3 shows the combined impact of each module on detection accuracy in the KITTI 3D object detection dataset.

[0065]

[0066]

[0067] After adding a search radius generation module based on center point category information, significant improvements were achieved in the Car, Pedestrian, and Cyclist classes, especially for small targets, where an accuracy improvement of over 1% was achieved. This is because after the network regresses the center point coordinates of the target, it no longer uses a uniform hyperparameter as the neighborhood search radius. This allows for a narrower neighborhood search range for small targets, thus focusing more on the geometric features of the target's point cloud itself, thereby improving detection accuracy.

[0068] By incorporating a feature aggregation module based on channel-level attention, different weights can be assigned to the channel-level features of neighboring nodes, rather than a simple max-pooling operation. This allows for a fuller exploration of the contributions of the center point's neighboring nodes to the center point and their relative geometric features. Furthermore, using only the embedding feature vector of each center point as the query vector does not impose an excessive memory burden on the network, thus ensuring its inference speed.

[0069] Adding a detection head based on point anchor boxes significantly improves the recall rate of the algorithm's detection results compared to the no-anchor-box mechanism. This is because for targets with severe occlusion in the dataset or those at a distance, the number of point clouds is very small, sometimes only around 10 points. The no-anchor-box mechanism only regresses a detection box at the center point, inevitably leading to false positives, as orientation regression is a difficult task for the network. Adding point anchor boxes allows for the prediction of more detection boxes at the center point, covering more possible orientations, and the number of boxes generated by point anchor boxes is far less than that of the traditional anchor box mechanism. Point anchor boxes only set anchor boxes at meaningful points, so they do not impact post-processing speed. In summary, the point anchor box mechanism combines the advantages of both methods, and experimental results demonstrate its effectiveness.

Claims

1. A 3D object detection method of category-guided center point neighborhood search radius, characterized in that, The method comprises the following steps: (1) the point-based representation form of the laser radar point cloud does not change the structure of the point cloud, and the original 3D geometric information of the point cloud is preserved to the maximum extent; in the data preprocessing stage, the point clouds collected in different scenes are all down-sampled to a fixed value N as the input of the neural network; (2) a neural network structure is established, the neural network structure is trained by using known target detection data sets, a total loss function is set to supervise the output of the neural network, and the parameter of the trained neural network structure is obtained; (3) the point cloud collected by the laser radar in the automatic driving scene is input into the trained neural network structure, the class information and the 3D prediction frame regression residual of the neural network are output, the 3D prediction frame is calculated according to the classification result and the regression residual of the current scene, and the final prediction result is obtained; in the step (2), the neural network structure comprises the following: a point-based feature extraction backbone network, a center point generation module, a center point neighborhood search radius generation module, a center point feature aggregation module based on a channel-level attention mechanism and a detection head module based on a point anchor frame mechanism; the original feature of the point cloud is input into the point-based feature extraction backbone network for feature extraction, the center point generation module and the center point neighborhood search radius generation module are parallel in the rear, the center point 3D coordinates and the center point category information are provided for the center point feature aggregation module based on the channel-level attention mechanism, the center point feature aggregation module based on the channel-level attention mechanism obtains the final center point instance-level feature vector, the detection head module based on the point anchor frame mechanism is connected with the center point feature aggregation module, the final center point instance-level feature vector is classified and predicted, and the output of the neural network is obtained; in the step (3), the 3D prediction frame classification confidence is calculated according to the output of the neural network, and the parameters of the 3D prediction frame are calculated according to the output of the 3D prediction frame, and the calculation formula is as follows: ； ； ； wherein, respectively represent the coordinates, size, orientation of the final 3D prediction box, respectively represent the coordinates, size, orientation of the 3D anchor box, respectively represent the output results of the network.

2. The 3D object detection method of claim 1, wherein, the point-based feature extraction backbone network comprises four down-sampling-feature aggregation blocks connected in a loop, the down-sampling strategy in the first two blocks adopts the farthest point sampling algorithm, the sampling strategy in the last two blocks utilizes semantic information guidance, and the feature aggregation block operation remains consistent.

3. The 3D object detection method of claim 2, wherein, The input point cloud is N points, which are first processed by a first downsampling module, and N1 points are retained after the farthest point sampling algorithm, and then feature aggregation is completed. Specifically, a uniform neighborhood search radius is first set for the N1 points, the search range is the N points of the initial point cloud, the search method is spherical search, and grouping is completed according to whether it is a neighbor node. Thus, the N1 retained points each have n1 neighbor nodes, and N1 groups are obtained. For each group, feature aggregation needs to be completed. First, the neighbor node features are input into a multi-layer perception network for dimension upgrading. In order to retain obvious features, a maximum pooling operation is performed in the point dimension to obtain the features of the current point. In order to obtain different levels of features, the neighborhood search process is performed twice, and the radii of the two neighborhood searches are different. The different levels of features obtained by the two groupings are input into a multi-layer perception network again to complete feature fusion. The N1 points and their features are input into a second downsampling block, which is the same as the previous block. First, the farthest point sampling algorithm needs to be performed to obtain N2 retained points, and feature aggregation is completed through the grouping process of two neighborhood searches. At this time, the retained point features already have rich semantic information, so the semantic information is used to supervise the downsampling process. Specifically, two multi-layer perception layers are added to the encoding layer to further determine the semantic category of each point. The point cloud in the ground truth box is labeled as a foreground point and used as label information of the point cloud to supervise the training process. The N2 points and their features are input into a third downsampling block, N3 points with the highest foreground point confidence are retained, and feature aggregation is completed through the grouping process of two neighborhood searches. The N3 points and their features are input into a fourth downsampling block, N4 points with the highest foreground point confidence are retained, and feature aggregation is completed through the grouping process of two neighborhood searches. Feature extraction is completed.

4. The 3D object detection method of claim 3, wherein, The center point generation module completes prediction by explicitly predicting the offset of the foreground point to the target positive center. The N4 points and their features are input into this module, and the output of the offset is obtained through two multi-layer perception networks. The offset is added to the foreground point coordinates to obtain the predicted value of the final target center coordinates.

5. The 3D object detection method of claim 3, wherein, The center point neighborhood search radius generation module completes the inference of the center point category by predicting the category information of the foreground point. The N4 points and their features are input into this module, and the category information prediction information is obtained through two multi-layer perception networks.

6. The 3D object detection method of claim 1, wherein, The input of the center point feature aggregation module based on the channel-level attention mechanism is the predicted value of the center point coordinate, the category information of the center point, N3 points and the features thereof. First, the spherical search is performed in the range of N3 points according to the coordinates of N4 center points and the corresponding neighborhood search radius of the category, the neighbor node search is completed, the grouping result is obtained, and after the grouping, for each group, the center point embedding vector feature is taken as a Query vector, the searched neighbor node feature is taken as a Key vector and a Value vector, first, the Query vector is transposed, the matrix product is performed on the transposed Query vector and the Key vector to obtain an N x 1 vector, then the N x 1 vector is repeated along the second dimension to an N x D vector, the dimension of the Key vector is kept consistent, the Hadamard product is performed to obtain an N x D matrix, the spatial information is propagated to each channel to keep the channel difference, the Hadamard product is performed again on the N x D matrix and the Value vector to obtain a center point instance-level feature vector; in order to obtain features at different levels, the grouping process is performed twice, the neighborhood search radius is different in the two times, and the center point features obtained in the two times are combined to obtain the final center point instance-level feature.

7. The class-guided center-point neighborhood search radius 3D object detection method of claim 1, wherein, The detection head module based on the point anchor box mechanism includes two branches of 3D prediction box classification and residual regression, after the final center point instance-level feature is obtained, the mean value (l, w, h) of the size statistics of different categories of target objects is sparsely laid at the position (x, y, z) of the center point, and the anchor boxes of eight orientations of 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees and 315 degrees are laid, therefore, each center point has eight anchor boxes, and the basis for judging the positive and negative samples in the classification task is the intersection over union between the anchor boxes and the true value boxes.

8. The 3D object detection method of claim 1, wherein, The total loss function includes a 3D prediction box classification loss function, a 3D prediction box regression loss function, an angle loss function, a foreground point classification loss function, a center point classification loss function and a center point offset loss function, and the calculation formula is as follows: ； wherein, represents the total loss function, represents the classification loss function of the 3D prediction box, represents the regression loss function of the 3D prediction box, represents the foreground point classification loss function, represents the offset loss function of the center point, represents the classification loss function of the center point; , are respectively weights of the loss functions.

Images