Keypoint-based 3D object detection and its parameterized radius learning method and system

Abstract

This invention discloses a keypoint-based 3D object detection method and system, along with its parameterized radius learning method. The method specifically involves: downsampling keypoints from a point cloud, voxelizing the original point cloud, and then inputting it into a 3D sparse convolutional network to obtain voxel features; using a point feature extraction method, aggregating the voxel features onto the keypoints to obtain keypoint features; learning the offset from the keypoints to the target center through target-aware feature pooling, and aggregating features within the target-aware neighborhood to obtain candidate points and their corresponding features; performing candidate box regression centered on the candidate points, extracting features, and performing secondary regression and score prediction on the candidate boxes to obtain the 3D object detection result; using the point cloud feature aggregation sampling radius as a learnable parameter, updating the sampling radius through backpropagation to obtain the optimal sampling radius parameter; and addressing the problems of high difficulty and inaccuracy in target center regression, as well as the need for manual adjustment of the feature aggregation sampling radius.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a method and system for 3D target detection based on key points and its parameterized radius learning. Background Technology

[0002] 3D object detection, based on input information such as images or point clouds, outputs the 3D position, size, yaw angle, and category of objects in a scene. Cameras and lidar are two commonly used sensors for this task. Camera-acquired images reflect the texture information of the scene, and during detection, the depth of the target needs to be predicted. In contrast, lidar, as an electro-optical ranging sensor, directly converts measurement data into point cloud data reflecting spatial geometry information through the laser emission angle and time-of-flight. With the widespread adoption of lidar, and the precise measurement of spatial information and insensitivity to changes in light in point clouds, 3D object detection based on pure point cloud data has become the mainstream research direction.

[0003] 3D object detection algorithms can be categorized into two types based on point cloud feature extraction methods: voxelization methods and point-based methods. Voxelization methods convert the point cloud into relatively dense voxels, resulting in high computational efficiency. However, their candidate box generation network, similar to that used in 2D detection, can negatively impact object center estimation, particularly in the height dimension. Specifically, voxelization methods use top-view features to generate anchor boxes, which are set to a constant height. During center regression, unlike residual learning along the X / Y axes, the detector needs to directly predict the target center's height above the ground. The target's distribution along the Z-axis exhibits significant variance, increasing the difficulty of height estimation.

[0004] In contrast, point-based methods use the original point cloud as input and then extract point-level features through a series of stacked sampling, grouping, and point set feature extraction networks. These point-level features are then used as input to a candidate box generation network to generate candidate boxes centered on the original points. However, while existing point-based methods achieve a more flexible receptive field, the stacked point set feature extraction modules are computationally intensive and time-consuming. Furthermore, the original points are distributed on the surface of the target, which also leads to a mismatch between the point cloud and the target center. Summary of the Invention

[0005] To address the challenges of high difficulty and inaccuracy in target center regression in existing technologies, as well as the need for manual adjustment of feature aggregation sampling radius during training, this invention provides a key-point-based 3D target detection method and its parameterized radius learning method.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a 3D target detection method based on key points and its parameterized radius learning method, comprising:

[0007] Using the original point cloud as input, key points are obtained through downsampling, and the original point cloud is voxelized to obtain initial voxel features.

[0008] The initial voxel features are input into a 3D sparse convolutional network to obtain voxel features. The 3D sparse convolutional network for extracting voxel features consists of four sparse convolutional modules. Each sparse convolutional module takes a non-empty voxel and its corresponding voxel features and index as input to construct a three-dimensional sparse tensor and uses a 3D sparse convolutional layer to extract features. At the same time, the sparse voxel feature map is downsampled by a factor of two. The sparse voxel features are concatenated in the height dimension and transformed into dense top-view features, i.e., voxel features.

[0009] A construct point feature extraction method is adopted to obtain key point features from sampling key points in the original point cloud. Specifically, construct points are generated at positions offset by a fixed distance in the positive and negative directions of the three-dimensional coordinates, centered on the key points. Using the voxel center as the coordinate, the voxel features of the construct points and the neighborhood of the key points are aggregated using a point set feature extraction method to obtain fusion features with orientation information enhancement. Combining the features of multiple sparse convolutional modules and key points in the original point cloud and dense top view, multi-level key point features are obtained.

[0010] The sampling radius of point cloud features is aggregated and used as a learnable parameter. The sampling radius is updated through backpropagation to obtain the optimal sampling radius parameter.

[0011] By using target-aware feature pooling, the offset of key points to the target center is learned, and feature aggregation is performed in the target-aware neighborhood to obtain candidate points and corresponding features. The key points corresponding to the candidate points are used for classification, and candidate box regression is performed with the candidate points as the center.

[0012] Based on candidate bounding boxes and multi-level key point features, the grid center point features are extracted through region of interest grid pooling, and the candidate bounding boxes are subjected to secondary regression and score prediction to obtain the final 3D object detection results.

[0013] The steps for downsampling key points from the original point cloud data are as follows: randomly select an initial point, and use the farthest point sampling method with Euclidean distance as the metric to uniformly sample from the original point cloud to obtain a sequence of key points.

[0014] The process of voxelizing point clouds and extracting initial voxel features is as follows: the measurement space is divided into three-dimensional voxels of equal size, and the point cloud is projected onto the corresponding voxels. A voxel index is constructed so that point clouds projected onto the same voxel have the same voxel index. A non-repeating voxel index is constructed, and the point clouds in non-empty voxels are dynamically encoded to obtain the initial features of non-empty voxels. The initial features include the three-dimensional coordinates of the point cloud within the voxel and the average reflection intensity.

[0015] The process of obtaining candidate points and corresponding features through target-aware feature pooling includes: learning the three-dimensional offset from the key point to the corresponding target center, adding the offset to the key point coordinates to obtain candidate points; forming a bidirectional mapping between the key points and candidate points, constructing a target-aware neighborhood in the candidate point space through the spherical neighborhood of the candidate points, and performing target-aware feature aggregation based on this neighborhood to obtain the corresponding features of the candidate points.

[0016] The process of predicting target candidate boxes using key points, candidate points and their corresponding features includes: constructing anchor boxes centered on the coordinates of candidate points; classifying candidate boxes by determining whether the key points are within the ground truth of the target box; and regressing the center position, size and orientation of candidate boxes using the features of candidate points based on the anchor boxes.

[0017] The process of obtaining the final 3D object detection result by grid pooling of the target region of interest includes: dividing the 3D candidate box into equally spaced grids, extracting point set features based on the grid center point, obtaining key point features in the neighborhood of the center point and aggregating them; and performing secondary regression of the target box and classification score prediction based on the grid center point features.

[0018] The process of learning the parameterized feature aggregation radius includes: simulating the point cloud sampling process using a truncation function, and introducing the point cloud feature aggregation sampling radius as a learnable parameter into the detection model; updating the sampling radius using the gradient of the objective function through backpropagation, and simultaneously learning the sampling radius during the detection model training process; constructing a truncated Smooth L1 loss function to constrain the range of the learned sampling radius; the expression for the truncated Smooth L1 loss function is:

[0019]

[0020] Where ε and ρ represent the center of symmetry and the cutoff range, the cutoff Smooth L1 loss function is symmetric to ε and has a loss of 0 in the interval [ε-ρ, ε+ρ].

[0021] This invention also provides a 3D target detection system based on key points and its parameterized radius learning system, including: an initial feature extraction module, a voxel feature extraction module, a key point feature extraction module, a candidate box generation module, a sampling radius parameter optimization module, and a two-stage target box regression module;

[0022] The initial feature extraction module is used to obtain key points by downsampling the original point cloud as input, and to perform voxelization on the original point cloud to obtain initial voxel features.

[0023] The voxel feature extraction module is used to input the initial voxel features into the 3D sparse convolutional network to obtain voxel features. The 3D sparse convolutional network for extracting voxel features consists of four sparse convolutional modules. Each sparse convolutional module takes a non-empty voxel and its corresponding voxel features and index as input to construct a three-dimensional sparse tensor and uses a 3D sparse convolutional layer to extract features. At the same time, the sparse voxel feature map is downsampled by a factor of two. The sparse voxel features are concatenated in the height dimension and transformed into dense top-view features, i.e., voxel features.

[0024] The keypoint feature extraction module employs a constructed point feature extraction method, sampling keypoints from the original point cloud to obtain keypoint features. Specifically, constructed points are generated at positions offset by a fixed distance in the positive and negative directions of the keypoint's 3D coordinates. Using the voxel center as coordinates, a point set feature extraction method is used to aggregate voxel features from the constructed points and the neighborhood of the keypoints to obtain fusion features with enhanced orientation information. Combining features from multiple sparse convolutional modules, the original point cloud, and the dense top-view keypoints, multi-level keypoint features are obtained. The sampling radius parameter optimization module aggregates the point cloud features and uses the sampling radius as a learnable parameter. The sampling radius is updated through backpropagation to obtain the optimal sampling radius parameter.

[0025] The candidate box generation module learns the offset of key points to the center of the target through target-aware feature pooling, and performs feature aggregation in the target-aware neighborhood to obtain candidate points and corresponding features; it then uses the key points corresponding to the candidate points for classification and performs candidate box regression with the candidate points as the center.

[0026] The two-stage target bounding box regression module, based on candidate bounding boxes and multi-level key point features, extracts grid center point features through region of interest grid pooling, and performs secondary regression and score prediction on candidate bounding boxes to obtain the final 3D target detection results.

[0027] Another computer device is provided, including a processor and a memory, wherein an executable program is stored in the memory, and when the processor executes the executable program, it can perform the key point-based 3D target detection and parameterized radius learning method of the present invention.

[0028] A computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the key point-based 3D target detection and parameterized radius learning method described in this invention.

[0029] Compared with existing technologies, this invention has at least the following advantages: Addressing the problem of inaccurate center regression in current 3D object detection, this invention proposes a two-stage keypoint-based 3D object detection framework, which simplifies the center estimation process and generates high-quality candidate boxes. To address the ambiguity of structural and orientation information during voxel feature to keypoint feature mapping, a construct point extraction method is proposed. By generating construct points around keypoints, features from different directions and local regions of the keypoints are aggregated, thereby enhancing the geometric information of local representations. To address the issue of deviation between keypoint and target center positions, a target-aware feature pooling method is proposed. This method learns the offset of keypoints from the target center, generating a target-aware neighborhood for extracting target-oriented features. To address the problem of needing multiple training and parameter tuning processes due to manual sampling radii during point cloud feature aggregation, a parameterized radius learning strategy is proposed. By simultaneously training the model and adjusting the sampling radius during training, the model training process is simplified, further improving detection performance. Attached Figure Description

[0030] Figure 1 This is a block diagram of a 3D target detection method based on key points provided in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the construction point feature extraction method provided in an embodiment of the present invention.

[0032] Figure 3 This is a schematic diagram of the target perception neighborhood provided by the target perception feature extraction module in an embodiment of the present invention.

[0033] Figure 4 This is a schematic diagram of the target perception feature extraction module and candidate box generator provided in an embodiment of the present invention.

[0034] Figure 5 This is a schematic diagram of the truncated Smooth L1 loss function in the parameterized radius learning strategy provided in this embodiment of the invention.

[0035] Figure 6 The 3D object detection method provided in this embodiment of the invention is shown to perform on the Waymo Open dataset. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0038] This invention provides a keypoint-based 3D object detection method and its parameterized radius learning method, simplifying the difficulty of object center regression and adaptively adjusting the sampling radius during training through a parameterized radius learning strategy. Figure 1 As shown, the original point cloud is first processed using a voxel-point backbone network: the point cloud is voxelized and feature encoding is performed using a 3D sparse convolutional network to obtain voxel features; simultaneously, keypoints are extracted using farthest point sampling, and anisotropic voxel features are aggregated onto the keypoints by constructing a point feature extraction method. Then, candidate points and corresponding features are generated using a target-aware feature pooling method. Anchor boxes are set with the candidate points as the center, and classification is performed using keypoint features. Candidate box regression is then performed based on the candidate points. Finally, target instance features are extracted using region-of-interest grid pooling, and target box secondary regression and score prediction are performed. Furthermore, during training, a parameterized radius learning strategy is used, simultaneously training the model and adjusting the sampling radius; the specific implementation steps are as follows:

[0039] Step 1: Construct a voxel-point integrated backbone network, extract voxel features, and use a point feature extraction method to aggregate multi-source voxel features onto key points. For example... Figure 1 As shown, the backbone network of the present invention includes two branches: a voxelization branch and a point-based branch.

[0040] In the voxelization branch, given a point cloud Where N is the number of point clouds, this invention divides the measurement space into voxels of equal size with a resolution of L×W×H, and projects the point clouds onto the voxels; dynamic voxel encoding is used to obtain initial voxel features. First, the point clouds are quantized by voxel size to obtain voxel indices, denoted as . Then, by removing duplicate elements from the voxel index, the non-duplicate index is obtained. Point cloud-voxel mapping Where M represents the number of non-empty graticules; subsequently, a sparse average pooling layer is used to obtain the initial features of the non-empty voxels:

[0041] F v =mean(P,V) idx )

[0042] in Features of non-empty initial voxels.

[0043] This invention uses a 3D sparse convolutional network with four sparse convolutional modules for voxel feature extraction, where each sparse convolutional module is a non-empty voxel. and their corresponding voxel characteristics As input, a 3D sparse tensor is constructed, and feature extraction is performed using 3D sparse convolutional layers. Simultaneously, the sparse voxel feature maps are downsampled by a factor of two. Finally, the sparse voxel features are concatenated along the height dimension and transformed into dense top-view features. Furthermore, a farthest-point sampling method is used to extract n uniformly distributed keypoints from the original point cloud.

[0044] Furthermore, to enhance the encoding of geometric and directional information in the local neighborhood of keypoints, anisotropic keypoint features are obtained using a constructed point feature extraction method, such as... Figure 2 As shown; for the l-th sparse convolutional module, 6 construction points are generated centered on the keypoints. The construction point can be represented as:

[0045]

[0046]

[0047] Among them, O l This represents the offset of the keypoint from the construction point in the l-th sparse convolutional module.

[0048] Subsequently, this invention uses non-empty voxel centers Coordinates representing voxels, combined with voxel features The Set Abstraction method is used to extract voxel features from the neighborhood of a constructed point, where M... l This represents the number of non-empty voxels in the l-th sparse convolutional module, and the process is expressed as:

[0049]

[0050]

[0051] in, S represents the neighborhood point set of the constructed point, and S(·) represents the selection function, which selects a fixed number of points from the neighborhood point set for subsequent feature extraction and pooling operations.

[0052] Similarly, feature extraction is performed centered on key points to obtain the neighborhood features of the key points. Subsequently, the neighborhood features of key points and the features of construction points are fused using a multilayer perceptron, as shown below:

[0053]

[0054] in, This represents the keypoint fusion feature of the l-th sparse convolution module. Specifically, in this embodiment of the invention, the last two modules of the 3D sparse convolution are used to generate the fusion feature, denoted as...

[0055] In addition, keypoint features are extracted from the original point cloud and the top-view feature map using point set feature extraction and bilinear interpolation, resulting in keypoint features corresponding to the original point cloud and the top-view feature map, denoted as f. i raw and f i bev Finally, keypoint features from different sources are concatenated and fused using a linear layer, as shown below:

[0056]

[0057] Key feature f i k As input, it is used for candidate box generation.

[0058] Step 2: Generate anchor boxes and candidate boxes using key points and their features.

[0059] Since key points are distributed on the surface of the target, far from the target center, this invention proposes a target perception feature aggregation method that uses a multilayer perceptron to predict the Euclidean distance offset from key points to the target center. Add the keypoint coordinates to the offset to obtain the point closer to the corresponding target center, and record it as the candidate point. Key points and candidate points form a bidirectional mapping, represented as:

[0060]

[0061]

[0062] Furthermore, since candidate points are concentrated in the neighborhood of the corresponding target center, this invention uses the candidate points as the center and employs a local feature aggregation module to extract features pointing to the target. The feature sampling region of the local feature aggregation module is defined as follows:

[0063]

[0064] Where r represents the sampling radius of the candidate point in Euclidean space.

[0065] In particular, such as Figure 3As shown, since candidate points and key points form a bidirectional mapping, the sampling neighborhood in the candidate point space is mapped to the target perception neighborhood in the key point space, thereby better extracting target-related key point features. In this invention example, PosPool is used as the feature aggregation method to extract key point features in the target perception neighborhood, which are the features corresponding to the candidate points, denoted as...

[0066] like Figure 4 As shown, candidate points and their corresponding features As input, candidate boxes are regressed; anchor boxes are set with the candidate points as centers, and the 3D dimensions of the anchor boxes are set to the mean of the dataset for each detection class; then, a regression network is used to predict the 3D center and size offset of the anchor boxes from the corresponding ground truth values ​​of the targets, and the sine and cosine values ​​of the orientation are regressed to decode the candidate boxes. Keypoints are used as input, classified using a classification network, and candidate boxes corresponding to foreground keypoints are retained.

[0067] Step 3: Based on the candidate bounding boxes and keypoint features, perform secondary regression and score prediction of the target bounding boxes using region-of-interest (ROI) grid pooling. Specifically, in this embodiment of the invention, each candidate bounding box is divided into a 6×6×6 grid, with the grid center as the grid point. The grid point features are extracted using a point set feature aggregation method. The grid point features are concatenated and fused using a linear layer. Based on the fused features, a regression and classification network is used to obtain the final 3D object detection result.

[0068] Step 4: Using a parameterized radius learning strategy, the feature sampling radius extracted from the point set features in the backbone network is treated as a learnable parameter and jointly trained with the algorithm using a deep convolutional network to obtain the optimal sampling radius. Specifically, in this invention example, the input point cloud and corresponding features of the point set feature extraction method are denoted as... To learn the sampling radius r using backpropagation, the sampling process is approximated using a truncation function, expressed as:

[0069]

[0070] Where q represents the sampling center point, τ reflects the sharpness of the cutoff function, and d(i,r) represents the sampling probability when the sampling point p i Within the sampling region, the sampling probability tends to 1; when sampling point p i Outside the sampling region, the sampling probability tends to 0. Multiply the sampling probability by the corresponding feature f. i ·d(i,r) serves as a new point cloud feature.

[0071] To constrain the sampling radius, a truncated Smooth L1 loss function is proposed, expressed as:

[0072]

[0073] Where ε and ρ represent the center of symmetry and the cutoff range, the cutoff Smooth L1 loss function is symmetric to ε and has a loss of 0 in the interval [ε-ρ, ε+ρ]. For example... Figure 5 As shown, in this embodiment of the invention, ε and ρ are both set to 5m, that is, the reasonable range of the sampling radius is set to [0, 10m].

[0074] The sampling radius is updated using the momentum method, expressed as:

[0075]

[0076] Where ε is the objective function, μ represents momentum, and ∈ represents the learning rate.

[0077] A 3D target detection system includes: a voxel feature encoding module, which takes the original point cloud as input, obtains key points through downsampling, and performs voxelization processing on the original point cloud to obtain initial voxel features;

[0078] The initial voxel features are input into a 3D sparse convolutional network to obtain voxel features. The 3D sparse convolutional network for extracting voxel features consists of four sparse convolutional modules. Each sparse convolutional module takes a non-empty voxel and its corresponding voxel features and index as input to construct a three-dimensional sparse tensor and uses a 3D sparse convolutional layer to extract features. At the same time, the sparse voxel feature map is downsampled by a factor of two. The sparse voxel features are concatenated in the height dimension and transformed into dense top-view features, i.e., voxel features.

[0079] The voxel feature to keypoint feature mapping module employs a construction point feature extraction method to sample keypoints from the original point cloud and obtain keypoint features. Specifically, construction points are generated at positions offset by a fixed distance in the positive and negative directions of the keypoint's 3D coordinates. Using the voxel center as the coordinate, a point set feature extraction method is used to aggregate the voxel features of the construction points and the neighborhood of the keypoints to obtain fusion features with orientation information enhancement. Combining the features of multiple sparse convolution modules and the keypoints in the original point cloud and dense top view, multi-level keypoint features are obtained.

[0080] The candidate box generation and target box regression module includes a first-stage candidate box generation unit (target-aware feature pooling method and candidate box regression, classification) and a second-stage target box regression unit (target of interest pooling, target box quadratic regression, score prediction).

[0081] The sampling radius of point cloud features is aggregated and used as a learnable parameter. The sampling radius is updated through backpropagation to obtain the optimal sampling radius parameter. The offset of key points to the target center is learned through target-aware feature pooling, and feature aggregation is performed in the target-aware neighborhood to obtain candidate points and corresponding features. The key points corresponding to the candidate points are used for classification, and candidate box regression is performed with the candidate points as the center. Through region of interest grid pooling, the features of the grid center point are extracted, and secondary regression and score prediction are performed on the candidate boxes to obtain the final 3D target detection result.

[0082] Experiments and Analysis: The method of this invention was evaluated on the Waymo Open 3D object detection dataset, which has the largest data scale, and compared with state-of-the-art methods. Ablation experiments were also conducted to analyze the effectiveness of each module.

[0083] The Waymo Open Dataset is the largest autonomous driving dataset, collecting data from multiple locations under various weather conditions. The dataset consists of 1000 labeled sequences, with 798 sequences (approximately 158,000 point cloud samples) split into a training set and the remaining 202 sequences (approximately 40,000 point cloud samples) forming a validation set. The point cloud data is truncated to within 75 meters. In addition to coordinates and reflection intensity, the point cloud data also includes elongation. The evaluation metrics for this dataset are mean accuracy (mAP) and orientation-weighted mean accuracy (mAPH), with IoU thresholds for vehicles and pedestrians set to 0.7 and 0.5, respectively. The dataset is divided into two difficulty levels: LEVEL_1 for targets with more than 5 points and LEVEL_2 for targets containing at least 1 point. The dataset can also be divided based on the distance between the target and the sensor coordinate system: 0-30m, 30-50m, and greater than 50m.

[0084] This invention presents key results and comparisons with state-of-the-art algorithms on the Waymo Open dataset. As shown in Table 1, the proposed KPDet achieves state-of-the-art results across all categories. For the vehicle category, the proposed method outperforms AFDetV2 at LEVEL_1 difficulty with 1.16% mAP and 1.17% mAPH, and shows a slight improvement in mAP / mAPH at LEVEL_2 difficulty. For the pedestrian category, although similar performance is achieved at different difficulty levels, the proposed method surpasses SST at LEVEL_1 and LEVEL_2 difficulties with 1.71% and 1.31% mAPH, respectively. Experiments demonstrate that the proposed method outperforms existing methods in direction prediction and detection of smaller targets such as pedestrians.

[0085] Furthermore, Tables 2 and 3 present a comparison of detection results at different distances with a LEVEL_1 difficulty level. While the present invention achieves slightly lower BEV metrics for the vehicle category compared to existing methods, it demonstrates significant progress in 3D metrics, proving its greater accuracy in target height regression. Further, the significant improvements across all metrics for the pedestrian category indicate the superiority of the present invention in center estimation. The present invention is also compared on the Waymo Open dataset test set. With single-frame point clouds as input and no test augmentations, as shown in Table 4, the present invention achieves the best performance across all metrics. Figure 6 The diagram shows the detection results of this invention on the Waymo Open dataset.

[0086] Table 1: Comparison of Validation Set Results for Waymo Open Dataset under Different Difficulty Levels

[0087]

[0088] Table 2: Comparison of vehicle detection results on the Waymo Open dataset validation set under different distance partitions

[0089]

[0090] Table 3: Comparison of pedestrian detection results on the Waymo Open dataset validation set under different distance divisions

[0091]

[0092] Table 4: Comparison of detection results on the Waymo Open dataset test set under different difficulty levels

[0093]

[0094] To verify the effectiveness of the method described in this invention, extensive experiments were conducted on the Waymo Open dataset. In terms of experimental setup, 20% of the data frames were uniformly sampled from the training set, and the results were evaluated on the complete validation set. As shown in Table 5, a keypoint-based baseline was first constructed. The baseline maintains a 3D sparse convolutional network and keypoint sampling as its backbone, where keypoint features are generated by Voxel Set Abstraction. Unlike existing voxelization methods, the baseline uses keypoints to generate candidate boxes. Subsequently, the effectiveness was independently evaluated by sequentially stacking improved modules on the baseline. As shown in rows 1 and 2 of Table 5, using the Object-Aware Feature Pooling (OFP) method, an improvement of approximately 0.2% in mAP / mAPH metrics was achieved at LEVEL_1 and LEVEL_2 difficulties. Furthermore, this invention retains Object-Aware Feature Pooling and explores the experimental effects of constructing Point Feature Extraction (SPA) and Parametric Radius Learning (PRL) strategies. Specifically, in this embodiment of the invention, when the parameterized radius learning strategy is not used, the radii of the keypoints and construct points are set to [(1.2m, 2.4m), (2.4m, 4.8m)] and [1.2m, 2.4m]. Rows 2-5 of Table 5 show that the construct point feature extraction and parameterized radius learning strategy can improve the mAP / mAPH performance at LEVEL_1 and LEVEL_2 difficulties by approximately 0.5%, 0.2%, and 1.5%, 0.5%, respectively.

[0095] Table 6 shows the impact of three different radius learning strategies on detection performance in this invention:

[0096] 1. "SP": Only the sampling radius of the construction points is set as a learnable parameter, while the radius of the key points is fixed.

[0097] 2. "KP": Only the sampling radius of the key points is set as a learnable parameter, while the construction points are fixed.

[0098] 3. "SP&KP": The radii of both the construction point and the key point are learnable.

[0099] Table 6 shows that optimal performance can be achieved by employing a parameterized radius learning strategy in both the construction points and keypoints. Experiments demonstrate that the proposed parameterized radius learning strategy can serve as a fundamental element for point set feature extraction and can be applied to different point-based feature aggregation processes. To demonstrate the robustness of the parameterized radius learning strategy, Table 7 explores the impact of initial values ​​on the sampling radius. The present invention achieves similar performance across all three initial value settings, proving that the parameterized radius learning strategy is insensitive to initial values. To avoid the introduction of prior information, the examples in this invention use the initial value settings in the first row of Table 7.

[0100] Table 5. Validation of the effectiveness of each module of the method described in the invention.

[0101]

[0102] Table 6 Comparison of different learning strategies

[0103]

[0104] Table 7 Comparison of detections with different initial sampling radii

[0105]

[0106] This invention also provides a 3D target detection system based on key points and its parameterized radius learning system, including: an initial feature extraction module, a voxel feature extraction module, a key point feature extraction module, a candidate box generation module, a sampling radius parameter optimization module, and a two-stage target box regression module;

[0107] The initial feature extraction module is used to obtain key points by downsampling the original point cloud as input, and to perform voxelization on the original point cloud to obtain initial voxel features.

[0108] The voxel feature extraction module is used to input the initial voxel features into the 3D sparse convolutional network to obtain voxel features. The 3D sparse convolutional network for extracting voxel features consists of four sparse convolutional modules. Each sparse convolutional module takes a non-empty voxel and its corresponding voxel features and index as input to construct a three-dimensional sparse tensor and uses a 3D sparse convolutional layer to extract features. At the same time, the sparse voxel feature map is downsampled by a factor of two. The sparse voxel features are concatenated in the height dimension and transformed into dense top-view features, i.e., voxel features.

[0109] The keypoint feature extraction module employs a constructed point feature extraction method, sampling keypoints from the original point cloud to obtain keypoint features. Specifically, constructed points are generated at positions offset by a fixed distance in the positive and negative directions of the keypoint's 3D coordinates. Using the voxel center as coordinates, a point set feature extraction method is used to aggregate voxel features from the constructed points and the neighborhood of the keypoints to obtain fusion features with enhanced orientation information. Combining features from multiple sparse convolutional modules, the original point cloud, and the dense top-view keypoints, multi-level keypoint features are obtained. The sampling radius parameter optimization module aggregates the point cloud features and uses the sampling radius as a learnable parameter. The sampling radius is updated through backpropagation to obtain the optimal sampling radius parameter.

[0110] The candidate box generation module learns the offset of key points to the center of the target through target-aware feature pooling, and performs feature aggregation in the target-aware neighborhood to obtain candidate points and corresponding features; it then uses the key points corresponding to the candidate points for classification and performs candidate box regression with the candidate points as the center.

[0111] The two-stage target bounding box regression module, based on candidate bounding boxes and multi-level key point features, extracts grid center point features through region of interest grid pooling, and performs secondary regression and score prediction on candidate bounding boxes to obtain the final 3D target detection results.

[0112] In addition, the present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and the processor can realize the key point-based 3D target detection and parameterized radius learning method of the present invention when executing part or all of the executable program.

[0113] The computer device may be a laptop, a desktop computer, or a workstation.

[0114] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).

[0115] A computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the key point-based 3D target detection and parameterized radius learning method described in this invention.

[0116] The memory described in this invention can be an internal storage unit of a laptop, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.

[0117] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0118] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A keypoint-based 3D object detection and its parameterized radius learning method, characterized in that, The method comprises the following steps: Taking the original point cloud as input, key points are obtained by downsampling, and the original point cloud is voxelized to obtain initial voxel features; The initial voxel features are input into a 3D sparse convolution network to obtain voxel features; specifically, the 3D sparse convolution network for extracting voxel features is composed of four sparse convolution modules, each of which takes non-empty voxels and their corresponding voxel features and indexes as input, constructs a three-dimensional sparse tensor, and uses a 3D sparse convolution layer to extract features while performing two times of downsampling on the sparse voxel feature map; after the last sparse convolution module, the sparse voxel features are cascaded in the height dimension and converted into a dense overhead view feature, i.e., voxel features; A structured point feature extraction method is used to obtain key point features from the key points sampled from the original point cloud; specifically, structured points are generated at positions offset by a fixed distance in the positive and negative directions of the three-dimensional coordinates of the key points; combined with the voxel features, the voxel features in the neighborhood of the structured points and the key points are aggregated using a point set feature extraction method to obtain fusion features enhanced by orientation information; The features of the key points at multiple levels are obtained by combining the features of the key points at multiple levels, the original point cloud, and the dense overhead view. The point cloud feature aggregation sampling radius is used as a learnable parameter, and the sampling radius is updated through backpropagation to obtain the optimal sampling radius parameter. Through target perception feature pooling, the offset of the key point to the target center is learned, and feature aggregation is performed in the target perception neighborhood to obtain candidate points and corresponding features; the key point corresponding to the candidate point is used for classification, and the candidate box is regressed with the candidate point as the center; based on the candidate box and the multi-level key point features, the region of interest grid pooling is used to extract the grid center point features and perform secondary regression and score prediction on the candidate box to obtain the final 3D target detection result. 2.The keypoint-based 3D object detection and its parameterized radius learning method according to claim 1, wherein, The processing steps of downsampling the original point cloud data to obtain key points are as follows: an initial point is randomly selected, and the farthest point sampling method is used to uniformly sample the original point cloud in the Euclidean distance as a measure to obtain a sequence of key points. 3.The keypoint-based 3D object detection and its parameterized radius learning method of claim 1, wherein, The process of voxelizing the point cloud and extracting initial voxel features is as follows: the measurement space is divided into three-dimensional voxels of equal size, and the point cloud is projected into the corresponding voxels to construct voxel indexes, so that the point cloud projected into the same voxel has the same voxel index; the non-repeated voxel index is constructed, and the point cloud in the non-empty voxel is dynamically encoded to obtain the initial features of the non-empty voxel, which includes the three-dimensional coordinates and average reflectivity of the point cloud in the voxel. 4.The keypoint-based 3D object detection and its parameterized radius learning method of claim 1, wherein, The process of obtaining candidate points and corresponding features through target perception feature pooling includes: learning the three-dimensional offset of the key point to the corresponding target center, adding the offset to the key point coordinates to obtain the candidate point; the bidirectional mapping is formed by the key point and the candidate point, the target perception neighborhood under the candidate point space is constructed through the spherical neighborhood of the candidate point, and the target perception feature aggregation is performed based on the neighborhood, so as to obtain the corresponding features of the candidate point. 5.The keypoint-based 3D object detection and its parameterized radius learning method of claim 1, wherein, The process of target candidate box prediction based on key points, candidate points and corresponding features includes: constructing an anchor box with the candidate point coordinates as the center; performing candidate box classification by judging whether the key points are within the target box ground truth; and performing candidate box center position, size and orientation regression based on the anchor box and using candidate point features. 6.The keypoint-based 3D object detection and its parameterized radius learning method of claim 1, wherein, The process of obtaining the final 3D target detection result through target region of interest grid pooling includes: dividing the 3D candidate box into equally spaced grids, using point set feature extraction based on the grid center points, obtaining key point features of the neighborhood of the center points and performing aggregation; and performing target box secondary regression and classification score prediction based on the grid center point features. 7.The keypoint-based 3D object detection and its parameterized radius learning method of claim 1, wherein, The process of learning the parameterized feature aggregation radius includes: introducing the point cloud feature aggregation sampling radius as a learnable parameter into the detection model by simulating the point cloud sampling process through a truncation function; updating the sampling radius using the gradient of the objective function through back propagation, and simultaneously learning the sampling radius during the detection model training process; constructing a truncated Smooth L1 loss function to constrain the range of the learned sampling radius; and the expression of the truncated Smooth L1 loss function is: where ε and ρ represent the symmetry center and the truncation range, and the truncated Smooth L1 loss function is symmetric to ε and has a loss of 0 in the interval [ε-ρ, ε+ρ].

8. A keypoint-based 3D object detection and its parameterized feature aggregation radius learning system, characterized in that, It comprises: an initial feature extraction module, a voxel feature extraction module, a key point feature extraction module, a candidate box generation module, a two-stage target box regression module, and a sampling radius parameter optimization module; the initial feature extraction module is configured to take the original point cloud as input, obtain key points through downsampling, and perform voxelization processing on the original point cloud to obtain voxel initial features; the voxel feature extraction module is configured to input the voxel initial features into a 3D sparse convolution network to obtain voxel features; the 3D sparse convolution network for extracting voxel features is composed of four sparse convolution modules; each sparse convolution module takes non-empty voxels and their corresponding voxel features and indexes as input, constructs a three-dimensional sparse tensor, and uses a 3D sparse convolution layer to extract features while performing two times downsampling on the sparse voxel feature map; the sparse voxel features are concatenated in the height dimension and converted into dense overhead view features, i.e., voxel features; the key point feature extraction module adopts a structured point feature extraction method to obtain key point features from the key points sampled from the original point cloud; specifically, structured points are generated at positions offset by a fixed distance in the positive and negative directions of the three-dimensional coordinates of the key points with the key points as the center; the voxel features in the neighborhood of the structured points and the key points are aggregated using a point set feature extraction method with the voxel centers as coordinates to obtain fusion features enhanced by orientation information; the key point features are obtained by combining the features of the multiple sparse convolution modules, the original point cloud and the key points of the dense overhead view; the sampling radius parameter optimization module updates the point cloud feature aggregation sampling radius as a learnable parameter through back propagation to obtain the optimal sampling radius parameter. The candidate box generation module learns the offset of the key point to the target center through target-aware feature pooling, and aggregates features within the target-aware neighborhood to obtain candidate points and corresponding features; the candidate points are classified using the corresponding key points, and candidate box regression is performed with the candidate points as the center; The two-stage target box regression module extracts the center point feature of the grid through the region of interest grid pooling, and performs secondary regression and score prediction on the candidate box to obtain the final 3D target detection result.

9. A computer device, comprising: The method comprises a processor and a memory, and the memory stores an executable program. When the processor executes the executable program, the method for 3D target detection based on key points and parameterized radius learning according to any one of claims 1 to 7 can be executed.

10. A computer-readable storage medium, characterized in that, A computer readable storage medium stores a computer program. When the computer program is executed by a processor, the method for 3D target detection based on key points and parameterized radius learning according to any one of claims 1 to 7 can be implemented.

Images