A three-dimensional target detection method based on key point cloud local features

Abstract

A 3D target detection method based on keypoint point cloud local features includes: acquiring point cloud data to be detected at the current time; inputting the acquired point cloud data into a trained multi-dimensional feature fusion feature encoding network to obtain target detection results; wherein the multi-dimensional feature fusion feature encoding network performs keypoint sampling and feature encoding on the point cloud data in the training samples to obtain keypoint point cloud local features, specifically acquiring N original point cloud data; sampling N' / 3 points from the N points using the spatial furthest point sampling method as keypoints; sampling N' / 3 points from the N points using the feature furthest point sampling method; sampling N' / 3 points from the N points; S5. performing feature extraction through a point set abstraction module to generate keypoint point cloud local features. This invention makes the extracted local features of the object more accurate and better expresses the semantic information of the object, making subsequent object detection and classification more accurate.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a three-dimensional target detection method based on local features of key point cloud. Background Technology

[0002] There are two commonly used methods in 3D object detection: point-based methods and voxel-based methods. Point-based methods treat 3D point cloud data as a series of scattered points and process each point directly. Their advantages lie in high geometric flexibility and scalability, while also accurately capturing the details and local features of the point cloud data. However, due to the sparsity and unordered nature of point cloud data, point-based methods suffer from efficiency and accuracy issues when processing large scenes. Representative research in this area includes PointNet and PointNet++. FPointNet first proposed applying PointNet to 3D detection using cropped point clouds based on 2D image bounding boxes. PointRCNN directly generates 3D candidate boxes from the entire point cloud, rather than using only the 2D image for 3D detection.

[0003] Voxel-based methods: This approach divides 3D point cloud data into a series of regular voxels (i.e., 3D pixels) and then processes each voxel. Compared to point-based methods, voxel-based methods have higher computational efficiency and stability, and are easily integrated with deep learning techniques such as convolutional neural networks (CNNs). However, voxel-based methods also suffer from high data dimensionality and significant information loss. Related research includes the VoxelNet and SECOND algorithms.

[0004] In existing approximation schemes, point-based keypoint sampling and point set feature extraction methods mainly use PointNet++'s farthest point sampling algorithm to sample keypoints, obtaining uniform global keypoints, and then extracting features through a feature abstraction layer. PointNet++ and PointRCNN use an ensemble abstraction layer (SA) to downsample keypoints for higher efficiency and a larger receptive field, obtaining local features in the neighborhood of the keypoints. The ensemble abstraction layer uses the farthest point sampling algorithm to downsample and obtain a subset of keypoints as representative points for downsampling. However, this sampling method only considers the relative positions between points. Since the number of points of interest is not as large as the number of background points, and due to the non-uniformity of LiDAR point clouds (dense points near objects and sparse points far away, with only a few points on distant objects), using this method results in the sampled keypoints actually being background points, while points on distant objects are filtered out and not detected. Most existing methods apply a feature propagation layer (FP) during the downsampling process to recall those discarded useful points, but they must incur longer computational overhead for inference time. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention proposes a three-dimensional target detection method based on local features of key point cloud. By sampling the farthest point in spatial distance, the farthest point in feature distance, and the distance from the point to the center, one-third of the key points are obtained from each of the three sampling methods. Finally, the key point cloud is generated by multi-dimensional feature fusion.

[0006] The technical solution of the present invention is as follows:

[0007] A 3D target detection method based on key point cloud local features includes: acquiring point cloud data to be detected at the current time; inputting the acquired point cloud data into a trained multi-dimensional feature fusion feature encoding network to obtain target detection results. The method is characterized in that the multi-dimensional feature fusion feature encoding network performs key point sampling and feature encoding on the point cloud data in the training samples to obtain key point local features of the point cloud. The specific steps are as follows:

[0008] S1. Obtain N raw point cloud data points;

[0009] S2. Using the farthest point sampling method, sample N' / 3 points from N points as key points;

[0010] S3. Using the feature distance furthest point sampling method, sample N' / 3 points from N points as key points;

[0011] S4. Convert the point cloud coordinate system to a coordinate system with the lidar as the origin, calculate the relative distance of each point from the origin, set different sampling probabilities according to the distance of the point to the origin, the probability of a point being randomly sampled is proportional to the distance of the point to the origin, and sample N' / 3 points from N points.

[0012] S5. Recombine the point clouds obtained from steps S2, S3 and S4 into a key point cloud set of N' points, and extract features through the point set abstraction module (SA) to generate local features of the key point cloud.

[0013] Furthermore, in step S2, N' / 3 points are sampled from N points using the farthest point sampling method. The specific steps are as follows:

[0014] Step S2.1 The input point cloud has N points. Select point P0 from the point cloud as the starting point to obtain the sampling point set S = {P0}.

[0015] Step S2.2 Calculate the distance from all points to point P0, form an N-dimensional array L, and select the point corresponding to the maximum value as point P1, and update the sampling point set S = {P0, P1}.

[0016] Step S2.3 Calculate the distance from all points to point P1. For each point Pi, if its distance to P1 is less than L[i], then update L[i] = d(Pi, P1). That is, the array L stores the shortest distance from each point to the set of sampling points S.

[0017] Step S2.4 Select the point corresponding to the maximum value in array L as point P2, and update the sampling point set S = {P0, P1, P2};

[0018] Step S2.5 Repeat steps S2.2-S2.4 until N' / 3 target sampling points are sampled.

[0019] Furthermore, in step S3, N' / 3 points are sampled from N points using the feature distance furthest point sampling method. The specific steps are as follows:

[0020] Step S3.1 Randomly select a point as the first sampling point and add it to the sampling point set;

[0021] Step S3.2 For the remaining points, calculate their feature distances to all points in the sampling point set, select the point with the farthest distance and the largest feature distance as the next sampling point, and add it to the sampling point set;

[0022] Step S3.3 Repeat step S3.2 until the number of points in the sampling point set reaches the specified number.

[0023] Furthermore, step S4, based on different sampling probabilities of the distance from the point to the center, specifically involves the following steps:

[0024] Step S4.1 Using the lidar as the origin, i.e. the center point, calculate the Euclidean distance d from each point to the center point;

[0025] Step S4.2 Sample the original point cloud. The probability of a point being sampled is proportional to the Euclidean distance of the point, with probability p = d / D, where D is the farthest distance from the point to the center.

[0026] Step S4.3 Repeat step S4.2 until a specified number of points have been sampled.

[0027] Furthermore, the feature extraction via the point set abstraction module (SA) involves the following specific steps:

[0028] Step S5.1 Perform region segmentation on the input point cloud: The point set abstraction module layer first segments the input point cloud data according to spatial location, treating the points in each region as a set. For each region, the point set abstraction module layer uses a learnable neural network to determine the center point of the region. These center points are used to represent the overall features of the region.

[0029] Step S5.2 Extract local features of the region and aggregate local features to the center point: Use the max pooling method to aggregate local features to the center point, that is, compare each local feature vector with the feature vector of the center point of the region, and select the maximum value as the final feature vector of the region;

[0030] Step S5.3 Merge the feature vectors of each region: Connect the final feature vectors of each region to form the global feature vector of the entire point cloud. This global feature vector can be passed to subsequent network layers for tasks such as classification or regression.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] 1) By using feature distance as the criterion in farthest point sampling, many similar useless background points will be largely removed, such as a large number of ground points. Even valid points for distant objects can be retained because the semantic features of points from different objects are different from each other. The keypoints obtained in this way take into account both spatial distance and semantic information of the points.

[0033] 2) Adjust the sampling probability based on the distance of the point from the coordinate center, appropriately increasing the probability of sampling points of distant objects. This increases the number of key points sampled from distant objects, enriching the point cloud features and improving the feature extraction performance of the SA (Set Abstraction) layer. Thus, during the SA layer, F-FPS, D-FPS, and sampling probability algorithms based on point-to-center distance are applied to retain more effective points for localization and also retain sufficient control points for classification.

[0034] 3) The point cloud features extracted in this invention, for the target object, are more accurate and better express the semantic information of the object by using three methods to fuse the key points of the sampled object. Using these features for subsequent object detection and classification is more accurate. Attached Figure Description

[0035] Figure 1 This is a flowchart of the three-dimensional target detection method based on local features of key point cloud according to the present invention;

[0036] Figure 2 This is a graph showing the object detection results on the KITTI dataset. Detailed Implementation

[0037] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the scope of protection of the present invention.

[0038] Please see Figure 1 , Figure 1This is a flowchart of the 3D target detection method based on key point cloud local features of the present invention. As shown in the figure, a 3D target detection method based on key point cloud local features includes: acquiring the point cloud data to be detected at the current time; inputting the acquired point cloud data into a trained multi-dimensional feature fusion feature encoding network to obtain the target detection result. The multi-dimensional feature fusion feature encoding network performs key point sampling and feature encoding on the point cloud data in the training samples to obtain point-by-point feature vectors. The specific steps are as follows:

[0039] (1a) Input the point cloud into the distance-based farthest point sampling algorithm module to sample N' / 3 points from N points.

[0040] (1b) Input the point cloud into the feature distance-based farthest point sampling algorithm module to sample N' / 3 points from N points. The feature distance-based farthest point sampling algorithm replaces the part in the above farthest point sampling algorithm that calculates the distance based on the input point set with the distance in the pre-calculated feature space. That is, it calculates the distance between two tensors [BNC] and [BSC]. In fact, the input of this algorithm is a point cloud feature tensor that integrates xyz and features. The distances here are all L2 norm distances.

[0041] (1c) Convert the point cloud coordinate system to a coordinate system with the lidar as the origin, and calculate the relative distance of each point from the origin. Set different sampling probabilities according to the distance of the point to the origin. The probability of a point being randomly sampled is proportional to the distance of the point to the origin, so as to sample N' / 3 points from N points.

[0042] (1d) Recombine the point clouds sampled in the three ways into a key point cloud set. Input the key point set into the Set Abstraction Module (SA) for feature extraction and output a feature vector of [N', Cm].

[0043] Detection results are as follows Figure 2 As shown, the red dots represent the keypoint set obtained after multi-dimensional feature fusion sampling (i.e., sampling using three methods) of the original point cloud. For objects with fewer points in the image, a larger number of keypoints are sampled, essentially covering the object's outline. Even for distant objects, although their point clouds are smaller, a considerable number of keypoints containing important information about the object are still retained after the keypoint sampling process. After passing through the detection network, objects with fewer point clouds and distant, sparsely populated objects are accurately detected.

Claims

1. A three-dimensional object detection method based on key point cloud local features, comprising: Obtain the point cloud data to be detected at the current moment; The acquired point cloud data is input into a trained multi-dimensional feature fusion feature encoding network to obtain the target detection result. The feature encoding network performs keypoint sampling and feature encoding on the point cloud data in the training samples to obtain local features of the keypoint point cloud. The specific steps are as follows: S1. Obtain N raw point cloud data points; S2. Using the farthest point sampling method, sample N' / 3 points from N points as key points; S3. Using the feature distance furthest point sampling method, sample N' / 3 points from N points as key points; S4. Convert the point cloud coordinate system to a coordinate system with the lidar as the origin, calculate the relative distance of each point from the origin, set different sampling probabilities according to the distance of the point to the origin, the probability of a point being randomly sampled is proportional to the distance of the point to the origin, and sample N' / 3 points from N points. S5. Recombine the point clouds obtained from steps S2, S3, and S4 into a key point cloud set of N' points, and extract features using the point set abstraction module (SA) to generate local features of the key point cloud. The specific steps are as follows: Step S5.1 Perform region segmentation on the input point cloud: The point set abstraction module layer first segments the input point cloud data according to spatial location, treating the points in each region as a set. For each region, the point set abstraction module layer uses a learnable neural network to determine the center point of the region. These center points are used to represent the overall features of the region. Step S5.2 Extract local features of the region and aggregate local features to the center point: Use the max pooling method to aggregate local features to the center point, that is, compare each local feature vector with the feature vector of the center point of the region, and select the maximum value as the final feature vector of the region; Step S5.3 Merge the feature vectors of each region: Connect the final feature vectors of each region to form the global feature vector of the entire point cloud. This global feature vector can be passed to subsequent network layers for classification or regression tasks.

2. The three-dimensional object detection method based on key point point cloud local features according to claim 1, characterized in that, Step S2 involves sampling N' / 3 points from N points using the farthest point sampling method. The specific steps are as follows: Step S2.1 The input point cloud has N points. Select point P0 from the point cloud as the starting point to obtain the sampling point set S={P0}; Step S2.2 Calculate the distance from all points to point P0, form an N-dimensional array L, and select the point corresponding to the maximum value as point P1, and update the sampling point set S={P0, P1}; Step S2.3 Calculate the distance from all points to point P1. For each point Pi, if its distance to P1 is less than L[i], then update L[i] = d(Pi, P1), that is, the array L stores the shortest distance from each point to the set of sampling points S. Step S2.4 Select the point corresponding to the maximum value in array L as point P2, and update the sampling point set S={P0, P1, P2}; Step S2.5 Repeat steps S2.2-S2.4 until N' / 3 target sampling points are sampled.

3. The three-dimensional target detection method based on local features of key point cloud as described in claim 1, characterized in that, Step S3. Using the feature distance furthest point sampling method, sample N' / 3 points from N points. The specific steps are as follows: Step S3.1 Randomly select a point as the first sampling point and add it to the sampling point set; Step S3.2 For the remaining points, calculate their feature distances to all points in the sampling point set, select the point with the farthest distance and the largest feature distance as the next sampling point, and add it to the sampling point set; Step S3.3 Repeat step S3.2 until the number of points in the sampling point set reaches the specified number.

4. The three-dimensional target detection method based on local features of key point cloud as described in claim 1, characterized in that... Step S4, based on the different sampling probabilities of the distance from the point to the center, specifically involves the following steps: Step S4.1 Using the lidar as the origin, i.e. the center point, calculate the Euclidean distance d from each point to the center point; Step S4.2 Sample the original point cloud. The probability of a point being sampled is proportional to the Euclidean distance of the point, with probability p = d / D. Where D is the farthest distance from the point to the center; Step S4.3 Repeat step S4.2 until a specified number of points have been sampled.

Images