A general lightweight component segmentation method for dense point clouds

Abstract

This invention relates to the field of point cloud component segmentation technology, specifically to a general lightweight component segmentation method for dense point clouds, comprising the following steps: Step 1: Let the returned point cloud scan data be P = {p i =(x i ,y i ,z i Step 1: Perform centering and scaling operations on point cloud Q to obtain point cloud Q; Step 2: Perform adaptive multi-scale voxel mesh downsampling on point cloud Q; Step 3: Perform data augmentation operations on point cloud D, performing random scaling, random translation, slight rotation, and random noise addition to obtain point cloud DP; Step 4: Perform component segmentation on point cloud DP. This invention maximizes the preservation of local features through adaptive multi-scale voxel mesh downsampling, introduces a multi-resolution domain feature extraction module, and incorporates feature Euclidean distance weights, effectively improving the component segmentation accuracy in dense point cloud domains and significantly reducing the requirements for GPU memory and computing power.

Description

Technical Field

[0001] This invention relates to the field of point cloud component segmentation technology, and specifically to a general lightweight component segmentation method for dense point clouds. Background Technology

[0002] In recent years, the field of point cloud component segmentation has developed rapidly and has been widely used in applications such as autonomous driving and 3D reconstruction, demonstrating significant advantages in operational efficiency and robustness compared to traditional methods. Point cloud component segmentation is further divided into dense and sparse point clouds based on point cloud density. Due to hardware limitations, most research has focused on sparse point clouds to address the feature loss problem caused by point cloud sparsity. However, with advancements in sensor technology, the accuracy of acquired point clouds is increasing, while existing deep learning models place a significant burden on GPU memory and computing power. This means that research on component segmentation on dense point clouds under limited computing power and GPU memory is urgently needed.

[0003] Current existing technologies related to point cloud component segmentation methods include: Patent Application No. 202210754650.X, titled "A Point Cloud Component Segmentation Method Based on Local Feature Enhancement and Similarity Measurement." This patent trains a three-branch network in parallel using feature matrices that extract and fuse global and local features. It then uses Euclidean distance to learn a similarity matrix to calculate a confidence matrix, removes redundant points from both matrices, and uses a multilayer perceptron and embedding space to fuse component features using a similarity measurement. This method does not consider multi-scale features and is difficult to achieve effective fusion of local and global features. Patent Application No. 202310349933.0, titled "A Geometrically Adaptive Point Cloud Classification and Segmentation Method and Apparatus Based on Point Cloud-Voxel," uses a voxel-based depthwise convolution operator and coordinate-adaptive pooling to construct a feature extraction module, enabling adaptive extraction and fusion of features at different scales in the point cloud. However, the voxel selection in this method has a significant impact on the algorithm results and exhibits poor robustness. Invention patent application number: 202410849292.X, title: Robust Component Segmentation Method for 3D Point Clouds Based on Semantic SAM Large Model. This patent projects 3D point cloud data onto a 2D depth map through multiple views, uses a large SAM model to fuse features from the 2D depth map under multiple views, and uses an adapter module to fine-tune the semantic SAM model, integrating the output back into the 3D point cloud data. However, this scheme processes the projected 2D image, which easily leads to the loss of the arrangement invariance of the point cloud, resulting in feature loss.

[0004] In summary, the current technical solutions have the following limitations: (1) The above algorithms are all relatively complex, the steps are cumbersome, the calculation difficulty is high, and the computing power and video memory requirements are high; (2) The applicable scope is relatively narrow and not universal; (3) The accuracy is generally not high. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a general lightweight component segmentation method for dense point clouds. It relies on lidar or structured light sensors and processes the returned point cloud scanning data through this invention to achieve high-speed component segmentation and low resource requirements.

[0006] The technical solution of the present invention is as follows:

[0007] A general lightweight component segmentation method for dense point clouds includes the following steps:

[0008] Step 1: Let the returned point cloud scan data be P = {p i =(x i ,y i ,z i )}, perform centering and scaling operations according to equations (1) and (2) to obtain point cloud Q; point cloud Q is shown in equation (3):

[0009]

[0010]

[0011] Q = {p i "=(x i ",y i ",z i ")} (3)

[0012] Among them, (x i ′,y i ′,z i ′) represents the coordinates of the i-th point in P after centering. i ′;(x i ",y i ",z i ") represents the coordinates of the i-th point in P after centering and scaling. i ";

[0013] Step 2: Perform adaptive multi-scale voxel mesh downsampling on the point cloud Q. The specific steps are as follows:

[0014] Step 2.1: Set the initial voxel group V, containing n different voxels and the number of sampling target points N;

[0015] Step 2.2: Find the minimum boundary value on each coordinate axis in the point cloud Q according to equation (4), and denot it as MinBound:

[0016] MinBound=(min{x′ i ′},min{y′ i ′},min{z′i (4)

[0017] Step 2.3: Traverse the initial voxel set V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to equation (5):

[0018]

[0019] Among them, V i Let p″-MinBound represent the i-th voxel in the initial voxel set V, and let p″-MinBound represent all points (x) i ",y i ",z i Subtract the corresponding three values ​​from the Minbound value;

[0020] Step 2.4: Assign points in the point cloud Q to the corresponding voxels VV according to their voxel indices. j As shown in equation (6):

[0021] VV j ={p′ i ′vIndex(p′ i ′)=j} (6)

[0022] Where vIndex(p i ) represents p i "Corresponding voxel index;

[0023] Step 2.5: For each voxel VV j The downsampling point d is determined according to formula (7). ij The calculations are combined to obtain a new point cloud D. Let L be the number of points in point cloud D, as shown in equation (8):

[0024]

[0025] D = {D i 0≤i≤n} (8)

[0026] Among them, VV j Let be all points in voxel j, and n be the number of voxels in voxel group V;

[0027] Step 2.6: Compare the number of points L in point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in voxel group V, discard the number of points sampled from L, and reduce it by a factor of k. Repeat steps 2.2 to 2.5. If L plus the new sampling points is greater than N, repeat step 2.6; if L plus the new sampling points is less than N, randomly sample to complete L until L equals N; if L is less than N, increase it by a factor of k, and repeat steps 2.2 to 2.5. If L plus the new sampling points is less than N, repeat step 2.6; if L plus the new sampling points is greater than N, randomly discard them until L equals N.

[0028] Step 3: Perform data augmentation operations on point cloud D, including random scaling, random translation, slight rotation, and random noise addition, to obtain point cloud DP;

[0029] Step 4: Perform component segmentation on the point cloud DP. The specific steps are as follows:

[0030] Step 4.1: Set up the feature extraction module group FE, with a length denoted as FEL; set up the feature recovery module group FR, with a length denoted as FRL;

[0031] Step 4.2: Select the i-th module in the feature extraction module group FE, and set the number of sampling points to sample. i Set the number of neighbors to be selected as neighbor i Set the multilayer perceptron parameters to mlpe i Samples were taken using the furthest point sampling method. i Use the KNN nearest neighbor algorithm to select neighbors. i For each neighbor, obtain the distance DF of the farthest neighbor and the distance DN of the nearest neighbor for the j-th sampling point, and divide them into three resolutions according to Equation (9):

[0032]

[0033] Among them, S r The r-th resolution in scale group S;

[0034] Step 4.3: Using the j-th sampling point as the center and the r-th resolution as the radius, perform a spherical neighbor lookup and collect the neighbor coordinate information SM. j and Feature Information FM j ;

[0035] Step 4.4: Perform coordinate information splicing SM according to formula (10). j and Feature Information FM j Obtain characteristic FEM j According to formula (11), a multilayer perceptron (MLPE) is used. i FEE is obtained by feature extraction. j ;

[0036] FEM j =Concat(SM) j FM j (10)

[0037] FEE j =mlpe i (FEM j (11)

[0038] Step 4.5: Apply formula (12) to the FEE j After max pooling, the features are concatenated to obtain the feature FEE. i Repeat steps 4.2-4.5 to complete feature extraction;

[0039] FEE i =Concat(MAXPool(FEE) j (12)

[0040] Step 4.6: Select the k-th module in the Feature Recovery Module Group (FR), and set the multilayer perceptron parameters to mlpr. k Calculate the coordinate information SM according to formula (13) k and SM k-1 Euclidean distance:

[0041]

[0042] Step 4.7: For feature information FM k and FM K-1 Based on their dimensionality, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between them is calculated according to equation (14):

[0043]

[0044] Step 4.8: Add DisXYZ and DisFM to get Dis, sort them in ascending order, take m items, and perform weight normalization according to equation (15):

[0045]

[0046] Step 4.9: Match Weights with FM K-1 Multiply, then with SM k-1 Perform splicing, perform multilayer perceptron feature recovery according to formula (16), perform max pooling according to formula (17) and then splice;

[0047] FREk=mlprk(Concat(Weights*FMk -1 ,SMk -1(16)

[0048] FRF k =Concat(MAXPool(FRE) k (17)

[0049] Step 4.10: Add FRF k Mapping to categories yields the segmentation results.

[0050] Compared with the prior art, the beneficial results of the present invention are as follows: The present invention maximizes the preservation of local features by adaptive multi-scale voxel grid downsampling, introduces a multi-resolution domain feature extraction module, and incorporates feature Euclidean distance weights, which effectively improves the component segmentation accuracy in dense point cloud domains and greatly reduces the requirements for video memory and computing power. Attached Figure Description

[0051] Figure 1 This is the original point cloud scanned by the lidar;

[0052] Figure 2 for Figure 1 Point cloud after centralization and scaling;

[0053] Figure 3 for Figure 2 Point cloud after adaptive multi-scale grid voxel downsampling;

[0054] Figure 4 for Figure 3 Data-augmented point cloud;

[0055] Figure 5 for Figure 4 Point cloud inference using a general lightweight component segmentation model for dense point clouds. Detailed Implementation

[0056] The following examples illustrate in detail the specific implementation of the general lightweight component segmentation method for dense point clouds of the present invention.

[0057] The general lightweight component segmentation method for dense point clouds of the present invention specifically includes the following steps:

[0058] Step 1: Record the point cloud scanning data returned by the line lidar as P = {p i =(x i ,y i ,z i )},like Figure 1 As shown, centering and scaling operations are performed according to equations (1) and (2) to obtain point cloud Q; point cloud Q is shown in equation (3), and the result is as follows. Figure 2 As shown:

[0059]

[0060] Q = {p i "=(x i ",y i ",z i ")} (3)

[0061] Among them, (x i ′,y i ′,z i ′) represents the coordinates of the i-th point in P after centering. i ′;(x i ",y i ",z i ") represents the coordinates of the i-th point in P after centering and scaling. i ";

[0062] Step 2: Adaptive multi-scale voxel mesh downsampling is performed on the point cloud Q, and the results are as follows. Figure 3 As shown, the specific steps are as follows:

[0063] Step 2.1: Set the initial voxel set V, which contains n different voxels and the number of sampling target points N. In this example, the initial voxel set V = {0.1, 0.05, 0.02}.

[0064] Step 2.2: Find the minimum boundary value on each coordinate axis in the point cloud Q according to equation (4), and denot it as MinBound:

[0065] MinBound=(min{x′ i ′},min{y′ i ′},min{z′ i (4)

[0066] Step 2.3: Traverse the initial voxel set V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to equation (5):

[0067]

[0068] Among them, V i Let p″-MinBound represent the i-th voxel in the initial voxel set V, and let p″-MinBound represent all points (x) i ",y i ",z i Subtract the corresponding three values ​​from the Minbound value;

[0069] Step 2.4: Assign points in the point cloud Q to the corresponding voxels VV according to their voxel indices. j As shown in equation (6):

[0070] VV j ={p′ i ′vIndex(p′ i ′)=j} (6)

[0071] Where vIndex(p i ) represents p i "Corresponding voxel index;

[0072] Step 2.5: For each voxel VV j The downsampling point d is determined according to formula (7). ij The calculations are combined to obtain a new point cloud D. Let L be the number of points in point cloud D, as shown in equation (8):

[0073]

[0074] D = {D i 0≤i≤n} (8)

[0075] Among them, VV j Let be all points in voxel j, and n be the number of voxels in voxel group V;

[0076] Step 2.6: Compare the number of points L in point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in voxel group V, discard its sampled points from L, and reduce it by a factor of k. Repeat steps 2.2 to 2.5. If L plus the new sampled points is greater than N, repeat step 2.6; if L plus the new sampled points is less than N, randomly sample to complete L until L equals N; if L is less than N, increase it by a factor of k, and repeat steps 2.2 to 2.5. If L plus the new sampled points is less than N, repeat step 2.6; if L plus the new sampled points is greater than N, randomly discard until L equals N. In this example, N = 102400, k = 1.1; Step 3: Perform data augmentation operations on point cloud D, performing random scaling, random translation, slight rotation, and random noise addition to obtain point cloud DP. The result is as follows. Figure 4 As shown;

[0077] Step 4: Create a general lightweight component segmentation model for dense point clouds to segment the point cloud DP into components; this model is a U-Net-like architecture, which includes an FE module for point cloud feature extraction and an FR module for point cloud feature recovery, plus residual connections to promote the fusion of global and local features; the specific steps are as follows:

[0078] Step 4.1: Set the feature extraction module group FE, with a length of FEL; set the feature recovery module group FR, with a length of FRL; in this example, FEL = FRL = 3;

[0079] Step 4.2: Select the i-th module in the feature extraction module group FE, and set the number of sampling points to sample. i Set the number of neighbors to be selected as neighbor i Set the multilayer perceptron parameters to mlpe i Samples were taken using the furthest point sampling method. i Use the KNN nearest neighbor algorithm to select neighbors. i For each neighbor, obtain the distance DF of the farthest neighbor and the distance DN of the nearest neighbor for the j-th sampling point, and divide them into three resolutions according to Equation (9):

[0080]

[0081] Among them, S r Let r be the resolution in scale group S; in this example, sample1 = 512, sample2 = 128, sample3 = 32, neighbor1 = 384, neighbor2 = 96, neighbor3 = 24, mlpe1 = [32, 64, 128], mlpe2 = [64, 128, 192], mlpe3 = [128, 128, 256];

[0082] Step 4.3: Using the j-th sampling point as the center and the r-th resolution as the radius, perform a spherical neighbor lookup and collect the neighbor coordinate information SM. j and Feature Information FM j ;

[0083] Step 4.4: Perform coordinate information splicing SM according to formula (10). j and Feature Information FM j Obtain characteristic FEM j According to formula (11), a multilayer perceptron (MLPE) is used. i FEE is obtained by feature extraction. j ;

[0084] FEM j =Concat(SM) j FM j (10)

[0085] FEE j =mlpe i (FEM j (11)

[0086] Step 4.5: Apply formula (12) to the FEE j After max pooling, the features are concatenated to obtain the feature FEE. i Repeat steps 4.2-4.5 to complete feature extraction;

[0087] FEE i =Concat(MAXPool(FEE) j (12)

[0088] Step 4.6: Select the k-th module in the Feature Recovery Module Group (FR), and set the multilayer perceptron parameters to mlpr. k Calculate the coordinate information SM according to formula (13) k and SM k-1 Euclidean distance:

[0089]

[0090] In this example, mlpr1 = [1024, 512, 512], mlpr2 = [512, 256, 256], and mlpr3 = [256, 128, 128].

[0091] Step 4.7: For feature information FM k and FM K-1 Based on their dimensionality, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between them is calculated according to equation (14):

[0092]

[0093] Step 4.8: Add DisXYZ and DisFM to get Dis, sort them in ascending order, take m items, and perform weight normalization according to equation (15):

[0094]

[0095] In this example, m = 4;

[0096] Step 4.9: Match Weights with FM K-1 Multiply, then with SM k-1 Perform splicing, perform multilayer perceptron feature recovery according to formula (16), perform max pooling according to formula (17) and then splice;

[0097] FREk=mlprk(Concat(Weights*FMk -1 ,SMk -1 (16)

[0098] FRF k =Concat(MAXPool(FRE) k (17)

[0099] Step 4.10: Add FRF kMapping to categories yields segmentation results, as shown below. Figure 5 As shown.

[0100] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A general lightweight component segmentation method for dense point clouds, characterized in that, Includes the following steps: Step 1: Denote the point cloud scan data as P={p i =(x i ,y i ,z i )}, perform centering and scaling operations according to equations (1) and (2) to obtain point cloud Q; point cloud Q is shown in equation (3): (1)； (2)； (3)； in, This represents the coordinates of the i-th point in P after centering. ; Let P be the coordinates of the i-th point after centering and scaling. ; Step 2: Perform adaptive multi-scale voxel mesh downsampling on point cloud Q to obtain point cloud D; Step 3: Perform data augmentation operations on point cloud D, including random scaling, random translation, slight rotation, and random noise addition, to obtain point cloud DP; Step 4: Perform component segmentation on the point cloud DP; The specific steps of step 4) are as follows: Step 4.1: Set up the feature extraction module group FE, with a length denoted as FEL; set up the feature recovery module group FR, with a length denoted as FRL; Step 4.2: Select the i-th module in the feature extraction module group FE, and set the number of sampling points to sample. i Set the number of neighbors to be selected as neighbor i Set the multilayer perceptron parameters to mlpe i Samples were taken using the furthest point sampling method. i Use the KNN nearest neighbor algorithm to select neighbors. i For each neighbor, obtain the distance DF of the farthest neighbor and the distance DN of the nearest neighbor for the j-th sampling point, and divide them into three resolutions according to Equation (9): (9)； Among them, S r The r-th resolution in scale group S; Step 4.3: Using the j-th sampling point as the center and the r-th resolution as the radius, perform a spherical neighbor lookup and collect the neighbor coordinate information SM. j and Feature Information FM j ; Step 4.4: Perform coordinate information splicing SM according to formula (10). j and Feature Information FM j Obtain characteristic FEM j According to formula (11), a multilayer perceptron (MLPE) is used. i FEE is obtained by feature extraction. j ; (10)； (11)； Step 4.5: Apply formula (12) to the FEE j After max pooling, the features are concatenated to obtain the feature FEE. i Repeat steps 4.2-4.5 to complete feature extraction; (12)； Step 4.6: Select the k-th module in the Feature Recovery Module Group (FR), and set the multilayer perceptron parameters to mlpr. k Calculate the coordinate information SM according to formula (13) k and SM k-1 Euclidean distance: (13)； Step 4.7: For feature information FM k and FM K-1 Based on their dimensionality, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between them is calculated according to equation (14): (14)； Step 4.8: Add DisXYZ and DisFM to get Dis, sort them in ascending order, take m items, and perform weight normalization according to equation (15): (15)； Step 4.9: Match Weights with FM K-1 Multiply, then with SM k-1 Perform splicing, perform multilayer perceptron feature recovery according to formula (16), perform max pooling according to formula (17) and then splice; (16)； (17)； Step 4.10: Add FRF k Mapping to categories yields the segmentation results.

2. The general lightweight component segmentation method for dense point clouds according to claim 1, characterized in that, The specific steps of step 2) are as follows: Step 2.1: Set the initial voxel group V, containing n different voxels and the number of sampling target points N; Step 2.2: Find the minimum boundary value on each coordinate axis in the point cloud Q according to equation (4), and denot it as MinBound: (4)； Step 2.3: Traverse the initial voxel set V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to equation (5): (5)； Among them, V i For the i-th voxel in the initial voxel set V, Represents all points Subtract the corresponding three values ​​from the Minbound; Step 2.4: Assign points in the point cloud Q to the corresponding voxels VV according to their voxel indices. j As shown in equation (6): (6)； Where vIndex(p i )express Corresponding voxel index; Step 2.5: For each voxel VV j The downsampling point d is determined according to formula (7). ij The calculations are combined to obtain a new point cloud D. Let L be the number of points in point cloud D, as shown in equation (8): (7)； (8)； Among them, VV j Let be all points in voxel j, and n be the number of voxels in voxel group V; Step 2.6: Compare the number of points L in point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in voxel group V, discard the number of points sampled from L, and reduce it by a factor of k. Repeat steps 2.2 to 2.

5. If L plus the new sampling points is greater than N, repeat step 2.

6. If L plus the new sampling points is less than N, randomly sample to complete L until L equals N. If L is less than N, increase it by a factor of k and repeat steps 2.2 to 2.

5. If L plus the new sampling points is less than N, repeat step 2.

6. If L plus the new sampling points is greater than N, randomly discard them until L equals N.

Images