Dense point cloud-oriented general lightweight component segmentation method

Abstract

The invention relates to the technical field of point cloud part segmentation, in particular to a dense point cloud-oriented universal lightweight part segmentation method, which comprises the following steps of: 1, recording returned point cloud scanning data as P = {pi = (xi, yi, zi)}, and performing centralization and size scaling operation to obtain a point cloud Q; step 2, adaptive multi-scale voxel grid downsampling is carried out on the point cloud Q; 3, performing data enhancement operation on the point cloud D, and executing random size scaling, random translation, slight rotation and random noise addition to obtain a point cloud DP; and step 4, carrying out part segmentation on the point cloud DP. According to the method, local features are reserved to the maximum extent through adaptive multi-scale voxel grid downsampling, a multi-resolution domain feature extraction module is introduced, and a feature Euclidean distance weight is fused, so that the component segmentation accuracy of the dense point cloud field is effectively improved, and the video memory and computing power requirements are greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud component segmentation, and in particular to a general lightweight component segmentation method for dense point clouds. Background Art

[0002] In recent years, the field of point cloud component segmentation has developed rapidly and has been widely used in applications such as autonomous driving and 3D reconstruction. It has shown obvious advantages in operating efficiency and robustness compared to traditional methods. The field of point cloud component segmentation is divided into dense point clouds and sparse point clouds according to the density of point clouds. Due to hardware limitations, most research focuses on sparse point clouds to solve the problem of missing features caused by sparse point clouds. However, with the advancement of sensor technology, the accuracy of the acquired point clouds is getting higher and higher, and the existing deep learning models will impose a heavy burden on video memory and computing power when running. This means that research on component segmentation on dense point clouds under limited computing power and video memory is imminent.

[0003] The existing technologies currently involved in point cloud component segmentation methods include: Invention patent application number: 202210754650.X, name: A point cloud component segmentation method based on local feature enhancement and similarity measurement. This patent extracts and fuses the feature matrices of global features and local features in parallel to perform three-branch network training, uses Euclidean distance to learn the similarity matrix to calculate the confidence matrix, removes redundant points in the two matrices, and relies on multi-layer perceptrons and embedding space to fuse the converted features with similarity measurement to the component features. This method does not consider multi-scale features, and it is difficult to achieve effective local and global feature fusion. Invention patent application number: 202310349933.0, name: Geometric adaptive point cloud classification and segmentation method and device based on point cloud-voxel, this method uses a voxel-based deep convolution operator and a coordinate adaptive pooling computing power to construct a feature extraction module to achieve adaptive extraction and fusion of features of different scales in the point cloud. However, the voxel selection of this method has a great influence on the algorithm results and poor robustness. Invention patent application number: 202410849292.X, name: Robust component segmentation method of 3D point cloud based on semantic SAM large model, this patent projects 3D point cloud data into 2D depth map through multi-view, uses SAM large model to fuse features of 2D depth map under multi-view, and uses adapter module to fine-tune semantic SAM large model, and integrates output results back into 3D point cloud data. This solution processes the projected 2D image, which is easy to lose the arrangement invariance of point cloud, resulting in feature loss.

[0004] In summary, the current technical solutions have the following limitations: (1) The above algorithms are relatively complex, the steps are cumbersome, the calculations are difficult, and the computing power and video memory requirements are high; (2) The scope of application is relatively narrow and not universal; (3) The accuracy is generally not high. Summary of the invention

[0005] In order to overcome the deficiencies in the prior art, the purpose of the present invention is to provide a general lightweight component segmentation method for dense point clouds, which relies on lidar or structured light sensors and processes the returned point cloud scanning data through the present invention to achieve high-speed component segmentation and lower resource requirements.

[0006] The technical solution of the present invention is as follows:

[0007] A general lightweight component segmentation method for dense point clouds includes the following steps:

[0008] Step 1: Record the returned point cloud scan data as P = {p i =(x i ,y i ,z i )}, perform centering and scaling operations according to equations (1) and (2) to obtain point cloud Q; point cloud Q is shown in equation (3):

[0009]

[0010]

[0011] Q = {p i ″=(x i ″,y i ″,z i ″)} (3)

[0012] Among them, (x i ′,y i ′,z i ′) represents the coordinate p of the i-th point in P after centralization i ′;(x i ″,y i ″,z i ″) is the coordinate p of the i-th point in P after centering and scaling i ″;

[0013] Step 2: Adaptively downsample the point cloud Q to a multi-scale voxel grid. The specific steps are as follows:

[0014] Step 2.1: Set the initial voxel group V, which contains n different voxels and the number of sampling target points N;

[0015] Step 2.2: In the point cloud Q, find the minimum boundary value on each coordinate axis according to formula (4), recorded as MinBound:

[0016] MinBound=(min{x′ i ′},min{y′ i ′},min{z′i ′}) (4)

[0017] Step 2.3: Traverse the initial voxel group V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to formula (5):

[0018]

[0019] Among them, V i is the i-th voxel in the initial voxel group V, p″-MinBound represents all points (x i ″,y i ″,z i ″) and the three values ​​in Minbound are subtracted accordingly;

[0020] Step 2.4: Assign the points in the point cloud Q to the corresponding voxels VV according to the voxel index j , as shown in formula (6):

[0021] VV j = {p′ i ′vIndex(p′ i ′)=j} (6)

[0022] Where vIndex(p i ) indicates p i ″The corresponding voxel index;

[0023] Step 2.5: For each voxel VV j , according to formula (7) to perform downsampling point d ij After merging, a new point cloud D is obtained. The number of points in the point cloud D is L, as shown in formula (8):

[0024]

[0025] D={D i 0≤i≤n} (8)

[0026] Among them, VV j are all the points in voxel j, and n is the number of voxels in voxel group V;

[0027] Step 2.6: Compare the number of points L of the point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in the voxel group V, discard its sampled points from L, and reduce it by k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is greater than N, repeat step 2.6; if L plus the new sampling points is less than N, randomly sample to complete until L is equal to N; if L is less than N, then, enlarge it k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is less than N, repeat step 2.6; if L plus the new sampling points is greater than N, randomly discard until L is equal to N;

[0028] Step 3: Perform data enhancement operations on the point cloud D, perform random resizing, random translation, slight rotation, and random noise addition to obtain the point cloud DP;

[0029] Step 4: Segment the point cloud DP into components. The specific steps are as follows:

[0030] Step 4.1: Set the feature extraction module group FE, the length of which is recorded as FEL; set the feature recovery module group FR, the length of which is recorded as FRL;

[0031] Step 4.2: Select the i-th module in the feature extraction module group FE and set the number of sampling points to sample i , set the number of selected neighbors to neighbor i , set the multi-layer perceptron parameters to mlpe i ; Use the farthest point sampling to take sample i points, use the KNN nearest neighbor algorithm to select neighbor i neighbors, obtain the farthest neighbor distance DF and the nearest neighbor distance DN of the jth sampling point, which are divided into three resolutions according to formula (9):

[0032]

[0033] Among them, S r is the rth resolution in the scale group S;

[0034] Step 4.3: With the jth sampling point as the center and the rth resolution as the radius, perform a spherical query of neighbors and collect neighbor coordinate information SM j and feature information FM j ;

[0035] Step 4.4: Use formula (10) to stitch coordinate information SM j and feature information FM j Characteristic FEM j , according to formula (11) using multi-layer perceptron mlpe i Perform feature extraction to obtain FEE j ;

[0036] FEM j =Concat(SM j ,FM j ) (10)

[0037] FEE j =mlpe i (FEM j ) (11)

[0038] Step 4.5: According to formula (12), FEE j After performing the maximum pooling, the concatenation is performed to obtain the feature FEE i ; Repeat steps 4.2 to 4.5 to complete feature extraction;

[0039] FEE i =Concat(MAXPool(FEE j )) (12)

[0040] Step 4.6: Select the kth module in the feature recovery module group FR and set the multilayer perceptron parameters to mlpr k ; Calculate the coordinate information SM according to formula (13) k and SM k-1 The Euclidean distance of:

[0041]

[0042] Step 4.7: For feature information FM k and FM K-1 , according to the dimension, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between the two is calculated according to formula (14):

[0043]

[0044] Step 4.8: Add DisXYZ and DisFM to get Dis, sort them from small to large, and take m of them. Normalize the weights according to formula (15):

[0045]

[0046] Step 4.9: Use weights with FM K-1 Multiply and then add to SM k-1 Perform splicing, perform multi-layer perceptron feature recovery according to formula (16), perform maximum pooling according to formula (17) and then perform splicing;

[0047] FREk=mlprk(Concat(Weights*FMk -1 ,SMk -1)) (16)

[0048] FRF k =Concat(MAXPool(FRE k )) (17)

[0049] Step 4.10: FRF k Mapped to categories and obtain segmentation results.

[0050] Compared with the prior art, the beneficial results of the present invention are as follows: the present invention maximizes local feature retention through adaptive multi-scale voxel grid downsampling, introduces a multi-resolution field feature extraction module, and incorporates feature Euclidean distance weights, thereby effectively improving the accuracy of component segmentation in the dense point cloud field and greatly reducing the requirements for video memory and computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Scan point cloud for raw LiDAR;

[0052] Figure 2 for Figure 1 The point cloud after centering and scaling;

[0053] Figure 3 for Figure 2 Point cloud downsampled by adaptive multi-scale grid voxels;

[0054] Figure 4 for Figure 3 Data-augmented point cloud;

[0055] Figure 5 for Figure 4 Point cloud inferred by the generic lightweight part segmentation model for dense point clouds. DETAILED DESCRIPTION

[0056] The specific implementation of the general lightweight component segmentation method for dense point clouds of the present invention is described in detail below with reference to examples.

[0057] The general lightweight component segmentation method for dense point clouds of the present invention specifically comprises the following steps:

[0058] Step 1: The point cloud scan data returned by the laser radar scan is P = {p i =(x i ,y i ,z i )},like Figure 1 As shown, according to formula (1) and (2), the centering and size scaling operations are performed to obtain the point cloud Q; the point cloud Q is shown in formula (3). The result is Figure 2 As shown:

[0059]

[0060] Q = {p i ″=(x i ″,y i ″,z i ″)} (3)

[0061] Among them, (x i ′,y i ′,z i ′) represents the coordinate p of the i-th point in P after centralization i ′;(x i ″,y i ″,z i ″) is the coordinate p of the i-th point in P after centering and scaling i ″;

[0062] Step 2: Adaptively downsample the point cloud Q to a multi-scale voxel grid. The result is as follows: Figure 3 As shown, the specific steps are as follows:

[0063] Step 2.1: Set an initial voxel group V, which includes n different voxels and the number of sampling target points N. In this example, the initial voxel group V = {0.1, 0.05, 0.02};

[0064] Step 2.2: In the point cloud Q, find the minimum boundary value on each coordinate axis according to formula (4), recorded as MinBound:

[0065] MinBound=(min{x′ i ′},min{y′ i ′},min{z′ i ′}) (4)

[0066] Step 2.3: Traverse the initial voxel group V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to formula (5):

[0067]

[0068] Among them, V i is the i-th voxel in the initial voxel group V, p″-MinBound represents all points (x i ″,y i ″,z i ″) and the three values ​​in Minbound are subtracted accordingly;

[0069] Step 2.4: Assign the points in the point cloud Q to the corresponding voxels VV according to the voxel index j , as shown in formula (6):

[0070] VV j ={p′ i ′vIndex(p′ i ′)=j} (6)

[0071] Where vIndex(p i ) indicates p i ″The corresponding voxel index;

[0072] Step 2.5: For each voxel VV j , according to formula (7) to perform downsampling point d ij After merging, a new point cloud D is obtained. The number of points in the point cloud D is L, as shown in formula (8):

[0073]

[0074] D={D i 0≤i≤n} (8)

[0075] Among them, VV j are all the points in voxel j, and n is the number of voxels in voxel group V;

[0076] Step 2.6: Compare the number of points L of point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in the voxel group V, discard its sampled points from L, and reduce it by k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is greater than N, repeat step 2.6; if L plus the new sampling points is less than N, randomly sample to complete until L is equal to N; if L is less than N, then, enlarge it k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is less than N, repeat step 2.6; if L plus the new sampling points is greater than N, randomly discard until L is equal to N; in this example, N = 102400, k = 1.1; Step 3: Perform data enhancement operations on point cloud D, perform random size scaling, random translation, slight rotation and random noise addition, and obtain point cloud DP. The result is as follows Figure 4 As shown;

[0077] Step 4: Create a general lightweight component segmentation model for dense point clouds to segment the point cloud DP. This model is a U-Net-like architecture, which contains a FE module for point cloud feature extraction and a FR module for point cloud feature recovery. In addition, the residual connection promotes the fusion of global and local features. The specific steps are as follows:

[0078] Step 4.1: Set the feature extraction module group FE, and record its length as FEL; set the feature recovery module group FR, and record its length as FRL; in this example, FEL = FRL = 3;

[0079] Step 4.2: Select the i-th module in the feature extraction module group FE and set the number of sampling points to sample i , set the number of selected neighbors to neighbor i , set the multi-layer perceptron parameters to mlpe i ; Use the farthest point sampling to take sample i points, use the KNN nearest neighbor algorithm to select neighbor i neighbors, obtain the farthest neighbor distance DF and the nearest neighbor distance DN of the jth sampling point, which are divided into three resolutions according to formula (9):

[0080]

[0081] Among them, S r is the rth resolution in the scale group S; in this example, sample1=512, sample2=128, sample3=32, neighbor1=384, neighbor2=96, neighbor3=24, mlpe1=[32,64,128], mlpe2=[64,128,192], mlpe3=[128,128,256];

[0082] Step 4.3: With the jth sampling point as the center and the rth resolution as the radius, perform a spherical query of neighbors and collect neighbor coordinate information SM j and feature information FM j ;

[0083] Step 4.4: Use formula (10) to stitch coordinate information SM j and feature information FM j Characteristic FEM j , according to formula (11) using multi-layer perceptron mlpe i Perform feature extraction to obtain FEE j ;

[0084] FEM j =Concat(SM j ,FM j ) (10)

[0085] FEE j =mlpe i (FEM j ) (11)

[0086] Step 4.5: According to formula (12), FEE j After performing the maximum pooling, the concatenation is performed to obtain the feature FEE i ; Repeat steps 4.2 to 4.5 to complete feature extraction;

[0087] FEE i =Concat(MAXPool(FEE j )) (12)

[0088] Step 4.6: Select the kth module in the feature recovery module group FR and set the multilayer perceptron parameters to mlpr k ; Calculate the coordinate information SM according to formula (13) k and SM k-1 The Euclidean distance of:

[0089]

[0090] In this example, mlpr1 = [1024, 512, 512], mlpr2 = [512, 256, 256], mlpr3 = [256, 128, 128];

[0091] Step 4.7: For feature information FM k and FM K-1 , according to the dimension, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between the two is calculated according to formula (14):

[0092]

[0093] Step 4.8: Add DisXYZ and DisFM to get Dis, sort them from small to large, and take m of them. Normalize the weights according to formula (15):

[0094]

[0095] Wherein, in this example, m=4;

[0096] Step 4.9: Use weights with FM K-1 Multiply and then add to SM k-1 Perform splicing, perform multi-layer perceptron feature recovery according to formula (16), perform maximum pooling according to formula (17) and then perform splicing;

[0097] FREk=mlprk(Concat(Weights*FMk -1 ,SMk -1 )) (16)

[0098] FRF k =Concat(MAXPool(FRE k )) (17)

[0099] Step 4.10: FRF kMapped to categories, the segmentation results are obtained, as shown Figure 5 shown.

[0100] The contents described in the embodiments of this specification are merely an enumeration of implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A general lightweight component segmentation method for dense point clouds, characterized in that: The steps include: Step 1: Record the point cloud scan data as P = {p i =(x i ,y i ,z i )}, perform centering and scaling operations according to equations (1) and (2) to obtain point cloud Q; point cloud Q is shown in equation (3): Q={p i ″=(x i ″,y i ″,z i ″)} (3) Among them, (x i ′,y i ′,z i ′) represents the coordinate p of the i-th point in P after centralization i ′;(x i ″,y i ″,z i ″) is the coordinate p of the i-th point in P after centering and scaling i ″; Step 2: Adaptively downsample the point cloud Q to a multi-scale voxel grid to obtain the point cloud D; Step 3: Perform data enhancement operations on the point cloud D, perform random resizing, random translation, slight rotation, and random noise addition to obtain the point cloud DP; Step 4: Segment the point cloud DP into components.

2. A general lightweight component segmentation method for dense point clouds, characterized in that: The specific steps of step 2) are as follows: Step 2.1: Set the initial voxel group V, which contains n different voxels and the number of sampling target points N; Step 2.2: In the point cloud Q, find the minimum boundary value on each coordinate axis according to formula (4), recorded as MinBound: MinBound=(min{x′ i ′},min{y′ i ′},min{z′ i ′}) (4) Step 2.3: Traverse the initial voxel group V and calculate the voxel index of all points in the point cloud Q under the i-th voxel according to formula (5): Among them, V i is the i-th voxel in the initial voxel group V, p″-MinBound represents all points (x i ″,y i ″,z i ″) and the three values ​​in Minbound are subtracted accordingly; Step 2.4: Assign the points in the point cloud Q to the corresponding voxels VV according to the voxel index j , as shown in formula (6): VV j ={p′ i ′vIndex(p′ i ′)=j} (6) Where vIndex(p i ) indicates p i ″The corresponding voxel index; Step 2.5: For each voxel VV j , according to formula (7) to perform downsampling point d ij After merging, a new point cloud D is obtained. The number of points in the point cloud D is L, as shown in formula (8): D={D i 0≤i≤n} (8) Among them, VV j are all the points in voxel j, and n is the number of voxels in voxel group V; Step 2.6: Compare the number of points L of the point cloud D with the target number of points N; if L is greater than N, take the smallest voxel vm in the voxel group V, discard its sampled points from L, and reduce it by k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is greater than N, repeat step 2.6; if L plus the new sampling points is less than N, randomly sample to complete until L is equal to N; if L is less than N, then, enlarge it by k times, repeat steps 2.2 to 2.5, if L plus the new sampling points is less than N, repeat step 2.6; if L plus the new sampling points is greater than N, randomly discard until L is equal to N.

3. A general lightweight component segmentation method for dense point clouds, characterized in that: The specific steps of step 4) are as follows: Step 4.1: Set the feature extraction module group FE, the length of which is recorded as FEL; set the feature recovery module group FR, the length of which is recorded as FRL; Step 4.2: Select the i-th module in the feature extraction module group FE and set the number of sampling points to sample i , set the number of selected neighbors to neighbor i , set the multi-layer perceptron parameters to mlpe i ; Use the farthest point sampling to take sample i points, use the KNN nearest neighbor algorithm to select neighbor i neighbors, obtain the farthest neighbor distance DF and the nearest neighbor distance DN of the jth sampling point, which are divided into three resolutions according to formula (9): Among them, S r is the rth resolution in the scale group S; Step 4.3: With the jth sampling point as the center and the rth resolution as the radius, perform a spherical query of neighbors and collect neighbor coordinate information SM j and feature information FM j ; Step 4.4: Use formula (10) to stitch coordinate information SM j and feature information FM j Characteristic FEM j , according to formula (11) using multi-layer perceptron mlpe i Perform feature extraction to obtain FEE j ; FEM j =Concat(SM j ,FM j ) (10) FEE j =mlpe i (FEM j ) (11) Step 4.5: According to formula (12), FEE j After performing the maximum pooling, the concatenation is performed to obtain the feature FEE i ; Repeat steps 4.2 to 4.5 to complete feature extraction; FEE i =Concat(MAXPool(FEE j )) (12) Step 4.6: Select the kth module in the feature recovery module group FR and set the multilayer perceptron parameters to mlpr k ; Calculate the coordinate information SM according to formula (13) k and SM k-1 The Euclidean distance of: Step 4.7: For feature information FM k and FM K-1 , according to the dimension, the low-dimensional features are mapped to the high-dimensional features, and then the Euclidean distance between the two is calculated according to formula (14): Step 4.8: Add DisXYZ and DisFM to get Dis, sort them from small to large, and take m of them. Normalize the weights according to formula (15): Step 4.9: Use weights with FM K-1 Multiply and then add to SM k-1 Perform splicing, perform multi-layer perceptron feature recovery according to formula (16), perform maximum pooling according to formula (17) and then perform splicing; FREk=mlprk(Concat(Weights*FMk -1 ,SMk -1 )) (16) FRF k =Concat(MAXPool(FRE k )) (17) Step 4.10: FRF k Mapped to categories and obtain segmentation results.

Images