Unsupervised semantic segmentation method for large-scale garden point cloud

Abstract

The invention discloses an unsupervised semantic segmentation method for a large-scale garden point cloud. The unsupervised semantic segmentation method comprises the following steps: 1, super-point construction: constructing super-point representation of a garden scene by using a super voxel clustering and region growing method; 2, point cloud enhancement: carrying out data enhancement on the garden scene point cloud, and enhancing the representation of the garden point cloud; 3, pseudo label generation: extracting features of the garden point cloud through features, guiding super-point growth to form new super-points with a larger scale, adaptively clustering the garden super-point features into semantic primitives, endowing the points with pseudo labels, and forming a feature classifier by using primitive centers; 4, network training: classifying point features extracted by the garden point cloud feature extraction network by using a feature classifier to obtain point-by-point logic values, and training the network through pseudo labels and logic value loss; and 5, interactive segmentation: for large-scale garden point clouds, extracting garden point cloud features according to parameters and models set by a user, clustering clusters of a specified number of categories, displaying center points of the cluster categories, and after the user marks the center points, endowing all garden point clouds in the clusters with point-by-point category labels according to marks.

Description

Technical Field

[0001] The present invention relates to the field of point cloud semantic segmentation, and in particular to an unsupervised semantic segmentation method for large-scale garden point clouds. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] The present invention is a semantic segmentation method for garden point cloud data, which aims to enable the machine to automatically discover the semantic combination in the complex garden point cloud and distinguish it according to category. Since the large-scale garden scene point cloud with a large number of point data contains a variety of scenes and classical art examples of various forms (such as various rockery, strange stones, buildings, pools, vegetation, etc.) scattered in the garden scene, and there is a lack of examination and related research on garden scenes from the perspective of computer processing at home and abroad, the semantic segmentation of garden scene point clouds has become a daunting challenge.

[0004] Traditional semantic segmentation is achieved by extracting manually set feature descriptors and assigning parameterized algorithms to specific structural responses, such as Reference 1: Object tracking using SIFT features and mean shift, Reference 2: Wang X, Han TX, Yan S. An HOG-LBP human detector with partial occlusion handling [C] / / 2009 IEEE 12th international conference on computer vision. IEEE, 2009: 32-39. However, these methods are generally oriented towards relatively low-level semantic features and cannot capture high-level semantic information. Moreover, most of these methods are oriented towards two-dimensional images. More suitable methods need to be developed for disordered three-dimensional point clouds. In recent years, with the continuous advancement of deep learning technology, more and more mature deep learning methods have been applied to the field of point cloud semantic segmentation, such as Reference 3: Qi CR, Su H, Mo K, et al. Pointnet: Deep learning on point setsfor 3d classification and segmentation[C] / / Proceedings of the IEEE conferenceon computer vision and pattern recognition.2017:652-660., Reference 4: Thomas H, Qi CR, Deschaud JE, et al. Kpconv: Flexible and deformable convolution for pointclouds[C] / / Proceedings of the IEEE / CVF international conference on computervision.2019:6411-6420., Reference 5: [1] Yang Bo, Zhang Zihui. An unsupervised three-dimensional point cloud semantic segmentation method and related equipment[P]. Guangdong Province: CN202310540780.8, 2023-08-29. These methods have achieved remarkable results on public datasets. The great success of these deep learning-based point cloud semantic segmentation methods is mainly based on fully supervised methods with fully labeled data. The model can learn feature representations that are closer to the training set from the loss of annotation and regression results. However, these methods rely on precise annotation information and can only have good fitting results for public data scenarios.Since public datasets are generally composed of dense indoor point cloud data collected by images and sparse urban outdoor point cloud data collected by lidar, these methods can only be applied to these labeled indoor and urban street scenes. However, these methods cannot be effectively used in special scenes without fine annotations, such as large-scale garden scenes.

[0005] In addition, there are many objects and regions in the garden point cloud. The differences between objects and regions of different categories may be relatively small, and it is difficult to distinguish them using conventional features and methods, especially the high similarity with natural landscapes, which makes it difficult to distinguish them from some backgrounds. At the same time, the characteristics of objects and regions of the same category in different scenes are different, and they need to be abstracted to higher-level semantics to distinguish them. None of the existing methods can well obtain the point-by-point categories in the garden point cloud without point-by-point labels. There are also some semantic segmentation methods for garden point cloud scenes, such as document 6: Sun Zhengxing, Sun Yunhan, Luo Shoutong, etc. A context-driven large-scale garden scene unsupervised point cloud semantic segmentation method [P]. Jiangsu Province: CN202310907329.5, 2023-11-10., document 7: Ai Qiao, Zhou Weishi, Yao Yang, etc. A garden image quantization method based on semantic segmentation [P]. Chongqing City: CN202111332979.9, 2022-03-18., but these methods require fine garden annotation information to obtain a good semantic segmentation model. For gardens with complex structures and exquisite constructions, obtaining accurate point-by-point annotations requires extremely high manpower costs and expert knowledge. Moreover, due to different understandings of garden art, it is impossible to achieve granularity and standard consistency for large-scale garden point cloud annotations, which will cause the performance of deep learning methods on such data to drop sharply or even fail to train. For large-scale garden scene point clouds, the existing technology lacks the means to segment the model without accurate annotations. Summary of the invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is that the current large-scale garden scene point cloud cannot train the semantic segmentation model without accurate annotation. A method for unsupervised semantic segmentation of large-scale garden point cloud is proposed, which includes the following steps:

[0007] Step 1, super-point construction: Use the super-voxel segmentation method to divide a single garden scene point cloud into a set of super-points that go beyond the point cloud representation.

[0008] Step 2, point cloud preprocessing: In view of the poor performance of the original garden point cloud on the existing model, it is necessary to downsample and enhance the garden point cloud, so as to convert the large-scale garden scene point cloud into point cloud data that is more conducive to feature extraction network learning.

[0009] Step 3, superpoint growth clustering: After several rounds of training, the original superpoints are grown into a new superpoint set with fewer points and more points, so as to obtain more accurate semantic primitives. In order to cope with the complex garden environment, the superpoints are adaptively clustered into primitives with more semantic characteristics, superpoints and point-by-point pseudo labels are assigned, and the primitive centers are extracted as the classifiers of point features.

[0010] Step 4, network training: Use the classifier to find the logical value of each label, make a loss with the point-by-point pseudo-label, back-propagate to train the point cloud extraction network, and converge the garden point cloud feature extraction network through multiple rounds of iterative training.

[0011] Step 5, interactive segmentation: Due to the complexity and variability of garden point clouds and the different needs of users, this method uses the garden point cloud segmentation model trained in steps 1 to 4 to segment and cluster the target garden point cloud into primitives of different categories according to the parameters set by the user and obtain the center points of primitives of different categories, cluster the points in the primitives into the corresponding number of categories, and assign category names with category annotations.

[0012] Furthermore, step 1 includes the following steps:

[0013] Step 1-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B). First, convert its coordinates into floating point type, subtract the mean to center it, and quantize the coordinates according to the voxel block size Vs scale, which can be expressed as:

[0014] Coord1=floor(float(Coord)-mean(float(Coords))) / Vs)

[0015] Among them, Coord is the input original point cloud coordinates, Coords is the input coordinates of all original point clouds, mean(.) is the mean, float(.) is converted to floating point number, and floor(.) is rounded down.

[0016] Step 1-2, use the sparse voxelization method (sparse_quantize) in the reference 8: Choy C, Gwak JY, Savarese S.4d spatio-temporalconvnets: Minkowski convolutional neural networks[C] / / Proceedings ofthe IEEE / CVF conference on computer vision and pattern recognition.2019:3075-3084. to convert the original point cloud data into sparse voxelized point cloud That is, the continuous coordinate values ​​are discretized into a fixed grid.

[0017] In steps 1-3, supervoxel clustering is performed on the voxelized point cloud coordinates and RGB values. The point cloud is processed using the Voxel Cloud Connectivity Segmentation (VCCS) algorithm in the literature 9: Papon J, Abramov A, Schoeler M, et al. Voxel cloud connectivity segmentation-supervoxels for point clouds [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2013: 2027-2034., and a voxel index is assigned to each point.

[0018] In step 1-4, the voxelized point cloud is further clustered using the region growing algorithm. The coordinates are processed using the region growing method (region_growing) function in the literature 11: Rusu RB, Cousins ​​S. 3d is here: Point cloud library (pcl) [C] / / 2011 IEEE international conference on robotics and automation. IEEE, 2011: 1-4. to obtain the region clustered by region growing.

[0019] In steps 1-5, the results of supervoxel clustering and region growing clustering are merged. For each point in a voxel, the region to which it belongs is counted, and the majority region is assigned to all points in the voxel so that each point obtains a superpoint index. The merged superpoint labels are converted into continuous labels.

[0020] In step 1-6, the super-point labels are remapped to the original point cloud data according to the inverse index, and the most frequently appearing category in each super-point is counted as the category label of the super-point.

[0021] Further, step 2 includes the following steps:

[0022] Step 2-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B). Due to its large size, it is randomly cropped at the center and voxelized to obtain a sparse voxel representation of the original point cloud data.

[0023] Step 2-2, perform data enhancement operations on the sparse voxelized point cloud V2, and transform the voxelized point cloud V2 into an enhanced point cloud V3 consisting of standardized coordinates Coord2 and a feature vector F1 containing color and standardized coordinates through data enhancement operations such as translation, rotation, scaling and blending.

[0024] Step 2-3, if the preprocessed data is used for super point growth clustering in step 3, then obtain the normal set of the standardized coordinate Coord2, and delete the small area according to the threshold, remap the label of the valid area, and initialize the pseudo label; if the preprocessed data is used for point cloud feature extraction network training in step 4, then initialize the normal to the zero vector and load the pseudo label from the file.

[0025] Furthermore, the random center cropping voxelization in step 2-1 includes the following steps:

[0026] Step 2-1-1, for the point cloud P1 of the input scene, obtain the size of the scene (W1, H1, D1), that is, the difference between the maximum and minimum values ​​of the coordinates in three dimensions, which can be expressed by the formula:

[0027]

[0028] Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates within a certain range. The minimum range is the minimum value of each dimension of the original three-dimensional plus 1% of the cropping length, and the maximum range is the maximum value of each dimension of the original three-dimensional minus 1% of the cropping length. The formula can be expressed as:

[0029]

[0030] Where random(·) is a random function.

[0031] Step 2-1-3, check the cropping center of each dimension d, if the coordinates of this dimension are outside the original point cloud, the coordinates of this dimension of the cropping center are changed to randomly selected coordinates in this dimension of the original point cloud, which can be expressed as:

[0032]

[0033] where d i represents the i-th dimension, which mainly refers to the three dimensions XYZ here, and random(·) is a random function.

[0034] Step 2-1-4, determine the relationship between the scene size (W2, H2, D2) and the crop length CL. If the crop length is greater than the maximum value of W2, H2, D2, no cropping is performed and the entire scene is voxelized; otherwise, the point cloud is voxelized with the crop center Center. (X,Y,Z) As the center, clip all points within the sphere with a radius of CL and voxelize them. The voxelization method uses the voxelization method in steps 1-2.

[0035] Furthermore, the data enhancement method in step 2-2 includes the following steps:

[0036] Step 2-2-1, for the sparse voxelized point cloud V2, perform coordinate translation enhancement. Generate a random value between 0 and 1 in each dimension, then multiply the random value by the translation factor SR to obtain the translation amount S of the coordinate in each dimension. x ,S y ,S z , and then the coordinates of each dimension are added with the displacement to obtain the new point cloud V4, where the translation factor SR is set to 50.

[0037] Step 2-2-2: For the point cloud V4 after translation enhancement, perform coordinate rotation enhancement. For the three dimensions of the point cloud coordinates, set the rotation angle range RB of the three dimensions x ,RB y ,RB z , randomly generate three rotation angles in the rotation angle range, thereby generating three rotation matrices M x ,M y ,M z , where the rotation axis is set to 1. Then the three rotation matrices are multiplied in random order to obtain the final rotation matrix M, and the coordinates of the point cloud V4 are multiplied with the rotation matrix to obtain the rotation-enhanced point cloud V5, where the rotation angle range RB x ∈(-π / 32,π / 32), RB y ∈(-π / 32,π / 32), RB z ∈(-π,π).

[0038] Step 2-2-3, scale and enhance the coordinates of the rotation-enhanced point cloud V5. For the coordinates of the point cloud, set a scaling range SB, randomly select a number in the scaling range as the scaling factor, and multiply the coordinates of the point cloud V5 by the scaling factor to obtain the scaled and enhanced point cloud V6, where the scaling range SB∈(0.9,1.1).

[0039] Step 2-2-4: Perform point cloud mixing enhancement on the scaled and enhanced point cloud V6. First, randomly select an original point cloud P2∈R from all scene point clouds. (N×6) Then, the random center cropping voxelization process of step 2-1 is used to obtain the randomly cropped sparse voxelized point cloud V7, and then the methods of steps 2-2-1 to 2-2-3 are used to perform data enhancement and splice it with the point cloud V6 in the coordinate and color dimensions to obtain the mixed enhanced point cloud V8.

[0040] Step 2-2-5, for the hybrid enhanced point cloud V8, first calculate its center coordinates, and use the coordinates of all points to subtract the center coordinates to obtain the standardized coordinates Coord2. The color information and the standardized coordinates Goord2 are concatenated in the last dimension to form the final feature vector F1. The feature vector F1 and the standardized coordinates Goord2 form the enhanced point cloud V3.

[0041] Furthermore, step 3 super point growth clustering includes the following steps:

[0042] The superpoint growth clustering step is mainly divided into two stages: superpoint clustering and superpoint growth clustering. In the pre-set first E1 epochs (a training cycle of a complete training set), only superpoint clustering is performed; in the set last E2 epochs, superpoint growth clustering is performed.

[0043] Step 3-1, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, calculate the number of clusters required for the current super point according to the current training round and configuration. The calculation formula is expressed as:

[0044] CG=int(sp start -((epoch-epoch start ) / epoch max )*(sp start -sp end ))

[0045] Where CG is the current number of superpoint clusters, int(·) represents rounding, sp start represents the number of clusters starting from the superpoint, sp end Represents the number of clusters at the end of the superpoint, epoch is the current epoch, epoch maxTo set the maximum epoch of the current stage. If the calculated number of current superpoint clusters is less than sp end , then set CG to sp end .

[0046] Step 3-2, use the point cloud feature extraction network FEN to extract point features F2 and super point features F3.

[0047] Step 3-3, perform primitive clustering on the features according to the super-point features F3, and use the KMeans algorithm in the literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006. to perform primitive clustering on the features, and generate the primitive category label L1 and the point index I1 in the primitive. According to the primitive clustering results, a pseudo label PL is assigned to each point as the basis for training network parameters.

[0048] Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, assign it as a weight to a linear layer as a classifier C, and freeze the parameters of the classifier.

[0049] Step 3-5, adaptively adjust the number of clustering primitives according to the feature distance of points in different primitives and the feature distance between primitives, and obtain the number of primitives n for the next primitive clustering primitive .

[0050] Furthermore, the extraction of super-point features in step 3-2 includes the following steps:

[0051] Step 3-2-1, obtain the coordinate Coord2, feature F1, normal vector N, label and other information from the super point cloud and enhanced point cloud, and prepare some auxiliary variables.

[0052] In step 3-2-2, use the TensorField method in reference 13: GwakJY, Choy C, Savarese S. Generative sparsedetection networks for 3d single-shot object detection[C] / / Computer Vision–ECCV 2020:16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020:297-313. to construct the feature F1 and coordinate Coord2 as input tensors, use the point cloud feature extraction network to extract features, and filter out invalid areas.

[0053] Step 3-2-3, extract RGB and coordinate information from the features, then build a regional association matrix to indicate which superpoint each point belongs to, calculate the number of points of each superpoint, and finally calculate the average feature F5 of each superpoint.

[0054] Step 3-2-4, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, perform super point growth to obtain the super point index I used next time. SP .

[0055] Step 3-2-5, use the normal vector N to construct the normal vector histogram (PFH) feature of each superpoint as described in reference 14: Rusu RB, Marton ZC, Blodow N, et al. Persistent point feature histograms for 3D point clouds [M] / / Intelligent Autonomous Systems 10. IOS Press, 2008: 119-128.

[0056] Step 3-2-6, normalize the average features F5 and PFH of the super point, and combine the RGB and normal vector histogram features to obtain the super point feature F3.

[0057] Furthermore, the super point growth in step 3-2-4 includes the following steps:

[0058] Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each super point, and extract the weight parameter W of the set RGB, XYZ and normal vector RGB ,W XYZ ,W normal .

[0059] Step 3-2-4-2, normalize the average feature F5 of the super point and add it to the weight parameter W PGB ,W XYZ ,W normal The weighted RGB, XYZ and normal vectors are combined to obtain the super-point growing feature F6.

[0060] Step 3-2-4-3, extract the number of superpoints. If the current number of superpoint clusters CG is greater than the number of superpoints, set the number of growing clusters to the number of superpoints; otherwise, set the number of growing clusters to the number of superpoint clusters CG.

[0061] Step 3-2-4-4, according to the superpoint growth feature F6, use the K-means clustering method to cluster the superpoints, where the number of clusters is CG, and get the superpoint index I used next time SP K-means clustering method uses literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0062] Furthermore, the adaptive adjustment of the number of clustering primitives in step 3-5 includes the following steps:

[0063] Step 3-5-1, combine the point feature F2 and the primitive center feature F4 to calculate the average feature distance FD between the points in the primitive Point The average characteristic distance FD between primitives primitive , and its calculation formula is as follows:

[0064]

[0065] Where FD is the feature distance, n is the number of points or primitives, d(f i ,f j ) represents the Euclidean distance between point features or primitive center features.

[0066] Step 3-5-2, by judging the average characteristic distance FD between points Point The average characteristic distance FD between primitives primitive The relationship between them is determined by the following formula:

[0067]

[0068] where n primitive is the number of primitives, the initial value is 450, n1 is 10, and n2 is 1.

[0069] Furthermore, step 4 network training includes the following steps:

[0070] Step 4-1, load the data, including the point cloud. If it is the first training, initialize the network parameters, optimizer, loss, and learning rate scheduler.

[0071] Step 4-2, the TensorField method in the literature: Gwak JY, Choy C, Savarese S. Generative sparse detection networks for 3d single-shot object detection [C] / / Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV16. Springer International Publishing, 2020: 297-313. constructs the features and coordinates of the enhanced point cloud as input tensors, and uses the point cloud feature extraction network to extract features F7.

[0072] Step 4-3, use the classifier C obtained in step 3-3 to classify the feature F7 extracted by the network so that each point obtains a logical value Logits.

[0073] Step 4-4, use the pseudo-label PL and logical value Logits obtained in step 3-5 to calculate the loss. The loss calculation formula is as follows:

[0074] Loss=mean(CrossEntropyLoss(2*logits,PL))

[0075] Where Loss represents the loss function, mean(·) represents the average, and CrossEntropyLoss(·) represents the cross entropy.

[0076] Step 4-5, backpropagate to the point cloud feature extraction network according to the loss function and update the parameters to train the network.

[0077] Step 4-6, return to step 3, and perform super-point clustering or super-point growth clustering according to the epoch.

[0078] Furthermore, step 5 interactive segmentation includes the following steps:

[0079] Step 5-1: The user sets the hyperparameters, which mainly include the number of primitives n that need to be clustered P , the number of categories n that need to be segmented class .

[0080] Step 5-2: Use the model to segment the input garden point cloud into the set number of primitives n using the trained garden scene point cloud semantic segmentation model P The segmentation method is to use the feature extraction network to extract features and use the clustering method to cluster the points into n P primitives, and then cluster the primitives into n class Clusters, extract the center point set P of the features according to the clustering results fc , no need to construct super points. The clustering method uses the KMeans clustering algorithm in the literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0081] Step 5-3, change n class The feature center point set P of the cluster fc Display it to the user, and the user labels the categories of these points. By propagating the same cluster, all points in the garden point cloud can be divided into n class categories, thereby completing the semantic segmentation of the garden point cloud.

[0082] The present invention focuses on the characteristics of existing garden point cloud data and designs an unsupervised semantic segmentation method for large-scale garden point clouds. The point cloud feature extraction network can be trained without annotation to learn the semantic features of the point cloud through pseudo labels of superpoint clustering, and cluster it into point cloud blocks of different categories according to user needs when used. Superpoint is a point cloud representation method for dividing a point cloud into points beyond a single point proposed in document 10: Landrieu L, Simonovsky M. Large-scale point cloud semantic segmentation with superpoint graphs [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4558-4567. Each superpoint is a new sub-point cloud composed of multiple points. The present invention realizes unsupervised training of the point cloud feature extraction network on the basis of this method, and the garden scene point cloud semantic segmentation model is mainly composed of a trained point cloud feature extraction network and a clustering module.

[0083] The present invention divides the point cloud in the garden scene into super points, and clusters the super points into primitives with semantic pseudo labels by clustering the point cloud features and super point features extracted by the point cloud feature extraction network. The primitives can be considered as the basic units constituting different objects. Then, the clustered primitive centers are used as classifiers to classify the point cloud and assign point-by-point logical values ​​(logits), and the point cloud feature extraction network is trained by the loss of logical values ​​and pseudo labels, and the previous step clustering is returned for continuous iteration. Then, when the point cloud feature extraction network has reached a certain convergence, the super points are allowed to grow continuously according to the scene requirements, so that the super points contain more and more points, so that the super points are getting closer and closer to the semantic primitives, and the primitives are getting closer and closer to objects of different categories, and finally a garden point cloud feature extraction network that can distinguish different categories of point cloud features can be obtained. In actual use, there is no need to reconstruct the super points, and the feature center points of the semantic primitives are directly obtained through the garden scene point cloud semantic segmentation model, and the primitives are clustered into different semantic categories through the user's annotation of the center, thereby realizing the semantic segmentation of large-scale garden scene point clouds.

[0084] The present invention does not require additional annotated data when training the model. When in use, the user only needs to provide the number of semantic categories and the semantic labels of a very small number of points, which greatly reduces the training cost and usage threshold of the model. The present invention can also adaptively adjust the granularity of semantic segmentation during training according to the actual situation of the scene, so as to adapt to complex and delicate garden scenes, so that the model can better respond to the user's requirements for semantic segmentation of different granularities of large-scale garden scene point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0086] Figure 1 It is a processing flow diagram of the present invention.

[0087] Figure 2 It is the input point cloud visualization result.

[0088] Figure 3 It is the visualization effect of constructing the input point cloud into super points.

[0089] Figure 4 It is a visualization result that requires users to annotate points in the point cloud.

[0090] Figure 5 It is the visualization result of the final semantic segmentation result. DETAILED DESCRIPTION

[0091] like Figure 1As shown, the unsupervised semantic segmentation method of a large-scale garden point cloud disclosed in the present invention is specifically implemented according to the following steps:

[0092] Step 1, super-point construction: Use the super-voxel segmentation method to divide a single garden scene point cloud into a set of super-points that go beyond the point cloud representation.

[0093] Step 2, point cloud preprocessing: downsample and enhance the original point cloud to convert the large-scale garden scene point cloud into point cloud data that is more conducive to feature extraction network learning.

[0094] Step 3, pseudo-label generation: After several rounds of training, the original superpoints are grown into a new superpoint set with fewer points and more points, so as to obtain more accurate semantic primitives. The superpoints are clustered into primitives with more semantic characteristics, and pseudo-labels are assigned to the superpoints and point by point. The primitive centers are extracted as classifiers of point features.

[0095] Step 4, network training: Use the classifier to find the logical value of each label, make a loss with the point-by-point pseudo-label, back-propagate to train the point cloud extraction network, and converge the garden point cloud feature extraction network through multiple rounds of iterative training.

[0096] Step 5, interactive segmentation: according to the parameters set by the user, the garden point cloud segmentation model trained in steps 1 to 4 is used to segment the primitives of the target garden, cluster the points in the primitives into a corresponding number of categories, and assign category names to the category labels.

[0097] Step 1 includes the following steps:

[0098] Step 1-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B), such as Figure 2 As shown, first convert its coordinates into floating point type, subtract the mean to center it, and quantize the coordinates according to the voxel block size Vs scale, which can be expressed as:

[0099] Coord1=floor(float(Coord)-mean(float(Coords))) / Vs)

[0100] Among them, Coord is the input original point cloud coordinates, Coords is the input coordinates of all original point clouds, mean(·) is the mean, float(·) is converted to floating point number, and floor(·) is rounded down.

[0101] Step 1-2, use the sparse voxelization method (sparse_quantize) in the reference 8: Choy C, Gwak JY, Savarese S.4d spatio-temporalconvnets: Minkowski convolutional neural networks[C] / / Proceedings ofthe IEEE / CVF conference on computer vision and pattern recognition.2019:3075-3084. to convert the original point cloud data into sparse voxelized point cloud That is, the continuous coordinate values ​​are discretized into a fixed grid.

[0102] In steps 1-3, supervoxel clustering is performed on the voxelized point cloud coordinates and RGB values. The point cloud is processed using the Voxel Cloud Connectivity Segmentation (VCCS) algorithm in the literature 9: Papon J, Abramov A, Schoeler M, et al. Voxel cloud connectivity segmentation-supervoxels for point clouds [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2013: 2027-2034., and a voxel index is assigned to each point.

[0103] In step 1-4, the voxelized point cloud is further clustered using the region growing algorithm. The coordinates are processed using the region growing method (region_growing) function in the literature 11: Rusu RB, Cousins ​​S. 3d is here: Point cloud library (pcl) [C] / / 2011 IEEE international conference on robotics and automation. IEEE, 2011: 1-4. to obtain the region clustered by region growing.

[0104] In steps 1-5, the results of supervoxel clustering and region growing clustering are merged. For each point in a voxel, the region to which it belongs is counted, and the majority region is assigned to all points in the voxel so that each point obtains a superpoint index. The merged superpoint labels are converted into continuous labels.

[0105] Step 1-6, remap the superpoint label to the original point cloud data according to the inverse index, count the most common categories in each superpoint, and use them as the category label of the superpoint. The generated superpoint division is as follows: Figure 3 shown.

[0106] Step 2 includes the following steps:

[0107] Step 2-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B). Due to its large size, it is randomly cropped at the center and voxelized to obtain a sparse voxel representation of the original point cloud data.

[0108] Step 2-2, perform data enhancement operations on the sparse voxelized point cloud V2, and transform the voxelized point cloud V2 into an enhanced point cloud V3 consisting of standardized coordinates Coord2 and a feature vector F1 containing color and standardized coordinates through data enhancement operations such as translation, rotation, scaling and blending.

[0109] Step 2-3, if the preprocessed data is used for super point growth clustering in step 3, then obtain the normal set of the standardized coordinate Coord2, and delete the small area according to the threshold, remap the label of the valid area, and initialize the pseudo label; if the preprocessed data is used for point cloud feature extraction network training in step 4, then initialize the normal to the zero vector and load the pseudo label from the file.

[0110] The random center crop voxelization in step 2-1 includes the following steps:

[0111] Step 2-1-1, for the point cloud P1 of the input scene, obtain the size of the scene (W2, H2, D2), that is, the difference between the maximum and minimum values ​​of the coordinates in three dimensions, which can be expressed by the formula:

[0112]

[0113] Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates within a certain range. The minimum range is the minimum value of each dimension of the original three-dimensional plus 1% of the cropping length, and the maximum range is the maximum value of each dimension of the original three-dimensional minus 1% of the cropping length. The formula can be expressed as:

[0114]

[0115] Where random(·) is a random function.

[0116] Step 2-1-3, check the cropping center of each dimension d, if the coordinates of this dimension are outside the original point cloud, the coordinates of this dimension of the cropping center are changed to randomly selected coordinates in this dimension of the original point cloud, which can be expressed as:

[0117]

[0118] where d i represents the i-th dimension, which mainly refers to the three dimensions XYZ here, and random(·) is a random function.

[0119] Step 2-1-4, determine the relationship between the scene size (W2, H2, D2) and the crop length CL. If the crop length is greater than the maximum value of W2, H2, D2, no cropping is performed and the entire scene is voxelized; otherwise, the point cloud is voxelized with the crop center Center. (X,Y,Z) As the center, clip all points within the sphere with a radius of CL and voxelize them. The voxelization method uses the voxelization method in steps 1-2.

[0120] The data enhancement method in step 2-2 includes the following steps:

[0121] Step 2-2-1, for the sparse voxelized point cloud V2, perform coordinate translation enhancement. Generate a random value between 0 and 1 in each dimension, then multiply the random value by the translation factor SR to obtain the translation amount S of the coordinate in each dimension. x ,S y ,S z , and then the coordinates of each dimension are added with the displacement to obtain the new point cloud V4, where the translation factor SR is set to 50.

[0122] Step 2-2-2: For the point cloud V4 after translation enhancement, perform coordinate rotation enhancement. For the three dimensions of the point cloud coordinates, set the rotation angle range RB of the three dimensions x ,RB y ,RB z , randomly generate three rotation angles in the rotation angle range, thereby generating three rotation matrices M x ,M y ,M z , where the rotation axis is set to 1. Then the three rotation matrices are multiplied in random order to obtain the final rotation matrix M, and the coordinates of the point cloud V4 are multiplied with the rotation matrix to obtain the rotation-enhanced point cloud V5, where the rotation angle range RB x ∈(-π / 32,π / 32), RB y ∈(-π / 32,π / 32), RB z ∈(-π,π).

[0123] Step 2-2-3, scale and enhance the coordinates of the rotation-enhanced point cloud V5. For the coordinates of the point cloud, set a scaling range SB, randomly select a number in the scaling range as the scaling factor, and multiply the coordinates of the point cloud V5 by the scaling factor to obtain the scaled and enhanced point cloud V6, where the scaling range SB∈(0.9,1.1).

[0124] Step 2-2-4: Perform point cloud mixing enhancement on the scaled and enhanced point cloud V6. First, randomly select an original point cloud P2∈R from all scene point clouds. (N×6) Then, the random center cropping voxelization process of step 2-1 is used to obtain the randomly cropped sparse voxelized point cloud V7, and then the methods of steps 2-2-1 to 2-2-3 are used to perform data enhancement and splice it with the point cloud V6 in the coordinate and color dimensions to obtain the mixed enhanced point cloud V8.

[0125] Step 2-2-5, for the hybrid enhanced point cloud V8, first calculate its center coordinates, and use the coordinates of all points to subtract the center coordinates to obtain the standardized coordinates Coord2. The color information and the standardized coordinates Coord2 are concatenated in the last dimension to form the final feature vector F1. The feature vector F1 and the standardized coordinates Coord2 form the enhanced point cloud V3.

[0126] Step 3 includes the following steps:

[0127] The superpoint growth clustering step is mainly divided into two stages: superpoint clustering and superpoint growth clustering. In the pre-set first E1 epochs (a training cycle of a complete training set), only superpoint clustering is performed; in the set last E2 epochs, superpoint growth clustering is performed.

[0128] Step 3-1, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, calculate the number of clusters required for the current super point according to the current training round and configuration. The calculation formula is expressed as:

[0129] CG=int(sp start -((epoch-epoch start ) / epoch max )*(sp start -sp end ))

[0130] Where CG is the current number of superpoint clusters, int(·) represents rounding, sp start represents the number of clusters starting from the superpoint, sp end Represents the number of clusters at the end of the superpoint, epoch is the current epoch, epoch maxTo set the maximum epoch of the current stage. If the calculated number of current superpoint clusters is less than sp end , then set CG to sp end .

[0131] Step 3-2, use the point cloud feature extraction network FEN to extract point features F2 and super point features F3.

[0132] Step 3-3, perform primitive clustering on the features according to the super-point features F3, and use the KMeans algorithm in the literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006. to perform primitive clustering on the features, and generate the primitive category label L1 and the point index I1 in the primitive. According to the primitive clustering result, a pseudo label PL is assigned to each point as the basis for training network parameters.

[0133] Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, assign it as a weight to a linear layer as a classifier C, and freeze the parameters of the classifier.

[0134] Step 3-5, adaptively adjust the number of clustering primitives according to the feature distance of points in different primitives and the feature distance between primitives, and obtain the number of primitives n for the next primitive clustering primitive .

[0135] The extraction of super-point features in step 3-2 includes the following steps:

[0136] Step 3-2-1, obtain the coordinate Coord2, feature F1, normal vector N, label and other information from the super point cloud and enhanced point cloud, and prepare some auxiliary variables.

[0137] In step 3-2-2, use the TensorField method in reference 13: GwakJY, Choy C, Savarese S. Generative sparsedetection networks for 3d single-shot object detection[C] / / Computer Vision–ECCV 2020:16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020:297-313. to construct the feature F1 and coordinate Coord2 as input tensors, use the point cloud feature extraction network to extract features, and filter out invalid areas.

[0138] Step 3-2-3, extract RGB and coordinate information from the features, then build a regional association matrix to indicate which superpoint each point belongs to, calculate the number of points of each superpoint, and finally calculate the average feature F5 of each superpoint.

[0139] Step 3-2-4, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, perform super point growth to obtain the super point index I used next time. SP .

[0140] Step 3-2-5, use the normal vector N to construct the normal vector histogram (PFH) feature of each superpoint as described in reference 14: Rusu RB, Marton ZC, Blodow N, et al. Persistent point feature histograms for 3D point clouds [M] / / Intelligent Autonomous Systems 10. IOS Press, 2008: 119-128.

[0141] Step 3-2-6, normalize the average features F5 and PFH of the super point, and combine the RGB and normal vector histogram features to obtain the super point feature F3.

[0142] The super point growth in step 3-2-4 includes the following steps:

[0143] Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each super point, and extract the weight parameter W of the set RGB, XYZ and normal vector RGB ,W XYZ ,W normal .

[0144] Step 3-2-4-2, normalize the average feature F5 of the super point and add it to the weight parameter W RGB ,W XYZ ,W normal The weighted RGB, XYZ and normal vectors are combined to obtain the super-point growing feature F6.

[0145] Step 3-2-4-3, extract the number of superpoints. If the current number of superpoint clusters CG is greater than the number of superpoints, set the number of growing clusters to the number of superpoints; otherwise, set the number of growing clusters to the number of superpoint clusters CG.

[0146] Step 3-2-4-4, according to the superpoint growth feature F6, use the K-means clustering method to cluster the superpoints, where the number of clusters is CG, and get the superpoint index I used next time SP K-means clustering method uses literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0147] The adaptive adjustment of the number of clustering primitives in step 3-5 includes the following steps:

[0148] Step 3-5-1, combine the point feature F2 and the primitive center feature F4 to calculate the average feature distance FD between the points in the primitive Point The average characteristic distance FD between primitives primitive , and its calculation formula is as follows:

[0149]

[0150] Where FD is the feature distance, n is the number of points or primitives, d(f i ,f j ) represents the Euclidean distance between point features or primitive center features.

[0151] Step 3-5-2, by judging the average characteristic distance FD between points Point The average characteristic distance FD between primitives primitive The relationship between them is determined by the following formula:

[0152]

[0153] where n primitive is the number of primitives, the initial value is 450, n1 is 10, and n2 is 1.

[0154] Step 4 includes the following steps:

[0155] Step 4-1, load the data, including the point cloud. If it is the first training, initialize the network parameters, optimizer, loss, and learning rate scheduler.

[0156] Step 4-2, the TensorField method in the literature: Gwak JY, Choy C, Savarese S. Generative sparse detection networks for 3d single-shot object detection[C] / / Computer Vision–ECCV 2020:16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV16. Springer International Publishing, 2020:297-313. constructs the features and coordinates of the enhanced point cloud as input tensors, and uses the point cloud feature extraction network to extract features F7.

[0157] Step 4-3, use the classifier C obtained in step 3-3 to classify the feature F7 extracted by the network, so that each point obtains a logical value Logits.

[0158] Step 4-4, use the pseudo-label PL and logical value Logits obtained in step 3-5 to calculate the loss. The loss calculation formula is as follows:

[0159] Loss=mean(CrossEntropyLoss(2*logits,PL))

[0160] Where Loss represents the loss function, mean(·) represents the average, and CrossEntropyLoss(·) represents the cross entropy.

[0161] Step 4-5, use the multi-label classification loss function (MultiLabelMarginLoss) in PyTorch to backpropagate to the point cloud feature extraction network and update the parameters to train the network.

[0162] Step 4-6, return to step 3, and perform super-point clustering or super-point growth clustering according to the epoch.

[0163] Step 5 includes the following steps:

[0164] Step 5-1: The user sets the hyperparameters, which mainly include the number of primitives n that need to be clustered P , the number of categories n that need to be segmented class .

[0165] Step 5-2: Use the model to segment the input garden point cloud into the set number of primitives n using the trained garden scene point cloud semantic segmentation model P The segmentation method is to use the feature extraction network to extract features and use the clustering method to cluster the points into n P primitives, and then cluster the primitives into n class Clusters, extract the center point set P of the features according to the clustering results fc ,like Figure 4 As shown, there is no need to construct super points. The clustering method uses the KMeans clustering algorithm in the literature 12: Arthur D, Vassilvitskii Sk-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0166] Step 5-3, change n class The feature center point set P of the cluster fc Display it to the user, and the user labels the categories of these points. By propagating the same cluster, all points in the garden point cloud can be divided into n class categories, thereby completing the semantic segmentation of the garden point cloud. The segmentation results are as follows Figure 5 shown.

[0167] The present invention provides an unsupervised semantic segmentation method for large-scale garden point clouds. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. An unsupervised semantic segmentation method for large-scale garden point clouds, characterized in that: The steps include: Step 1, super-point construction: Use the super-voxel segmentation method to divide the garden scene point cloud into a super-point set; Step 2, point cloud preprocessing: downsampling and data enhancement processing are performed on the original point cloud to transform the garden scene point cloud into enhanced point cloud data that is conducive to feature extraction network learning; Step 3, superpoint growth clustering: using the superpoint set obtained in step 1 and the enhanced point cloud obtained in step 2, cluster the superpoints into primitives, assign superpoints and point-by-point pseudo labels, extract the primitive centers as classifiers of point features, train the data and obtain a new superpoint set; Step 4, network training: use the classifier obtained in step 3 to calculate the point-by-point label logic value, and make a loss with the point-by-point pseudo-label obtained in step 3, back-propagate and train the point cloud extraction network, return to step 3 and generate a garden point cloud feature extraction network through multiple rounds of iterations; Step 5, interactive segmentation: according to the parameters set by the user, the garden point cloud segmentation model trained in steps 1 to 4 is used to segment the primitives of the target garden, cluster the points in the primitives into a corresponding number of categories, and assign category names to the category labels.

2. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 1, characterized in that: Step 1 Superpoint construction includes the following steps: Step 1-1, for an input garden scene point cloud, standardize it according to the voxel block size; Step 1-2, sparse voxelizing the garden point cloud obtained in step 1-1; Step 1-3, using the point cloud voxel connectivity segmentation algorithm to perform supervoxel clustering on the voxelized point cloud coordinates and RGB values ​​obtained in step 1-2; Step 1-4, using the region growing algorithm to further cluster the voxelized point cloud obtained in step 1-2; Step 1-5, merging the results of the supervoxel clustering obtained in step 1-3 and the region growing clustering obtained in step 1-4, so that each point is in a superpoint; Step 1-6, assign a continuous superpoint label to each superpoint obtained in step 1-5.

3. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 1, characterized in that: Step 2 Point cloud preprocessing includes the following steps: Step 2-1, for an input garden scene point cloud, perform random center cropping and then voxelization; Step 2-2, performing a data enhancement operation on the voxelized point cloud obtained in step 2-1, converting the voxelized point cloud into an enhanced point cloud consisting of standardized coordinates and feature vectors, wherein the feature vectors include colors and standardized coordinates; In step 2-3, the preprocessed data is processed for subsequent steps, the normal set of the standardized coordinates obtained in step 2-2 is obtained, and small areas are deleted according to the threshold, the labels of the valid areas are remapped, and the pseudo labels are initialized for super point growth clustering in step 3; the normal of the standardized coordinates obtained in step 2-2 is initialized to a zero vector, and the pseudo labels are loaded from the file for point cloud feature extraction network training in step 4.

4. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 3, characterized in that: The random center cropping of voxelized point cloud data in step 2-1 includes the following steps: Step 2-1-1, for the point cloud P1 of the input scene, obtain the size of the scene (W1, H1, D1), which is the difference between the maximum and minimum values ​​of the coordinates in the three dimensions, which can be expressed by the formula: Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates within a certain range. The minimum range is the minimum value of each dimension of the original three-dimensional plus one percent of the cropping length, and the maximum range is the maximum value of each dimension of the original three-dimensional minus one percent of the cropping length. It can be expressed as follows: Where random(·) is a random function, Center x , Center y , Center z is the center coordinate value; Step 2-1-3, check the cropping center of each dimension d , if the coordinates of this dimension are outside the original point cloud, the coordinates of this dimension of the cropping center are changed to randomly selected coordinates in this dimension of the original point cloud, which can be expressed as: where d i represents dimension, mainly XYZ dimensions here, and random(·) is a random function; Step 2-1-4, determine the relationship between the scene size (W2, H2, D2) and the crop length CL. If the crop length is greater than the maximum value of W2, H2, D2, no cropping is performed and the entire scene is voxelized; otherwise, the point cloud is voxelized with the crop center Center. (X,Y,Z) As the center, clip all points within the sphere with a radius of CL and voxelize them. The voxelization method uses the voxelization method in steps 1-2.

5. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 3, characterized in that: The data enhancement method in step 2-2 includes the following steps: Step 2-2-1, for the sparse voxelized point cloud V2, the coordinates are translated and enhanced, a random value is generated in each dimension, and the random value is multiplied by the translation factor SR to obtain the translation amount S of the coordinates in each dimension. x , S y , S z , then add the displacement to the coordinates of each dimension to obtain the new point cloud V4; Step 2-2-2: For the point cloud V4 after translation enhancement, the coordinate rotation is enhanced. For the three dimensions of the point cloud coordinates, the rotation angle range RB of the three dimensions is set. x , R.B. y and RB z , randomly generate three rotation angles in the rotation angle range, thereby generating three rotation matrices M x , M y , M z , where the rotation axis is set to 1, the three rotation matrices are multiplied in a random order to obtain the final rotation matrix M, and the coordinates of the point cloud V4 obtained in the previous step are multiplied by the rotation matrix to obtain the rotation-enhanced point cloud V5. Step 2-2-3, scale and enhance the coordinates of the rotation-enhanced point cloud V5. For the coordinates of the point cloud, set a scaling range SB, randomly select a value in the scaling range as the scaling factor, and multiply the coordinates of the point cloud V5 by the scaling factor to obtain the scaled and enhanced point cloud V6. In step 2-2-4, the point cloud V6 after scaling enhancement is subjected to point cloud hybrid enhancement. An original point cloud is randomly selected from the scene point cloud, and the random center cropping voxelization process of step 2-1 is used to obtain a randomly cropped sparse voxelized point cloud V7. Then, the data is enhanced using the methods of steps 2-2-1 to 2-2-3, and it is concatenated with the point cloud V6 in the coordinate and color dimensions to obtain a hybrid enhanced point cloud V8. Step 2-2-5, for the mixed enhanced point cloud V8, its coordinates are standardized, and the color information and the standardized coordinates Coord2 are concatenated to form the final feature vector F1. The feature vector F1 and the standardized coordinates Coord2 constitute the enhanced point cloud V3.

6. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 1, characterized in that: Step 3: Super point growth clustering includes the following steps: The superpoint growth clustering step includes superpoint clustering and superpoint growth clustering. In the first E1 epochs, that is, in a training cycle of a complete training set, only superpoint clustering is performed; in the next E2 epochs, superpoint growth clustering is performed; Step 3-1, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, calculate the number of clusters required for the current super point according to the current training round and configuration. The calculation formula is expressed as: CG=int(sp start -((epoch-epoch start ) / epoch max )*(sp start -sp end )); Where CG is the current number of superpoint clusters, int(·) represents rounding, sp start represents the number of clusters starting from the superpoint, sp end Represents the number of clusters at the end of the superpoint, epoch is the current epoch, epoch max To set the maximum epoch of the current stage, if the calculated number of current superpoint clusters is less than spe nd , then set CG to spen d ; Step 3-2, using the point cloud feature extraction network FEN to extract point features F2 and super point features F3; Step 3-3, perform primitive clustering on the features obtained in step 3-2, and assign a pseudo label PL to each point according to the primitive clustering result as the basis for training network parameters; Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, assign it as a weight to a linear layer as a classifier C, and freeze the parameters of the classifier; Step 3-5, adaptively adjust the number of clustering primitives according to the feature distance of points in different primitives and the feature distance between primitives, and obtain the number of primitives n for the next primitive clustering primitive .

7. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 6, characterized in that: The extraction of super-point features in step 3-2 includes the following steps: Step 3-2--1, obtain coordinates, features, normal vectors and label information from super point cloud and enhanced point cloud; Step 3-2-2, construct the features and coordinates as input tensors, use the point cloud feature extraction network FEN to extract features, and filter invalid areas; Step 3-2-3, extract RGB and coordinate information from the features, construct a regional association matrix to represent the superpoint to which each point belongs, calculate the number of points of each superpoint, and then calculate the average feature F5 of each superpoint; Step 3-2-4, determine whether it is in the super point growth clustering stage. If it is in the super point growth clustering stage, perform super point growth to obtain the super point index I used next time. SP ; Step 3-2-5, use the normal vector N to construct the normal vector histogram feature PFH of each super point; Step 3-2-6, normalize the average features F5 and PFH of the super point, and combine the RGB and normal vector histogram features to obtain the super point feature F3. The super point growth in step 3-2-4 comprises the following steps: Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each super point, and extract the weight parameter W of the set RGB, XYZ and normal vector RaB ,W XYZ ,W normal , where W RaB =0.9~1.2,W XYZ =0.5~0.7,W normal =0.3~0.5: Step 3-2-4-2, normalize the average feature F5 of the super point and add it to the weight parameter W RGB ,W XYZ , W normal The weighted RGB, XYZ and normal vectors are combined to obtain the super point growth feature F6; Step 3-2-4-3, extract the number of superpoints. If the current number of superpoint clusters CG is greater than the number of superpoints, set the number of growing clusters to the number of superpoints; otherwise, set the number of growing clusters to the number of superpoint clusters CG. Step 3-2-4-4, cluster the superpoints according to the superpoint growth feature F6, where the number of clusters is CG, and obtain the superpoint index I used next time SP .

8. The unsupervised semantic segmentation method for large-scale garden point clouds according to claim 6, characterized in that: The adaptive adjustment of the number of clustering primitives in step 3-5 includes the following steps: Step 3-5-1, combine the point feature F2 and the primitive center feature F4 to calculate the average feature distance FD between the points in the primitive Point The average characteristic distance FD between primitives primitive , and its calculation formula is as follows: Where FD is the feature distance, n is the number of points or primitives, d(f i , f j ) represents the Euclidean distance between point features or primitive center features; Step 3-5-2, by judging the average characteristic distance FD between points Point The average characteristic distance FD between primitives primitive The relationship between the number of primitives n primitive Adjustment is made, and the judgment formula is as follows: where n primitive is the number of primitives, the initial value is 450, n1 is 10, and n2 is 1.

9. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 1, characterized in that: Step 4 Network training includes the following steps: Step 4-1, load data, including point cloud. If it is the first training, initialize network parameters, optimizer, loss, and learning rate scheduler; Step 4-2, construct the features and coordinates of the enhanced point cloud as an input tensor, and use the point cloud feature extraction network FEN to extract feature F7; Step 4-3, use the classifier C obtained in step 3-3 to classify the feature F7 extracted by the network, so that each point gets a logical value Logits; Step 4-4, use the pseudo-label PL and logical value Logits obtained in step 3-5 as loss, and the loss calculation formula is as follows: Loss=mean(CrossEntropyLoss(2*logits,PL)) Where Loss is the loss function, mean(·) represents the average, and CrossEntropyLoss(·) represents the cross entropy; Step 4-5, back propagate to the point cloud feature extraction network according to the loss function and update the parameters to train the network; Step 4-6, return to step 3, perform super-point clustering or super-point growth clustering according to the epoch, and iterate steps 3 and 4 until E1+E2 epochs are executed, where E1 is the epoch number of super-point clustering and E2 is the epoch number of super-point growth clustering.

10. The unsupervised semantic segmentation method of large-scale garden point cloud according to claim 1, characterized in that: Step 5 Interactive segmentation includes the following steps: Step 5-1, the user sets the hyperparameters, including the number of primitives n to be clustered P And the number of categories n that need to be segmented class ; Step 5-2: Use the model to segment the input garden point cloud into the set number of primitives n using the trained garden scene point cloud semantic segmentation model P , and cluster the primitives into n class clusters, and obtain the feature center point set of each cluster; Step 5-3, change n class The characteristic center point set of each cluster is displayed to the user, and the user labels the categories of these points. By propagating the same cluster, all points in the garden point cloud can be divided into n class categories, thereby completing the semantic segmentation of the garden point cloud.

Images