An unsupervised semantic segmentation method for large-scale garden point cloud

Abstract

The application discloses a kind of large-scale garden point cloud unattended semantic segmentation method, comprising the following steps:1.supernode construction: using super voxel clustering and region growing method constructs the superpoint representation of garden scene;2.point cloud enhancement: the data enhancement is done to garden scene point cloud, and the representation of garden point cloud is enhanced;3.pseudo-label generation: by extracting the features of garden point cloud, guide superpoint grow into larger new superpoint, using garden superpoint feature adaptive clustering into semantic primitive, thereby giving point pseudo-label, and using primitive center constitutes feature classifier;4.network training: using feature classifier to the point feature classification that garden point cloud feature extraction network extracts, obtains the logic value of point by point, and trains network by pseudo-label and logic value loss;5.interactive segmentation: face large-scale garden point cloud, according to the parameters set by user, model extracts garden point cloud feature and clusters out the cluster of specified number of categories and shows the center point of cluster category, after user labels center point, according to labeling gives the class label of all garden point cloud in cluster point by point.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of point cloud semantic segmentation, and particularly relates to an unsupervised semantic segmentation method for large-scale garden point cloud. BACKGROUND

[0002] This part provides only background information related to the present disclosure, which is not necessarily prior art.

[0003] Suzhou gardens, as a world-renowned Chinese cultural heritage, is the artistic gem of Chinese classical splendid civilization, and has a high reputation at home and abroad. Its artistic characteristics of "though made by people, it seems to be opened by nature" and "imitating nature" and gardening techniques are unique in the history of world gardens. The garden point cloud data generated by digitalization also has a high research value. The present application is a semantic segmentation method for garden point cloud data, aiming to enable the machine to automatically discover the semantic combination in the complex garden point cloud, and to distinguish them according to the categories. Due to the large number of large-scale garden scene point clouds containing a large amount of point data, various scenes and various forms of classical art examples (such as various rockeries, strange stones, buildings, water pools, vegetation, etc.) are scattered in the garden scene, and there is a lack of computer processing perspective on the garden scene and related research at home and abroad, which makes the semantic segmentation of the garden scene point cloud a great challenge.

[0004] Traditional semantic segmentation is through the extraction of manually set feature descriptors and the parameterization algorithm response to specific structures, such as document 1: Object tracking using SIFT features and mean shift, document 2: Wang X, Han T X, Yan S. An HOG-LBP human detector with partial occlusion handling [C] / / 2009IEEE 12th international conference on computer vision. IEEE, 2009: 32-39. However, these methods are generally low-level semantic features, and cannot capture high-level semantic information. Moreover, these methods are mostly two-dimensional image-oriented, and need to develop more suitable methods for unordered three-dimensional point clouds. In recent years, with the continuous progress of deep learning technology, more and more mature deep learning methods have been applied to point cloud semantic segmentation, such as document 3: Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 652-660. Document 4: Thomas H, Qi CR, Deschaud J E, et al. Kpconv: Flexible and deformable convolution for point clouds [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019: 6411-6420. Document 5: [1] Yang B, Zhang Z H. An unsupervised three-dimensional point cloud semantic segmentation method and related equipment [P]. Guangdong: CN202310540780.8, 2023-08-29. These methods have achieved outstanding results on public data sets. The great success of these deep learning-based point cloud semantic segmentation methods is mainly based on fully supervised methods with complete labeled data. The model can learn a feature representation closer to the training set from the loss of the labeled and regressed results. However, these methods rely on accurate labeling information and can only have good fitting results for the scenes of public data.Since public datasets are generally dense indoor point clouds collected by image acquisition and sparse outdoor point clouds collected by lidar, these methods can only be applied to these labeled indoor and urban street scenes, but in the face of special scenes such as large-scale garden scenes without fine labeling, these methods cannot be effectively used.

[0005] In addition, there are many objects and regions in the garden point cloud, and the differences between objects and regions of different categories may be small, which is difficult to distinguish with conventional features and methods, especially with high similarity to natural landscapes, which is difficult to distinguish from some background. At the same time, the characteristics of the same category of objects and regions in different scenes are different, and need to be abstracted to higher semantics for distinction. None of the existing methods can well obtain the point-by-point category in the garden point cloud without point-by-point labels. There are also some semantic segmentation methods for garden point cloud scenes, such as document 6: Sun Zhengxing, Sun Yunhan, Luo Shoutong, et al. A context-driven unsupervised point cloud semantic segmentation method for large-scale garden scenes [P]. Jiangsu province: CN202310907329.5, 2023-11-10, document 7: Ai Qiao, Zhou Weishi, Yao Yang, et al. A garden image quantification method based on semantic segmentation [P]. Chongqing: CN202111332979.9, 2022-03-18, but these methods need fine garden labeling information to obtain a good semantic segmentation model. For complex and delicate garden structures, obtaining accurate point-by-point labeling requires high labor costs and expert knowledge. Moreover, due to different understandings of garden art, large-scale garden point cloud labeling cannot achieve consistency in granularity and standard, which will cause the performance of deep learning methods on such data to drop sharply or even fail to train. For large-scale garden scene point clouds, existing technologies lack means to segment models without accurate labeling. SUMMARY

[0006] The technical problem to be solved by the present application is that current large-scale garden scene point clouds cannot train semantic segmentation models without accurate labeling. A large-scale garden point cloud unsupervised semantic segmentation method is proposed, including the following steps:

[0007] Step 1, super point construction: using super voxel segmentation method to divide single garden scene point cloud into super point set represented by super point cloud.

[0008] Step 2, point cloud preprocessing: since the original garden point cloud performs poorly on existing models, it is necessary to downsample and data augment the garden point cloud, thereby converting the large-scale garden scene point cloud into point cloud data more conducive to feature extraction network learning.

[0009] Step 3, super point growth clustering: After several rounds of training, the original super points grow into a new super point set with fewer numbers and more points, so as to obtain more accurate semantic primitives. In order to deal with complex garden environment, the super points are adaptively clustered into primitives with more semantic characteristics, the super points and the point-wise pseudo-labels are assigned, and the primitive centers are extracted as the point feature classifier.

[0010] Step 4, network training: using the classifier to obtain the point-wise label logic value, and the loss of the point-wise pseudo-label, the point cloud extraction network is trained by back propagation, and the garden point cloud feature extraction network is trained through multiple iterations.

[0011] Step 5, interactive segmentation: due to the complexity and variability of the garden point cloud and the different needs of users, the method uses the garden point cloud segmentation model trained by steps 1-4 to segment and cluster the target garden point cloud into different categories of primitives and obtain different categories of primitive center points, and the points in the primitives are clustered into corresponding categories, and the category name is assigned.

[0012] Further, step 1 includes the following steps:

[0013] Step 1-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B), first convert its coordinates to float type, and subtract the mean to center it, and adjust the coordinates according to the size of the voxel block Vs scale, which can be expressed as:

[0014] Coord1=floor(float(Coord)-mean(float(Coords))) / Vs)

[0015] Wherein, Coord is the input original point cloud coordinate, Coords is the input all original point cloud coordinate, mean(.) is the mean, float(.) is the conversion to float, floor(.) is the floor.

[0016] Step 1-2, convert the original point cloud data to sparse voxelized point cloud using the sparse_quantize method in Literature 8: Choy C, Gwak JY, Savarese S. 4d spatio-temporal convnets: Minkowski convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 3075-3084. That is, discretize the continuous coordinate values into a fixed grid.

[0017] Step 1-3, super voxel clustering is performed on the voxelized point cloud coordinates and RGB values, and the point cloud is processed using the Voxel Cloud Connectivity Segmentation (VCCS) algorithm in Literature 9: Papon J, Abramov A, Schoeler M, et al. Voxel cloud connectivity segmentation-supervoxels for point clouds [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2013: 2027-2034., and each point is assigned a voxel index.

[0018] Step 1-4, further clustering is performed on the voxelized point cloud using the region growing algorithm, and the coordinates are processed using the region_growing function in Literature 11: Rusu RB, Cousins S. 3d is here: Point cloud library (pcl) [C] / / 2011 IEEE international conference on robotics and automation. IEEE, 2011: 1-4., to obtain the regions clustered by region growing.

[0019] Step 1-5, merge the results of super voxel clustering and region growing clustering, for each point in a voxel, count the region it belongs to, assign the majority region to all points in the voxel, so that each point obtains a super point index, and convert the merged super point label to a continuous label.

[0020] Step 1-6, according to the inverse index, remap the super point label to the original point cloud data, and count the most frequently appearing category in each super point as the category label of the super point.

[0021] Further, step 2 includes the following steps:

[0022] Step 2-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B), due to its large scale, after random center cropping, voxelization is performed to obtain the sparse voxel representation of the original point cloud data

[0023] Step 2-2, data augmentation is performed on the sparse voxelized point cloud V2, and through translation, rotation, scaling and mixing data augmentation operations, the voxelized point cloud V2 is converted into an enhanced point cloud V3 composed of normalized coordinates Coord2, and a feature vector F1 containing color and normalized coordinates.

[0024] Step 2-3, if the preprocessed data is used for super point growth clustering in step 3, the normal set of normalized coordinates Coord2 is calculated, and according to the threshold, the small area is deleted, the label of the effective area is remapped, and the pseudo label is initialized; if the preprocessed data is used for point cloud feature extraction network training in step 4, the normal is initialized as a zero vector, and the pseudo label is loaded from the file.

[0025] Further, the random center cropping voxelization in step 2-1 includes the following steps:

[0026] Step 2-1-1, for the point cloud P1 of the input scene, the size (W2, H2, D2) of the scene is obtained, that is, the difference between the maximum and minimum values of the coordinates in three dimensions, which can be expressed by the formula:

[0027]

[0028] Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates in a certain range. The minimum range is the minimum value of each dimension of the original three dimensions plus one percent of the cropping length, and the maximum range is the maximum value of each dimension of the original three dimensions minus one percent of the cropping length, which can be expressed by the formula:

[0029]

[0030] Where random(·) is a random function.

[0031] Step 2-1-3, check each dimension of the cropping center Center dIf the dimension coordinate is outside the original point cloud, the coordinate of the clipping center in this dimension is changed to a random coordinate in this dimension of the original point cloud, which is expressed by the formula:

[0032]

[0033] where d i represents the i-th dimension, here mainly refers to the three dimensions of XYZ, and random(·) is a random function.

[0034] Step 2-1-4, judge the relationship between the scene size (W2, H2, D2) and the clipping length CL. If the clipping length is greater than the maximum value in W2, H 2, D2, do not clip, and voxelize the entire scene; otherwise, clip all points in the sphere with the clipping center Center (X,Y,Z) as the center and the clipping radius CL, and voxelize. The voxelization method uses the voxelization method in step 1-2,

[0035] Further, the data augmentation method in step 2-2 includes the following steps:

[0036] Step 2-2-1, for the sparse voxelized point cloud V2, do coordinate translation augmentation. In each dimension, generate a random value between 0 and 1, then multiply the random value by the translation factor SR to obtain the translation amount S x ,S y ,S z in each dimension of the coordinate, and then add the displacement amount to the coordinate in each dimension to obtain a new point cloud V4, wherein the translation factor SR is set to 50.

[0037] Step 2-2-2, for the point cloud V4 after translation augmentation, do coordinate rotation augmentation. For the three dimensions of the coordinates of the point cloud, set the rotation angle range RB x ,RB y ,RB z , randomly generate three rotation angles in the rotation angle range, thereby generating three rotation matrices M x ,M y ,M z , wherein the rotation axis is set to 1. Then multiply the three rotation matrices in a random order to obtain the final rotation matrix M, and multiply the coordinates of the point cloud V4 by the rotation matrix, thereby obtaining the point cloud V5 after rotation augmentation, wherein the rotation angle range RB x ∈(-π / 32,π / 32), RB y ∈(-π / 32,π / 32), RB z ∈(-π,π).

[0038] Step 2-2-3, for the point cloud V5 after rotation enhancement, coordinate scaling enhancement is performed. For the coordinates of the point cloud, a scaling range SB is set, a number in the scaling range is randomly taken as a scaling factor, and the point cloud V5 coordinates are multiplied by the scaling factor to obtain the scaled enhanced point cloud V6, wherein the scaling range SB ∈ (0.9, 1.1).

[0039] Step 2-2-4, for the scaled enhanced point cloud V6, point cloud mixing enhancement is performed. First, an original point cloud P2 ∈ R (N×6) is randomly selected from all scene point clouds, and then the random center cropping voxelization processing of step 2-1 is used to obtain a randomly cropped sparse voxelization point cloud V7, and then the method of steps 2-2-1 to 2-2-3 is used to perform data enhancement and splice in the coordinate and color dimensions with the point cloud V6 to obtain a mixed enhanced point cloud V8.

[0040] Step 2-2-5, for the mixed enhanced point cloud V8, the center coordinates are first calculated, and the normalized coordinates Coord2 are obtained by subtracting the center coordinates from all point coordinates. The color information and the normalized coordinates Coord2 are spliced in the last dimension to form the final feature vector F1, and the feature vector F1 and the normalized coordinates Coord2 form the enhanced point cloud V3.

[0041] Further, step 3 super point growth clustering includes the following steps:

[0042] The super point growth clustering step mainly includes two stages: super point clustering and super point growth clustering. In the first E1 epochs (a complete training set training period), only super point clustering is performed; in the last E2 epochs, super point growth clustering is performed.

[0043] Step 3-1, it is judged whether it is in the super point growth clustering stage, if it is in the super point growth clustering stage, the number of current super points to be clustered is calculated according to the current training round and the configuration, and the calculation formula is represented as:

[0044] CG = int(sp start - ((epoch - epoch start ) / epoch max ) * (sp start - sp end ))

[0045] Wherein CG is the current super point clustering number, int(·) represents rounding, sp start represents the starting clustering number of super points, sp end represents the ending clustering number of super points, epoch is the current epoch, and epoch startFor the initial epoch, epoch max Set the maximum epoch of the current stage. If the calculated current hyperpoint cluster number is less than sp end , set CG to sp end .

[0046] Step 3-2, extract point features F2 and hyperpoint features F3 using point cloud feature extraction network FEN.

[0047] Step 3-3, according to the hyperpoint feature F3, the feature is primitive clustered, using the KMeans algorithm in the literature 12: Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding [R]. Stanford, 2006. to cluster the features, generate primitive class labels L1 and point indexes I1 in the primitive. According to the primitive clustering result, each point is assigned a pseudo label PL as the basis for training network parameters.

[0048] Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, assign it as a weight to a linear layer as a classifier C, and freeze the parameters of the classifier.

[0049] Step 3-5, according to the distance of point features in different primitives and the distance of features between primitives, adaptively adjust the number of clustered primitives, and get the number of primitives n primitive .

[0050] Further, the extraction of hyperpoint features in step 3-2 includes the following steps:

[0051] Step 3-2-1, obtain the coordinates Coord2, features F1, normal vectors N, labels and other information from the hyperpoint point cloud and the enhanced point cloud, and prepare some auxiliary variables.

[0052] Step 3-2-2, use the TensorField method in document 13: Gwak JY, Choy C, Savarese S. Generative sparsedetection networks for 3d single-shot object detection [C] / / Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020: 297-313. to build the feature F1 and the coordinates Coord2 into the input tensor, extract the feature using the point cloud feature extraction network, and filter out the invalid area.

[0053] Step 3-2-3, extract the RGB and coordinate information from the feature, then build the region association matrix to represent which super point each point belongs to, calculate the number of points of each super point, and finally calculate the average feature F5 of each super point.

[0054] Step 3-2-4, determine whether it is in the super point growth clustering stage, if it is in the super point growth clustering stage, perform super point growth to obtain the super point index I used next time SP .

[0055] Step 3-2-5, use the normal vector N to build the normal vector histogram (PFH) feature of each super point as described in document 14: Rusu R B, Marton Z C, Blodow N, et al. Persistent point feature histograms for 3D point clouds [M] / / Intelligent Autonomous Systems 10. IOS Press, 2008: 119-128.

[0056] Step 3-2-6, normalize the average feature F5 and PFH of the super point, and combine the RGB and normal vector histogram features to obtain the super point feature F3.

[0057] Further, the super point growth in step 3-2-4 includes the following steps:

[0058] Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each super point, and extract the set RGB, XYZ and normal vector weight parameters W RGB ,W XYZ ,W normal .

[0059] Step 3-2-4-2, normalize the average feature F5 of the super point, and combine it with the weighted parameter W RGB XYZ normal The weighted RGB, XYZ and normal vector are combined together to obtain the super point growth feature F6.

[0060] Step 3-2-4-3, extract the number of super points, if the current super point cluster number CG is greater than the number of super points, set the growth cluster number to the number of super points; otherwise, set the growth cluster number to the super point cluster number CG.

[0061] Step 3-2-4-4, according to the super point growth feature F6, use the K-means clustering method to cluster the super points, where the cluster number is CG, to obtain the super point index I SP for next use. The K-means clustering method uses document 12: Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0062] Further, the adaptive adjustment of the number of cluster primitives in step 3-5 includes the following steps:

[0063] Step 3-5-1, calculate the average feature distance FD Point between points in the primitive and the average feature distance FD primitive between primitives based on the point feature F2 and the primitive center feature F4, and the calculation formula is as follows:

[0064]

[0065] where FD is the feature distance, n is the number of points or primitives, and d(f i ,f j ) represents the Euclidean distance between two points or primitive center features.

[0066] Step 3-5-2, determine the relationship between the average feature distance FD Point between points and the average feature distance FD primitive between primitives by the following formula:

[0067]

[0068] where n primitive is the number of primitives, the initial value is 450, n1 is 10, and n2 is 1.

[0069] ​​Further, step 4 network training includes the following steps:

[0070] Step 4-1, load data, including point cloud, if it is the first training, initialize network parameters, optimizer, loss, learning rate scheduler.

[0071] Step 4-2, the TensorField method in the literature: Gwak J Y, Choy C, Savarese S. Generative sparse detection networks for 3d single-shot object detection [C] / / Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020: 297-313. will enhance the features and coordinates of point cloud to build input tensor, and use point cloud feature extraction network to extract features F7.

[0072] Step 4-3, use the classifier C obtained in step 3-4 to classify the features F7 extracted by the network, so that each point gets a logical value Logits.

[0073] Step 4-4, use the pseudo label PL obtained in step 3-3 and the logical value Logits to do loss, and its loss calculation formula is as follows:

[0074] Loss = mean(CrossEntropyLoss(2*logits, PL))

[0075] Where Loss represents the loss function, mean(·) represents taking the average, and CrossEntropyLoss(·) represents cross entropy.

[0076] Step 4-5, according to the loss function, back propagation to the point cloud feature extraction network and update the parameters, train the network.

[0077] Step 4-6, return to step 3, according to the epoch it is in, do super point clustering or super point growth clustering.

[0078] Further, step 5 interactive segmentation includes the following steps:

[0079] Step 5-1, the user sets the hyperparameters, mainly including the number of primitives n P that need to be clustered class .

[0080] Step 5-2, using the model, using the trained garden scene point cloud semantic segmentation model to segment the input garden point cloud into a set number of primitives n P , the segmentation method is to extract features using a feature extraction network, and use clustering method to cluster points into n P primitives, and then cluster the primitives into n class clusters, and extract the center point set P fc of the features according to the clustering results. No need to construct super points. The clustering method uses the KMeans clustering algorithm in document 12: Arthur D, Vassilvitskii S. k-means ++: The advantages of careful seeding [R]. Stanford, 2006.

[0081] Step 5-3, the feature center point set P class of n fc clusters is displayed to the user, and the user labels the categories of these points, and through the propagation of the same cluster, all points in the garden point cloud can be divided into n class categories, thereby completing the semantic segmentation of the garden point cloud.

[0082] The present application focuses on the characteristics of existing garden point cloud data, and designs an unsupervised semantic segmentation method for large-scale garden point cloud, which can train the point cloud feature extraction network to learn the semantic features of the point cloud through the pseudo-label of super point clustering without labeling, and cluster it into different categories of point cloud blocks according to the user's demand when used. Super point is a point cloud representation method that divides point cloud into super points beyond single points, each super point is a new sub-point cloud composed of multiple points, and the present application realizes unsupervised training of point cloud feature extraction network on the basis of this method, and the garden scene point cloud semantic segmentation model mainly consists of a trained point cloud feature extraction network and a clustering module.

[0083] The application divides the point cloud in the garden scene into super points, clusters the super points into primitives with semantic pseudo labels by clustering the point cloud features extracted by the point cloud feature extraction network and the super point features, and the primitives can be considered as the basic units of different objects. Then, the cluster centers of the primitives are used as the classifier to classify the point cloud and assign point-by-point logical values (logits), the point cloud feature extraction network is trained through the loss of the logical values and the pseudo labels, and the previous clustering is returned for continuous iteration. Then, under the condition that the point cloud feature extraction network converges, the super points are continuously grown according to the scene requirements, so that the points contained in the super points are more and more, and the super points are more and more close to the semantic primitives, and the primitives are more and more close to objects of different categories. Finally, a garden point cloud feature extraction network capable of distinguishing point cloud features of different categories can be obtained. In actual use, the super points do not need to be reconstructed, and the feature center points of the semantic primitives are obtained directly through the garden scene point cloud semantic segmentation model, and the primitives are clustered into different semantic categories through the user's labeling of the center, so as to realize the semantic segmentation of large-scale garden scene point cloud.

[0084] The application does not need additional labeled data when training the model, and only needs the user to provide the number of semantic categories and the label of the semantics of a few points when using, greatly reducing the training cost and use threshold of the model. The application can also adaptively adjust the granularity of semantic segmentation in training according to the actual situation of the scene, so as to adapt to complex and delicate garden scenes, so that the model can better meet the user's requirements for different granularity semantic segmentation of large-scale garden scene point cloud. BRIEF DESCRIPTION OF DRAWINGS

[0085] The above and / or other aspects of the application will become more apparent by describing in detail the preferred embodiments thereof with reference to the attached drawings.

[0086] Figure 1 is a processing flow schematic diagram of the application.

[0087] Figure 2 is a visualization result of the input point cloud.

[0088] Figure 3 is a visualization result of the input point cloud constructed into super points.

[0089] Figure 4 is a visualization result of the point cloud that needs to be labeled by the user.

[0090] Figure 5 is a visualization result of the final semantic segmentation result. DETAILED DESCRIPTION

[0091] As Figure 1As shown, the present application discloses a large-scale garden point cloud unsupervised semantic segmentation method, which is implemented according to the following steps:

[0092] Step 1, super point construction: using the super voxel segmentation method to cut the single garden scene point cloud into a super point set represented by a super point cloud.

[0093] Step 2, point cloud preprocessing: downsampling and data enhancement are performed on the original point cloud, so as to convert the large-scale garden scene point cloud into point cloud data more conducive to feature extraction network learning.

[0094] Step 3, pseudo label generation: after several rounds of training, the original super point is grown into a new super point set with fewer quantities and more points, so as to obtain more accurate semantic primitives. The super points are clustered into primitives with more semantic characteristics, the super points and the point-by-point pseudo labels are assigned, and the primitive centers are extracted as the point feature classifier.

[0095] Step 4, network training: using the classifier to obtain the point-by-point label logic value, and the point-by-point pseudo label loss, the point cloud extraction network is trained by back propagation, and the garden point cloud feature extraction network is converged through multiple rounds of iterative training.

[0096] Step 5, interactive segmentation: according to the user-set parameters, the garden point cloud segmentation model trained by steps 1-4 is used to segment the target garden primitives, the points in the primitives are clustered into corresponding categories, and the category names are assigned.

[0097] Step 1 includes the following steps:

[0098] Step 1-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B), as shown in Figure 2 , first convert the coordinates to float type, and subtract the mean to center it, and quantize the coordinates according to the size of the voxel block Vs, which can be represented by the formula:

[0099] Coord1=floor(float(Coord)-mean(float(Coords))) / Vs)

[0100] Wherein, Coord is the input original point cloud coordinate, Coords is the input all original point cloud coordinate, mean(·) is the mean, float(·) is the float number, floor(·) is the floor.

[0101] Step 1-2, convert the original point cloud data to sparse voxelized point cloud using the sparse_quantize method in Literature 8: Choy C, Gwak JY, Savarese S. 4d spatio-temporal convnets: Minkowski convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 3075-3084. That is, discretize the continuous coordinate values into a fixed grid.

[0102] Step 1-3, super voxel clustering is performed on the voxelized point cloud coordinates and RGB values, and the point cloud is processed using the Voxel Cloud Connectivity Segmentation (VCCS) algorithm in Literature 9: Papon J, Abramov A, Schoeler M, et al. Voxel cloud connectivity segmentation-supervoxels for point clouds [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2013: 2027-2034., and each point is assigned a voxel index.

[0103] Step 1-4, further clustering is performed on the voxelized point cloud using the region growing algorithm, and the coordinates are processed using the region_growing function in Literature 11: Rusu RB, Cousins S. 3d is here: Point cloud library (pcl) [C] / / 2011 IEEE international conference on robotics and automation. IEEE, 2011: 1-4., to obtain the regions clustered by region growing.

[0104] Step 1-5, merge the results of super voxel clustering and region growing clustering, for each point in a voxel, count the region it belongs to, assign the majority region to all points in the voxel, so that each point obtains a super point index, and convert the merged super point label to a continuous label.

[0105] Step 1-6, according to the inverse index, remap the super point label to the original point cloud data, count the most frequently appearing category in each super point as the category label of the super point, and generate the super point division as shown in Figure 3

[0106] Step 2 includes the following steps:

[0107] Step 2-1, for an input garden scene point cloud P1∈R (N×6) , containing its coordinates (X, Y, Z) and color information (R, G, B), due to its large scale, after random center cropping, voxelization is performed to obtain the sparse voxel representation of the original point cloud data

[0108] Step 2-2, data augmentation is performed on the sparse voxelized point cloud V2, and through translation, rotation, scaling and mixing, etc. Data augmentation operations are used to convert the voxelized point cloud V2 into an enhanced point cloud V3 composed of normalized coordinates Coord2, and feature vectors F1 containing color and normalized coordinates.

[0109] Step 2-3, if the preprocessed data is used for super point growth clustering in step 3, the normal set of normalized coordinates Coord2 is calculated, and according to the threshold, the small area is deleted, the label of the effective area is remapped, and the pseudo label is initialized; if the preprocessed data is used for point cloud feature extraction network training in step 4, the normal is initialized as a zero vector, and the pseudo label is loaded from the file.

[0110] Random center cropping voxelization in step 2-1 includes the following steps:

[0111] Step 2-1-1, for the point cloud P1 of the input scene, the size (W2, H2, D2) of the scene is obtained, that is, the difference between the maximum and minimum values of the coordinates in three dimensions, which can be expressed by the formula:

[0112]

[0113] Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates in a certain range. The minimum range is the minimum value of each dimension of the original three dimensions plus one percent of the cropping length, and the maximum range is the maximum value of each dimension of the original three dimensions minus one percent of the cropping length, which can be expressed by the formula:

[0114]

[0115] Where random(·) is a random function.

[0116] Step 2-1-3, check each dimension of the cropping center Center d ​If the dimension coordinate is outside the original point cloud, the coordinate of the clipping center in this dimension is changed to a random coordinate in this dimension of the original point cloud, which is expressed by the formula:

[0117]

[0118] where d i represents the ith dimension, here mainly refers to the three dimensions of XYZ, and random(·) is a random function.

[0119] Step 2-1-4, judge the relationship between the scene size (W2, H2, D2) and the clipping length CL, if the clipping length is greater than the maximum value in W2, H 2, D2, do not clip, and voxelize the entire scene; otherwise, clip all points in the sphere with the clipping center Center (X,Y,Z) as the center and the clipping radius CL, and voxelize. The voxelization method uses the voxelization method in step 1-2,

[0120] The data augmentation method in step 2-2 includes the following steps:

[0121] Step 2-2-1, for the sparse voxelized point cloud V2, do coordinate translation augmentation. In each dimension, generate a random value between 0 and 1, then multiply the random value by the translation factor SR to obtain the translation amount S x ,S y ,S z in each dimension of the coordinate, and then add the displacement amount to the coordinate of each dimension to obtain the new point cloud V4, where the translation factor SR is set to 50.

[0122] Step 2-2-2, for the point cloud V4 after translation augmentation, do coordinate rotation augmentation. For the three dimensions of the coordinates of the point cloud, set the rotation angle range RB x ,RB y ,RB z of the three dimensions, randomly generate three rotation angles in the rotation angle range, thereby generating three rotation matrices M x ,M y ,M z , where the rotation axis is set to 1. Then multiply the three rotation matrices in a random order to obtain the final rotation matrix M, and multiply the coordinates of the point cloud V4 by the rotation matrix, thereby obtaining the point cloud V5 after rotation augmentation, where the rotation angle range RB x ∈(-π / 32,π / 32), RB y ∈(-π / 32,π / 32), RB z ∈(-π,π).

[0123] Step 2-2-3, for the point cloud V5 after rotation enhancement, coordinate scaling enhancement is performed. For the coordinates of the point cloud, a scaling range SB is set, a number in the scaling range is randomly taken as a scaling factor, and the point cloud V5 coordinates are multiplied by the scaling factor to obtain the scaled enhanced point cloud V6, wherein the scaling range SB ∈ (0.9, 1.1).

[0124] Step 2-2-4, for the scaled enhanced point cloud V6, point cloud mixing enhancement is performed. First, an original point cloud P2 ∈ R (N×6) is randomly selected from all scene point clouds, then the random center cropping voxelization processing of step 2-1 is used to obtain a randomly cropped sparse voxelization point cloud V7, and then the method of steps 2-2-1 to 2-2-3 is used to perform data enhancement and splice in the coordinate and color dimensions with the point cloud V6 to obtain a mixed enhanced point cloud V8.

[0125] Step 2-2-5, for the mixed enhanced point cloud V8, the center coordinates are first calculated, and the normalized coordinates Coord2 are obtained by subtracting the center coordinates from all point coordinates. The color information and the normalized coordinates Coord2 are spliced in the last dimension to form the final feature vector F1, and the feature vector F1 and the normalized coordinates Coord2 form the enhanced point cloud V3.

[0126] Step 3 includes the following steps:

[0127] The super point growth clustering step mainly includes two stages: super point clustering and super point growth clustering. In the first E1 epochs (a complete training set training period), only super point clustering is performed; in the last E2 epochs, super point growth clustering is performed.

[0128] Step 3-1, it is judged whether it is in the super point growth clustering stage, if it is in the super point growth clustering stage, the number of current super points to be clustered is calculated according to the current training round and the configuration, and the calculation formula is represented as:

[0129] CG = int(sp start - ((epoch - epoch start ) / epoch max ) * (sp start - sp end ))

[0130] Wherein CG is the current super point clustering number, int(·) represents rounding, sp start represents the starting clustering number of super points, sp end represents the ending clustering number of super points, epoch is the current epoch, epoch start is the starting epoch, and epochmax Set the maximum epoch of the current stage. If the calculated current super point cluster number is less than sp end , set CG as sp end .

[0131] Step 3-2, extract point features F2 and super point features F3 using point cloud feature extraction network FEN.

[0132] Step 3-3, according to the super point feature F3, the feature is primitive clustered, using the KMeans algorithm in the literature 12: Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding [R]. Stanford, 2006. to cluster the features, generate primitive class labels L1 and point indexes I1 in the primitive. According to the primitive clustering result, each point is assigned a pseudo label PL as the basis for training network parameters.

[0133] Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, and assign it as a weight to a linear layer as a classifier C, and freeze the parameters of the classifier.

[0134] Step 3-5, according to the distance of point features in different primitives and the distance of features between primitives, adaptively adjust the number of clustered primitives, and get the number of primitives n primitive .

[0135] The extraction of super point features in step 3-2 includes the following steps:

[0136] Step 3-2-1, get the coordinates Coord2, features F1, normal vectors N, labels and other information from the super point cloud and enhanced point cloud, and prepare some auxiliary variables.

[0137] Step 3-2-2, use the TensorField method in document 13: Gwak JY, Choy C, Savarese S. Generative sparsedetection networks for 3d single-shot object detection [C] / / Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020: 297-313. to build the feature F1 and the coordinates Coord2 into the input tensor, extract the feature using the point cloud feature extraction network, and filter out the invalid area.

[0138] Step 3-2-3, extract the RGB and coordinate information from the feature, then build the region association matrix to represent which superpoint each point belongs to, calculate the number of points of each superpoint, and finally calculate the average feature F5 of each superpoint.

[0139] Step 3-2-4, determine whether it is in the superpoint growth clustering stage, if it is in the superpoint growth clustering stage, perform superpoint growth to obtain the superpoint index I used next time SP .

[0140] Step 3-2-5, use the normal vector N to build the normal vector histogram (PFH) feature of each superpoint as described in document 14: Rusu R B, Marton Z C, Blodow N, et al. Persistent point feature histograms for 3D point clouds [M] / / Intelligent Autonomous Systems 10. IOS Press, 2008: 119-128.

[0141] Step 3-2-6, normalize the average feature F5 and PFH of the superpoint, and combine the RGB and normal vector histogram features to obtain the superpoint feature F3.

[0142] The superpoint growth in step 3-2-4 includes the following steps:

[0143] Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each superpoint, and extract the set RGB, XYZ and normal vector weight parameters W RGB ,W XYZ ,W normal .

[0144] Step 3-2-4-2, normalize the average feature F5 of the super point, and combine it with the weight parameter W RGB XYZ normal The weighted RGB, XYZ and normal vector are combined together to obtain the super point growth feature F6.

[0145] Step 3-2-4-3, extract the number of super points, if the current super point cluster number CG is greater than the number of super points, set the growth cluster number to the number of super points; otherwise, set the growth cluster number to the super point cluster number CG.

[0146] Step 3-2-4-4, according to the super point growth feature F6, use the K-means clustering method to cluster the super points, where the cluster number is CG, to obtain the super point index I SP next used. The K-means clustering method uses document 12: Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0147] The adaptive adjustment of the number of cluster primitives in step 3-5 includes the following steps:

[0148] Step 3-5-1, calculate the average feature distance FD Point between points in the primitive and the average feature distance FD primitive between primitives based on the point feature F2 and the primitive center feature F4, whose calculation formula is as follows:

[0149]

[0150] Where FD is the feature distance, n is the number of points or primitives, and d(f i ,f j ) represents the Euclidean distance between each pair of point features or primitive center features.

[0151] Step 3-5-2, judge the relationship between the average feature distance FD Point between points and the average feature distance FD primitive between primitives, whose judgment formula is as follows:

[0152]

[0153] Where n primitive is the number of primitives, the initial value is 450, n1 is 10, and n2 is 1.

[0154] Step 4 includes the following steps:​​

[0155] Step 4-1: Load data, including point cloud. If this is the first training, initialize network parameters, optimizer, loss, learning rate scheduler.

[0156] Step 4-2, the TensorField method in the literature: Gwak JY, Choy C, Savarese S. Generative sparse detection networks for 3d single-shot object detection[C] / / Computer Vision–ECCV 2020:16th European Conference, Glasgow,UK,August 23–28,2020,Proceedings,Part IV16.Springer International Publishing,2020:297-313. constructs the features and coordinates of the enhanced point cloud as the input tensor, and uses the point cloud feature extraction network to extract features F7.

[0157] Step 4-3: Use the classifier C obtained in step 3-4 to classify the features F7 extracted by the network, so that each point gets a logical value Logits.

[0158] Step 4-4: Use the pseudo-label PL and logistic values ​​Logits obtained in Step 3-3 to calculate the loss. The loss calculation formula is as follows:

[0159] Loss=mean(CrossEntropyLoss(2*logits,PL))

[0160] Where Loss represents the loss function, mean(·) represents taking the average, and CrossEntropyLoss(·) represents the cross-entropy.

[0161] Steps 4-5: Use the multi-label classification loss function (MultiLabelMarginLoss) in PyTorch to backpropagate to the point cloud feature extraction network and update the parameters to train the network.

[0162] Steps 4-6: Return to step 3 and perform super-point clustering or super-point growing clustering based on the current epoch.

[0163] Step 5 includes the following steps:

[0164] Step 5-1: The user sets hyperparameters, mainly including the number of primitives n to be clustered. P The number of categories that need to be segmented, n class .

[0165] Step 5-2, using the model, using the trained garden scene point cloud semantic segmentation model to segment the input garden point cloud into a set number of primitives n P , the segmentation method is to extract features using a feature extraction network, and to cluster points into n P primitives using a clustering method, and then to cluster the primitives into n class clusters, and to extract the center point set P fc from the clustering results. Figure 4 As shown in , there is no need to construct a super point. The clustering method uses the KMeans clustering algorithm in document 12: Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding [R]. Stanford, 2006.

[0166] Figure 5 Step 5-3, the feature center point set P class of the n fc clusters is displayed to the user, and the user labels the classes of these points, and through the propagation of the same cluster, all points in the garden point cloud can be divided into n class classes, thereby completing the semantic segmentation of the garden point cloud, and the segmentation result is as shown in .

[0167] The present application provides an unsupervised semantic segmentation method for large-scale garden point clouds, and there are many methods and ways to realize this technical solution. The above description is only the preferred embodiment of the present application, and it should be pointed out that for ordinary technical personnel in this technical field, without departing from the principle of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application. The components not explicitly described in the embodiments can be realized by existing technology.

Claims

1. An unsupervised semantic segmentation method for large-scale garden point clouds, characterized in that, Comprising the following steps: Step 1, super point construction: using the super voxel segmentation method, the garden scene point cloud is divided into a super point set; Step 2, point cloud preprocessing: the original point cloud is down-sampled and data enhanced, and the garden scene point cloud is converted into enhanced point cloud data for feature extraction network learning; Step 3, super point growth clustering: using the super point set obtained in step 1 and the enhanced point cloud obtained in step 2, the super points are clustered into primitives, and the super points and the point-by-point pseudo-labels are assigned, the primitive centers are extracted as the point feature classifier, the data is trained and a new super point set is obtained; Step 4, network training: using the classifier obtained in step 3 to calculate the point-by-point label logic value, and using the point-by-point pseudo-label obtained in step 3 to do loss, the point cloud extraction network is trained by back propagation, and the garden point cloud feature extraction network is generated by multiple iterations in step 3; Step 5, interactive segmentation: according to the user's setting parameters, using the garden point cloud segmentation model trained in steps 1-4 to segment the target garden primitives, the points in the primitives are clustered into corresponding categories, and the category names are assigned to the category labels; Step 1, super point construction, comprising the following steps: Step 1-1, for an input garden scene point cloud, it is standardized according to the voxel block size; Step 1-2, the garden point cloud obtained in step 1-1 is sparse voxelized; Step 1-3, using a point cloud voxel connectivity segmentation algorithm to cluster the voxelized point cloud coordinates and RGB values obtained in step 1-2 into super voxels; Step 1-4, using a region growing algorithm to further cluster the voxelized point cloud obtained in step 1-2; Step 1-5, merging the super voxel clustering obtained in step 1-3 and the region growing clustering obtained in step 1-4, so that each point is in a super point; Step 1-6, assigning a continuous super point label to each super point obtained in step 1-5.

2. The method of claim 1, wherein, Step 2, point cloud preprocessing, comprising the following steps: Step 2-1, for an input garden scene point cloud, after random center cropping, it is voxelized to obtain a sparse voxelized point cloud V2; Step 2-2, performing data enhancement on the voxelized point cloud obtained in step 2-1, converting the voxelized point cloud into an enhanced point cloud composed of standardized coordinates and feature vectors, the feature vectors including color and standardized coordinates; Step 2-3, respectively processing the preprocessed data for subsequent steps, calculating the normal set of standardized coordinates obtained in step 2-2, and deleting small regions according to a threshold, re-mapping the labels of the effective regions, initializing the pseudo-labels for step 3 super point growth clustering; initializing the normal of the standardized coordinates obtained in step 2-2 as a zero vector, and loading the pseudo-labels from a file for step 4 point cloud feature extraction network training.

3. The method of claim 2, wherein, The random center cropping voxelized point cloud data in step 2-1 comprises the following steps: Step 2-1-1, for the input scene point cloud P1, the size (W2, H2, D2) of the scene is obtained, which is the difference between the maximum and minimum values of the coordinates in three dimensions, which can be expressed by the formula: Step 2-1-2, set the cropping length CL, and randomly obtain the cropping center coordinates within a certain range, the minimum of which is the minimum value of each dimension of the original three-dimensional plus one percent of the cropping length, and the maximum is the maximum value of each dimension of the original three-dimensional minus one percent of the cropping length, which can be expressed by the formula: where random(·) is a random function, Center x , Center y , Center z is a center coordinate value; Step 2-1-3, check each dimension clipping center Center d If the dimension coordinate is outside the original point cloud, change the coordinate of the clipping center in this dimension to a random coordinate in this dimension of the original point cloud, which can be expressed by the formula: where d i represents the dimension, here mainly refers to the XYZ three dimensions, and random(·) is a random function; Step 2-1-4, judge the relationship between the scene size (W2, H2, D2) and the clipping length CL, if the clipping length is greater than the maximum value in W2, H2, D2, then do not clip, and voxelize the entire scene; otherwise, take the point cloud in the sphere with the clipping center Center 2, (X,Y,Z) as the center, and all points within the sphere with the clipping radius CL, and voxelize, the voxelization method uses the voxelization method in step 1-2.​ 4. The method of claim 2, wherein, The data enhancement method in step 2-2 includes the following steps: Step 2-2-1, for the sparse voxelized point cloud V2, coordinate translation enhancement is performed, a random value is generated in each dimension, and the random value is multiplied by a translation factor SR to obtain a translation amount S of the coordinates in each dimension x y z Then the coordinates of each dimension are added to the displacement amount to obtain a new point cloud V4;​​ Step 2-2-2, for the point cloud V4 after the translation enhancement, the coordinate rotation enhancement is performed, for the three dimensions of the coordinates of the point cloud, the rotation angle range RB x , RB y and RB z are set, three rotation angles are randomly generated in the rotation angle range, thereby generating three rotation matrices M x ,M y ,M z , the rotation axis is set to 1, the three rotation matrices are multiplied in a random order to obtain a final rotation matrix M, and the coordinates of the point cloud V4 obtained in the previous step are multiplied by the rotation matrix, thereby obtaining the point cloud V 5； Step 2-2-3, for the point cloud V5 after rotation enhancement, coordinate scaling enhancement is performed, for the coordinates of the point cloud, a scaling range SB is set, a scaling factor is randomly taken in the scaling range, and the point cloud V5 coordinates are multiplied by the scaling factor to obtain the point cloud V 6； Step 2-2-4, for the scaled enhanced point cloud V6, a random original point cloud is selected in the scene point cloud, and a random center cropping voxelization processing is performed using the random center cropping voxelization processing of step 2-1 to obtain a sparse voxelization point cloud V7 after random cropping. Then, the method of steps 2-2-1 to 2-2-3 is used to perform data enhancement on the sparse voxelization point cloud V7, and the sparse voxelization point cloud V7 is spliced with the point cloud V6 in the coordinate and color dimensions to obtain a mixed enhanced point cloud V8. 8； Step 2-2-5, for the mixed enhanced point cloud V8, standardize its coordinates, concatenate the color information and the standardized coordinates Coord2 to form the final feature vector F1, and the feature vector F1 and the standardized coordinates Coord2 form the enhanced point cloud V3.

5. The method of claim 1, wherein, Step 3 super point growth clustering includes the following steps: The super point growth clustering step includes super point clustering and super point growth clustering. In the first E1 epochs, i.e. a complete training cycle of the training set, only super point clustering is performed; in the last E2 epochs, super point growth clustering is performed; Step 3-1, determine whether it is in the super point growth clustering stage, if it is in the super point growth clustering stage, calculate the number of super points that need to be clustered according to the current training round and the configuration, and the calculation formula is: CG = int(sp start - ((epoch - epoch start ) / epoch max ) * (sp start - sp end )) ; where CG is the current superpoint cluster number, int( ) represents rounding, sp start represents the cluster number where the superpoint starts, sp end represents the cluster number where the superpoint ends, epoch is the current epoch, epoch start is the starting epoch, epoch max is the maximum epoch of the current stage, if the calculated current superpoint cluster number is less than sp end , then set CG as sp end ; Step 3-2, use the point cloud feature extraction network FEN to extract point features F2 and super point features F3; Step 3-3, primitive clustering is performed on the features obtained in step 3-2, and each point is assigned a pseudo label PL according to the primitive clustering result as the basis for training network parameters; Step 3-4, calculate the center of each clustered primitive, normalize the feature F4 of the primitive center, and assign a linear layer as a classifier C as a weight, and freeze the parameters of the classifier; Step 3-5, the number of clustering primitives is adaptively adjusted according to the distance of point features in different primitives and the distance of features between primitives, to obtain the number of primitives n of the next primitive clustering primitive .

6. The method of unsupervised semantic segmentation of large-scale garden point clouds according to claim 5, characterized in that, The extraction of super point features in step 3-2 includes the following steps: Step 3-2-1, obtain coordinates, features, normal vectors and label information from the super point cloud and the enhanced point cloud; Step 3-2-2, construct the input tensor from the features and coordinates, use the point cloud feature extraction network FEN to extract the features, and filter the invalid area; Step 3-2-3, extract RGB and coordinate information from the features, construct a region association matrix to represent the super point to which each point belongs, calculate the number of points in each super point, and then calculate the average feature F5 of each super point; Step 3-2-4, judging whether in the super point growing clustering stage, if in the super point growing clustering stage, performing super point growing to obtain the super point index I used next time SP ; Step 3-2-5, use the normal vector N to construct the normal vector histogram feature PFH of each super point; Step 3-2-6, normalize the average features F5 and PFH of the super point, and combine the RGB and normal vector histogram features to obtain the super point feature F 3； The super point growth in step 3-2-4 includes the following steps: Step 3-2-4-1, calculate the average RGB, XYZ and normal vector of each super point, and extract the set RGB, XYZ and normal vector weight parameters W RGB ,W XYZ ,W normal , wherein W RGB = 0.9~1.2, W XYZ = 0.5~0.7, W normal = 0.3~0.5; Step 3-2-4-2, normalize the average feature F5 of the superpoint, and combine it with the weighted parameter W RGB ,W XYZ ,W normal The weighted RGB, XYZ and normal vector are combined together to obtain the superpoint growth feature F6; Step 3-2-4-3, extract the number of super points, if the current super point clustering number CG is greater than the number of super points, set the growth clustering number to the number of super points; otherwise, set the growth clustering number to the super point clustering number CG; Step 3-2-4-4, clustering super points according to the super point growth feature F6, where the number of clustering clusters is CG, to obtain the super point index I for next use SP .

7. The method of claim 5, wherein, The adaptive adjustment of the number of clustered primitives in step 3-5 includes the following steps: Step 3 - 5-1, compute the average feature distance FD between points in a primitive combining point feature F2 and primitive center feature F4 Point and the average feature distance FD between primitives primitive which is computed as follows: where FD is the feature distance, n is the number of points or primitives, d(f i ,f j ) represents the Euclidean distance between each pair of point features or primitive center features. Step 3 - 5 - 2, adjust the number of primitives n by judging the average feature distance FD between points Point and the average feature distance FD between primitives primitive The judging formula is as follows: primitive Step 3 - 5 - 3, adjust the number of primitives n by judging the relationship between the average feature distance FD between points and the average feature distance FD between primitives where n primitive is the number of primitives, with an initial value of 450, n1 is 10, and n2 is 1.

8. The method of claim 5, wherein, Step 4 network training includes the following steps: Step 4-1, load data, including point cloud, if it is the first training, initialize network parameters, optimizer, loss, learning rate scheduler; Step 4-2, construct the input tensor from the features and coordinates of the enhanced point cloud, and use the point cloud feature extraction network FEN to extract the features F7; Step 4-3, using the classifier C obtained in step 3-4 to classify the features F7 extracted by the network, so that each point gets a logical value Logits; Step 4-4, using the pseudo label PL obtained in step 3-3 and the logical value Logits to do loss, and the loss calculation formula is as follows: Loss = mean(CrossEntropyLoss(2*logits, PL)) Where Loss is the loss function, mean(·) represents taking the average, and CrossEntropyLoss(·) represents cross entropy; Step 4-5, according to the loss function, the point cloud feature extraction network is back propagated and the parameters are updated, and the network is trained; Step 4-6, return to step 3, according to the epoch, perform super point clustering or super point growth clustering, and iterate steps 3 and 4 until E1+E2 epochs are executed, where E1 is the number of epochs for performing super point clustering, and E2 is the number of epochs for performing super point growth clustering.

9. The method of claim 1, wherein, Step 5 interactive segmentation includes the following steps: Step 5-1, user sets hyperparameters, including the number of primitives n that need to be clustered P and the number of classes n that need to be segmented class ; Step 5-2, using the model, segmenting the input garden point cloud into a set number of primitives n using the trained garden scene point cloud semantic segmentation model P and clustering the primitives into n class clusters, obtaining a feature center point set for each cluster; Step 5-3, the feature center point set of n class clusters is displayed to the user, the user labels the categories of the points, and all points in the garden point cloud can be divided into n class categories through the same cluster propagation, thereby completing semantic segmentation of the garden point cloud.

Images