Unsupervised semantic segmentation method for separating branches and leaves from forest point cloud

Abstract

The invention discloses an unsupervised semantic segmentation method for separating branches and leaves from forest point cloud. The method comprises the following steps: S1, acquiring three-dimensional point cloud data of a forest scene; s2, performing feature extraction on the three-dimensional point cloud data based on an unsupervised semantic segmentation deep learning network to obtain comprehensive feature representation of the point cloud; s3, performing super-point generation and clustering based on the comprehensive feature representation to generate a pseudo tag, and realizing separation of branches and leaves; the unsupervised semantic segmentation deep learning network does not need to carry out priori training through a manually labeled point cloud with a semantic tag. According to the method, automatic separation of forest point cloud branches and leaves is realized through the unsupervised semantic segmentation deep learning network, manual data annotation is not needed, and the method has relatively high segmentation precision and robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an unsupervised semantic segmentation method for separating branches and leaves from forest point clouds. BACKGROUND

[0002] Light Detection and Ranging (LiDAR) provides an efficient and accurate means for forest resource investigation. As an active remote sensing technology, LiDAR can quickly acquire three-dimensional point cloud data of forest scenes, accurately depicting the three-dimensional structural information and parameters of forests.

[0003] Extracting various semantic information and components (including tree leaves, branches, understory shrubs, and ground point clouds) from forest scene point clouds enables accurate separation of branches and leaves (hereinafter referred to as branch-leaf separation), thereby enabling accurate estimation and inversion of various forest structure parameters (such as tree location, height, crown size, and shape). Therefore, branch-leaf separation of forest point clouds is one of the core tasks for forest resource investigation and management. Due to the large amount of forest point cloud data and disordered distribution, point cloud missing, occlusion, or overlap often occurs under complex terrain and stand conditions, making branch-leaf separation of forest scenes a challenging task.

[0004] Past research has proposed various branch-leaf separation methods for complex forest scenes, mainly using supervised learning methods to assign semantic labels (such as branches or leaves) to each point. Representative methods include machine learning methods such as support vector machines, random forests, Gaussian mixture models (GMM), and deep learning methods. For example, Vicari et al. combined the shortest path algorithm with a Gaussian mixture model for branch-leaf separation; Lalonde et al. fitted a GMM based on geometric features to classify tree trunks, branches, and crowns; and Xing et al. used XGBoost to distinguish ground, tree trunks, and branches in a Mongolian oak plantation. Other researchers have conducted forest point cloud branch-leaf separation research based on laser intensity, local convex hull, K-value search, and other methods. In addition, with the development of convolutional neural networks, deep learning has made significant progress in image processing and point cloud segmentation. Supervised point cloud semantic segmentation networks such as PointNet, PointNet++, and PointneXt have been proposed, providing new ideas for branch-leaf separation in forest scenes. However, compared to structured objects, naturally growing trees have complex morphological structures, and tree point clouds are sparse and irregular, making supervised learning methods more challenging in capturing branch-leaf feature details. In particular, the interlaced structure of branches and leaves makes it difficult to accurately distinguish their boundaries, limiting the application of such methods in large-scale forest point cloud processing.

[0005] More importantly, the branch and leaf separation method based on supervised learning relies on a large amount of annotated training data, and manual annotation of branch and leaf for large-scale tree point cloud is not only time-consuming and laborious, but also prone to errors. Currently, there is a lack of unsupervised point cloud semantic segmentation method for branch and leaf separation task in forest scene, which has become an important bottleneck for the further development of forest resource inventory and management.

[0006] Therefore, the skilled in the art is committed to developing an unsupervised semantic segmentation method for separating branch and leaf from forest point cloud, aiming at automatically, efficiently and accurately separating branch and leaf components from unlabeled forest scene three-dimensional point cloud. SUMMARY

[0007] In view of the above defects of the prior art, the present application at least solves the following technical problems:

[0008] 1) The existing forest point cloud branch and leaf separation method based on supervised learning relies on a large amount of manually annotated point cloud data with semantic labels, and manual annotation of large-scale tree point cloud is time-consuming, laborious and prone to errors, which limits the practicability of the method;

[0009] 2) The natural growth of trees has complex morphological structure, and the tree point cloud has sparsity and irregularity, and the branch and leaf structure is intertwined, which makes it difficult for supervised learning method to accurately capture branch and leaf feature details and distinguish boundaries, and limits its application in large-scale forest point cloud processing;

[0010] 3) The existing technology lacks an unsupervised point cloud semantic segmentation method for branch and leaf separation task in forest scene, which has become an important bottleneck for the development of forest resource inventory and management.

[0011] To achieve the above purpose, the present application discloses an unsupervised semantic segmentation method for separating branch and leaf from forest point cloud,

[0012] S1: obtaining three-dimensional point cloud data of forest scene;

[0013] S2: performing feature extraction on the three-dimensional point cloud data based on an unsupervised semantic segmentation deep learning network to obtain a comprehensive feature representation of the point cloud;

[0014] S3: generating and clustering hyperpoints based on the comprehensive feature representation to generate pseudo labels and realize separation of branch and leaf; the unsupervised semantic segmentation deep learning network does not need to be trained in advance by manually annotated point cloud with semantic labels;

[0015] Further, each point in the three-dimensional point cloud data in step S1 contains 8 feature dimensions, which are spatial coordinates {x, y, z}, color features {R, G, B}, normal vector and curvature;

[0016] Furthermore, the feature extraction in step S2 includes: using a sparse convolutional neural network as a backbone network, combining a point cloud feature convolution integrator to fuse the eight feature dimensions, and introducing a dual-focus attention module to improve global feature attention;

[0017] Furthermore, the generation and clustering of superpoints in step S3 includes: constructing initial superpoints, combining points with similar geometric features into initial superpoints; during the non-growth period and growth period of the superpoints, expanding the initial superpoints to form larger superpoints based on feature similarity; and aggregating the superpoints using a K-means algorithm;

[0018] Furthermore, the processing process of the point cloud feature convolution integrator includes:

[0019] S21: Divide the eight feature dimensions of the three-dimensional point cloud data into two parts, the first part is the spatial coordinates {x, y, z}, and the second part is the color features {R, G, B}, normal vector and curvature;

[0020] S22: Process the first part using a VecKM module to obtain a first feature output;

[0021] S23: Processing the second part through linear transformation and nonlinear activation function to obtain a second feature output;

[0022] S24: Concatenate the first feature output and the second feature output to obtain the comprehensive feature representation;

[0023] Furthermore, in step S22, the processing of the VecKM module includes: after processing the spatial coordinates {x, y, z}, converting the complex vector into a real vector by calculating the square norm of the complex vector to obtain the first feature output;

[0024] Furthermore, in step S2, the dual-focus attention module includes a channel attention module and a spatial attention module, and the processing process can be expressed as:

[0025]

[0026] in, Represents element multiplication, F is the intermediate feature map in the feature extraction process, is the channel attention value, M s Spatial attention value, F′ is the final refined output;

[0027] Furthermore, the constructing of the initial superpoint includes: dividing the three-dimensional point cloud data into a plurality of fragment blocks, each fragment block containing points with similar geometric features, and each fragment block serving as an initial superpoint;

[0028] Furthermore, the training parameters of the non-growth period and the growth period of the superpoint are different, wherein the max_epoch of the non-growth period is 500 and the number of iterations is 10,000; the max_epoch of the growth period is 800 and the number of iterations is 30,000;

[0029] Furthermore, the initial number of super points is 80, and the number of super points after cluster fusion is no less than 20.

[0030] Compared with the prior art, the above technical solution conceived by the present invention has at least the following beneficial effects:

[0031] 1) No prior training is required using semantically labeled point clouds, which avoids the time-consuming, labor-intensive, and error-prone manual annotation of large-scale tree point clouds. This reduces the reliance on labeled data and is more suitable for actual large-scale forest point cloud processing scenarios.

[0032] 2) It focuses more on extracting important features, suppresses unnecessary features, and enhances the ability to capture branch-leaf feature details. This helps improve the accuracy of boundary distinction, especially in complex scenes with intertwined branches and leaves.

[0033] 3) Better extract and fuse the geometric features, color features and auxiliary features of the point cloud to generate a comprehensive feature representation, which improves the adaptability to the sparsity and irregularity of tree point clouds;

[0034] 4) It can tolerate a large amount of loss and incompleteness of point clouds, thus having good robustness;

[0035] 5) It has sufficient generalization ability and is not only suitable for forest point cloud branch and leaf separation, but can also be extended to other point cloud semantic segmentation scenarios, with a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the method architecture of the present invention;

[0037] Figure 2 Schematic diagram of the network architecture of the present invention;

[0038] Figure 3 Schematic diagram of the structure of the dual-focus attention module of the present invention;

[0039] Figure 4 Schematic diagram of the structure of the point cloud feature convolution integrator module of the present invention;

[0040] Figure 5 This is a schematic diagram of the point cloud data used in the present invention;

[0041] Figure 6 This is a schematic diagram of the visualization result of the separation of branches and leaves of the entire forest in an embodiment of the present invention;

[0042] Figure 7 A single tree segmentation result visualization diagram in an embodiment of the present application;

[0043] Figure 8 Different network single segmentation result visualization in an embodiment of the present application;

[0044] Figure 9 Single tree segmentation visualization result schematic diagram under different occlusion rates in an embodiment of the present application;

[0045] Figure 10 S3DIS dataset model comparison visualization result in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following reference to the drawings of the specification introduces a plurality of preferred embodiments of the present application, so that its technical content is more clear and convenient to understand. The present application can be embodied in many different forms of embodiments, the protection scope of the present application is not limited to the embodiments mentioned in the text.

[0047] In the drawings, the same components are denoted by the same reference numerals, and components having similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application does not limit the size and thickness of each component. In order to make the drawing clearer, the thickness of some components is appropriately exaggerated in some places in the drawing.

[0048] The present application discloses an unsupervised semantic segmentation method for separating branches and leaves from forest point clouds, as shown in Figure 1 The method comprises the following steps:

[0049] S1: Obtain three-dimensional point cloud data of a forest scene; each point in the three-dimensional point cloud data contains 8 feature dimensions, which are spatial coordinates {x, y, z}, color features {R, G, B}, normal vector and curvature;

[0050] S2: Based on the unsupervised semantic segmentation deep learning network, the feature of the three-dimensional point cloud data is extracted to obtain the comprehensive feature representation of the point cloud;

[0051] Preferably, a sparse convolutional neural network is used as the backbone network, the 8 feature dimensions are integrated by combining the point cloud feature convolution integrator, and a double-point attention module is introduced to improve the global feature attention degree;

[0052] S3: Generate and cluster superpoints based on the comprehensive feature representation, including constructing initial superpoints and combining points with similar geometric features into initial superpoints; during the non-growth and growth periods of the superpoints, expand the initial superpoints based on feature similarity to form larger superpoints; use the K-means algorithm to aggregate the superpoints and generate pseudo-labels, which are used for unsupervised semantic segmentation deep learning network training to achieve the separation of branches and leaves; the unsupervised semantic segmentation deep learning network does not need to be trained a priori on manually labeled point clouds with semantic labels.

[0053] In another preferred embodiment, Figure 2 The network of the present invention is an unsupervised semantic segmentation network that can perform point-by-point feature learning and clustering without any artificial labels. The network of the present invention can simultaneously input 6 forest point cloud datasets {q 1 …q 6}. Each point has 8 feature dimensions: spatial coordinates {x, y, z}, color features {R, G, B}, normal vector and curvature. The network of the present invention has two working stages: feature extraction and super-point clustering: in the point feature extraction stage, powerful sparse convolution is used in combination with the auxiliary point cloud feature convolution integrator (PFCI) PFCI module to extract features. Another auxiliary module, the dual point attention module (DPA) is used to improve the accuracy of feature extraction, so that feature extraction can focus more on important details; in the super-point generation and clustering stage, points with similar features are gradually clustered into super-points, and gradually increase with the number of iterations. The super-points are sent to the semantic primitive clustering module to generate pseudo-labels for the super-points, and then the corresponding centroids estimated by K-means are used as classifiers to classify all individual points. These pseudo-labels will also be used to optimize feature extraction.

[0054] About point feature extraction

[0055] Due to uneven sampling and the sparsity and irregularity of point clouds, a sparse convolutional neural network framework, well-suited for processing sparse point clouds, was modified as the backbone network. This framework, combined with the PFCI auxiliary module, fuses point spatial coordinates {x, y, z}, color features {R, G, B}, normal vectors, and curvature features to generate a comprehensive feature representation. Furthermore, a DPA module was introduced to increase global feature attention, enabling the unsupervised semantic segmentation deep learning network to focus more on extracting important features and less on unimportant ones.

[0056] About superpoint generation and clustering:

[0057] Compared with single point data features, single block has more collective features and meanings, and is easier to extract high-level semantic features, so this paper aims to build an initial super point, that is, to divide each input point cloud into each fragment block, that is, to combine points with similar geometric features into a super point. In this way, each fragment will be divided into the same category. Given a sparse point cloud composed of multiple semantic categories, it can be easily observed that a relatively small local point set almost contains no different semantic information. However, in the non-growth and growth periods of the super point, the fragment block will naturally appear as the basic element of a specific semantic category. With continuous training, the output features gradually have higher-level semantic information, and the super points of the same category are continuously expanded according to the similarity of the features, gradually covering the similar semantic areas to form larger super points, covering more features, so as to make the accuracy of this category higher and the precision higher.

[0058] For each training round, each input point cloud is included in a certain number of super points, and each super point represents a specific category. Most super points can be divided into two categories, leaves or trunks, and for the entire data set, all super points are integrated into a tree. In order to effectively group these super points, it is selected to continuously group these super points into a cluster of relatively more super points during the training phase. In this step, the super point features of the entire data set (initial super points in the early stage of unsupervised semantic segmentation deep learning network training, and super points after growth) are clustered. The super points are aggregated into multiple basic semantic units, and pseudo labels are generated for model training. After training, the semantic primitives are aggregated into categories, and the K-means method is used for aggregation.

[0059] About dual point attention module

[0060] In the branch and leaf separation task, an important challenge is how to extract features from sparse and irregular tree point clouds. In order to enable the unsupervised semantic segmentation deep learning network to focus on important features and suppress unnecessary features, point attention in the channel (referred to as channel attention CA) and point attention in space (referred to as spatial attention SA) are combined. The two attentions are used to emphasize meaningful features along the two dimensions of channel and space axes, and attention pooling is used in channel attention. Based on this, an efficient global information attention module is proposed, which is called dual point attention module (DPA), as shown in the structure of Figure 3 .

[0061] Given the intermediate feature map F∈R C×H×W As input, the DPA module sequentially infers a one-dimensional channel attention map M C ∈RC ×1×1 and two-dimensional spatial attention map M s ∈R 1×H×W The processing procedure can be represented as:

[0062]

[0063] where, represents element-wise multiplication, F is the intermediate feature map in the feature extraction process, is the channel attention value, M s is the spatial attention value, F' is the final refined output. In the multiplication process, the attention value is broadcasted accordingly: where the channel attention value is and then broadcasted along the spatial dimension. Figure 3 The calculation process of each attention map is depicted.

[0064] Point cloud feature convolution integrator

[0065] In order to better extract the point cloud spatial coordinate features and color features, as well as the normal vector and curvature features, and integrate these features through a specific network and generate a comprehensive feature representation, the present application proposes a point cloud feature convolution integrator (PFCI), namely a PFCI module, whose complete structure is shown in Figure 4 The PFCI module acts on the feature extraction stage, not only can skip the sampling stage, but also can improve the performance of the segmentation task, and has stronger robustness to noise.

[0066] The PFCI module is good at capturing local geometric features, and it can generate a series of complex data containing rich geometric information. When processing these complex data, a series of calculation layers will be used, which can be regarded as a series of processing units. They analyze and process the output of the encoder through specific algorithms. These processing units usually include linear transformation and nonlinear activation functions such as ReLU, which help the network extract useful features from the data and enhance the understanding of object shape. Finally, in order to ensure that the output of the encoder can be compatible with the standard data processing architecture, there will be a conversion step to convert the complex vector into a real vector. This is achieved by calculating the square norm of the complex vector, and the real vector obtained can be used by subsequent systems or algorithms, so as to complete the complete understanding and analysis of the three-dimensional object shape. The whole process is like transforming complex three-dimensional shapes into information that can be intuitively understood.

[0067] In addition, the VecKM module is a core module inside the PFCI module. The VecKM module can only process three-dimensional point cloud data, but the initial point cloud data has eight dimensions. Therefore, the VecKM module is modified to be able to process higher-dimensional data. First, the data is processed in two parts, one part of the data only contains spatial coordinates {x, y, z}, and the other part only contains all other features except spatial coordinates {x, y, z}. Then, the data with spatial features is processed through the VecKM module to finally obtain the output R n×256 . The data with other features is processed through a series of linear layers and Relu layers to finally obtain the output R n×256 . Finally, the two parts are spliced to obtain the required comprehensive feature representation and the required dimension R n×512 .

[0068] The data source, experimental configuration and evaluation index of the present application are described in detail as follows:

[0069] Data source

[0070] The present application carries out branch-leaf separation based on a large public data set FOR-instance. The data set includes five subsets, and the data is collected from representative forests in Norway, Czech Republic, Australia, New Zealand and Austria. All the data is collected by laser radar, but there are significant differences in plot design, forest type, sensor type, data collection mode, etc. Figure 5 The point cloud data used in the present application is shown in the schematic diagram. Figure 5 The first five data sets in the present application are shown in the schematic diagram. Figure 5 (a)-(e) are European pine forest, spruce forest, original hard-leaf eucalyptus forest, radiata pine forest and deciduous alluvial forest, respectively. Figure 5 (f) shows the simulated tree point cloud data synthesized by a simulator. The simulator can generate custom tree point cloud and capture the geometric features of the trunk and leaves, thereby automatically generating a large amount of training data.

[0071] Experimental conditions

[0072] The experimental hardware and software configuration of the present application is as follows: the CPU is Intel Xeon(R)Platinum 8362, the GPU is a single NVIDIA RTX 3090 with 24GB of video memory, the operating system is ubuntu 18.04, the deep learning framework is PyTorch1.9.0, and the development environment is Python3.8.

[0073] The initial learning rate of the experiment of the application is set to 0.1, the batch size is set to 10, the number of threads during training is 10, the momentum parameter of batch normalization is 0.02, and the stochastic gradient descent (SGD) optimizer is configured. In the non-growth period and growth period of the super point, different max_epoch are set, which are 500 and 800 respectively, and the number of iterations of training is also set differently, which are 10000 and 30000 respectively. The number of threads used to load training data and the number of threads used to cluster data are 10 and 4 respectively. The voxel size used in sparse convolution is 0.05, and the number of original units used during training is 50. The initial number of super points is 80; the number of super points after clustering and fusion is not less than 20.

[0074] Regarding evaluation indicators

[0075] Several precision evaluation indicators are selected for performance evaluation of the unsupervised semantic segmentation deep learning network, including overall segmentation accuracy overall accuracy (oAcc), average class accuracy mean Accuracy (mAcc), and average intersection over union mean intersection over union (mIoU). The definitions of the indicators are as follows:

[0076] (1) oAcc: The proportion of samples correctly classified among all samples.

[0077] (2) mAcc: Calculate the accuracy for each class separately, and then take the average.

[0078] (3) mIoU: It is the average value of Intersection over union (IoU) calculated for all classes, which provides a quantitative indicator of the overall performance of the model on different classes, measuring the degree of overlap between the predicted results and the true labels.

[0079] The specific formula of the indicators is shown in Table 1, where C represents the number of segmentation categories, i represents the i-th semantic category, and the value range of i is {0, 1, …, C}. TP i is the true positive of the i-th category, TN i is the true negative of the i-th category, FP i is the false positive of the i-th category, and FN i is the false negative of the i-th category. The higher the mIoU, the closer to the true value.

[0080] Table 1 Common evaluation indicators for point cloud segmentation

[0081]

[0082]

[0083] The implementation effects of the unsupervised semantic segmentation method of the present application are described in detail below:

[0084] The branch-leaf separation results

[0085] Based on the above six point cloud datasets, the network of the present application obtained an OA average of 66.842%, an mAcc average of 49.240%, and an mIoU average of 37.438%. As shown in Figure 6 The visualized point cloud segmentation results show that the method of the present application can perform the branch-leaf separation task, and obtains good segmentation results on different datasets or different tree species. Figure 6 (a), (b) and (d) are the segmentation results of coniferous trees. Obviously, the network of the present application can accurately separate the trunk and leaves from the point cloud, but the discrimination for small branches is not high. Figure 6 (c) presents a poor segmentation result, because the dataset is collected by an airborne laser radar, and the point cloud density is sparse and the quality is poor. Figure 6 The segmentation result of (e) shows that the method of the present application has high segmentation accuracy for high-density broadleaf forest point clouds collected by a ground-based laser radar, and can accurately distinguish the leaf, trunk and branch components of the forest. Even for Figure 6 For the point cloud generated by the simulator in (f), the network of the present application also obtains a certain separation effect.

[0086] In addition, the network of the present application is also used for branch-leaf separation of a single tree, and obtains higher segmentation accuracy than a forest: the oAcc average is 76.97%, the mAcc average is 64.65%, and the mIoU average is 50.63%. As shown in Figure 7 The branch-leaf separation results of a single tree are shown, and each tree here is randomly selected from each dataset, including: coniferous trees (a, b and d), broadleaf trees (c and e), and simulated trees (f). The broadleaf trees in the deciduous alluvial forest still obtain the best segmentation effect. Figure 7 Figure 7 Figure 7

[0087] Comparison of the network of the present application with other networks

[0088] Table 2 is a performance comparison of the network of the present application with the mainstream unsupervised point cloud segmentation networks Growsp and PointDC in the branch-leaf separation task. In terms of indicators, the network of the present application has the best segmentation effect compared with Growsp and PointDC.

[0089] Comparison of unsupervised point cloud segmentation networks

[0090] ​​​

[0091] Figure 8 Further details of the segmentation of single trees are given for the three methods described above. Considering the differences between tree species and the geometric differences between the tree trunk and leaf components caused by different tree species, one tree is selected from each of the three data sets for comparison. Tree1 is a Norway spruce; Tree2 is a radiata pine forest; and Tree3 is a deciduous alluvial forest. In order to highlight the superiority of the network model of the present application over other models, we selected the original point cloud data, the original model Growsp and PointDC for comparison experiments. Among them Figure 8 The segmentation results of different networks are visualized and compared, and the segmentation details of different tree species are compared. One of the difficulties in separating the branches and leaves of a tree point cloud is how to distinguish the intersection part of the woody component and the leaf component. The segmentation effect of the network of the present application at the pure wood position, the pure leaf position and the branch-leaf intersection position is better than that of the other two networks. However, there is still a lot of room for improvement in the differentiation of lateral branches of needle-leaved trees for the three networks. In addition, Table 3 shows the segmentation accuracy of a single tree in the three methods.

[0092] Table 3 Comparison of single tree segmentation accuracy of different networks

[0093]

[0094] Regarding the ablation experiment

[0095] In order to test the effectiveness of the method described in the present application and verify the role and contribution of which module in the proposed unsupervised semantic segmentation deep learning network, the DPA module and the PFCI module are removed from the network respectively. Table 4 shows the quantitative comparison after removing different modules: Model I is the baseline (Growsp), which takes the sparse convolutional neural network as the backbone network (oAcc is 56.632, mAcc is 46.986, and mIoU is 32.234); Model II only includes the DPA module (oAcc is 64.682, mAcc is 50.154, and mIoU is 36.742); Model III only includes the PFCI module (oAcc is 66.142, mAcc is 55.920, and mIoU is 36.860); and Model IV is the complete network of the present application (oAcc is 68.842, mAcc is 49.240, and mIoU is 37.438).

[0096] Table 4 Ablation experiment results

[0097]

[0098] The ablation experiment results show that the accuracy of the unsupervised semantic segmentation deep learning network model is greatly reduced after deleting the DPA module. The DPA module plays an important role in extracting and focusing on the neighborhood geometric features. Deleting the PFCI module will also cause the segmentation accuracy to be greatly reduced. The PFCI module is not only conducive to extracting the geometric features and color features of the point cloud, but also further fuses these features and generates a comprehensive feature representation. However, the network model of the present application still has a greater space for improvement in identifying needle forest branches.

[0099] Robustness test of the network

[0100] In fact, the point cloud data obtained in real scenes is usually incomplete, and may be affected by the mutual occlusion of the ground terrain and vegetation regardless of the data acquisition method. Therefore, it is necessary to analyze the robustness of the network. In the present application, different percentages of random occlusion (30%, 60% and 90%) are performed on a single tree point cloud to verify the robustness of the method described in the present application.

[0101] Table 5 quantitatively shows the single tree segmentation accuracy under different occlusion rates. When the point cloud is 30% occluded, the segmentation accuracy of the unsupervised semantic segmentation deep learning network (oAcc = 80.546%, mAcc = 53.921% and mIoU = 43.254%) is only slightly lower than that of the complete point cloud (oAcc = 81.688%, mAcc = 54.598% and mIoU = 45.406%). When the point cloud is 60% occluded, the segmentation accuracy of the network decreases to 59.630% of oAcc, 49.184% of mAcc and 35.760% of mIoU. But when the point cloud is 90% occluded, the segmentation accuracy of the network does not continue to decrease, but slightly increases (oAcc = 63.434%, mAcc = 50.980%, mIoU = 30.423%). The results show that the method described in the present application exhibits very good robustness. The present application can tolerate all point cloud occlusions below 30%, and even when the point cloud is randomly occluded by 60%-90%, it can still maintain a certain degree of segmentation accuracy. When the occlusion ratio of the point cloud is large, most of the neighborhood information and edge information will be lost, which may be the main reason for the decrease in accuracy. On the other hand, the unsupervised semantic segmentation deep learning network uses spatial geometric features as the basis for point-by-point classification, which to some extent makes up for the loss of information. Therefore, the network of the present application can still maintain a certain degree of branch identification even under a large occlusion rate (more than 60%), and the visual segmentation results are as shown in Figure 9

[0102] Table 5 shows the single tree point cloud segmentation accuracy under different occlusion rates

[0103]

[0104] ​Test on network generalization

[0105] To verify the generalization of the unsupervised semantic segmentation deep learning network of the present application, tests were conducted on the public dataset S3DIS. S3DIS is a large-scale indoor RGB-D dataset containing 250 million points, which was collected in 6 indoor areas in 3 buildings, including classrooms and offices, etc., with a total coverage area of more than 600 m 2 The annotated objects are divided into 13 categories (such as ceiling, floor, wall, door, etc.) for training and evaluating the semantic segmentation model. In this embodiment, Area_5 is taken as the test set, and the rest is taken as the training set. Figure 10 The comparison of the visual results (3 scenes are randomly selected) is shown. The results show that the network model of the present application has sufficient generalization ability and can achieve good segmentation effect on various datasets.

[0106] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes to the present application without creative labor based on the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning or limited experiments based on the existing technology according to the concept of the present application shall be within the protection scope defined by the claims.

Claims

1. An unsupervised semantic segmentation method for separating branches and leaves from forest point clouds, characterized by: The method comprises the following steps: S1: Obtain 3D point cloud data of the forest scene; S2: extracting features from the three-dimensional point cloud data based on an unsupervised semantic segmentation deep learning network to obtain a comprehensive feature representation of the point cloud; S3: Generate and cluster super points based on the comprehensive feature representation to generate pseudo labels to achieve the separation of branches and leaves; the unsupervised semantic segmentation deep learning network does not require prior training through manually annotated point clouds with semantic labels.

2. The unsupervised semantic segmentation method according to claim 1, characterized in that Each point in the three-dimensional point cloud data in step S1 contains eight feature dimensions, namely spatial coordinates {x, y, z}, color features {R, G, B}, normal vector and curvature.

3. The unsupervised semantic segmentation method according to claim 2, characterized in that The feature extraction in step S2 includes: using a sparse convolutional neural network as the backbone network, combining a point cloud feature convolution integrator to fuse the eight feature dimensions, and introducing a dual-focus attention module to improve global feature attention.

4. The unsupervised semantic segmentation method according to claim 1, characterized in that The generation and clustering of superpoints in step S3 includes: constructing initial superpoints and combining points with similar geometric features into initial superpoints; during the non-growth period and growth period of the superpoints, expanding the initial superpoints to form larger superpoints based on feature similarity; and aggregating the superpoints using the K-means algorithm.

5. The unsupervised semantic segmentation method according to claim 3, characterized in that The processing process of the point cloud feature convolution integrator includes: S21: Divide the eight feature dimensions of the three-dimensional point cloud data into two parts, the first part is the spatial coordinates {x, y, z}, and the second part is the color features {R, G, B}, normal vector and curvature; S22: Process the first part using a VecKM module to obtain a first feature output; S23: Processing the second part through linear transformation and nonlinear activation function to obtain a second feature output; S24: Concatenate the first feature output and the second feature output to obtain the comprehensive feature representation.

6. The unsupervised semantic segmentation method according to claim 5, characterized in that In step S22, the processing of the VecKM module includes: after processing the spatial coordinates {x, y, z}, converting the complex vector into a real vector by calculating the square norm of the complex vector to obtain the first feature output.

7. The unsupervised semantic segmentation method according to claim 3, characterized in that In step S2, the dual-focus attention module includes a channel attention module and a spatial attention module, and the processing process can be expressed as: in, Represents element multiplication, F is the intermediate feature map in the feature extraction process, is the channel attention value, M s The spatial attention value, F′ is the final refined output.

8. The unsupervised semantic segmentation method according to claim 4, characterized in that The constructing of the initial super point includes dividing the three-dimensional point cloud data into a plurality of fragment blocks, each fragment block contains points with similar geometric features, and each fragment block serves as an initial super point.

9. The unsupervised semantic segmentation method according to claim 4, characterized in that The training parameters of the non-growth period and the growth period of the super point are different, wherein the max_epoch of the non-growth period is 500 and the number of iterations is 10,000; the max_epoch of the growth period is 800 and the number of iterations is 30,000.

10. The unsupervised semantic segmentation method according to claim 4, characterized in that The initial number of super points is 80, and the number of super points after cluster fusion is no less than 20.

Images