A point cloud data leaf separation method

Abstract

The application discloses a point cloud data wood and leaf separation method, acquires tree three-dimensional point clouds, and pre-processes to obtain normalized point clouds; a point feature vector containing coordinates, curvatures and linearity is constructed, a point cloud Transformer segmentation network integrating a scale self-adaptive deformable attention is input, and each point trunk, branchlet or leaf label is output, so that wood point cloud and leaf point cloud separation results are obtained; in the training stage, a branchlet F1 guide loss is introduced and combined with a reweighted or focal loss; after reasoning, a skeleton is extracted based on voxelization refinement, connectivity constraints are applied to correct misclassification and remove isolated points, and a structure-continuous wood point cloud is output.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional point cloud data processing and intelligent segmentation technology, specifically to a method for separating leaf data from point cloud data. Background Technology

[0002] With the deepening application of 3D LiDAR technology in forestry surveys, fine tree modeling, landscape asset management, and plant structural phenotypic analysis, acquiring high-precision and resolvable 3D tree structural data has become a crucial industry requirement. Especially with the widespread adoption of acquisition methods such as terrestrial LiDAR (TLS), handheld LiDAR, backpack LiDAR, and Mobile Measurement Systems (MLS), high-density point cloud data containing elements such as trunks, main branches, twigs, and leaves can be rapidly obtained. However, tree point clouds typically contain both woody and leaf points, which differ significantly in geometric morphology, spatial distribution, and structural continuity. Accurately distinguishing between woody and leaf points is often an indispensable prerequisite for tree structural modeling (e.g., skeleton extraction, branch topology reconstruction), parameter inversion (e.g., diameter at breast height, branch gradation, branching angle, canopy volume), and physiological and ecological analysis (e.g., canopy light interception efficiency, leaf area index estimation). If leaf and woody point clouds are mixed, it will directly affect the accuracy of structural reconstruction and even lead to topological errors, significantly interfering with the reliability of subsequent parameter calculations.

[0003] In existing technologies, the separation of woody components and leaves from tree point clouds typically employs rule-based segmentation based on geometric statistical features, classification methods based on artificial features, and point cloud semantic segmentation methods based on deep learning. Although some progress has been made in related research, this task still faces significant challenges due to the high complexity and multi-scale characteristics of tree structures. On the one hand, leaf morphology is highly variable, and point clouds are dense and disordered, exhibiting significant overlap with twigs in terms of scale, density, curvature, and spatial distribution. Traditional geometric statistical methods lack stability under different tree species and growing seasons. On the other hand, twigs and small-scale leaves often exhibit similar sizes and local curvatures in point clouds, leading existing algorithms to easily misclassify twigs as leaves, resulting in incomplete branch structures and affecting skeleton and topology reconstruction. Simultaneously, different acquisition devices show significant differences in point cloud density, noise levels, and viewpoint coverage, making the model's generalization ability across devices and scenarios a major obstacle to engineering applications. Furthermore, in complex scenarios such as urban greening or forest understory, factors such as background vegetation, ground reflection, and occlusion can amplify classification uncertainty, making it more difficult for the separation results to meet the accuracy requirements of subsequent structural modeling.

[0004] Therefore, in order to improve the reliability of tree 3D structural modeling, enhance the accuracy of separating wood components from leaves, and especially ensure the complete identification of trunks, main branches, and twigs, it is necessary to provide a more robust, automated tree point cloud separation technology with strong generalization ability to better support the demand for high-quality basic point cloud data in applications such as digital forestry and urban greening management. Summary of the Invention

[0005] The purpose of this application is to propose a method for separating tree trunks and leaves from point cloud data. By preprocessing the 3D point cloud of trees, constructing point features, and performing point-level classification based on an improved point cloud Transformer segmentation network, the method can accurately distinguish between the trunk, branches, and leaves. Furthermore, by combining post-processing with connectivity constraints, isolated misclassified points can be corrected and the continuity of the branch structure can be enhanced. This forms a closed-loop processing mechanism of "acquisition-segmentation-error correction-output", thereby improving the reliability and usability of the tree trunk and leaf separation results in applications such as fine tree modeling, skeleton extraction, and parameter inversion.

[0006] A method for separating leaf nodes from point cloud data, comprising:

[0007] S1. Obtain the three-dimensional point cloud data of the tree to be processed, wherein the three-dimensional point cloud data of the tree includes at least points representing wood and leaf points;

[0008] S2. Preprocess the three-dimensional point cloud data of the trees to obtain a normalized point cloud;

[0009] S3. Construct point feature vectors for each point in the normalized point cloud. The point feature vectors include at least spatial coordinate features and further include multi-scale geometric features.

[0010] S4. Input the point feature vector of each point into the point cloud Transformer segmentation network. The point cloud Transformer segmentation network includes a scale-adaptive deformable attention module and performs downsampling on the point cloud during the encoding process to obtain a set of sampling points in order to extract multi-scale contextual features of thick trunk, thin branches and leaves.

[0011] S5. Based on the classification probability output by the point cloud Transformer segmentation network, assign a category label of trunk, twig or leaf to each point, and obtain the separation result of wood point cloud and leaf point cloud based on the category label.

[0012] S6. Post-process the separation results, use connectivity constraints to remove isolated misclassified points and enhance the spatial coherence of branches, and output a woody point cloud with a continuous structure.

[0013] In some implementations, the woody point cloud is a set of points labeled with both trunk and twig tags.

[0014] In some implementations, local curvature and linearity are introduced as geometric features in or after step S3, and the local curvature and linearity are encoded into the network input layer.

[0015] In some implementations, the scale-adaptive deformable attention module includes:

[0016] Multiple candidate neighborhoods are established for the sampled points. The candidate neighborhoods are established using a radius or a K1 value, where K1 is the number of K nearest neighbor points.

[0017] The output scale weight vector is composed of a multilayer perceptron (MLP) layer and a softmax layer, wherein the input of the MLP layer is point features.

[0018] In some implementations, in step S4, deformable sampling is used to select a small set of sampling points, so that each query point interacts only with key points in the set of sampling points.

[0019] In some implementations, in step S4, K2 offset vectors are predicted for each point at scale s, and several matching points are selected in the neighborhood based on the offset vectors, where K2 is 8 or 16.

[0020] In some implementations, in step S4, attention outputs are calculated for each set of sampling points at each scale, and the attention outputs at multiple scales are weighted and fused according to the scale weight vector to obtain the output of the scale-adaptive deformable attention module.

[0021] In some implementations, the point cloud Transformer segmentation network is built on the PointTransformerV3 framework, and the scale-adaptive deformable attention module is integrated into the attention layer of the framework.

[0022] In some implementations, the point cloud Transformer segmentation network employs a hierarchical structure of a five-stage encoder and a four-stage decoder.

[0023] In some implementations, the block depth of each stage of the encoder is 2, 2, 2, 6, 2, the number of channels is 32, 64, 128, 256, 512, and the number of heads is 2, 4, 8, 16, 32.

[0024] In some implementations, the block depth of each stage is 2, 2, 2, 2, the number of channels is 64, 64, 128, 256, and the number of heads is 4, 4, 8, 16; the downsampling step size is 2, 2, 2, 2; and the input is a six-dimensional point feature, which includes xyz, intensity, curvature, and linearity.

[0025] In some implementations, an F1-score or equivalent F1-guided loss for the twig category is introduced into the loss function to encourage the model to improve both precision and recall. Reweighting or focus loss is used to address class imbalance, preventing the model from being overly biased towards the leaf class. The wood point cloud is voxelized and refined to extract tree skeleton nodes. Spatial connectivity between skeleton points is established through distance and angle constraints to ensure the spatial connectivity of branches, where nodes with a degree of 1 are less than 0.2m apart and have an directional angle less than 60 degrees. Based on the skeleton point connectivity, misclassified twig points are corrected and assigned to the twig label, while isolated misclassified points are removed.

[0026] Compared with the prior art, the present invention has the following significant advantages:

[0027] 1. Enhanced multi-scale structural adaptability: By introducing a scale-adaptive deformable attention mechanism into the point cloud Transformer segmentation network, and weighting and fusing the multi-scale attention output based on the scale weight vector generated by the candidate neighborhood, the model can simultaneously take into account the global context of the trunk and the local details of the branches and leaves, thereby improving the ability to distinguish between the trunk, branches and leaves and reducing misclassification and omission caused by the mismatch of fixed neighborhood scales.

[0028] 2. More complete twig recognition and more continuous wood structure: By using deformable sampling to select a small number of key points for interaction and predicting the offset vector at the scale to select the best matching point in the neighborhood for attention calculation, redundant calculation is reduced while strengthening the feature aggregation of the slender structure of branches. This is conducive to improving the recall rate of twig regions, reducing the problem of branch breakage caused by twigs being misjudged as leaves, and enhancing the structural integrity of wood point cloud.

[0029] 3. Effectively alleviate training bias caused by class imbalance: To address the class imbalance problem in tree point clouds, where leaf points account for a high proportion and twig points account for a low proportion, an F1-guided loss for twigs is introduced into the loss function. This is combined with reweighting or focus loss to suppress the dominant influence of easily classified samples on the gradient. As a result, the accuracy and recall of twigs are improved simultaneously during the training phase, avoiding excessive bias of the model towards leaf categories and improving the stability of the separation results.

[0030] 4. The output results can directly support structural modeling and parameter inversion: By introducing voxelization and refinement-based skeleton node extraction after inference, and using distance and angle constraints to establish skeleton connectivity, misclassified points are corrected and isolated misclassified points are removed, so that the output wood point cloud has better spatial coherence and topological consistency, making it more suitable for subsequent applications such as skeleton extraction, branch topology reconstruction, calculation of trunk parameters such as diameter at breast height, and canopy structure analysis.

[0031] 5. Improved engineering adaptability and robustness: The method in this application combines point-level probability output with connectivity constraint post-processing, which can maintain good separation performance under different point densities, occlusion and noise levels. This is conducive to improving the applicability of the method under multi-source acquisition conditions such as TLS, handheld, backpack and mobile measurement, and enhances the method's ability to be implemented and its continued use value in scenarios such as greening maintenance, digital forestry and urban garden asset management. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a point cloud data leaf separation method according to the present invention.

[0033] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Example 1:

[0035] like Figure 1 As shown, this embodiment further proposes a point cloud data leaf separation method to separate the wood point cloud and leaf point cloud in the three-dimensional point cloud of a tree, and output a structurally continuous wood point cloud; wherein, the method can be executed by the processor of an electronic device, and the processor calls the program instructions in the memory to complete S1 to S6; in the traditional solution, the branches and leaves highly overlap in scale and local geometric features, which easily causes misclassification of branches and structural breakage of wood components. This application solves the above problems through the process of "multi-scale feature construction - scale adaptive attention segmentation - connectivity constraint error correction".

[0036] S1. Obtaining 3D point cloud data of the tree to be processed. This embodiment further proposes to obtain 3D point cloud data of the tree to be processed. The 3D point cloud data of the tree includes at least points representing wood and leaf points. Each point in the point cloud includes at least 3D coordinate information (xyz) and may include an intensity attribute. Its function is to provide raw input data for the subsequent segmentation network. In traditional schemes, if the input boundary is unclear, it can easily lead to unstable separation targets. This application solves the above problem by clarifying the input type and the object to be separated in S1.

[0037] S2. Preprocessing to obtain a normalized point cloud. This embodiment further proposes to preprocess the three-dimensional point cloud data of the tree to obtain a normalized point cloud. The preprocessing is used to convert the point cloud into a normalized point cloud that can be directly input into the network. Its function is to suppress misclassification caused by noise and density differences and improve the stability of network inference. In traditional schemes, unnormalized point clouds will amplify the errors caused by uneven point density and occlusion. This application solves the above problems by outputting a normalized point cloud through S2.

[0038] S3. Constructing point feature vectors and introducing multi-scale geometric features. This embodiment further proposes to construct point feature vectors for each point in the normalized point cloud; wherein, the point feature vectors include at least spatial coordinate features and further include multi-scale geometric features; in or after step S3, local curvature and linearity are introduced as geometric features, and the local curvature and linearity are encoded into the network input layer; its function is to enhance the separability of woody components (trunks, twigs) and leaves when their local morphology is similar; in traditional schemes, relying only on coordinates or weak geometric features will lead to twigs being misclassified as leaves. This application solves the above problem by strengthening the curvature and linearity.

[0039] In some implementations, the input is a six-dimensional point feature, which includes xyz, intensity, curvature, and linearity, so that the network can simultaneously acquire spatial location, reflection intensity, and local geometric property information at the input layer.

[0040] S4. Inputting the point cloud into a Transformer segmentation network and extracting multi-scale contextual features: This embodiment further proposes inputting the point feature vectors of each point into the point cloud Transformer segmentation network; wherein, the point cloud Transformer segmentation network includes a scale-adaptive deformable attention module, and performs downsampling on the point cloud during the encoding process to obtain a set of sampled points, so as to extract multi-scale contextual features of thick trunks, thin branches, and leaves; its function is to reduce computational redundancy and enhance the aggregation effect on thin branch structures while ensuring multi-scale modeling capability; in traditional schemes, fixed neighborhood scale or full attention interaction is prone to introducing redundancy and noise, and this application solves the above problems through downsampling and deformable attention.

[0041] In some implementations, the point cloud Transformer segmentation network is built on the PointTransformerV3 framework, and the scale-adaptive deformable attention module is integrated in the attention layer of the framework; its function is to improve the recognition stability of thick trunks and thin branches under complex canopy occlusion conditions by utilizing the Transformer's ability to model long-range dependencies and contextual relationships.

[0042] In some implementations, the point cloud Transformer segmentation network adopts a hierarchical structure of a five-stage encoder and a four-stage decoder. The encoder has block depths of 2, 2, 2, 6, and 2 at each stage, with 32, 64, 128, 256, and 512 channels, and 2, 4, 8, 16, and 32 multi-head numbers, respectively. The decoder has block depths of 2, 2, 2, and 2 at each stage, with 64, 64, 128, and 256 channels, and 4, 4, 8, and 16 multi-head numbers, respectively. The downsampling stride is 2, 2, 2, 2. Its function is to gradually abstract structural semantics during the encoding stage and restore spatial details during the decoding stage, thereby balancing the coarse-grained global context with the fine-grained local details.

[0043] (1) Implementation of the scale-adaptive deformable attention module: This embodiment further proposes an implementation method for the scale-adaptive deformable attention module; wherein:

[0044] (a) Multiple candidate neighborhoods are established for the sampled points. The candidate neighborhoods are established using a radius or a K1 value, where K1 is the number of neighborhood points of the K nearest neighbors. Its function is to provide neighborhood context candidates at different scales.

[0045] (b) Output scale weight vector, which is composed of a multilayer perceptron (MLP) layer and a softmax layer, wherein the input of the MLP layer is point features; its function is to adaptively select a more appropriate scale contribution based on the structural properties of the points.

[0046] (c) In step S4, deformable sampling is used to select a small set of sampling points so that each query point interacts only with the key points in the set of sampling points; its purpose is to reduce the computational redundancy of full neighborhood attention and reduce noise interference.

[0047] (d) In step S4, K2 offset vectors are predicted for each point at scale s, and several matching points are selected in the neighborhood according to the offset vectors, where K2 is 8 or 16; its function is to locate key interaction points in the neighborhood by offset guidance, thereby enhancing the aggregation ability of sparse and slender structures such as branches.

[0048] (e) In step S4, attention output is calculated for the sampling point set at each scale, and the attention outputs at multiple scales are weighted and fused according to the scale weight vector to obtain the output of the scale adaptive deformable attention module; its function is to realize the adaptive fusion of multi-scale context information and improve the differentiation effect of thick trunk, thin branches and leaves.

[0049] S5. Generate category labels and output the separation results of woody components and leaves. This embodiment further proposes to assign category labels of trunk, twig or leaf to each point based on the classification probability output by the point cloud Transformer segmentation network, and obtain the separation results of woody point cloud and leaf point cloud based on the category labels. Its function is to form a woody point set and a leaf point set that can be directly used for structural modeling. In traditional schemes, binary classification is prone to sacrificing the twig recognition effect. This application improves the twig differentiation ability and ensures the integrity of woody components by outputting three types of labels.

[0050] In some implementations, the wood point cloud is a set of points labeled with trunk and twig tags, and the leaf point cloud is a set of points labeled with leaf tags.

[0051] In some implementations, to improve the learning performance of branch categories under imbalanced sample conditions, an F1-score or equivalent F1-guided loss for branch categories is introduced into the loss function to encourage the model to improve both precision and recall. Furthermore, reweighting or focus loss is used to address class imbalance and prevent the model from being overly biased towards leaf categories. This improves branch recall and suppresses training bias caused by the dominant gradient of leaf categories. Conventional loss methods in traditional schemes tend to bias the model towards leaf categories; this application addresses these issues through F1-guided and imbalance suppression mechanisms.

[0052] S6. Post-processing and connectivity constraint error correction, outputting a structurally continuous wooden point cloud. This embodiment further proposes to post-process the separation results, using connectivity constraints to remove isolated misclassified points and enhance the spatial coherence of branches, outputting a structurally continuous wooden point cloud; its function is to perform structural prior error correction on local misclassifications in the network output and improve the continuity of branches; in traditional schemes, isolated misclassified points and branch breaks are prone to occur in the inference output. This application solves the above problems through connectivity constraint post-processing.

[0053] In some implementations, the wood point cloud is voxelized and refined to extract tree skeleton nodes. Spatial connectivity of the skeleton points is established through distance and angle constraints to ensure the spatial connectivity of branches. Nodes with a degree of 1 are less than 0.2m apart and have an directional angle of less than 60 degrees. Based on the skeleton point connectivity, twigs that are mistakenly classified as leaves are corrected and assigned to the twig label. Isolated misclassified points are removed, thereby outputting a structurally continuous wood point cloud.

[0054] Although this application discloses several aspects and embodiments, other aspects and embodiments will be obvious to those skilled in the art. Various modifications and improvements can be made without departing from the concept of this application, and these all fall within the scope of protection of this application. The various aspects and embodiments disclosed in this application are for illustrative purposes only and are not intended to limit this application. The actual scope of protection of this application is determined by the claims.

Claims

1. A method for separating leaf elements from point cloud data, characterized in that, include: S1. Obtain the three-dimensional point cloud data of the tree to be processed, wherein the three-dimensional point cloud data of the tree includes at least points representing wood and leaf points; S2. Preprocess the three-dimensional point cloud data of the trees to obtain a normalized point cloud; S3. Construct point feature vectors for each point in the normalized point cloud. The point feature vectors include at least spatial coordinate features and further include multi-scale geometric features. S4. Input the point feature vector of each point into the point cloud Transformer segmentation network. The point cloud Transformer segmentation network includes a scale-adaptive deformable attention module and performs downsampling on the point cloud during the encoding process to obtain a set of sampling points in order to extract multi-scale contextual features of thick trunk, thin branches and leaves. S5. Based on the classification probability output by the point cloud Transformer segmentation network, assign a category label of trunk, twig or leaf to each point, and obtain the separation result of wood point cloud and leaf point cloud based on the category label. S6. Post-process the separation results, use connectivity constraints to remove isolated misclassified points and enhance the spatial coherence of branches, and output a woody point cloud with a continuous structure.

2. The point cloud data leaf separation method as described in claim 1, characterized in that, The woody point cloud is a set of points labeled with both trunk and twig labels.

3. The point cloud data leaf separation method as described in claim 1, characterized in that, In or after step S3, local curvature and linearity are introduced as geometric features, and the local curvature and linearity are encoded into the network input layer.

4. The point cloud data leaf separation method as described in claim 1, characterized in that, The scale-adaptive deformable attention module includes: establishing multiple candidate neighborhoods for the sampled points, wherein the candidate neighborhoods are established using a radius or a K1 value, where K1 is the number of K nearest neighbor points; and outputting a scale weight vector, wherein the scale weight vector is composed of a multilayer perceptron (MLP) layer and a softmax layer, wherein the input of the MLP layer is point features.

5. The point cloud data leaf separation method as described in claim 1, characterized in that, In step S4, deformable sampling is used to select a small set of sampling points, so that each query point interacts only with the key points in the set of sampling points.

6. The point cloud data leaf separation method as described in claim 1, characterized in that, In step S4, K2 offset vectors are predicted for each point at scale s, and several matching points are selected in the neighborhood based on the offset vectors, where K2 is 8 or 16.

7. The point cloud data leaf separation method as described in claim 4, characterized in that, In step S4, attention outputs are calculated for each set of sampling points at each scale, and the attention outputs at multiple scales are weighted and fused according to the scale weight vector to obtain the output of the scale-adaptive deformable attention module.

8. The point cloud data leaf separation method as described in claim 1, characterized in that, The point cloud Transformer segmentation network is built on the PointTransformerV3 framework, and the scale-adaptive deformable attention module is integrated into the attention layer of the framework.

9. The point cloud data leaf separation method as described in claim 1, characterized in that, The point cloud Transformer segmentation network adopts a hierarchical structure of a five-stage encoder and a four-stage decoder.

10. The point cloud data leaf separation method as described in claim 1, characterized in that, The encoder has block depths of 2, 2, 2, 6, and 2 at each stage, channels of 32, 64, 128, 256, and 512, and multiple heads of 2, 4, 8, 16, and 32.

Images