Forest individual tree segmentation method and device based on trunk structure auxiliary labeling and multi-task global clustering

Abstract

The invention provides a forest individual tree segmentation method and device based on trunk structure auxiliary labeling and multi-task global clustering, and the method comprises the steps: firstly obtaining a forest sample point cloud, carrying out the preprocessing of the forest sample point cloud, carrying out the preliminary segmentation of the preprocessed forest sample point cloud, and obtaining an individual tree segmentation result of each point; then, at least one round of regrowth operation based on the trunk center position and the trunk radius is executed on each single tree, and the final target point cloud of each single tree is obtained; and finally, performing semantic-level prediction and offset vector prediction by using the double-task prediction model, and generating individual tree instances based on semantic prediction and offset prediction results. The point-level semantic category and the offset vector pointing to the trunk center line are predicted at the same time under a unified framework, and high-precision and end-to-end automatic segmentation of the single tree in the complex forest point cloud is realized in combination with a global clustering strategy oriented to forest structure characteristics.

Description

A forest tree segmentation method and apparatus based on trunk structure-assisted annotation and multi-task global clustering Technical Field

[0001] This disclosure relates to the field of forestry information technology, and in particular to a method and apparatus for forest tree segmentation based on trunk structure-assisted annotation and multi-task global clustering. Background Technology

[0002] Forest resource surveys are a crucial part of forestry management. Traditional surveys rely on manual plot measurements, which are time-consuming, labor-intensive, and difficult to achieve large-scale coverage. With the development of LiDAR technology, acquiring high-precision 3D forest point cloud data has become possible. How to automatically and accurately segment individual trees from the point cloud directly determines the quality of subsequent acquisition of tree parameters such as diameter at breast height (DBH), height, volume, and canopy closure. Traditional methods mostly rely on algorithms such as trunk detection, graph segmentation, or region growing. However, the segmentation results significantly decrease in scenarios with dense, interwoven canopies and severe foliage occlusion, and the algorithms also exhibit poor robustness. Summary of the Invention

[0003] This disclosure provides at least one forest tree segmentation method and apparatus based on trunk structure-assisted annotation and multi-task global clustering, to improve the accuracy and robustness of tree segmentation.

[0004] According to one aspect of this disclosure, a forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering is provided, including:

[0005] Forest sample point cloud is acquired and preprocessed; the preprocessed forest sample point cloud is then preliminarily segmented to obtain the individual tree segmentation result for each point;

[0006] For each individual tree, perform at least one round of the following operations until a confirmation signal from the annotator is obtained, resulting in the final target point cloud for each tree: Based on the tree trunk seed points at different heights and the tree segmentation results for each point, determine the trunk center position and trunk radius at different height layers corresponding to each tree trunk seed point; determine the trunk centerline based on each trunk center position, and determine the trunk radius model based on each trunk radius; based on the trunk centerline and trunk radius model, re-segment the region point cloud of the tree along the vertical direction of the trunk from the tree trunk seed point to obtain the target point cloud for the tree.

[0007] For each individual tree, the following operations are performed to determine the final point cloud of each tree: The target point cloud of the tree and its point cloud features are input into a dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; For each point, the offset coordinates of the point are determined based on the offset vector of the point relative to the trunk centerline, and a joint feature vector of the point is constructed using the semantic features, the offset coordinates, and the normalized height; Based on the joint feature vector, the points whose semantic feature is the trunk are clustered to obtain a new trunk centerline, and the trunk cluster to which the points whose semantic feature is the branch belong are determined based on the new trunk centerline and the joint feature vector.

[0008] In one possible implementation, the determination of the trunk center position and trunk radius at different height layers corresponding to each trunk seed point, based on the trunk seed points at different heights and the single-tree segmentation results at each point, includes:

[0009] Obtain tree trunk seed points at different heights as determined by the annotators; determine the local cross-sectional neighborhood corresponding to each tree trunk seed point; extract points in the local cross-sectional neighborhood from the points corresponding to each tree based on the single-tree segmentation results; and fit the points in the local cross-sectional neighborhood to obtain the tree trunk center position and tree trunk radius on the height layer corresponding to each tree trunk seed point.

[0010] In one possible implementation, the step of resegmenting the region point cloud of the single tree based on the trunk centerline and trunk radius model, starting from the seed point on the trunk and moving up and down along the trunk, to obtain the target point cloud of the single tree, includes:

[0011] Points belonging to the single tree and points whose shortest distance to the center line of the tree trunk is less than the first preset distance are selected as candidate points;

[0012] For each candidate point, determine the shortest distance from the candidate point to the center line of the tree trunk, the height of the candidate point, and the local point density features of the candidate point; select candidate points whose shortest distance to the center line of the tree trunk is less than a radial distance threshold, whose height is higher than a low height threshold, and whose local point density features meet the local geometric consistency condition as the target point cloud of the single tree.

[0013] In one possible implementation, the semantic features are used to characterize the probability distribution of the category to which a point belongs; the category includes tree trunk, branch, and non-tree.

[0014] The point cloud features include the point's position, intensity, normal vector, local point density, curvature, and distance from the corresponding tree trunk centerline;

[0015] The semantic feature is that the offset vector of the point on the tree trunk is the offset vector between the point and the first target position; the first target position is the point on the center line of the trunk of the single tree to which the point belongs, which is closest in height to the point;

[0016] The semantic feature is that the offset vector of the point of the branch is the offset vector between the point and the second target position; the second target position is the point on the center line of the trunk of the single tree to which the point belongs, which is the point at the second preset distance from the point.

[0017] In one possible implementation, the loss function of the dual-task prediction model includes a semantic segmentation loss term, a offset regression loss term, and an offset field smoothing regularization term; the semantic segmentation loss term is used to characterize the distance between the predicted value of the semantic feature and the true value of the semantic feature; the offset regression loss term is used to characterize the distance between the predicted value of the offset vector and the true value of the offset vector; the offset field smoothing regularization term is used to constrain the smoothness of the change of the offset vector of spatially neighboring points.

[0018] The weight of each point in the offset regression loss term is determined based on the semantic features of the corresponding point; the weight of each point in the offset field smoothing regularization term is determined based on the distance and height difference between spatially adjacent points.

[0019] In one possible implementation, the dual-task prediction model includes a semantic segmentation branch and an offset branch;

[0020] The semantic segmentation branch is trained in the first training phase; the semantic segmentation branch and the offset branch are jointly trained in the second training phase.

[0021] In the training of the offset branch, the weights of high-confidence points in the offset regression loss term are amplified by a factor of N based on a high-confidence mask; N is a positive number greater than 1; the high-confidence mask includes information about high-confidence points, which are points whose semantic features are tree trunks or branches, and whose tree trunk confidence or branch confidence is greater than a confidence threshold.

[0022] The high-confidence mask is updated after every M joint training iterations of the offset branch and the semantic segmentation branch.

[0023] In one possible implementation, it also includes:

[0024] When training the offset branch, the offset field smoothing regularization term is applied to the semantic boundary point and the points in the neighborhood of the semantic boundary point based on the boundary mask.

[0025] The boundary mask includes information about semantic boundary points, which are determined according to the following steps:

[0026] Determine the local neighborhood region of each point;

[0027] Points within a local neighborhood region that represent different categories by semantic features are used as semantic boundary points.

[0028] The boundary mask is updated after every M joint training iterations of the offset branch and the semantic segmentation branch.

[0029] In one possible implementation, the step of clustering points whose semantic feature is the tree trunk based on the joint feature vector to obtain a new tree trunk centerline includes:

[0030] The cluster radius of a point is determined based on its local neighborhood distance and height variance.

[0031] Set the clustering weight of points whose trunk confidence corresponding to semantic features is less than a preset value to be lower than the preset weight;

[0032] Using the determined cluster radius and cluster weights, clustering is performed on points whose semantic feature is the tree trunk based on the joint feature vector to obtain a new tree trunk centerline.

[0033] In one possible implementation, determining the trunk cluster to which the point with the semantic feature of a branch belongs, based on the new trunk centerline and joint feature vector, includes:

[0034] Based on the joint feature vector, the nearest distance between the point whose semantic feature is a branch and the new trunk centerline is determined, and the point whose nearest distance is less than a third preset distance is taken as the branch point of the single tree.

[0035] Based on the height of the branch point, determine the branch cluster to which the branch point belongs; different branch clusters correspond to different height ranges.

[0036] According to another aspect of this disclosure, a forest tree segmentation device based on trunk structure-assisted annotation and multi-task global clustering is provided, comprising:

[0037] The preprocessing module is used to acquire forest sample point clouds and preprocess the forest sample point clouds; the preprocessed forest sample point clouds are then subjected to preliminary segmentation processing to obtain the individual tree segmentation results for each point;

[0038] The auxiliary segmentation module performs at least one round of the following operations on each individual tree until a confirmation signal from the annotator is obtained, resulting in the final target point cloud for each tree: Based on the tree trunk seed points at different heights and the tree segmentation results for each point, the center position and radius of the trunk at different height layers corresponding to each tree trunk seed point are determined; the center line of the trunk is determined based on the center position of each tree trunk, and the trunk radius model is determined based on the radius of each tree trunk; based on the center line and the trunk radius model, the region point cloud of the tree is re-segmented along the vertical direction of the trunk from the tree trunk seed point to obtain the target point cloud of the tree.

[0039] The inference segmentation module performs the following operations on each individual tree to determine the final point cloud of each tree: It inputs the target point cloud of the tree and its point cloud features into a dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; for each point, it determines the offset coordinates of the point based on the offset vector relative to the trunk centerline, and constructs a joint feature vector for the point using its semantic features, offset coordinates, and normalized height; based on the joint feature vector, it clusters the points whose semantic feature is "trunk" to obtain a new trunk centerline; and, based on the new trunk centerline and the joint feature vector, it determines the trunk cluster to which the points whose semantic feature is "branch" belong.

[0040] This disclosure presents a forest tree segmentation method and apparatus based on trunk structure-assisted annotation and multi-task global clustering. It abandons the traditional two-stage process of trunk detection followed by region growth segmentation, instead simultaneously performing semantic feature recognition and trunk centerline offset prediction within a unified framework. Combined with density clustering, it directly outputs individual tree instances, significantly improving the automation and overall efficiency of the segmentation process. This disclosure utilizes offset vectors pointing to the trunk centerline to aggregate and cluster point clouds across the entire map of a single tree, avoiding instance clipping and boundary misclassification caused by spatial block prediction, achieving consistent segmentation in complex forest stand scenarios. In this disclosure, the offset target is the trunk centerline instead of a simple centroid, ensuring that even with overlapping canopies and curved trunks, the point cloud of the same tree can still accurately cluster near the corresponding trunk skeleton, significantly improving segmentation accuracy compared to previous methods. In summary, this disclosure simultaneously predicts point-level semantic categories and offset vectors pointing to the trunk centerline within a unified framework, and combines this with a global clustering strategy oriented towards forest structural characteristics to achieve high-precision, end-to-end automatic segmentation of individual trees in complex forest point clouds.

[0041] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0042] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0043] Figure 1 is a flowchart of the forest single tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering in an embodiment of this disclosure;

[0044] Figure 2 is a schematic diagram of the forest single tree segmentation device based on trunk structure-assisted annotation and multi-task global clustering in an embodiment of this disclosure. Detailed Implementation

[0045] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0046] This disclosure addresses the shortcomings of current segmentation schemes, including low integrity of canopy and trunk segmentation, boundary misclassification and instance fragmentation in block prediction, lack of end-to-end consistency, extremely high cost of high-quality single-tree instance annotation, and the large workload, strong subjectivity, and difficulty in ensuring consistency across large forest stands of traditional purely manual annotation methods. It proposes a forest single-tree segmentation method and apparatus based on trunk structure-assisted annotation and multi-task global clustering. The scheme of this disclosure achieves rapid construction of high-quality single-tree segmentation datasets and end-to-end automated processing from semantic recognition to single-tree instance partitioning, avoiding the accumulation of errors caused by block prediction and post-processing. Compared with existing technologies, this invention has the following fundamental differences:

[0047] Regarding data construction: This publication proposes a semi-automatic annotation process of "automatic trunk detection and initial labeling + rule filtering + manual visual fine-tuning + quality inspection", which makes full use of trunk structure information and greatly improves the construction efficiency and accuracy of forest point cloud single tree instances and semantic annotations, providing a high-quality data foundation for model training and promotion applications.

[0048] In terms of semantic-geometric modeling: This disclosure uses "semantic segmentation + offset vector pointing to the trunk centerline" for joint prediction, which not only distinguishes between the trunk, branches and non-tree points, but also explicitly encodes the trunk skeleton structure, making the offset field highly consistent with the forest structure, thus breaking through the limitations of single semantic classification and simple centroid offset methods.

[0049] Regarding instance generation: This disclosure introduces height and semantic constraints into the offset coordinate space, constructs a joint feature space containing multi-dimensional features such as offset coordinates, normalized height, and semantic confidence, and adopts an adaptive scale full-map density clustering strategy to obtain complete single-tree instances at once, avoiding block merging and boundary breakage problems.

[0050] The technical solution of this disclosure will be described below through specific embodiments.

[0051] Figure 1 shows a flowchart of the forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering in this embodiment. The execution subject of this embodiment is a computing device or component with data processing capabilities. The specific method of this embodiment may include the following steps:

[0052] S110. Obtain the forest sample location cloud and preprocess the forest sample location cloud; perform preliminary segmentation processing on the preprocessed forest sample location cloud to obtain the single tree segmentation result for each point.

[0053] S120. Perform at least one round of the following operations on each tree until a confirmation signal from the annotator is obtained, and obtain the final target point cloud of each tree: Based on the tree trunk seed points at different heights and the tree segmentation results of each point, determine the trunk center position and trunk radius at different height layers corresponding to each tree trunk seed point; determine the trunk centerline based on each tree trunk center position, and determine the trunk radius model based on each tree trunk radius; based on the trunk centerline and trunk radius model, re-segment the region point cloud of the tree from the tree trunk seed point along the vertical direction of the trunk to obtain the target point cloud of the tree.

[0054] Based on the above-mentioned tree trunk seed points at different heights and the tree segmentation results at each point, the center position and radius of the tree trunk at different height layers corresponding to each tree trunk seed point are determined. This can be achieved using the following steps:

[0055] Obtain tree trunk seed points at different heights as determined by the annotators; determine the local cross-sectional neighborhood corresponding to each tree trunk seed point; extract points in the local cross-sectional neighborhood from the points corresponding to each tree based on the single-tree segmentation results; and fit the points in the local cross-sectional neighborhood to obtain the tree trunk center position and tree trunk radius on the height layer corresponding to each tree trunk seed point.

[0056] The above-mentioned re-segmentation of the regional point cloud of a single tree is performed from the seed point on the trunk along the vertical direction of the trunk, based on the trunk centerline and trunk radius model, to obtain the target point cloud of the single tree. This includes: selecting points belonging to the single tree and points whose shortest distance to the trunk centerline of the single tree is less than a first preset distance as candidate points; for each candidate point, determining the shortest distance from the candidate point to the trunk centerline, the height of the candidate point, and the local point density features of the candidate point; and selecting candidate points whose shortest distance to the trunk centerline is less than a radial distance threshold, whose height is higher than a low height threshold, and whose local point density features meet the local geometric consistency condition as the target point cloud of the single tree.

[0057] The aforementioned local point density features are determined according to the following steps: First, obtain the fitted trunk centerline and trunk radius model; calculate the shortest radial distance and corresponding height of each candidate point to the trunk centerline; adaptively broaden the distance threshold according to the trunk radius at that height, and take the effective height range of the trunk as the height threshold; the local point density is the number of statistical points in the neighborhood / neighborhood volume; this neighborhood is determined based on the distance threshold and height threshold.

[0058] The "local geometric consistency condition" mentioned above refers to the point cloud geometry in the neighborhood of the candidate point being consistent with the tree trunk point cloud (e.g., exhibiting columnar / linear distribution, with normal / principal directions consistent with the tree trunk axis), used to exclude non-tree trunk points such as branches, leaves, and the ground.

[0059] S130. Perform the following operations on each individual tree to determine the final point cloud of each tree: input the target point cloud of the tree and the point cloud features of the target point cloud into the dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; for each point, determine the offset coordinates of the point based on the offset vector of the point relative to the trunk centerline, and construct the joint feature vector of the point using the semantic features, the offset coordinates, and the normalized height; based on the joint feature vector, perform density clustering on the points whose semantic feature is the trunk to obtain a new trunk centerline, and determine the trunk cluster to which the points whose semantic feature is the branch belong based on the new trunk centerline and the joint feature vector.

[0060] The aforementioned semantic features are used to characterize the probability distribution of the category to which a point belongs; the categories include tree trunk, branch, and non-tree. Point cloud features include the point's position, intensity, normal vector, local point density, curvature, distance, etc., which can be determined based on the preprocessed point cloud.

[0061] The offset vector of a point whose semantic feature is a tree trunk is the offset vector between that point and the first target position; the first target position is the point on the center line of the trunk of the tree to which that point belongs, which is closest in height to that point; the offset vector of a point whose semantic feature is a branch is the offset vector between that point and the second target position; the second target position is the point at a second preset distance from the point on the center line of the trunk of the tree to which that point belongs, which is closest in height to that point.

[0062] The loss function of the dual-task prediction model includes a semantic segmentation loss term, a offset regression loss term, and an offset field smoothing regularization term. The semantic segmentation loss term represents the distance between the predicted value and the ground truth value of a semantic feature. The offset regression loss term represents the distance between the predicted value and the ground truth value of an offset vector. The offset field smoothing regularization term constrains the smoothing of the offset vector changes between spatially neighboring points. The weight of each point in the offset regression loss term is determined based on the semantic features of that point; the weight of each point in the offset field smoothing regularization term is determined based on the distance and height difference between spatially neighboring points.

[0063] The dual-task prediction model includes a semantic segmentation branch and an offset branch. The semantic segmentation branch is trained in the first training phase, and the semantic segmentation branch and the offset branch are jointly trained in the second training phase. Specifically, when training the offset branch, the weights of high-confidence points in the offset regression loss term are amplified by a factor of N based on a high-confidence mask; N is a positive number greater than 1. The high-confidence mask includes information about high-confidence points, which are points whose semantic features are tree trunks or branches, and whose tree trunk confidence or branch confidence is greater than a confidence threshold. The high-confidence mask is updated after every M iterations of the joint training of the offset branch and the semantic segmentation branch.

[0064] In addition, during the training of the offset branch, based on the boundary mask, the offset field smoothing regularization term is applied to the semantic boundary point and the points in the neighborhood of the semantic boundary point; the boundary mask includes information about the semantic boundary point, which is determined according to the following steps: determining the local neighborhood region of each point; taking the points in the local neighborhood region that have different categories represented by semantic features as semantic boundary points; the boundary mask is updated after every M joint training cycles of the offset branch and the semantic segmentation branch.

[0065] In some embodiments, clustering points whose semantic features are tree trunks based on joint feature vectors to obtain new tree trunk centerlines includes: determining the clustering radius of points based on the local neighborhood distance and height variance of points; setting the clustering weight of points whose tree trunk confidence corresponding to semantic features is less than a preset value to be lower than the preset weight; and using the determined clustering radius and clustering weights, clustering points whose semantic features are tree trunks based on joint feature vectors to obtain new tree trunk centerlines.

[0066] In some embodiments, determining the trunk cluster to which a point whose semantic feature is a branch belongs, based on the new trunk centerline and the joint feature vector, includes: determining the nearest distance between the point whose semantic feature is a branch and the new trunk centerline based on the joint feature vector, and taking points whose nearest distance is less than a third preset distance as the branch points of the single tree; determining the branch cluster to which the branch point belongs based on the height of the branch point; wherein different branch clusters correspond to different height ranges.

[0067] The following describes the disclosed forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering through more specific embodiments.

[0068] Step 1: Data Acquisition and Preprocessing

[0069] (1) Use ground-based lidar (TLS) to acquire forest sample point cloud, which can be extended to multi-source data such as UAV-LiDAR and airborne lidar (ALS) as needed; (2) Denoise, segment and resample the forest sample point cloud, remove isolated noise points and ground points, and retain points related to the forest structure; (3) Normalize the height of the forest sample point cloud with local ground elevation or set reference surface to facilitate unified modeling under different terrain conditions; (4) Point cloud feature extraction: Extract the geometric and echo features such as position (x, y, z), distance R, intensity I, normal vector (n_x, n_y, n_z), local point density, and curvature for each point to provide rich input for the subsequent deep network (i.e., dual-task prediction model).

[0070] Step 2: Intelligent assisted annotation of individual trees in forest point cloud with trunk structure constraints. Here, the target point cloud of each individual tree is obtained.

[0071] In addition, this step can be used to label point clouds, and the labeled point clouds can be used to train a dual-task prediction model.

[0072] To construct a high-quality single-tree instance segmentation and annotation dataset, this disclosure fully utilizes the trunk-based single-tree segmentation function in existing 3D point cloud processing software (such as LiDAR360). Initial single-tree segmentation results are first obtained. Then, through a small amount of manual selection of "trunk seed points," combined with automatic trunk cross-section fitting and region growth along the trunk skeleton, the single-tree instances are finely corrected. This achieves high-precision single-tree instance annotation with minimal manual intervention. Annotators do not need to perform point-by-point semantic annotation; they only need to select clean locations on the trunk and instantly view the segmentation results. If the results are unsatisfactory, seed points can be added for iterative correction.

[0073] Specifically as follows:

[0074] (1) Initial single-tree segmentation

[0075] First, in professional point cloud processing software such as LiDAR360, the tree trunk-based single-tree segmentation function is used to perform initial automatic single-tree segmentation on the preprocessed forest sample point cloud, resulting in an initial instance result with a "single-tree segmentation result" for each point, that is, each point has an initial single-tree ID.

[0076] In complex forest stands, especially in scenarios where there is a large amount of low vegetation under the tree trunk and obvious overlapping of tree crowns, the initial segmentation results often have the following problems: low shrubs and herbaceous vegetation are easily misclassified into individual tree instances corresponding to adjacent tree trunks; the assignment of branches is not accurate enough, with some branches being assigned to the wrong individual trees, or instances sticking together in the overlapping area of ​​tree crowns.

[0077] Here, the initial single-tree segmentation results are used as the basis for coarse annotation, and then fine-tuning of the trunk structure constraints is performed on top of that.

[0078] (2) Selection of seed points on tree trunk and extraction of cross sections

[0079] Annotators review the initial tree segmentation results one by one in the 3D visualization interface. For a specific target tree, annotators select one or more trunk seed points at a height where the trunk shape is relatively regular and there is little surrounding interference. For each trunk seed point, a local cross-sectional neighborhood is constructed near the height of that point, and the point cloud set belonging to the current initial tree ID within this local cross-sectional neighborhood is extracted as input data for trunk cross-section fitting. Annotators only need to perform simple point selection operations and do not need to annotate the semantic category of each point individually.

[0080] (3) Trunk section fitting and centerline construction based on trunk seed points

[0081] For the point set extracted from the height of the layer where the tree trunk seed point is located, geometric parameters such as the center position and radius of the tree trunk cross section on that height layer are estimated by using methods such as circle fitting, line fitting or cylinder fitting, so as to obtain the center position and radius of the tree trunk.

[0082] When multiple tree trunk seed points exist at different heights, cross-sectional fitting can be performed at each height level, and the obtained tree trunk center positions can be connected along the height direction to construct the trunk centerline of the individual tree. Simultaneously, the curve showing the change in trunk radius with height can be obtained based on the trunk radius at different height levels. Points that significantly deviate from the cylindrical shape or are too far from the centerline can be removed during the fitting process, treating them as low-lying vegetation or other interference points to geometrically ensure the cleanliness and stability of the tree trunk model.

[0083] (4) Regional growth re-segmentation with the trunk as the framework

[0084] After obtaining the trunk centerline and trunk radius model, the trunk centerline is used as the growth framework. Starting from the seed point, the individual tree instances are re-segmented along the vertical direction of the trunk: For each candidate point, its nearest distance to the trunk centerline, its corresponding height position, and local point density are calculated; Based on the preset radial distance threshold, height range constraint (i.e., low height threshold), and local geometric consistency condition, it is determined whether the point belongs to the trunk or crown structure of the current individual tree; For points that are too far from the trunk centerline, located at a significant low height threshold, or significantly inconsistent with the current trunk geometric model, they are removed from the individual tree instance or marked as points to be reassigned; For points that belong to the tree but were missed in the original initial segmentation, they are supplemented and merged into the current individual tree instance based on their spatial relationship with the trunk centerline.

[0085] Through the aforementioned regional growth process based on tree trunk structure, a refined single-tree segmentation result can be generated that is highly consistent with the tree trunk geometry, significantly reduces interference from low-lying vegetation at the bottom, and provides more accurate branch attribution.

[0086] (5) Interactive iterative correction and result merging

[0087] After completing an automatic resegmentation, the annotator can immediately view the detailed segmentation results of the individual tree in the 3D view. If any omissions, adhesions, or unsatisfactory local boundaries are found, the annotator can continue to select new trunk seed points at other heights on the trunk. This disclosure refits or updates the trunk centerline and trunk radius model based on the newly added trunk seed points, and then performs region growth resegmentation again on this basis.

[0088] By repeating the lightweight interaction of "selecting seed point - automatically fitting tree trunk - resegmenting region growth - visual inspection", annotators can gradually obtain satisfactory segmentation results for a single tree with fewer clicks, without having to perform tedious point-by-point semantic annotation.

[0089] Within the entire sample plot, the above operations are performed on all or any individual trees that require correction. The resulting refined individual tree instance results are then merged into a unified data structure, ultimately forming a high-precision individual tree instance segmentation and labeling result with a unique individual tree ID for each tree. This provides reliable ground truth data for the subsequent training and performance evaluation of the deep learning model (i.e., the dual-task prediction model).

[0090] The intelligent auxiliary annotation of individual trees in forest point clouds constrained by the above-mentioned tree trunk structure significantly reduces the amount of manual work in the process of annotating individual trees in forest point clouds. It enables annotators to obtain high-quality and immediately verifiable individual tree instance annotation results by selecting only a small number of tree trunk seed points. At the same time, it effectively reduces the problems of misclassification of low vegetation at the bottom of the tree trunk and inaccurate attribution of branches.

[0091] Step 3: Training and Using the Dual-Task Prediction Model

[0092] The input to the dual-task prediction model is the preprocessed point cloud features. The backbone network uses a 3D sparse convolutional network as the feature extraction backbone, extracting multi-scale spatial-semantic features layer by layer to obtain a high-dimensional representation capable of identifying forest structures. The output of the dual-task prediction model includes the output of a semantic head (corresponding to the semantic segmentation branch) and an offset head (corresponding to the offset branch). The semantic head outputs the probability distribution of each point belonging to categories such as "trunk," "branch," and "non-tree," used to distinguish woody structures from background vegetation, noise points, etc., providing semantic constraints for offset prediction and subsequent instance generation. The offset head outputs a 3D offset vector for each point, representing the target displacement of that point towards the centerline of the tree trunk to which it belongs. In this disclosure, the offset label is not a simple instance centroid, but is constructed based on the tree trunk centerline: for any point, the point on the tree trunk centerline closest to its height is selected as the target position; the 3D vector pointing from that point to the target position is used as the offset label.

[0093] In this way, the offset field naturally shrinks along the tree trunk skeleton, which is beneficial for aggregating points of the same tree to the vicinity of its trunk centerline in complex structures such as overlapping canopies and curved trunks, significantly improving the separability of instance clustering.

[0094] To avoid offsets crossing different trees, this disclosure introduces height neighborhood and radius pruning constraints in the offset label construction: points are only allowed to offset to points on the trunk centerline within a certain height difference threshold, and an upper limit is set on the offset length to restrict the offset vector to a reasonable canopy range.

[0095] This disclosure employs a joint loss function to collaboratively optimize the semantic segmentation branch and the offset branch. The overall loss can be expressed as:

[0096] L = L_sem + λ1 * L_off + λ2 * L_smooth

[0097] in:

[0098] L_sem is the semantic segmentation loss term, which can be class-weighted cross-entropy or focus loss, used to address the imbalance in the number of tree trunks, branches, and non-tree samples in forest scenes; L_off is the offset regression loss term, which uses L1 or Smooth L1 loss to measure the difference between the predicted offset vector and the label offset vector; L_smooth is the offset field smoothing regularization term, used to constrain the smooth changes in the offset vectors of spatially neighboring points, reducing local abrupt changes.

[0099] To enhance the synergistic effect of semantics and offset, this disclosure introduces a semantic confidence-weighted offset loss: for the i-th point, weights w_i are constructed using the trunk and branch confidences p_i^trunk and p_i^branch from the semantic header output, for example:

[0100] w_i = α * p_i^trunk + β * p_i^branch (where α > β > 0)

[0101] The migration regression loss is weighted by w_i, which makes the model pay more attention to the migration prediction of the trunk and main branch points during training, and reduces the interference of non-tree or low confidence regions on the migration field.

[0102] The offset smoothing term L_smooth can be defined through spatial adjacency relationships. For example, for the K nearest neighbor pairs (i, j), weights are set according to the distance and height difference between the points, and the difference in the offset vectors of the neighboring points is penalized, thereby improving the overall continuity of the offset field while maintaining the clarity of the tree trunk boundary.

[0103] The training of the dual-task prediction model includes data augmentation and hard example mining. Data augmentation involves random rotation, scaling, translation, and noise addition to the point cloud to improve the model's robustness to different tree species, different sampling postures, and noise conditions. Hard example sampling focuses on sampling training samples from areas with overlapping tree canopies and severe occlusion to enhance the model's ability to distinguish complex scenes. Optimizer: Adaptive optimization algorithms such as AdamW can be used, combined with cosine annealing learning rate scheduling to improve convergence performance.

[0104] The enhancement steps for offset vector prediction in semantic segmentation specifically include:

[0105] 1. High-Confidence Point Selection: Based on the semantic head output, high-confidence trunk and branch points are assigned higher offset loss weights, guiding the offset branch to focus on learning skeleton regions with more reliable structural information. Specifically, in forward inference, the semantic segmentation branch outputs the probability of each category for each point, and takes the highest probability as the confidence of that point. A "high-confidence mask" is generated for points predicted as "trunk" or "branch" with a confidence higher than a threshold. When training the offset branch, a loss weight map is constructed based on this mask, and the offset loss of high-confidence trunk / branch points is multiplied by an amplification factor, thereby guiding the offset branch to focus on learning the offset of skeleton regions.

[0106] 2. Category-Guided Constraints: Semantic labels are used to suppress cross-class offsets. For example, "trunk points" are only allowed to offset to the trunk centerline, while "branch points" are preferentially offset to the vicinity of their corresponding trunks, avoiding confusion between offsets of different categories. When constructing offset labels, different objectives are adopted for different semantic categories: the offset label of a trunk point points to the trunk centerline or a local center point; the offset target of a branch point preferentially points to the vicinity of the trunk of its tree. During the inference phase, the offset points are further constrained by combining semantic labels: trunk points are only clustered among candidate trunk centers, and branch points are only allowed to belong to the nearest semantically consistent trunk instance, thus avoiding confusion between offsets of different categories in implementation.

[0107] 3. Boundary Consistency Correction: Local smoothing constraints are introduced at semantic boundaries to smooth the offset field within its neighborhood, reducing abrupt vector changes caused by semantic uncertainty;

[0108] Based on semantic prediction and local neighborhood relationships (such as through kNN or radius neighborhood), points within the neighborhood with different semantic labels are labeled as "semantic boundary points," forming a boundary mask. When training the offset branch, offset smoothing terms are added only to these boundary points and their neighborhoods, constraining the offset vectors of neighboring points to be as continuous as possible, avoiding abrupt offsets in semantically uncertain regions, thereby improving the stability of the boundary regions.

[0109] 4. Iterative joint optimization: In the early stage of training, the semantic branch can be pre-trained first, and then the offset branch can be jointly optimized. In several training stages, the offset loss weights and boundary masks are periodically updated according to the latest semantic output, so that the two branches promote each other in the iteration.

[0110] In the training process, the semantic segmentation branch is pre-trained separately first to stabilize the semantic results. Then, an offset branch is added, and semantic segmentation loss, offset regression loss, and offset field smoothing regularization are jointly optimized. During joint training, every few epochs, the high-confidence mask and boundary mask are recalculated based on the latest semantic output, and the weights and effective regions of the offset loss are updated accordingly. This ensures that the offset branch always depends on the latest semantic results for optimization, forming an iterative mutual promotion between semantics and offset.

[0111] 5. Enhanced instance center extraction: During the inference phase, by combining semantic constraints and smooth offset fields, the clustering of the offset point set can more accurately aggregate to the center line of the corresponding single tree trunk, thereby improving the completeness and boundary accuracy of instance segmentation.

[0112] During inference, the network outputs semantic features and offset vectors for each point. First, the point coordinates are "relocated" by adding the offset vector. Then, for points whose semantics are tree trunks, clustering or voxel aggregation methods are used to extract the trunk centerline or central cluster; points whose semantics are branches are assigned to the corresponding trunk clusters based on their offset positions, shortest distance, and semantic consistency. Finally, the original point cloud is merged according to the clustering results and semantic labels to obtain a single-tree-level instance mask and relatively complete and accurate instance boundaries.

[0113] Step 4: Reasoning and Global Clustering

[0114] After model training is complete, a global clustering strategy based on forest structure characteristics is adopted during the inference phase to combine semantic prediction and offset prediction results to generate single-tree instances, specifically including:

[0115] (1) Semantic filtering

[0116] First, based on the category probabilities output by the semantic header, points that clearly belong to non-tree categories are removed, and only the trunk and branch points are retained as candidate instance points to reduce the interference of background noise on the clustering results.

[0117] (2) Offset projection and feature construction

[0118] For each retained point, its coordinates (x, y, z) are translated to the offset coordinates (x', y', z') based on the offset vector Δv output by the offset head, so that points of the same tree cluster near the trunk centerline. Simultaneously, a joint feature vector is constructed for each point:

[0119] f_i = [x'_i, y'_i, z'_i, h_i, p_i^trunk, p_i^branch]

[0120] Where h_i is the normalized height, and p_i^trunk and p_i^branch are the trunk confidence and branch confidence of the semantic header output. This joint feature encodes geometric location, height hierarchy, and semantic information in a unified manner, providing stronger discriminative power for subsequent clustering.

[0121] (3) Adaptive global density clustering

[0122] In the joint feature space, a density-based clustering algorithm (such as the improved DBSCAN / HDBSCAN) is adopted, and an adaptive mechanism is introduced for the clustering radius ε and the minimum number of samples min_samples: ε can be dynamically adjusted according to the local average neighborhood distance and height variance of the points, so that a smaller clustering scale is used in dense canopy areas and a slightly larger scale is used in sparse areas; when constructing the neighborhood, point pairs with a height difference exceeding a given threshold are not considered as neighbors to avoid misclustering between upper and lower canopies; for points with low semantic confidence or inconsistent with the local structure, lower weights are given or the assignment is delayed during the clustering process to reduce their interference.

[0123] Through the above design, this disclosure can obtain the partitioning of individual tree instances in a single full-graph clustering operation, avoiding the problems of boundary discontinuity and instance sticking caused by block prediction and then splicing.

[0124] (4) Post-processing and quality control

[0125] The present disclosure further processes the clustering results as follows:

[0126] Small cluster filtering: Remove noisy clusters with too few points or whose spatial range does not clearly conform to the single tree scale; Height and geometric consistency check: Statistically analyze the geometric features such as height range and trunk continuity of each instance, and remove instances with abnormal height or whose shape does not clearly conform to the tree shape; Confidence filtering: Combine semantic confidence and overall instance stability, merge, remove or mark instances with low confidence as awaiting manual review.

[0127] This disclosure explicitly introduces height and semantic constraints in the offset construction and clustering stages, adaptively adjusting the clustering scale, and exhibits better adaptability to complex scenarios with interlaced canopies, small tree spacing, and high stand density. Even in stands with large variations in canopy height and severe shading, it maintains high integrity and boundary accuracy of individual tree instances, making it applicable to various forest types, including natural forests, plantations, and mixed forests. Intelligent auxiliary annotation based on trunk structure constraints significantly improves the efficiency and quality of individual tree data construction. Regarding the construction of labeled data, the intelligent auxiliary annotation scheme for individual trees in forest point clouds based on trunk structure constraints proposed in this disclosure fully utilizes the trunk-based individual tree segmentation function of software such as LiDAR360 to obtain initial individual tree instances. Based on this, annotators can trigger automatic trunk cross-section fitting, trunk centerline construction, and regional growth resegmentation along the trunk framework simply by selecting a few "trunk seed points" in clean locations on the trunk. This disclosure can automatically remove low-lying vegetation points at the bottom of the trunk that are mistakenly classified as individual trees, and correct the problem of inaccurate branch classification in the overlapping area of ​​the tree canopy, thereby generating clean and reliable fine individual tree instances that are highly consistent with the geometric structure of the trunk. On the other hand, annotators do not need to perform point-by-point semantic annotation or manually select complex tree canopies. They only need to add or adjust a small number of seed points as needed based on the visualization results to quickly iterate and obtain satisfactory individual tree segmentation and annotation results.

[0128] This disclosure is designed to fully consider the characteristics of different types of lidar data, making it applicable not only to high-density ground-based lidar (TLS) data but also applicable to multi-source point cloud data such as UAV-LiDAR and airborne lidar (ALS). When changing data sources, only preprocessing and some parameters need to be adjusted, allowing for cross-platform deployment while maintaining the overall framework, demonstrating strong versatility and engineering application value.

[0129] The above embodiments implement a data acquisition and preprocessing process for forest scenarios, utilizing a semi-automatic single-tree and semantic annotation method with trunk structure constraints; implement a dual-task prediction network structure (semantic head + trunk centerline offset head) and a collaborative training mechanism, and integrate a global density clustering strategy that combines offset coordinates, height, and semantic confidence; and simultaneously implement post-processing and quality control for single-tree instances.

[0130] Based on the same inventive concept, this disclosure provides a forest tree segmentation device based on trunk structure-assisted annotation and multi-task global clustering. The steps performed by the components of this device are the same as or similar to those of the method described above, therefore, similar parts will not be repeated. As shown in Figure 2, the forest tree segmentation device based on trunk structure-assisted annotation and multi-task global clustering in this embodiment includes:

[0131] The preprocessing module 210 is used to acquire forest sample location clouds and preprocess the forest sample location clouds; and to perform preliminary segmentation processing on the preprocessed forest sample location clouds to obtain the individual tree segmentation results for each point.

[0132] The auxiliary segmentation module 220 is used to perform at least one round of the following operations on each tree until a confirmation signal from the annotator is obtained, to obtain the final target point cloud of each tree: Based on the tree trunk seed points at different heights and the tree segmentation results of each point, the trunk center position and trunk radius at different height layers corresponding to each tree trunk seed point are determined respectively; the trunk center line is determined based on each tree trunk center position, and the trunk radius model is determined based on each tree trunk radius; based on the trunk center line and trunk radius model, the region point cloud of the tree is re-segmented along the vertical direction of the trunk from the tree trunk seed point to obtain the target point cloud of the tree.

[0133] The inference segmentation module 230 is used to perform the following operations on each tree to determine the final point cloud of each tree: input the target point cloud of the tree and the point cloud features of the target point cloud into the dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; for each point, determine the offset coordinates of the point according to the offset vector of the point relative to the trunk centerline, and construct the joint feature vector of the point using the semantic features, the offset coordinates, and the normalized height; based on the joint feature vector, perform density clustering on the points whose semantic feature is the trunk to obtain a new trunk centerline, and determine the trunk cluster to which the points whose semantic feature is the branch belong according to the new trunk centerline and the joint feature vector.

[0134] The various embodiments of the techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0138] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0139] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0140] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering, characterized in that, include: Acquire forest sample location clouds and preprocess the forest sample location clouds; Preliminary segmentation of the preprocessed forest sample cloud was performed to obtain the individual tree segmentation results for each point; For each individual tree, perform at least one round of the following operations until a confirmation signal from the annotator is obtained, resulting in the final target point cloud for each tree: Based on the trunk seed points at different heights and the tree segmentation results for each point, determine the trunk center position and trunk radius at different height layers corresponding to each trunk seed point; determine the trunk centerline based on each trunk center position, and determine the trunk radius model based on each trunk radius; based on the trunk centerline and trunk radius model, re-segment the region point cloud of the tree from the trunk seed point along the vertical direction of the trunk to obtain the target point cloud for that tree; perform the following operations for each individual tree to determine... The final point cloud of each individual tree: The target point cloud of the individual tree and its point cloud features are input into the dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; For each point, the offset coordinates of the point are determined according to the offset vector of the point relative to the trunk centerline, and the joint feature vector of the point is constructed using the semantic features, the offset coordinates, and the normalized height; Based on the joint feature vector, density clustering is performed on the points whose semantic feature is trunk to obtain a new trunk centerline, and the trunk cluster to which the points whose semantic feature is branch belong is determined according to the new trunk centerline and the joint feature vector.

2. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 1, characterized in that, The process involves determining the trunk center position and trunk radius at different heights based on the tree seed points at different heights and the tree segmentation results at each point. This includes: obtaining the tree seed points at different heights as determined by the annotators; determining the local cross-sectional neighborhood corresponding to each tree seed point; extracting points within each local cross-sectional neighborhood from the points corresponding to the tree based on the tree segmentation results; and fitting the points within each local cross-sectional neighborhood to obtain the trunk center position and trunk radius at the height level corresponding to each tree seed point.

3. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 1, characterized in that, The method of resegmenting the point cloud of a single tree along the vertical direction of the trunk from the seed point on the trunk, based on the trunk centerline and trunk radius model, to obtain the target point cloud of the single tree, includes: selecting points belonging to the single tree and points whose shortest distance to the trunk centerline of the single tree is less than a first preset distance as candidate points; for each candidate point, determining the shortest distance from the candidate point to the trunk centerline, the height of the candidate point, and the local point density features of the candidate point; and selecting candidate points whose shortest distance to the trunk centerline is less than a radial distance threshold, whose height is higher than a low height threshold, and whose local point density features meet the local geometric consistency condition as the target point cloud of the single tree.

4. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 1, characterized in that, The semantic features are used to characterize the probability distribution of the category to which a point belongs; the categories include tree trunk, branch, and non-tree; the point cloud features include the point's position, intensity, normal vector, local point density, curvature, and distance from the corresponding tree trunk centerline; The semantic feature is that the offset vector of the point on the tree trunk is the offset vector between the point and the first target position; the first target position is the point on the center line of the trunk of the single tree to which the point belongs, which is closest in height to the point; The semantic feature is that the offset vector of the point of the branch is the offset vector between the point and the second target position; the second target position is the point on the center line of the trunk of the single tree to which the point belongs, which is the point at the second preset distance from the point.

5. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 4, characterized in that, The loss function of the dual-task prediction model includes a semantic segmentation loss term, a offset regression loss term, and an offset field smoothing regularization term. The semantic segmentation loss term is used to characterize the distance between the predicted value and the true value of the semantic feature. The offset regression loss term is used to characterize the distance between the predicted value and the true value of the offset vector. The offset field smoothing regularization term is used to constrain the smoothing of the offset vector changes of spatially neighboring points. The weight of each point corresponding to the offset regression loss term is determined according to the semantic features of the corresponding point. The weight of each point corresponding to the offset field smoothing regularization term is determined according to the distance and height difference between spatially neighboring points.

6. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 5, characterized in that, The dual-task prediction model includes a semantic segmentation branch and an offset branch; the semantic segmentation branch is trained in the first training phase. In the second training phase, the semantic segmentation branch and the offset branch are jointly trained. When training the offset branch, the weights of high-confidence points in the offset regression loss term are amplified by a factor of N based on a high-confidence mask, where N is a positive number greater than 1. The high-confidence mask includes information about high-confidence points, which are points whose semantic features are tree trunks or branches, and whose tree trunk confidence or branch confidence is greater than a confidence threshold. The high-confidence mask is updated after every M joint training iterations of the offset branch and the semantic segmentation branch.

7. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 6, characterized in that, Also includes: During the training of the offset branch, based on the boundary mask, the offset field smoothing regularization term is applied to the semantic boundary point and the points in the neighborhood of the semantic boundary point; the boundary mask includes information about the semantic boundary point, which is determined according to the following steps: determining the local neighborhood region of each point; taking the points in the local neighborhood region that have different categories represented by semantic features as semantic boundary points; the boundary mask is updated after every M joint training cycles of the offset branch and the semantic segmentation branch.

8. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 1, characterized in that, The method of clustering points whose semantic feature is the trunk based on the joint feature vector to obtain a new trunk centerline includes: determining the clustering radius of the points based on the local neighborhood distance and height variance of the points; setting the clustering weight of points whose trunk confidence corresponding to the semantic feature is less than a preset value to be lower than the preset weight; and using the determined clustering radius and clustering weight, clustering the points whose semantic feature is the trunk based on the joint feature vector to obtain a new trunk centerline.

9. The forest tree segmentation method based on trunk structure-assisted annotation and multi-task global clustering according to claim 1, characterized in that, The step of determining the trunk cluster to which the point with the semantic feature of a branch belongs based on the new trunk centerline and the joint feature vector includes: determining the nearest distance between the point with the semantic feature of a branch and the new trunk centerline based on the joint feature vector, and taking the point whose nearest distance is less than a third preset distance as the branch point of the single tree; determining the branch cluster to which the branch point belongs based on the height of the branch point; wherein different branch clusters correspond to different height ranges.

10. A forest tree segmentation device based on trunk structure-assisted annotation and multi-task global clustering, characterized in that, include: A preprocessing module is used to acquire forest sample location clouds and preprocess the forest sample location clouds; Preliminary segmentation of the preprocessed forest sample cloud was performed to obtain the individual tree segmentation results for each point; The auxiliary segmentation module performs at least one round of the following operations on each individual tree until a confirmation signal from the annotator is obtained, resulting in the final target point cloud for each tree: Based on the trunk seed points at different heights of the tree and the segmentation results of each point, the center position and radius of the trunk at different height layers corresponding to each trunk seed point are determined; the trunk centerline is determined based on the center positions of each trunk, and the trunk radius model is determined based on the radii of each trunk; based on the trunk centerline and the trunk radius model, the region point cloud is re-segmented along the vertical direction of the trunk from the trunk seed point to obtain the target point cloud for that tree; the inference segmentation module performs the following operations on each individual tree: The following operations are performed to determine the final point cloud of each individual tree: The target point cloud of the individual tree and its point cloud features are input into a dual-task prediction model to obtain the semantic features of each point and the offset vector of each point relative to the trunk centerline; For each point, the offset coordinates of the point are determined based on the offset vector of the point relative to the trunk centerline, and a joint feature vector of the point is constructed using the semantic features, the offset coordinates, and the normalized height; Based on the joint feature vector, density clustering is performed on the points whose semantic feature is trunk to obtain a new trunk centerline, and the trunk cluster to which the points whose semantic feature is branch belong is determined based on the new trunk centerline and the joint feature vector.

Images