End-to-end network-based complex plant structure point cloud completion method and system
By combining end-to-end network with data collected by UAVs and ground-based LiDAR, and using a hierarchical geometric encoder and tree growth module, high-quality complete tree point clouds are generated, solving the problem of missing tree point clouds in real forest environments and achieving a reasonable structure and low noise completion effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG FORESTRY UNIVERSITY
- Filing Date
- 2026-04-09
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies are insufficient to effectively fill in the missing tree point clouds caused by foliage shading in real forest environments, and deep learning models lack high-quality training and evaluation data, resulting in inaccurate tree structure reconstruction.
An end-to-end network is used, combined with point cloud data collected by UAVs and ground-based LiDAR. High-quality complete tree point clouds are generated through a hierarchical geometric encoder and a tree growth module. Multi-scale feature fusion and dynamic deformation constraints are introduced to ensure that the generated point cloud structure is reasonable and the noise is low.
The generated point cloud has uniform density and clear structure, which can effectively complete the tree branch structure, maintain the original morphological authenticity and branch connectivity, and has good generalization ability.
Smart Images
Figure CN122244124A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of 3D computer vision and deep learning technology, and in particular relates to a method and system for completing point clouds of complex plant structures based on end-to-end networks. Background Technology
[0002] Accurate 3D tree point cloud models are crucial for forest resource surveys, carbon sink estimation, and ecological research. However, in real forest environments, point clouds obtained through laser scanning often contain significant gaps due to foliage occlusion, posing a significant challenge to accurately reconstructing tree structures. Traditional point cloud completion methods (such as those based on prior models or curve skeletons) struggle to capture the fine local features of different tree species, are sensitive to noise, and have limited generalization capabilities.
[0003] With the development of deep learning, point cloud completion technology has become increasingly mature, giving rise to networks such as PCN, GRNet, and SnowflakeNet. However, these models are mainly designed and optimized for man-made objects with characteristics such as continuity and symmetry. When directly applied to the complex, highly random, and detailed branch structures of trees, their performance is poor, failing to maintain the original morphological realism and branch connectivity of the trees. Furthermore, the training and evaluation of deep learning models heavily rely on large-scale, high-quality complete point cloud data as supervision signals. Existing technologies generally face the problem of an extreme scarcity of real, complete tree point cloud data. This is because using a single scanning platform (such as ground-only or aerial-only) cannot overcome the occlusion of the trees themselves, making it difficult to obtain structurally complete point clouds; while relying on simulation data to train models makes it difficult to guarantee their generalization ability in real, complex scenes. Therefore, there is an urgent need for a technical solution that can effectively solve the point cloud completion problem while providing high-quality training and evaluation data. Summary of the Invention
[0004] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies. To this end, this invention proposes a method and system for completing complex plant structure point clouds based on end-to-end networks. This method not only effectively integrates multi-scale geometric features of tree point clouds, strictly preserving the original morphology of the input portion while completing missing parts and generating structures that conform to plant growth patterns, but also provides a technical means to construct a high-quality, complete tree point cloud dataset, thereby solving the core bottleneck of scarce training and evaluation data.
[0005] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:
[0006] In a first aspect, the present invention provides a method for completing point clouds of complex plant structures based on end-to-end networks, comprising the following steps:
[0007] S1. Acquire point cloud data of the area above the tree canopy by using a drone equipped with a lidar, and acquire point cloud data of the area below the tree canopy by using a ground-based remote-controlled vehicle equipped with a solid-state lidar; then, register and fuse the two sets of point cloud data to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset.
[0008] S2. Train an end-to-end network using an encoder-decoder framework on the tree point cloud dataset, and use the trained end-to-end network as the tree point cloud completion network. The encoder is a hierarchical geometric encoder, which downsamples and aggregates local features of the incomplete tree point cloud through an abstraction layer, and performs cross-scale fusion of features from different levels through a geometric enhancement transformer unit to extract global features. Then, the global features are used as input to the seed generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. At the same time, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological skeleton of the trees. Finally, the point generation module progressively upsamples the coarse point cloud and introduces dynamic deformation constraints that conform to the growth law of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud.
[0009] S3. Input the incomplete tree point cloud to be completed into the tree point cloud completion network to complete the point cloud, and output the completed tree point cloud.
[0010] Based on the above scheme, each step can be implemented in the following preferred manner.
[0011] As a preferred embodiment of the first aspect, the specific processing flow in the hierarchical geometric encoder of step S2 is as follows: the input incomplete tree point cloud undergoes four levels of hierarchical processing. Each of the first three levels consists of a set abstraction layer and a geometric enhancement transformer unit cascaded sequentially, while the last level contains only a set abstraction layer. Specifically, the first enhanced feature output by the first-level geometric enhancement transformer unit is used as the input of the second-level set abstraction layer. The second enhanced feature output by the second-level geometric enhancement transformer unit is aggregated with the first enhanced feature to obtain the first aggregated feature, which is used as the input of the third-level set abstraction layer. The third enhanced feature output by the third-level geometric enhancement transformer unit is aggregated with the first aggregated feature to obtain the second aggregated feature, which is used as the input of the fourth-level set abstraction layer. Finally, the global shape context feature output by the fourth-level set abstraction layer is used as the global feature.
[0012] As a preferred embodiment of the first aspect mentioned above, the specific processing flow in the seed generator of step S2 is as follows: First, the feature acceptor layer adjusts the global features through the GELU activation function to obtain enhanced global features; then, the point splitting module expands the enhanced global features and increases the number of points to the target number of points; next, an MLP fuses the expanded global features and performs multi-scale feature mapping to obtain the mapped features; then, the feature rejection layer transforms the features mapped by the MLP through the GELU activation function to suppress the generation of unreasonable structures and obtain the transformed features; finally, the transformed features are passed through an MLP for coordinate regression to obtain the intermediate point cloud, and then the intermediate point cloud is stitched together with the incomplete tree point cloud, and the farthest point is sampled from the stitched point cloud to output a coarse point cloud.
[0013] As a preferred embodiment of the first aspect mentioned above, in the point generation module of step S2, progressive upsampling is achieved through multiple cascaded tree growth modules. The specific processing flow in the tree growth module is as follows: The seed point cloud output by the tree growth module is processed by two MLPs to generate initial features. These initial features are then interacted with the output features of the first MLP through a cross-attention mechanism. The interacted features are then processed by another MLP to obtain enhanced interactive features. Finally, the seed point cloud is processed by the first MLP. The displacement features and interaction enhancement features output by the tree growth module are concatenated and input into a geometric enhancement transformer unit to obtain context features. Subsequently, the context features are upsampled by point splitting and copying operations. The upsampled context features are then processed by an MLP to generate the first tree growth module. The displacement characteristics corresponding to each tree growth module are analyzed, and the point displacements are calculated. Then, the point displacements are compared with the first tree growth module. The seed point clouds output by each tree growth module are summed to obtain the first... Seed point cloud output by a tree growth module.
[0014] As a preferred embodiment of the first aspect mentioned above, the specific processing flow in each geometry enhancement transformer unit is as follows: the features input to the geometry enhancement transformer unit are first fused through MLP to obtain the first intermediate feature. Then, the first intermediate feature is processed by a multi-head self-attention mechanism, layer normalization and feedforward network to obtain the second intermediate feature. Finally, the second intermediate feature is added to the first intermediate feature through residual connection to output the enhanced feature.
[0015] As a preferred embodiment of the first aspect above, the point displacement generated by the tree growth module... It is calculated using the following formula:
[0016] ;
[0017] in, It is the hyperbolic tangent activation function; It is a multilayer perceptron; For the first Displacement features generated by a tree growth module; The dynamic constraint coefficient is greater than 1; Index for the tree growth module.
[0018] As a preferred embodiment of the first aspect, in the end-to-end network training of step S2, the loss function used is composed of the sum of the completion loss and the shape preservation loss; wherein, the completion loss is the sum of the chamfer distances between the output point cloud and the corresponding real point cloud at each stage, and the output point cloud is a coarse point cloud or a seed point cloud output by each tree growth module; the shape preservation loss is a partial matching loss with unidirectional constraints, used to force the output complete tree point cloud to partially match the input incomplete tree point cloud.
[0019] Secondly, the present invention provides a point cloud completion system for complex plant structures based on an end-to-end network, comprising:
[0020] The data acquisition module is used to acquire point cloud data of the area above the tree canopy collected by a drone equipped with a lidar, and point cloud data of the area below the tree canopy collected by a ground-based remote-controlled vehicle equipped with a solid-state lidar. Then, the two sets of point cloud data are registered and fused to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset.
[0021] The model training module trains an end-to-end network using an encoder-decoder framework on a tree point cloud dataset, and uses the trained end-to-end network as a tree point cloud completion network. The encoder is a hierarchical geometric encoder, which downsamples and aggregates local features of incomplete tree point clouds through an abstraction layer, and performs cross-scale fusion of features from different levels through a geometric enhancement transformer unit to extract global features. Then, the global features are used as input to the seed generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. At the same time, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological skeleton of the trees. Finally, the point generation module progressively upsamples the coarse point cloud and introduces dynamic deformation constraints that conform to the growth law of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud.
[0022] The result acquisition module is used to input the incomplete tree point cloud to be completed into the tree point cloud completion network for point cloud completion, and output the completed tree point cloud.
[0023] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for completing point clouds of complex plant structures based on end-to-end networks as described in any of the solutions of the first aspect above.
[0024] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;
[0025] The memory is used to store computer programs;
[0026] The processor is configured to, when executing the computer program, implement the method for completing point clouds of complex plant structures based on end-to-end networks as described in any of the solutions of the first aspect above.
[0027] Compared with the prior art, the present invention has the following advantages:
[0028] The TreeFormerNet network proposed in this invention achieves cross-scale feature fusion through a hierarchical geometric encoder, simultaneously capturing both local details and global shape of trees, providing a rich semantic foundation for subsequent completion. Through a topology-aware seed generator and its feature acceptance / rejection mechanism, it generates a structurally sound and low-noise coarse point cloud skeleton, laying a solid topological foundation for fine completion. The innovative tree growth module introduces dynamic deformation constraints, guiding the point upsampling (growth) process in accordance with botanical principles, significantly improving the morphological realism and branch continuity of the generated point cloud, effectively solving the problems of over-smoothing or geometric distortion caused by existing general methods. This invention not only performs excellently on synthetic data but also demonstrates powerful completion capabilities and good generalization on real-world tree point cloud data, providing a reliable technical tool for 3D modeling in forestry digitization. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the overall architecture of the TreeFormerNet network model in this invention;
[0030] Figure 2 This is a schematic diagram of the architecture of the hierarchical geometric encoder in this invention;
[0031] Figure 3 This is a schematic diagram of the geometry enhancement converter unit in this invention.
[0032] Figure 4 This is a schematic diagram of the seed generator architecture in this invention;
[0033] Figure 5 This is a schematic diagram of the tree growth module architecture in this invention;
[0034] Figure 6 This is a qualitative diagram illustrating the Treeformernet completion effect in this embodiment;
[0035] Figure 7 This is a system block diagram of the present invention;
[0036] Figure 8 This is a schematic diagram of the components of a computer electronic device according to the present invention. Detailed Implementation
[0037] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.
[0038] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.
[0039] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned method for completing point clouds of complex plant structures based on end-to-end networks includes the following steps S1 to S3. The specific implementation process of each step will be described in detail below.
[0040] S1. Acquire point cloud data of the area above the tree canopy by using a drone equipped with a lidar, and acquire point cloud data of the area below the tree canopy by using a ground-based remote-controlled vehicle equipped with a solid-state lidar; then, register and fuse the two sets of point cloud data to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset.
[0041] It should be noted that in step S1 of this invention, in order to solve the key problem of the scarcity of real and complete point cloud data that restricts the performance of deep learning models, this invention constructs a high-quality and complete tree point cloud dataset through air-ground collaborative laser scanning and precise registration, which can be used for model training and performance evaluation for tasks such as point cloud completion, segmentation, and classification.
[0042] In this embodiment, the method for constructing the above-mentioned tree point cloud dataset is as follows:
[0043] 1) Air-ground collaborative data acquisition
[0044] Data acquisition was conducted in two coordinated phases: aerial and ground-based. The aerial platform consisted of a DJI Matrice 350 RTK drone equipped with a Zenmuse L2 LiDAR. Flight parameters were configured as follows: relative altitude 30 meters, flight speed 6.3 m / s, and lateral overlap of the LiDAR scan set to 90% to ensure adequate coverage of the tree canopy. The ground platform used a remotely operated vehicle (ROV) equipped with a Livox Mid-360 solid-state LiDAR. This device has a horizontal field of view of 360° and a vertical field of view of -7° to 52°, allowing it to move flexibly beneath the tree canopy and scan the trunk, main branches, and obscured structures within the canopy from multiple angles, with a scan frequency of 200,000 points / second.
[0045] 2) Point cloud registration and fusion
[0046] After acquiring point cloud data of the area above the tree canopy (from an aerial platform) and the area below the tree canopy (from a ground platform), the Iterative Closest Point (ICP) algorithm is used for automatic and accurate point cloud registration. This algorithm, which is a current technology, iteratively calculates the optimal rotation and translation transformation matrices to align two sets of point clouds originating from different coordinate systems and perspectives into a unified spatial reference system. Ultimately, these are merged to form a single, structurally complete point cloud of a single tree, from root to treetop. The specific process is not detailed here.
[0047] The tree point cloud dataset constructed using the above method is structurally complete and rich in detail, making it suitable as high-precision ground truth data. The TreeFormerNet model of this invention is trained and tested on such high-quality datasets, ensuring that the model can learn realistic tree geometry and topological priors, thus achieving excellent results in completion tasks. This dataset is also applicable to other forestry 3D analysis tasks such as point cloud segmentation, tree species classification, and biomass estimation, and has significant research and application value.
[0048] S2. Train an end-to-end network with an encoder-decoder framework on the tree point cloud dataset, and use the trained end-to-end network as the tree point cloud completion network TreeFormerNet;
[0049] The encoder is a hierarchical geometric encoder, which uses a set abstraction (SA) layer to process incomplete tree point clouds. Downsampling and local feature aggregation are performed, and features from different levels are fused across scales using a Geometric-enhanced Transformer (GT) unit to extract global features. ;
[0050] Next, the global features are used as input to the seed point generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. Simultaneously, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological framework of trees. ;
[0051] Finally, the Point Generation Module progressively upsamples the coarse point cloud, introducing dynamic deformation constraints that conform to the growth patterns of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud. .
[0052] It should be noted that, as Figure 2 As shown, in the hierarchical geometric encoder of step S2 of the present invention, the specific processing flow is as follows: the input incomplete tree point cloud undergoes four levels of hierarchical processing. Each of the first three levels consists of a set abstraction layer and a geometric enhancement transformer unit cascaded sequentially, and the last level contains only a set abstraction layer. The first enhanced feature output by the first-level geometric enhancement transformer unit is used as the input of the second-level set abstraction layer. The second enhanced feature output by the second-level geometric enhancement transformer unit is aggregated with the first enhanced feature to obtain the first aggregated feature, which is used as the input of the third-level set abstraction layer. The third enhanced feature output by the third-level geometric enhancement transformer unit is aggregated with the first aggregated feature to obtain the second aggregated feature, which is used as the input of the fourth-level set abstraction layer. Finally, the global shape context feature output by the fourth-level set abstraction layer is used as the global feature.
[0053] Furthermore, such as Figure 3 As shown, the specific processing flow in each geometry enhancement transformer unit is as follows: the features input to the geometry enhancement transformer unit are first fused through MLP to obtain the first intermediate feature. Then, the first intermediate feature is processed by a multi-head self-attention mechanism, layer normalization and feedforward network to obtain the second intermediate feature. Finally, the second intermediate feature is added to the first intermediate feature through residual connection to output the enhanced feature.
[0054] In this embodiment, the hierarchical geometric encoder acts as a feature extractor, where the incomplete tree point cloud... First, after the initial processing, the ensemble abstraction layer performs downsampling and local feature aggregation through farthest point sampling and multi-scale grouping, outputting the multi-scale local geometric features corresponding to the first level. With global shape context features Then and The first-stage geometric enhancement transformer unit is concatenated and input, and the first enhanced feature is output. Then, Input the second-level set abstraction layer, output the corresponding multi-scale local geometric features of the second level. With global shape context features Then and The second-stage geometric enhancement transformer unit is concatenated and input to output the second enhanced feature. Next, and Aggregation is performed to obtain the first aggregated feature; then, the first aggregated feature is input into the third-level set abstraction layer, which outputs the corresponding multi-scale local geometric features of the third level. With global shape context features Then and The third-level geometric enhancement transformer unit is concatenated and input to output the third enhancement feature. Next, The first aggregated feature is aggregated to obtain the second aggregated feature; finally, the second aggregated feature is input into the fourth-level set abstraction layer, which outputs the global shape context feature. Global features as the final output The processing within the abstraction layer is existing technology and will not be elaborated further. The geometry enhancement transformer unit is an innovative design of this invention, which enables cross-layer and cross-scale information flow and fusion. The specific processing steps are as described above and will not be repeated here.
[0055] It should be noted that, as Figure 4As shown, the specific processing flow in the seed generator of step S2 of this invention is as follows: First, the feature adjust layer (FA) adjusts the global features through the GELU activation function to obtain enhanced global features; then, the point split module (PS) expands the enhanced global features and increases the number of points to the target number; next, an MLP fuses the expanded global features and performs multi-scale feature mapping to obtain mapped features; then, the feature refuse layer (FR) transforms the features mapped by the MLP through the GELU activation function to suppress the generation of unreasonable structures and obtain transformed features; finally, the transformed features are passed through an MLP for coordinate regression to obtain intermediate point clouds, which are then stitched together with the incomplete tree point clouds, and the stitched point clouds are subjected to farthest point sampling (FPS) to output a coarse point cloud. The processing procedures within the feature adjust layer, point split module, and feature refuse layer are all existing technologies and will not be described in detail here.
[0056] It should be noted that, as Figure 5 As shown, in the point generation module of step S2 of the present invention, progressive upsampling is achieved through multiple cascaded tree growth modules. The specific processing flow in the tree growth module is as follows: Seed point cloud output by each tree growth module After processing by two MLPs, initial features are generated. These initial features are then interacted with the output features of the first MLP through a cross-attention mechanism. Finally, the interacted features are processed by another MLP to obtain the interactively enhanced features. Then the first Displacement characteristics output by each tree growth module With interaction enhancement features After concatenation, input a geometry enhancement transformer unit to obtain contextual features. Subsequently, contextual features were analyzed. Point splitting and copying operations are performed for upsampling. The upsampled context features are then processed by an MLP to generate the first... Displacement characteristics corresponding to each tree growth module And calculate the point displacement Then the point displacement is compared with the first Seed point cloud output by each tree growth module Add them together to get the first one. Seed point cloud output by each tree growth module .
[0057] Furthermore, the point displacement generated by the tree growth module It is calculated using the following formula:
[0058]
[0059] in, It is the hyperbolic tangent activation function; It is a multilayer perceptron; For the first Displacement features generated by a tree growth module; The dynamic constraint coefficient is greater than 1; Index the tree growth modules (e.g., 1, 2, 3).
[0060] As can be seen from the above formula, with the deepening of the upsampling stage (i.e. (increase) By reducing the displacement range of the point, the deformation gradually decreases from the trunk to the branches during tree growth, thus ensuring the rationality of the generated structure's morphology.
[0061] In this embodiment, as Figure 1 As shown, the point generation module consists of three cascaded tree growth modules (TGMs), with upsampling factors of respectively. , and The structure of each TGM is as follows: Figure 5 As shown. For the first TGM, its input is a coarse point cloud. The displacement features used are randomly generated; for the second and third TGMs, the inputs are seed point clouds, respectively. , The displacement features used are derived from the first and second TGMs, respectively. Each tree growth module learns contextual information through a cross-attention mechanism and a geometry enhancement transformer unit, and generates new points based on displacements with dynamic constraints, which gradually tighten as the upsampling phase progresses. After processing according to the above process, the seed point cloud output by the third tree growth module is... This serves as the complete tree point cloud after completion.
[0062] It should be noted that the loss function used in the end-to-end network training in step S2 of this invention is... From the compensation loss and shape retention loss Addition constitutes:
[0063]
[0064] Among them, the completion loss The chamfer distance (CD) is the sum of the output point clouds at each stage and the corresponding ground truth point clouds. The output point clouds are either coarse point clouds or seed point clouds output by each tree growth module. In this embodiment, it is necessary to calculate the chamfer distance between the coarse point cloud and the ground truth point cloud downsampled to the corresponding density, as well as the seed point clouds output by the three TGMs. , , The chamfer distances between the output tree point cloud and the corresponding density of the real point cloud are calculated, and then the four chamfer distances are summed to obtain the completion loss. The shape preservation loss is a one-way constrained partial matching loss, which is used to force the output complete tree point cloud to partially match the input incomplete tree point cloud, thereby strictly preserving the original shape of the input incomplete tree point cloud while completing the missing parts.
[0065] S3. Input the incomplete tree point cloud to be completed into the tree point cloud completion network to complete the point cloud, and output the completed tree point cloud.
[0066] Qualitative results on complete point cloud datasets of synthetic walnut trees, real holly trees, and ginkgo trees constructed using a co-location method are as follows: Figure 6 As shown in Table 1, the qualitative results demonstrate that the TreeFormerNet proposed in this invention significantly outperforms existing mainstream methods in point cloud completion tasks. In terms of quantitative evaluation, taking the most challenging Ginkgo tree dataset as an example, TreeFormerNet achieves a CD-L1 score of 15.92, a CD-L2 score of 0.98, and an EMD of 39.66, representing improvements of 21.4%, 44.9%, and 20.9% respectively compared to the second-best SnowflakeNet. It also maintains a leading advantage on walnut and holly tree datasets. Qualitative visualization results further confirm that TreeFormerNet can completely reconstruct the trunk and branches at all levels, accurately restoring details. The generated point cloud has uniform density and clear structure, highly consistent with the real point cloud, while the comparative methods generally suffer from problems such as missing details, uneven density, or structural distortion.
[0067] Table 1. TreeFormerNet Completion Results
[0068]
[0069] Ablation experiments verified the effectiveness of each core module of this invention. The results are shown in Table 2. Compared with the optimal two-module combination, the complete model improved various indicators by 9.5% to 30.0%. In summary, TreeFormerNet, through the collaborative design of multi-level geometric encoding, topology-aware seed generation, and dynamic constraint growth, achieves an ideal completion effect with accurate global shape, rich local details, reasonable structural connectivity, and uniform density distribution.
[0070] Table 2. Ablation Experiment Results of TreeFormerNet
[0071]
[0072] It should also be noted that the point cloud completion method for complex plant structures based on end-to-end networks in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a point cloud completion system for complex plant structures based on end-to-end networks, corresponding to the point cloud completion method for complex plant structures provided in the above embodiments, such as... Figure 7 As shown, it includes:
[0073] The data acquisition module is used to acquire point cloud data of the area above the tree canopy collected by a drone equipped with a lidar, and point cloud data of the area below the tree canopy collected by a ground-based remote-controlled vehicle equipped with a solid-state lidar. Then, the two sets of point cloud data are registered and fused to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset.
[0074] The model training module trains an end-to-end network using an encoder-decoder framework on a tree point cloud dataset, and uses the trained end-to-end network as a tree point cloud completion network. The encoder is a hierarchical geometric encoder, which downsamples and aggregates local features of incomplete tree point clouds through an abstraction layer, and performs cross-scale fusion of features from different levels through a geometric enhancement transformer unit to extract global features. Then, the global features are used as input to the seed generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. At the same time, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological skeleton of the trees. Finally, the point generation module progressively upsamples the coarse point cloud and introduces dynamic deformation constraints that conform to the growth law of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud.
[0075] The result acquisition module is used to input the incomplete tree point cloud to be completed into the tree point cloud completion network for point cloud completion, and output the completed tree point cloud.
[0076] It is understood that the point cloud completion method for complex plant structures based on end-to-end networks described in S1-S3 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the point cloud completion method for complex plant structures based on end-to-end networks provided in the above embodiments, such as... Figure 8 As shown, it includes a memory and a processor;
[0077] The memory is used to store computer programs;
[0078] The processor is configured to implement the method for completing point clouds of complex plant structures based on end-to-end networks in the above embodiments when executing the computer program.
[0079] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.
[0080] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the method for completing point clouds of complex plant structures based on end-to-end networks provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the method for completing point clouds of complex plant structures based on end-to-end networks in the above embodiments.
[0081] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.
[0082] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0083] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.
[0084] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.
Claims
1. A method for completing point clouds of complex plant structures based on end-to-end networks, characterized in that, Includes the following steps: S1. Acquire point cloud data of the area above the tree canopy by using a drone equipped with a lidar, and acquire point cloud data of the area below the tree canopy by using a ground-based remote-controlled vehicle equipped with a solid-state lidar; then, register and fuse the two sets of point cloud data to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset. S2. Train an end-to-end network using an encoder-decoder framework on the tree point cloud dataset, and use the trained end-to-end network as the tree point cloud completion network. The encoder is a hierarchical geometric encoder, which downsamples and aggregates local features of the incomplete tree point cloud through an abstraction layer, and performs cross-scale fusion of features from different levels through a geometric enhancement transformer unit to extract global features. Then, the global features are used as input to the seed generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. At the same time, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological skeleton of the trees. Finally, the point generation module progressively upsamples the coarse point cloud and introduces dynamic deformation constraints that conform to the growth law of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud. S3. Input the incomplete tree point cloud to be completed into the tree point cloud completion network to complete the point cloud, and output the completed tree point cloud.
2. The method for point cloud completion of complex plant structures based on end-to-end networks as described in claim 1, characterized in that, In the hierarchical geometric encoder of step S2, the specific processing flow is as follows: the input incomplete tree point cloud undergoes four levels of hierarchical processing. Each of the first three levels consists of a set abstraction layer and a geometric enhancement transformer unit cascaded sequentially, while the last level contains only a set abstraction layer. Specifically, the first enhanced feature output by the first-level geometric enhancement transformer unit is used as the input of the second-level set abstraction layer. The second enhanced feature output by the second-level geometric enhancement transformer unit is aggregated with the first enhanced feature to obtain the first aggregated feature, which is used as the input of the third-level set abstraction layer. The third enhanced feature output by the third-level geometric enhancement transformer unit is aggregated with the first aggregated feature to obtain the second aggregated feature, which is used as the input of the fourth-level set abstraction layer. Finally, the global shape context feature output by the fourth-level set abstraction layer is used as the global feature.
3. The method for point cloud completion of complex plant structures based on end-to-end networks as described in claim 1, characterized in that, In the seed generator of step S2, the specific processing flow is as follows: First, the feature acceptor layer adjusts the global features through the GELU activation function to obtain the enhanced global features; then, the point splitting module expands the enhanced global features and increases the number of points to the target number of points; next, an MLP fuses the expanded global features and performs multi-scale feature mapping to obtain the mapped features; then, the feature rejection layer transforms the MLP-mapped features through the GELU activation function to suppress the generation of unreasonable structures and obtain the transformed features; finally, the transformed features are passed through an MLP for coordinate regression to obtain the intermediate point cloud, and then the intermediate point cloud is stitched together with the incomplete tree point cloud, and the farthest point is sampled from the stitched point cloud to output the coarse point cloud.
4. The method for completing point clouds of complex plant structures based on end-to-end networks as described in claim 1, characterized in that, In step S2, the point generation module progressively upsampling is achieved through multiple cascaded tree growth modules. The specific processing flow in the tree growth module is as follows: The seed point cloud output by the tree growth module is processed by two MLPs to generate initial features. These initial features are then interacted with the output features of the first MLP through a cross-attention mechanism. The interacted features are then processed by another MLP to obtain enhanced interactive features. Finally, the seed point cloud is processed by the first MLP. The displacement features and interaction enhancement features output by each tree growth module are concatenated and then input into a geometric enhancement transformer unit to obtain context features. Subsequently, the context features are upsampled by performing point splitting and copying operations. The upsampled context features are then processed by an MLP to generate the first... The displacement characteristics corresponding to each tree growth module are analyzed, and the point displacements are calculated. Then, the point displacements are compared with the first tree growth module. The seed point clouds output by each tree growth module are summed to obtain the first... Seed point cloud output by a tree growth module.
5. The method for point cloud completion of complex plant structures based on end-to-end networks as described in claim 2 or 4, characterized in that, In each geometry enhancement transformer unit, the specific processing flow is as follows: the features input to the geometry enhancement transformer unit are first fused through MLP to obtain the first intermediate feature. Then, the first intermediate feature is processed by a multi-head self-attention mechanism, layer normalization and feedforward network to obtain the second intermediate feature. Finally, the second intermediate feature is added to the first intermediate feature through residual connection to output the enhanced feature.
6. The method for point cloud completion of complex plant structures based on end-to-end networks as described in claim 4, characterized in that, The point displacement generated by the tree growth module It is calculated using the following formula: ； in, It is the hyperbolic tangent activation function; It is a multilayer perceptron; For the first Displacement features generated by a tree growth module; The dynamic constraint coefficient is greater than 1; Index for the tree growth module.
7. The method for point cloud completion of complex plant structures based on end-to-end networks as described in claim 4, characterized in that, In the end-to-end network training of step S2, the loss function used consists of the sum of the completion loss and the shape preservation loss. The completion loss is the sum of the chamfer distances between the output point cloud and the corresponding real point cloud at each stage. The output point cloud is either a coarse point cloud or a seed point cloud output by each tree growth module. The shape preservation loss is a partial matching loss with unidirectional constraints, used to force the output complete tree point cloud to partially match the input incomplete tree point cloud.
8. A point cloud completion system for complex plant structures based on an end-to-end network, characterized in that, include: The data acquisition module is used to acquire point cloud data of the area above the tree canopy collected by a drone equipped with a lidar, and point cloud data of the area below the tree canopy collected by a ground-based remote-controlled vehicle equipped with a solid-state lidar. Then, the two sets of point cloud data are registered and fused to generate a complete single tree point cloud from the tree root to the treetop, forming a tree point cloud dataset. The model training module trains an end-to-end network using an encoder-decoder framework on a tree point cloud dataset, and uses the trained end-to-end network as a tree point cloud completion network. The encoder is a hierarchical geometric encoder, which downsamples and aggregates local features of incomplete tree point clouds through an abstraction layer, and performs cross-scale fusion of features from different levels through a geometric enhancement transformer unit to extract global features. Then, the global features are used as input to the seed generator. After nonlinear enhancement of the global features, the enhanced global features are expanded to the target number of points through a point splitting operation. At the same time, a feature rejection mechanism is used to suppress the generation of unreasonable structures, thereby obtaining a low-density coarse point cloud containing the main topological skeleton of the trees. Finally, the point generation module progressively upsamples the coarse point cloud and introduces dynamic deformation constraints that conform to the growth law of trees during the upsampling process, ultimately generating a high-density, structurally complete tree point cloud. The result acquisition module is used to input the incomplete tree point cloud to be completed into the tree point cloud completion network for point cloud completion, and output the completed tree point cloud.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for completing point clouds of complex plant structures based on end-to-end networks as described in any one of claims 1 to 7.
10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the method for completing point clouds of complex plant structures based on end-to-end networks as described in any one of claims 1 to 7.