Backpack and unmanned aerial vehicle laser radar single tree segmentation method and system based on deep learning

Abstract

The invention discloses a backpack and unmanned aerial vehicle laser radar single tree segmentation method and system based on deep learning. The method comprises the steps that tree multi-platform point cloud data is acquired and fused based on an unmanned aerial vehicle and a backpack type laser radar system; generating individual tree label data; constructing a single-tree segmentation model which comprises a feature extraction network, a feature sorting and clustering module and a feature screening module; and single-tree segmentation is carried out after the single-tree segmentation model is trained. In order to solve the problem of insufficient feature extraction capability of a traditional method under complex forest stand conditions such as canopy embedding and form approaching, a spatial attention mechanism and a hierarchical density clustering module are introduced, feature weights of different regions are dynamically allocated in a coding stage, and the sensitivity to a local geometric structure of a canopy embedding region is enhanced; and meanwhile, multilevel semantic information is reserved by utilizing a feature fusion strategy in a decoding stage, so that the three-dimensional space topological feature distinguishing capability and the crown boundary recognition capability are enhanced, and the accuracy of single tree segmentation is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing science and computer vision, and specifically relates to a backpack and unmanned aerial vehicle lidar single tree segmentation method and system based on deep learning. Background Art

[0002] Point cloud data, acquired through advanced technologies such as LiDAR, provides high-precision three-dimensional structural information about trees. Point cloud data can accurately measure geometric features such as tree height, canopy shape, and trunk diameter, which vary significantly between tree species. Point cloud data can be used not only to identify individual trees but also to analyze and monitor forest structure over large areas. With the advancement of LiDAR technology, the cost of acquiring point cloud data has decreased and its accuracy has increased, making its application in tree species identification increasingly widespread. Furthermore, by extracting quantified intra-unit point cloud features, particularly elevation features, a wealth of tree information that is difficult to analyze directly can be captured, providing a rich data source for tree species identification. The development of this technology provides important support for improving forest management and promoting ecological protection.

[0003] The use of drones has greatly facilitated data collection. Drones offer multiple advantages for collecting point cloud data. First, their flexible flight capabilities enable them to efficiently complete data collection tasks in complex terrain and hard-to-reach areas, making them suitable for scenarios such as forests and mountainous areas. Drones can also be equipped with lidar (LiDAR) equipment to generate high-precision 3D point cloud data in real time, accurately capturing structural information about terrain, vegetation, and other target objects. Furthermore, drones' flight altitude and path are controllable, allowing the density and coverage of collected point cloud data to be flexibly adjusted to meet application requirements of varying resolution and range. Compared to traditional terrestrial laser scanning, drone point cloud data collection is faster and more efficient, while also reducing labor costs. Combined with automated route planning and multi-sensor fusion technology, point cloud data collected by drones can be widely used in fields such as forestry monitoring, geological surveys, and infrastructure management.

[0004] However, traditional methods are insufficient in feature extraction capabilities under complex forest stand conditions such as interlocking canopies and similar morphologies, and will also cause accuracy loss in the data conversion process. Summary of the Invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a backpack and drone lidar single tree segmentation method and system based on deep learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: One aspect of the present invention provides a deep learning-based backpack and drone lidar single tree segmentation method, comprising the following steps: The LiDAR point cloud data of target trees is collected using a UAV and backpack LiDAR system, and then processed using point cloud denoising, point cloud filtering, and ground point normalization. The processed LiDAR point cloud data of target trees is then registered and fused to generate multi-platform point cloud data of the trees. Matching the data containing tree species information and individual tree location coordinate information with the multi-platform point cloud data of the tree, assigning the tree species information to the multi-platform point cloud data of the tree, and generating individual tree label data; A single tree segmentation model was constructed, comprising a feature extraction network, a feature sorting and clustering module, and a feature screening module. The feature extraction network extracted point cloud features from multi-platform tree point cloud data using a submanifold sparse convolution kernel and output them to the feature sorting and clustering module's semantic label, center offset, and instance embedding branches for feature sorting and clustering. The feature screening module was used to perform instance screening and single tree segmentation based on the clustering results of the three branches. The single tree segmentation model is trained using multi-platform point cloud data of trees with tree species information and single tree label data; Use the trained single tree segmentation model to perform single tree segmentation.

[0007] As a preferred technical solution, after collecting LiDAR point cloud data of target trees, the raw data is converted into laser file format, and a real-time differential positioning device is used to record the coordinates of each tree and the tree species information; The point cloud denoising method specifically includes: constructing a point cloud elevation density model based on kernel density estimation; removing outliers in the clustered point cloud that are farther than a set threshold; and using a neighboring point search method to remove points that are closer to the ground than a set threshold. The point cloud filtering adopts a cloth simulation method, specifically: the original point cloud is inverted in the vertical direction so that the ground points are located below the cloth; a virtual cloth grid is generated above the inverted point cloud, and each cloth grid node is assigned an initial position; an iterative simulation is performed, wherein the cloth grid nodes are moved downward by gravity, and the displacement of the cloth grid nodes is constrained by adjacent grid nodes. The cloth grid nodes stop moving when they encounter a ground point in the point cloud. The point cloud area corresponding to the final position of the cloth is the ground point, and the rest are non-ground points. The ground point normalization is specifically as follows: subtracting the height value of the ground point obtained by point cloud filtering from the elevation value of each point to obtain a normalized point cloud; The processed LiDAR point cloud data of the target tree are registered and fused, specifically by using an iterative closest point registration algorithm to register and fuse the two point clouds.

[0008] As a preferred technical solution, the data containing tree species information and single tree position coordinate information is matched with the tree multi-platform point cloud data, the tree species information is assigned to the tree multi-platform point cloud data, and single tree label data is generated. The specific implementation process is as follows: The multi-platform point cloud data of trees and the data containing tree species information and individual tree position coordinate information are unified into the same coordinate system. A KD tree is constructed for the multi-platform point cloud data of trees. The space is recursively divided into hyperplanes along a certain dimension to form a binary tree structure for search. Search the multi-platform point cloud data of trees in sequence according to the position of a single tree, find the point closest to the position of the single tree, and assign the tree species information to the point closest to the position of the single tree; The assigned tree multi-platform point cloud data are annotated with single tree point clouds one by one to generate single tree label data.

[0009] As a preferred technical solution, before the multi-platform point cloud data of trees is input into the single tree segmentation model, the entire point cloud voxel grid is first down-sampled to sparse overly dense areas and achieve uniform point density; random sampling method is adopted during sampling, and cylinder is selected as the neighborhood definition.

[0010] As a preferred technical solution, the feature extraction network includes an input layer, an encoding layer and a decoding layer; The input layer includes a convolution kernel of 5×5×5×1; The encoding layer has two layers, each layer includes four 3×3×3+3 sparse convolution kernels; The encoding layer has three layers, each of which includes four 3×3×3+3 sparse convolution kernels; In this method, layers with the same step amplitude are connected through skip connections.

[0011] As a preferred technical solution, in the feature sorting and clustering module: The semantic label branch is used to classify the point cloud into "tree" and "non-tree" and output the "tree" point cloud for single tree segmentation. It includes a multi-layer perceptron with a single hidden layer and a spatial attention mechanism. softmax Activate and output the semantic category probability of each point; the semantic label branch is trained using the standard cross entropy loss and obtains the semantic label by taking the argmax of the predicted category probability; The center offset branch encodes the feature vector to produce N points, the corresponding offset is expressed as , and then predict a three-dimensional offset vector for each point so that the point can be moved to the center of the instance; the center offset branch is obtained by L 1 regression loss to constrain the learned offset, as follows: ; ; in, m i =1 indicates that i In the example, m i =0 means the i The point is not on the instance; O i For the i The offset of the points, p i 、 p j is the origin coordinate of the corresponding point, Center of mass; g ( i ) is the mapping point i The corresponding grounding instance index, that is, the i An example of a point, is an instance g ( i ) in the number of points; The instance embedding branch maps each point to a 5D feature space for instance-level differentiation. The instance embedding branch is supervised by contrastive loss during training, specifically by minimizing the distance between points of the same instance in the corresponding embedding space and maximizing the distance between points of different instances in the corresponding embedding space. Region growing clustering, mean shift clustering, HDBSCAN algorithm, watershed algorithm or BIRCH algorithm are used to generate candidate instances for the outputs of the three branches.

[0012] As a preferred technical solution, the feature screening module uses one of 3DuNet, PointNet, PointNet++ and KpConv to predict the matching degree between candidate instances and true labels, and outputs normalized scores through pooling and fully connected layers. θ ∈(0,1] completes single tree segmentation.

[0013] As a preferred technical solution, the single tree segmentation model is trained using multi-platform point cloud data of trees with tree species information and single tree label data. The specific implementation process is as follows: The multi-platform tree point cloud data with tree species information and individual tree label data is fed into a single tree segmentation model for training. The single tree segmentation model is based on cylindrical sampling and uses Gaussian random noise, random rotation, random voxel sampling, and random stretching for data augmentation. Iterate the pseudo-labels based on the training results to obtain the tree species and single tree point cloud results and training weights.

[0014] Another aspect of the present invention provides a deep learning-based backpack and UAV LiDAR single tree segmentation system, which is applied to the above-mentioned deep learning-based backpack and UAV LiDAR single tree segmentation method, including a data acquisition and fusion module, a data annotation module, a model construction module, a model training module, and a single tree segmentation module; The data acquisition and fusion module is used to collect LiDAR point cloud data of target trees based on the UAV and backpack LiDAR system and perform point cloud denoising, point cloud filtering, and ground point normalization processing; the processed LiDAR point cloud data of the target trees is aligning and fused to generate multi-platform point cloud data of the trees; The data annotation module is used to match the data containing tree species information and single tree position coordinate information with the tree multi-platform point cloud data, assign the tree species information to the tree multi-platform point cloud data, and generate single tree label data; The model construction module is used to construct a single tree segmentation model, including a feature extraction network, a feature sorting and clustering module, and a feature screening module. The feature extraction network extracts point cloud features from multi-platform point cloud data of trees using a submanifold sparse convolution kernel and outputs them to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering. The feature screening module is used to perform instance screening and single tree segmentation based on the clustering results of the three branches. The model training module is used to train the single tree segmentation model based on the multi-platform point cloud data of trees with tree species information and single tree label data; The single tree segmentation module is used to perform single tree segmentation according to the trained single tree segmentation model.

[0015] Another aspect of the present invention provides a storage medium storing a program, which, when executed by a processor, implements the above-mentioned deep learning-based backpack and drone lidar single tree segmentation method.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) To address the problem of insufficient feature extraction capabilities of traditional methods under complex forest stand conditions such as canopy interlocking and morphological convergence, the present invention introduces a spatial attention mechanism and a hierarchical density clustering module to dynamically allocate feature weights of different regions in the encoding stage, thereby enhancing the sensitivity to the local geometric structure of the canopy interlocking area; at the same time, the feature fusion strategy in the decoding stage is used to retain multi-level semantic information, enhance the ability to distinguish three-dimensional spatial topological features and crown boundary recognition, and improve the accuracy of single tree segmentation.

[0017] (2) The present invention compensates for the accuracy loss caused by the current single-tree segmentation method based on the canopy height model during the data conversion process; compared with the single-tree segmentation method based on machine learning, it can better cope with the situation where the point cloud features of adjacent trees in forest scenes are highly similar. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of a method for single tree segmentation using a backpack and UAV lidar based on deep learning according to an embodiment of the present invention; Figure 2 2 is a schematic structural diagram of a single wood segmentation model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] Example: like Figure 1 As shown, this embodiment provides a deep learning-based backpack and drone lidar single tree segmentation method, including the following steps: S1. Based on the UAV and backpack LiDAR system, the LiDAR point cloud data of the target tree is collected. The collected LiDAR point cloud data is pre-processed and then aligned and fused to generate multi-platform point cloud data of the tree.

[0021] In one or more preferred embodiments, the specific implementation process of step S1 is as follows: S11. Use backpack and airborne LiDAR equipment to collect sample point cloud data, and use supporting software to convert the raw data into Laser File Format (LAS) point cloud files. At the same time, use real-time differential positioning (RTK) equipment to record the coordinates of each tree and record tree species information.

[0022] S12. Preprocess the backpack and airborne LiDAR point cloud files, including point cloud denoising, point cloud filtering, and ground point normalization.

[0023] Furthermore, the point cloud denoising is specifically as follows: when using airborne LiDAR or backpack LiDAR to collect point cloud data, the point cloud data will inevitably be contaminated by noise points due to the influence of the instrument itself and various external factors. Therefore, the point cloud needs to be denoised, including the following steps: first, a point cloud elevation density model is constructed based on the sum density estimation, and then outliers of the clustered point cloud that are far away from the set threshold are removed, and then the neighboring point search method is used to remove noise points that are close to the set threshold from the ground.

[0024] Furthermore, the main purpose of the point cloud filtering is to separate the data into ground points and non-ground points, using a cloth simulation method, including the following steps: 1) Point cloud inversion: Invert the original point cloud vertically so that the ground points are below the "cloth".

[0025] 2) Cloth initialization: Generate a virtual cloth mesh above the inverted point cloud, and assign an initial position to each cloth mesh node.

[0026] 3) Iterative simulation: The cloth mesh nodes move downward due to gravity. The displacement of the cloth mesh nodes is constrained by the adjacent mesh nodes (simulating the cloth stiffness). The cloth mesh nodes stop moving when they encounter a ground point in the point cloud.

[0027] 4) Ground point determination: The point cloud area corresponding to the final position of the cloth is the ground point, and the rest are non-ground points.

[0028] Furthermore, the ground points obtained after filtering can be used to normalize the point cloud data, including the following steps: The normalized point cloud is obtained by subtracting the elevation value of the corresponding ground point from the elevation value of each point. The elevation value in the point cloud data after this processing is actually the relative elevation relative to the ground point.

[0029] S13. The pre-processed point cloud data is fused by using an iterative closest point (ICP) registration algorithm to register the two point clouds.

[0030] S2. Match the data containing tree species information and individual tree location coordinate information with the fused multi-platform tree point cloud data, assign the tree species information to the point cloud data, and generate high-precision individual tree label data based on seed points and manual annotation; In one or more preferred embodiments, step S2 is specifically implemented as follows: S21. Unify the multi-platform point cloud data of trees and the data containing tree species information and individual tree position coordinate information into the same coordinate system, construct a KD tree for the multi-platform point cloud data of trees, and accelerate the point cloud search capability; among them, the data containing tree species and individual tree position coordinate information is completed at the same time when the radar data is collected.

[0031] Furthermore, the KD tree aims to partition high-dimensional data (such as 3D point clouds) into a spatial partition, accelerating operations like nearest neighbor search and range search. The principle is to recursively partition the space along a certain dimension into hyperplanes, forming a binary tree structure. This involves the following steps: 1) Data structure definition. For each node structure, the following information is included: Split Axis: the current split dimension (such as X / Y / Z axis); Split Point: The point corresponding to the current node; Left Child: the subspace to the left of the split hyperplane; Right Child: The subspace to the right of the split hyperplane.

[0032] 2) Recursive segmentation process, including the following steps: a) Input: the set of points to be segmented (e.g., points in a 3D point cloud); b) Termination conditions: The number of points in the current node is less than the preset threshold (for example, a leaf node contains a maximum of 10 points), or the coordinates of all points are exactly the same in a certain dimension (no further division is possible); c) Select the split dimension: Usually the dimension with the largest variance of the current point set is selected (to ensure a balanced distribution of the subspace after division). For example, for a 3D point cloud, the variance of the X, Y, and Z axes is calculated, and the axis with the largest variance is selected as the division dimension; d) Select the split point: In the split dimension, the median point of the current point set is taken as the split point (to ensure the balance of the tree); If you want efficiency rather than strict balance, you can also randomly select the split point; e) Recursively build subtrees: Left subtree: the set of points whose split dimension coordinates are less than or equal to the split point; Right subtree: the set of points whose split dimension coordinates are greater than the split point; The above process is repeated recursively until the termination condition is reached.

[0033] S22. Search the multi-platform point cloud data of trees in sequence according to the position of the single tree, find the point closest to the position of the single tree, and assign the tree species information to the point closest to the position of the single tree.

[0034] S23. Load the assigned multi-platform point cloud data into the point cloud processing software and annotate the individual tree point clouds one by one to generate high-precision individual tree label data.

[0035] S3, build a single tree segmentation model, such as Figure 2 As shown, it includes a feature extraction network, a feature sorting and clustering module, and a feature screening module; the feature extraction network extracts point cloud features of multi-platform point cloud data of trees through a submanifold sparse convolution kernel, and then outputs a 16-dimensional feature vector to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering; the feature screening module is used to perform instance screening and single tree segmentation according to the clustering results of the three branches.

[0036] In one or more preferred embodiments, before the tree multi-platform point cloud data is input into the single tree segmentation model, the entire point cloud voxel grid is first downsampled to sparse overly dense areas and achieve uniform point density; a random sampling method is used during sampling, and a cylinder is selected as the neighborhood definition.

[0037] In one or more preferred embodiments, the feature extraction network includes an input layer, an encoding layer, and a decoding layer; The input layer includes a convolution kernel of 5×5×5×1; The encoding layer has two layers, each layer includes four 3×3×3+3 sparse convolution kernels; The encoding layer has three layers, each of which includes four 3×3×3+3 sparse convolution kernels; The overall structure is [64, 64, 128, 256, 512, 512, 512, 2, 1], and the final output is the predicted label. Multiple sparse convolutions with strides and sparse transposed convolutions with strides are added to the basic residual network, and layers with the same stride are connected through jump connections.

[0038] Furthermore, the feature extraction network can also be modeled using one of 3DuNet, PointNet, PointNet++ and KpConv.

[0039] In one or more preferred embodiments, in the feature ranking and clustering module, the semantic label branch is used to estimate point-by-point semantic labels, the center offset branch is used to regress the offset to the instance center, and the instance embedding branch is used to extract embedding features that distinguish instances, as follows: 1) The semantic label branch is used to classify the point cloud into “tree” and “non-tree” and output the “tree” point cloud for single tree segmentation. It includes a single hidden layer multi-layer perceptron (MLP) and a spatial attention mechanism (SAM). The hidden layer has softmax Activate and output the semantic category probability of each point; before passing the point cloud features downward, a spatial attention mechanism is introduced to readjust the point cloud features by focusing on the effective information on the feature map; the semantic label branch is trained using the standard cross entropy loss, and the semantic label is obtained by taking the argmax of the predicted category probability.

[0040] 2) The center offset branch encodes the feature vector to generate N points, the corresponding offset is expressed as , and then predict a three-dimensional offset vector for each point so that the point can be moved to the center of the instance. In short, if the prediction is correct, all points of each instance will converge to a single point; the center offset branch is calculated by L 1 regression loss to constrain the learned offset, as follows: ; ; in, m i =1 indicates that i In the example, m i =0 means the i The point is not on the instance; O i For the i The offset of the point, p i 、 p j is the origin coordinate of the corresponding point, yes i belong ie The center of mass; g ( i ) is the mapping point i The corresponding grounding instance index, that is, the i An example of a point, is an instance g ( i ) in the number of points.

[0041] 3) Unlike directly finding the geometric center, the instance embedding branch maps each point to a 5D feature space for differentiation at the instance level. The instance embedding branch is supervised by contrastive loss during training. Specifically, for points of the same instance, the distance between them in the corresponding embedding space is encouraged to be as small as possible; for points of different instances, the distance between them in the corresponding embedding space is encouraged to be as large as possible. The most important point is that the embedding space has more than three dimensions, so in addition to the compact cluster around the 3D center point, it also has some spare capacity to represent object properties.

[0042] Finally, the region growing clustering, mean shift clustering, HDBSCAN algorithm, watershed algorithm or BIRCH algorithm are used to generate candidate instances for the output of the three branches, specifically: The feature vector generated by the center shift branch will shift the point cloud to the center point of the predicted instance. Region growing clustering is preferably used to generate candidate instances under this feature vector. When the instance is embedded in the branch, the distance has no direct geometric meaning, so it is preferred to use mean shift clustering to generate candidate instances; The tree species features extracted by the semantic label branch are preferably generated using the HDBSCAN algorithm on the original point cloud coordinates. Among the three redundant candidate sets generated, the subset that best explains the scene is selected. In particular, the output of each branch can use any algorithm to generate candidate instances.

[0043] In one or more preferred embodiments, the feature screening module uses one of 3DuNet, PointNet, PointNet++ and KpConv to predict the matching degree between the candidate instance and the true label, and outputs the normalized score through the pooling and fully connected layers. θ ∈(0,1] completes single tree segmentation.

[0044] S4. Using the multi-platform point cloud data of trees with tree species information and single tree label data, the single tree segmentation model is trained.

[0045] In one or more preferred embodiments, step S4 is specifically implemented as follows: S41, feeding the point cloud data with tree species information and individual tree labels into a single tree segmentation model for training, wherein the single tree segmentation model is based on cylindrical sampling and adds Gaussian random noise, random rotation, random voxel sampling, and random stretching for data enhancement; S42. Iterate the pseudo-labels according to the training results to obtain high-quality tree species and single tree point cloud results and save them in LAS point cloud file format; obtain the training weights after the training is completed.

[0046] S5. Use the trained single tree segmentation model to perform single tree segmentation.

[0047] In addition, the above embodiment provides a detailed description of the deep learning-based backpack and drone lidar single tree segmentation method provided by this embodiment, and expounds on the technical principles and implementation methods of the present invention. The description of the above embodiment is only intended to help understand the method and core concept of the present invention; at the same time, for ordinary technicians in this field, based on the concept of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of the above embodiment should not be understood as limiting the present invention.

[0048] In another embodiment of the present application, a deep learning-based backpack and UAV lidar single tree segmentation system is provided, which includes a data acquisition and fusion module, a data annotation module, a model construction module, a model training module, and a single tree segmentation module; The data acquisition and fusion module is used to collect LiDAR point cloud data of target trees based on the UAV and backpack LiDAR system and perform point cloud denoising, point cloud filtering, and ground point normalization processing; the processed LiDAR point cloud data of the target trees is aligning and fused to generate multi-platform point cloud data of the trees; The data annotation module is used to match the data containing tree species information and single tree position coordinate information with the tree multi-platform point cloud data, assign the tree species information to the tree multi-platform point cloud data, and generate single tree label data; The model construction module is used to construct a single tree segmentation model, including a feature extraction network, a feature sorting and clustering module, and a feature screening module. The feature extraction network extracts point cloud features from multi-platform point cloud data of trees using a submanifold sparse convolution kernel and outputs them to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering. The feature screening module is used to perform instance screening and single tree segmentation based on the clustering results of the three branches. The model training module is used to train the single tree segmentation model based on the multi-platform point cloud data of trees with tree species information and single tree label data; The single tree segmentation module is used to perform single tree segmentation according to the trained single tree segmentation model.

[0049] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. The system can be applied to the deep learning-based backpack and drone lidar single tree segmentation method in the above embodiment.

[0050] In another embodiment of the present application, a storage medium is provided, storing a program. When the program is executed by a processor, the method for single tree segmentation using a backpack and a drone lidar based on deep learning in the above embodiment is implemented, specifically: S1. Using a UAV and backpack LiDAR system, collect LiDAR point cloud data of target trees and perform point cloud denoising, point cloud filtering, and ground point normalization. The processed LiDAR point cloud data of target trees is then registered and fused to generate multi-platform point cloud data of the trees. S2. Matching the data containing tree species information and individual tree position coordinate information with the multi-platform point cloud data of the tree, assigning the tree species information to the multi-platform point cloud data of the tree, and generating individual tree label data; S3. Construct a single tree segmentation model, including a feature extraction network, a feature sorting and clustering module, and a feature screening module. The feature extraction network extracts point cloud features from multi-platform point cloud data of trees using a submanifold sparse convolution kernel and outputs them to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering. The feature screening module is used to perform instance screening and single tree segmentation based on the clustering results of the three branches. S4. Training the single tree segmentation model using multi-platform point cloud data of trees with tree species information and single tree label data; S5. Use the trained single tree segmentation model to perform single tree segmentation.

[0051] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0052] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A deep learning-based backpack and drone lidar single tree segmentation method, characterized by: The steps include: The LiDAR point cloud data of target trees is collected based on UAVs and backpack LiDAR systems, and point cloud denoising, point cloud filtering, and ground point normalization are performed. The processed LiDAR point cloud data of the target tree is registered and fused to generate multi-platform point cloud data of the tree; Matching the data containing tree species information and individual tree location coordinate information with the multi-platform point cloud data of the tree, assigning the tree species information to the multi-platform point cloud data of the tree, and generating individual tree label data; Construct a single tree segmentation model, including a feature extraction network, a feature sorting and clustering module, and a feature screening module; The feature extraction network extracts point cloud features from multi-platform point cloud data of trees through submanifold sparse convolution kernels and outputs them to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering; the feature screening module is used to perform instance screening and single tree segmentation based on the clustering results of the three branches; The single tree segmentation model is trained using multi-platform point cloud data of trees with tree species information and single tree label data; Use the trained single tree segmentation model to perform single tree segmentation.

2. The deep learning-based backpack and drone laser radar single tree segmentation method according to claim 1 is characterized in that: After collecting LiDAR point cloud data of the target trees, the raw data is converted into a laser file format. At the same time, a real-time differential positioning device is used to record the coordinates of each tree and record the tree species information; The point cloud denoising method specifically includes: constructing a point cloud elevation density model based on kernel density estimation; removing outliers in the clustered point cloud that are farther than a set threshold; and using a neighboring point search method to remove points that are closer to the ground than a set threshold. The point cloud filtering adopts a cloth simulation method, specifically: the original point cloud is inverted in the vertical direction so that the ground points are located below the cloth; a virtual cloth grid is generated above the inverted point cloud, and each cloth grid node is assigned an initial position; Perform an iterative simulation, in which the cloth mesh nodes move downward due to gravity. The displacement of the cloth mesh nodes is constrained by the adjacent mesh nodes. The cloth mesh nodes stop moving when they encounter a ground point in the point cloud. The point cloud area corresponding to the final position of the cloth is the ground point, and the rest are non-ground points. The ground point normalization is specifically as follows: subtracting the height value of the ground point obtained by point cloud filtering from the elevation value of each point to obtain a normalized point cloud; The processed LiDAR point cloud data of the target tree are registered and fused, specifically: the two point clouds are registered and fused using an iterative closest point registration algorithm.

3. The method for single tree segmentation based on backpack and UAV laser radar based on deep learning according to claim 1 is characterized in that: The data containing tree species information and single tree position coordinate information is matched with the tree multi-platform point cloud data, the tree species information is assigned to the tree multi-platform point cloud data, and single tree label data is generated. The specific implementation process is as follows: The multi-platform point cloud data of trees and the data containing tree species information and individual tree position coordinate information are unified into the same coordinate system. A KD tree is constructed for the multi-platform point cloud data of trees. The space is recursively divided into hyperplanes along a certain dimension to form a binary tree structure for search. Search the multi-platform point cloud data of trees in sequence according to the position of a single tree, find the point closest to the position of the single tree, and assign the tree species information to the point closest to the position of the single tree; The assigned tree multi-platform point cloud data are annotated with single tree point clouds one by one to generate single tree label data.

4. The deep learning-based backpack and drone laser radar single tree segmentation method according to claim 1 is characterized in that: Before the multi-platform tree point cloud data is input into the single tree segmentation model, the entire point cloud voxel grid is first downsampled to thin out overly dense areas and achieve a uniform point density; a random sampling method is used during sampling, and a cylinder is selected as the neighborhood definition.

5. The method for single tree segmentation based on backpack and UAV laser radar based on deep learning according to claim 1 is characterized in that: The feature extraction network includes an input layer, an encoding layer and a decoding layer; The input layer includes a convolution kernel of 5×5×5×1; The encoding layer has two layers, each layer includes four 3×3×3+3 sparse convolution kernels; The encoding layer has three layers, each of which includes four 3×3×3+3 sparse convolution kernels; In this method, layers with the same step amplitude are connected through skip connections.

6. The deep learning-based backpack and drone laser radar single tree segmentation method according to claim 1 is characterized in that: In the feature sorting and clustering module: The semantic labeling branch is used to classify the point cloud into "tree" and "non-tree" points and output the "tree" point cloud for single tree segmentation. It consists of a single hidden layer multi-layer perceptron with a softmax activation and a spatial attention mechanism, which outputs the semantic category probability of each point. The semantic labeling branch is trained using a standard cross-entropy loss and obtains the semantic label by taking the argmax of the predicted category probabilities. The center offset branch encodes the feature vector to produce N points, the corresponding offset is expressed as , and then predict a three-dimensional offset vector for each point so that the point can be moved to the center of the instance; the center offset branch is obtained by L 1 regression loss to constrain the learned offset, as follows: ； ； in, m i =1 indicates that i In the example, m i =0 means the i The point is not on the instance; O i For the i The offset of the points, p i 、 p j is the origin coordinate of the corresponding point, Center of mass; g ( i ) is the mapping point i The corresponding grounding instance index, that is, the i An example of a point, is an instance g ( i ) in the number of points; The instance embedding branch maps each point to a 5D feature space for instance-level differentiation. The instance embedding branch is supervised by contrastive loss during training. Region growing clustering, mean shift clustering, HDBSCAN algorithm, watershed algorithm or BIRCH algorithm are used to generate candidate instances for the outputs of the three branches.

7. The deep learning-based backpack and drone laser radar single tree segmentation method according to claim 1 is characterized in that: The feature screening module uses one of 3DuNet, PointNet, PointNet++ and KpConv to predict the matching degree between candidate instances and true labels, and outputs normalized scores through pooling and fully connected layers. θ ∈(0,1] completes single tree segmentation.

8. The method for single tree segmentation based on backpack and UAV laser radar based on deep learning according to claim 1 is characterized in that: The single tree segmentation model is trained using the multi-platform point cloud data of trees with tree species information and single tree label data. The specific implementation process is as follows: The multi-platform tree point cloud data with tree species information and individual tree label data is fed into a single tree segmentation model for training. The single tree segmentation model is based on cylindrical sampling and uses Gaussian random noise, random rotation, random voxel sampling, and random stretching for data augmentation. Iterate the pseudo-labels based on the training results to obtain the tree species and single tree point cloud results and training weights.

9. A deep learning-based backpack and drone lidar single tree segmentation system, characterized by: A deep learning-based backpack and UAV lidar single tree segmentation method applied to any one of claims 1-8, comprising a data acquisition and fusion module, a data annotation module, a model construction module, a model training module, and a single tree segmentation module; The data acquisition and fusion module is used to collect LiDAR point cloud data of target trees based on the UAV and backpack LiDAR system and perform point cloud denoising, point cloud filtering, and ground point normalization processing; the processed LiDAR point cloud data of the target trees is aligning and fused to generate multi-platform point cloud data of the trees; The data annotation module is used to match the data containing tree species information and single tree position coordinate information with the tree multi-platform point cloud data, assign the tree species information to the tree multi-platform point cloud data, and generate single tree label data; The model construction module is used to construct a single tree segmentation model, including a feature extraction network, a feature sorting and clustering module, and a feature screening module. The feature extraction network extracts point cloud features from multi-platform point cloud data of trees using a submanifold sparse convolution kernel and outputs them to the semantic label, center offset, and instance embedding branches of the feature sorting and clustering module for feature sorting and clustering. The feature screening module is used to perform instance screening and single tree segmentation based on the clustering results of the three branches. The model training module is used to train the single tree segmentation model based on the multi-platform point cloud data of trees with tree species information and single tree label data; The single tree segmentation module is used to perform single tree segmentation according to the trained single tree segmentation model.

10. A storage medium storing a program, characterized in that: When the program is executed by the processor, the backpack and drone lidar single tree segmentation method based on deep learning according to any one of claims 1 to 8 is implemented.

Images