Single tree segmentation method based on machine learning

Abstract

The invention discloses an individual tree segmentation method based on machine learning, and particularly relates to the field of individual tree segmentation. The method comprises the following steps: respectively acquiring forest point cloud data through a ground scanner and an unmanned aerial vehicle, registering the forest point cloud data into a complete data set, preprocessing the point cloud data, extracting geometric, reflective and spatial distribution multi-dimensional features, and fusing the geometric, reflective and spatial distribution multi-dimensional features into a comprehensive feature vector; training a model by using a machine learning method based on the labeled data, taking the feature vector as input, and outputting whether the point belongs to a tree or not; predicting a point category by using the trained model, performing individual tree instance segmentation by using a connected component clustering and graph segmentation method through CRF post-processing optimization, constructing a 3D model and calculating key parameters; and finally, performing post-processing on a preliminary segmentation result, filtering noise by using connected region analysis, filling the deficiency by using a three-dimensional hole, optimizing a boundary based on graph segmentation, and optimizing a block seam by using an ICP (Inductively Coupled Plasma) algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of single tree segmentation, and more particularly to a single tree segmentation method based on machine learning. BACKGROUND

[0002] With the rapid development of laser radar technology and unmanned aerial vehicle technology, forest resource investigation and monitoring based on point cloud data has become possible. The high density and high precision characteristics of point cloud data make it possible to capture single trees in the forest. However, due to the different characteristics and limitations of point cloud data obtained by ground scanners and unmanned aerial vehicles, how to fuse these two kinds of data and accurately realize single tree segmentation is still a technical problem. In the prior art, the method of single tree segmentation based on a single data source still has many defects, such as in complex terrain and densely wooded areas, it is difficult to achieve high-precision segmentation results.

[0003] However, it still has some shortcomings in actual use, such as traditional forest resource investigation mainly relies on manual sample plot measurement, which is not only low in efficiency and high in cost, but also difficult to implement in complex terrain areas. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a single tree segmentation method based on machine learning to solve the problems raised in the background art.

[0005] To achieve the above object, the present application provides the following technical scheme:

[0006] Step A1: using a ground scanner and an unmanned aerial vehicle to obtain point cloud data of a forest area respectively; the data of the two are matched by a space matching method to form a complete forest point cloud data set;

[0007] Step A2: performing noise filtering, outlier rejection and resolution uniform preprocessing operations on the collected ground scanner and unmanned aerial vehicle point cloud data, and dividing the preprocessed point cloud data into multiple small blocks;

[0008] Step A3: extracting multi-dimensional feature information from the preprocessed point cloud data, fusing the multi-dimensional feature information, and generating a comprehensive feature vector;

[0009] Step A4: based on the collected labeled data, using a machine learning method to train a model; in the training process, the input is the fused feature vector, and the output is whether each point belongs to a part of a tree;

[0010] Step A5: using the trained machine learning model to predict the fused point cloud data, judging the category of each point, connecting all points of the same tree into a complete tree model, and performing single tree segmentation.

[0011] Step A6: Post-processing is performed on the preliminary segmentation result.

[0012] Preferably, in step A1, first, ground point cloud data of the target forest area is collected using a ground laser scanner; the ground scanner obtains fine three-dimensional point cloud information of the bottom of the trees and the near-ground area by emitting a laser beam and receiving a reflected signal, and a UAV equipped with a laser radar performs aerial scanning of the same area to obtain point cloud data of the tree canopy and upper structure.

[0013] Preferably, in step A2, for the ground scanner data, due to its close-range scanning characteristics, it is easily disturbed by non-target objects such as ground vegetation and fallen branches, so a statistical-based outlier removal algorithm is used to remove noise points that deviate significantly from the main body by calculating the average distance in the neighborhood of each point and setting a threshold; for the UAV point cloud data, the flying point noise caused by atmospheric interference or sensor error during flight is processed, and a radius filtering method is used to detect and delete isolated abnormal points in three-dimensional space.

[0014] Preferably, in step A3, multi-dimensional feature information is first extracted from each point cloud block, including three categories of geometric features, reflection features and spatial distribution features; in terms of geometric features, the normal vector, curvature, local surface roughness and elevation coefficient of variation in the neighborhood of each point are calculated, which can effectively represent the branch shape and canopy structure of the trees;

[0015] Reflection features are obtained by analyzing laser echo intensity and multispectral information to obtain material properties and physiological state features of the trees; spatial distribution features include point density distribution, vertical profile features and voxel-based occupancy rate, etc. Global statistics are used to describe the spatial distribution law of different tree species.

[0016] Preferably, in step A4, after feature extraction and fusion, a machine learning model is trained using a labeled point cloud dataset; each point in the training data is manually labeled as "tree" or "non-tree" category, and these labels serve as a supervisory signal to guide model learning; first, the dataset is divided into training set, validation set and test set according to the ratio of 7:2:1; during training, the fused multi-dimensional feature vector is used as the input feature X, and the corresponding category label is used as the output y to construct a classification model; taking random forest as an example, the model performs ensemble learning by constructing multiple decision trees, each tree selects the optimal feature based on Gini impurity or information gain to split the node, and finally determines the category through a voting mechanism; for more complex point cloud data, a deep learning model PointNet is used, which can automatically learn hierarchical feature representation of point cloud; during training, a cross-entropy loss function is used for optimization;

[0017] To prevent overfitting, early stopping strategy and regularization techniques are adopted, and accuracy, recall rate and F1-score and other indicators are monitored on the validation set; after training is completed, the model performance is evaluated on the test set to ensure that it has good generalization ability.

[0018] Preferably, in step A5, after the model training is completed, it is deployed to the actual point cloud segmentation process. First, the preprocessed fused point cloud data is input into the trained model for point-by-point prediction, and the model outputs the probability value of each point belonging to the tree, and sets 0.5 as the classification threshold, and the points with probability greater than the threshold are determined as tree points; in order to improve the segmentation accuracy, conditional random field (CRF) is used for post-processing optimization, and the spatial relationship between points is considered by establishing an energy function;

[0019] Single tree instance segmentation is performed, and a clustering algorithm based on connected components is used: first, the classified tree point cloud is clustered in Euclidean space, and the distance threshold is set to cluster the points connected in space into a class; then, each cluster is processed twice, the vertical distribution characteristics of the point cloud are analyzed, and multiple adjacent trees that may exist are identified and segmented; for overlapping canopies in complex scenes, an optimization method based on graph cut is introduced, and fine segmentation is performed combined with the geometric characteristics of the trees.

[0020] Preferably, in step A6, after obtaining the preliminary segmentation result, fine processing is needed for the segmentation result to improve the quality of the final output. First, a noise filtering algorithm based on connected region analysis is used, and by calculating the volume and point cloud density of each segmentation cluster, discrete point clusters that do not obviously meet the characteristics of single trees are removed, and in specific implementation, the volume threshold V min and the point number threshold N min are set, and clusters that meet the following conditions are removed;

[0021] V i <V min or N i <N min

[0022] For the problem of missing point cloud in the tree canopy due to occlusion, a three-dimensional hole filling algorithm is used, which first detects the topological holes of the point cloud surface, and then uses radial basis function (RBF) for surface interpolation reconstruction.

[0023] Technical effects and advantages of the present application:

[0024] The present application fuses the point cloud data of the ground scanner and the unmanned aerial vehicle, fully utilizes the data advantages of the two, and improves the accuracy and robustness of single tree segmentation; compared with the prior art, the present application has the following advantages:

[0025] High precision: Fusion of data from two different data sources can more comprehensively capture the structural information of trees and significantly improve segmentation accuracy.

[0026] Strong adaptability: It can adapt to environments with different terrains and tree distribution densities and has strong generalization ability.

[0027] High degree of automation: Through automatic training and prediction of machine learning models, the single tree splitting process is automated, reducing manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0030] See also Figure 1 As shown, the present invention provides a single tree segmentation method based on machine learning, comprising the following steps:

[0031] Step A1: Use a terrestrial scanner and a drone to obtain point cloud data of the forest area. The two data are aligned using a spatial matching method to form a complete forest point cloud dataset.

[0032] In step A1, first, a terrestrial laser scanner is used to collect ground point cloud data of the target forest area. The terrestrial scanner emits laser beams and receives reflected signals to obtain detailed three-dimensional point cloud information of the tree base and near-ground area. Simultaneously, a drone equipped with a lidar performs aerial scanning of the same area to obtain point cloud data of the tree canopy and upper structure.

[0033] After the acquisition is completed, the point cloud data from the two sources are spatially aligned; the specific methods include:

[0034] Feature matching: Extract common feature points from the ground scanner and drone point clouds and perform preliminary alignment using an iterative closest point algorithm.

[0035] Coordinate system 1: Use ground control point measurement data to transform the two point clouds into the same coordinate system to ensure spatial consistency;

[0036] Optimization and adjustment: Combining the point cloud density and overlapping area, the least squares method or nonlinear optimization method is used to further optimize the registration accuracy, and finally a complete forest point cloud dataset is generated.

[0037] Step A2: Noise filtering, outlier removal and resolution unification preprocessing operations are performed on the collected ground scanner and unmanned aerial vehicle point cloud data, and the preprocessed point cloud data is segmented into multiple small blocks;

[0038] In the step A2, for the ground scanner data, due to its close-range scanning characteristics, it is easy to be disturbed by non-target objects such as ground vegetation and fallen leaves, therefore a statistical-based outlier removal algorithm is adopted, by calculating the average distance in the neighborhood of each point and setting a threshold, the noise points obviously deviating from the main body are removed; for the unmanned aerial vehicle point cloud data, the flying point noise generated by atmospheric interference or sensor error in the flight process is processed, and the radius filtering method is used to detect and delete isolated abnormal points in three-dimensional space.

[0039] After completing the noise filtering, the resolution unification processing is performed on the point cloud data of two different sources, the high-density ground scanning data and the relatively sparse unmanned aerial vehicle data are adjusted to the same scale through voxel grid downsampling, to ensure the spatial consistency in data fusion; considering the calculation efficiency of large-scale point cloud data, the complete point cloud data set after preprocessing is further segmented into regular spatial blocks, an octree-based region division method is adopted to divide the forest area into several cubic small blocks with edge length of 5-10 meters, and appropriate overlapping boundaries are reserved for each block to avoid trees being cut off at the segmentation edge.

[0040] Step A3: Multi-dimensional feature information is extracted from the preprocessed point cloud data, and the multi-dimensional feature information is fused to generate a comprehensive feature vector;

[0041] In the step A3, multi-dimensional feature information is first extracted from each point cloud block, including three categories of geometric features, reflection features and spatial distribution features; in terms of geometric features, the normal vector, curvature, local surface roughness and elevation coefficient of variation in the neighborhood of each point are calculated, which can effectively represent the branch shape and crown structure of the tree;

[0042] Among them, the normal vector is obtained by PCA feature decomposition of the neighborhood point covariance matrix, the feature vector corresponding to the minimum eigenvalue is the normal vector, and the covariance matrix of the neighborhood point is calculated as follows:

[0043]

[0044] C represents the covariance matrix of the neighborhood points, N represents the set of neighborhood points, p j represents the jth point in the neighborhood point set N, represents the centroid or average position of the point p, that is, the average value of the neighborhood points N;

[0045] The calculation method of the local surface roughness is as follows:

[0046]

[0047] where σ i denotes the local surface roughness of the i-th point, N denotes the number of neighborhood points, z j denotes the vertical coordinate of the j-th point, denotes the mean value in the vertical direction;

[0048] curvature k i calculated from the eigenvalues of the covariance matrix, the height variation coefficient CV z using the vertical coordinates of the points;

[0049] The reflection feature is obtained by analyzing the laser echo intensity and the multi-spectral information to obtain the material properties and physiological state features of the trees; the spatial distribution feature includes the point density distribution, the vertical profile feature, and the voxel-based occupancy rate and other global statistics, which are used to describe the spatial distribution law of different tree species.

[0050] The calculation method of the point density distribution is specifically as follows:

[0051]

[0052] where ρ i denotes the density of the i-th point, N denotes the set of neighborhood points, and r denotes the neighborhood radius;

[0053] The vertical profile feature is used to statistically describe the distribution of the point cloud in the vertical direction, and the calculation method of the vertical skewness is specifically as follows:

[0054]

[0055] where s z denotes the vertical skewness, N denotes the set of neighborhood points, and z j denotes the vertical coordinate of the j-th point, denotes the mean value in the vertical direction, and σ z denotes the standard deviation in the vertical direction;

[0056] The vertical skewness is used to distinguish between trees (with larger skewness) and shrubs (with smaller skewness).

[0057] The calculation method of the vertical kurtosis is specifically as follows:

[0058]

[0059] where k z denotes the vertical kurtosis, N denotes the set of neighborhood points, and z j denotes the vertical coordinate of the j-th point, denotes the mean value in the vertical direction, and σz standard deviation in the vertical direction;

[0060] The above features are combined into a comprehensive feature vector:

[0061]

[0062] where F i is the comprehensive feature vector, k i is the curvature, σ i is the local surface roughness of the i-th point, CV z is the coefficient of variation in height, is the laser return intensity, NDVI is the normalized difference vegetation index, ρ i is the density of the i-th point, s z is the vertical skewness, k z is the vertical kurtosis, O v is the point cloud occupancy rate of each voxel.

[0063] Step A4: Based on the collected labeled data, a machine learning method is used for model training; during the training process, the input is the fused feature vector, and the output is whether each point belongs to a part of the tree;

[0064] In step A4, after feature extraction and fusion, a machine learning model is trained using a labeled point cloud dataset; each point in the training data is manually labeled as "tree" or "non-tree" category, and these labels serve as a supervisory signal to guide model learning; first, the dataset is divided according to a 7:2:1 ratio into a training set, a validation set, and a test set; during training, the fused multi-dimensional feature vector is used as the input feature X, and the corresponding class label is used as the output y to build a classification model; using a random forest as an example, the model performs ensemble learning by building multiple decision trees, each tree selects the optimal feature based on Gini impurity or information gain to split the nodes, and finally determines the class through a voting mechanism; for more complex point cloud data, a deep learning model called PointNet is used, which can automatically learn hierarchical feature representations of point clouds; during training, the cross-entropy loss function is used for optimization, and its mathematical expression is:

[0065]

[0066] where L is the weighted cross-entropy loss value, M is the total number of samples, y a is the true label of the a-th sample, is the class weight, is the probability that the model predicts the a-th sample as class y a ​

[0067] To prevent overfitting, early stopping strategy and regularization techniques are adopted, and accuracy, recall rate and F1-score are monitored on the validation set. After training, the model performance is evaluated on the test set to ensure its good generalization ability.

[0068] The final model can make point-by-point classification prediction on new input point cloud data and output the probability value of each point belonging to a tree.

[0069] Step A5: Through the trained machine learning model, the fused point cloud data is predicted to determine the category of each point, and all points of the same tree are connected into a complete tree model for single tree segmentation.

[0070] In step A5, after the model training is completed, it is deployed to the actual point cloud segmentation process. First, the preprocessed fused point cloud data is input into the trained model for point-by-point prediction, and the model outputs the probability value of each point belonging to a tree. Set 0.5 as the classification threshold, and judge the points with probability greater than the threshold as tree points. In order to improve the segmentation accuracy, we use conditional random field (CRF) for post-processing optimization, and establish an energy function to consider the spatial relationship between points.

[0071]

[0072] Where E(x) represents the total energy function, x represents the set of points, x b represents the feature of the bth point, ψ u (x b ) represents the unary potential, which represents the energy of a single point; ψ p (x b , x c ) represents the binary potential, which represents the relationship energy between points.

[0073] After CRF optimization, the isolated noise points are significantly reduced and the segmentation boundary is smoothed.

[0074] Single tree instance segmentation is performed using a clustering algorithm based on connected components: first, the classified tree point cloud is clustered in Euclidean space, and the distance threshold is set to cluster the points connected in space into a class; then, each cluster is processed twice to identify and segment possible multiple adjacent trees by analyzing the vertical distribution characteristics of the point cloud; for overlapping tree crowns in complex scenes, an optimization method based on graph cut is introduced to perform fine segmentation combined with the geometric characteristics of trees.

[0075] After segmentation, a complete 3D model is constructed for each tree; first, the trunk axis of the tree is extracted, and a RANSAC algorithm is used to fit a cylindrical model; then the tree crown surface is reconstructed, and a Poisson reconstruction or convex hull algorithm is used to generate a closed triangular mesh; finally, the key parameters of each tree are calculated, including tree height (vertical distance from the lowest point to the highest point), breast diameter (trunk diameter at 1.3 meters above the ground), crown width (horizontal projection area of the tree crown), etc. These parameters are stored in the forest resource database to support subsequent analysis and application.

[0076] Step A6: post-processing of the preliminary segmentation results.

[0077] In step A6, after obtaining the preliminary segmentation results, fine processing is needed to improve the quality of the final output. First, a noise filtering algorithm based on connected region analysis is used to remove discrete point clusters that do not meet the characteristics of single trees by calculating the volume and point cloud density of each segmentation cluster. In implementation, set volume threshold V min and point number threshold N min , clusters that meet the following conditions will be removed;

[0078] V i <V min or N i <N min

[0079] For the problem of missing point cloud in the tree canopy due to occlusion, a three-dimensional hole filling algorithm is used, which first detects the topological holes of the point cloud surface, and then uses radial basis function (RBF) for surface interpolation reconstruction;

[0080] For the jagged and non-smooth phenomenon existing in the segmentation boundary, an energy minimization method based on graph cut is used for optimization. To improve the efficiency of the algorithm, the entire point cloud space is divided into an octree structure for processing, and each voxel node stores feature statistics. The processing speed is improved by 3-5 times while maintaining accuracy. Finally, the iterative closest point (ICP) algorithm is used to register and optimize the overlapping area of adjacent blocks, eliminating the misalignment phenomenon at the block joint, and the registration error is controlled within 0.1m.

[0081] Finally: the above only for the preferred embodiments of the present application, and not for limiting the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application, should be included in the protection scope of the present application.

Claims

1. A single tree segmentation method based on machine learning, characterized in that: include: Step A1: Use a terrestrial scanner and a drone to obtain point cloud data of the forest area. The data of the two are registered through spatial matching method to form a complete forest point cloud dataset; Step A2: Perform noise filtering, outlier removal, and resolution unification preprocessing on the collected ground scanner and drone point cloud data, and then segment the preprocessed point cloud data into multiple small blocks; Step A3: extracting multidimensional feature information from the preprocessed point cloud data, fusing the multidimensional feature information, and generating a comprehensive feature vector; Step A4: Based on the collected labeled data, use machine learning methods to train the model; During the training process, the input is the fused feature vector, and the output is whether each point is part of a tree; Step A5: Use the trained machine learning model to predict the fused point cloud data, determine the category of each point, connect all points of the same tree into a complete tree model, and perform single tree segmentation; Step A6: Post-process the results of the preliminary segmentation.

2. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A1, first, a terrestrial laser scanner is used to collect ground point cloud data of the target forest area. The terrestrial scanner obtains detailed three-dimensional point cloud information of the tree base and near-ground area by emitting laser beams and receiving reflected signals. At the same time, a drone equipped with a lidar performs aerial scanning of the same area to obtain point cloud data of the tree canopy and upper structure.

3. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A2, for ground scanner data, due to its close-range scanning characteristics, it is easily interfered with by non-target objects such as surface vegetation, dead branches and fallen leaves. Therefore, a statistical outlier removal algorithm is adopted to remove noise points that obviously deviate from the main subject by calculating the average distance within the neighborhood of each point and setting a threshold. For UAV point cloud data, flying point noise generated by atmospheric interference or sensor errors during flight is processed, and a radius filtering method is used to detect and delete isolated outliers in three-dimensional space.

4. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A3, multi-dimensional feature information is first extracted from each point cloud block, including three categories: geometric features, reflection features, and spatial distribution features; In terms of geometric features, the normal vector, curvature, local surface roughness and elevation variation coefficient within the neighborhood are calculated for each point; The neighborhood point covariance matrix is ​​decomposed by PCA. The eigenvector corresponding to the minimum eigenvalue is the normal vector. The covariance matrix of the neighborhood points is calculated as follows: C represents the covariance matrix of the neighborhood points, N represents the set of neighborhood points, and p j Represented as the jth point in the neighborhood point set N, It is expressed as the centroid or average position of point p, that is, the average value of the neighborhood points N; The calculation method of local surface roughness is as follows: Among them, σ i is the local surface roughness of the i-th point, N is the number of neighborhood points, z j Expressed as the vertical coordinate of the j-th point, Expressed as the mean in the vertical direction.

5. The method for single tree segmentation based on machine learning according to claim 4, characterized in that: Reflection characteristics are analyzed by laser echo intensity and multispectral information to obtain the material properties and physiological state characteristics of trees. Spatial distribution characteristics include point density distribution, vertical profile characteristics, and global statistics such as voxel occupancy rate, which are used to describe the spatial distribution patterns of different tree species. The calculation method of point density distribution is as follows: Among them, ρ i It is represented as the density of the i-th point, N is the set of neighborhood points, and r is the neighborhood radius; The vertical profile feature statistics point cloud is distributed in the vertical direction. The vertical skewness is calculated as follows: Among them, s z It is expressed as vertical skewness, N is the set of neighborhood points, z j Expressed as the vertical coordinate of the j-th point, Expressed as the mean in the vertical direction, σ z The standard deviation in the vertical direction.

6. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A4, after feature extraction and fusion, the machine learning model is trained using the labeled point cloud dataset. Each point in the training data is manually labeled as "tree" or "non-tree," and these labels serve as supervisory signals to guide model learning. During the training process, the cross entropy loss function is used for optimization, and its mathematical expression is: Among them, L represents the weighted cross entropy loss value, M represents the total number of samples, and y a Represented as the true label of the a-th sample, Expressed as category weight, It is represented as the model predicting the category y for the a-th sample a probability.

7. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A5, after model training is complete, we deploy it in the actual point cloud segmentation process. First, the pre-processed fused point cloud data is input into the trained model for point-by-point prediction. The model outputs the probability value of each point belonging to a tree. We set 0.5 as the classification threshold and identify points with probabilities greater than the threshold as tree points. Post-processing optimization is performed using conditional random fields, and the spatial relationship between points is considered by establishing an energy function. To perform single tree instance segmentation, a connected component-based clustering algorithm is used. First, the classified tree point cloud is Euclidean clustering, and a distance threshold is set to group spatially connected points into one category. Each cluster is then processed secondary to identify and segment multiple adjacent trees by analyzing the vertical distribution characteristics of the point cloud. For overlapping tree crowns in complex scenes, a graph cut-based optimization method is introduced, combining the geometric characteristics of the trees for fine segmentation.

8. The method for single tree segmentation based on machine learning according to claim 1, characterized in that: In step A6, after obtaining the preliminary segmentation results, the segmentation results need to be refined to improve the final output quality. First, a noise filtering algorithm based on connected region analysis is used to calculate the volume and point cloud density of each segmentation cluster and remove discrete point clusters that do not meet the characteristics of a single tree. In the specific implementation, a volume threshold V is set. min Sum point number threshold N min ; To address the problem of missing point clouds caused by occlusion in tree canopies, a three-dimensional hole filling algorithm is used. This algorithm first detects topological holes on the point cloud surface and then uses radial basis functions to perform surface interpolation reconstruction.

Images