Onboard lidar single tree segmentation method based on extreme value offset deep learning

Abstract

The present application relates to a kind of based on extreme value offset deep learning airborne laser radar single tree segmentation method, comprising: S1, pre-processing;S2, extreme value offset: using extreme value offset deep learning algorithm, the spatial transformation of forest point cloud after pre-processing is carried out, each point is offset to corresponding tree vertex, and offset point cloud is obtained;S3, adaptive clustering: using the adaptive mean shift clustering algorithm based on average adjacent distance to the clustering of offset point cloud, offset point cloud set is divided, and marking is completed;S4, space mapping: after clustering annotation, offset point cloud is mapped back to original point cloud space, and the preliminary segmentation of single tree is completed;S5, ITS point set aggregation: adjacent crown layer with gentle gradient change is merged, to reduce over-segmentation error;S6, post-processing.The present application has higher universality, even in high canopy density and complex stand environment can also realize accurate single tree segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of single tree segmentation, and particularly relates to an airborne laser radar single tree segmentation method based on extreme value offset deep learning. BACKGROUND

[0002] It is of great significance to obtain forest growth information in time and effectively for protecting forest resources and formulating reasonable forest management plan. Airborne radar overcomes the defects of traditional artificial field investigation which is time-consuming and laborious, therefore, in recent decades, ALS (Airborne laser scanning) is widely applied in the field of forestry investigation. However, due to the complex structure of forest stand, the traditional ITS (Individual tree segmentation) algorithm which is set by human segmentation rules is difficult to cover various situations, and the generalization and segmentation accuracy of the single tree segmentation model still have a lot of room for improvement. Therefore, it is particularly important to design a single tree segmentation method with high generalization and high accuracy. SUMMARY

[0003] The technical problem to be solved by the present application is to overcome the defects of the prior art, and provide an airborne laser radar single tree segmentation method based on extreme value offset deep learning, which has high generalization, and can realize accurate single tree segmentation even in a high canopy density and complex forest stand environment.

[0004] In order to solve the above technical problems, the technical scheme of the present application is as follows: an airborne laser radar single tree segmentation method based on extreme value offset deep learning, comprising:

[0005] S1, preprocessing: sequentially performing point cloud filtering, height normalization, dividing subplots, point cloud denoising, downsampling and coordinate normalization on the forest point cloud, wherein the forest point cloud is obtained based on airborne laser radar;

[0006] S2, extreme value offset: using an extreme value offset deep learning algorithm, performing spatial transformation on the preprocessed forest point cloud, offsetting each point to the corresponding tree top point, and obtaining an offset point cloud;

[0007] S3, adaptive clustering: using an adaptive mean shift clustering algorithm based on average neighborhood distance to cluster the offset point cloud, dividing the offset point cloud set, and completing labeling;

[0008] S4, spatial mapping: mapping the labeled offset point cloud back to the original point cloud space to complete the preliminary segmentation of single trees;

[0009] S5, ITS point set aggregation: merging adjacent crown layers with gentle gradient changes to reduce over-segmentation errors;

[0010] S6, post-processing: the results of point set aggregation are up-sampled and coordinate de-normalized to complete segmentation.

[0011] Further, step S2 is based on a PointTransformer network, which has three core modules: a down-sampling module, an up-sampling module, and a Transformer module; wherein,

[0012] The down-sampling module reduces the number of data points while extracting deeper feature information, and the working steps include: first, using the FPS algorithm to sample the input forest point cloud to obtain a sampled point cloud; then, taking each sampling point as a center point, using the kNN graph to find the k nearest neighbors around the center point; finally, each center point and its neighbors form a point set, which is input into a deep network for deep feature extraction;

[0013] The role of the up-sampling module is to assign deep features to non-sampling points, and the working steps include: first, the point cloud composed of center points is subjected to feature extraction by MLP, BN and ReLU; then, the feature of each center point is interpolated to its neighboring point set using the inverse distance weight interpolation algorithm; finally, the features corresponding to the down-sampling module are aggregated to the corresponding up-sampling module through a jump connection;

[0014] The role of the Transformer module is to enhance the local semantic perception ability of each point to its surrounding points, which includes two MLPs and a Transformer layer, and the Transformer layer is as shown in formula (1):

[0015]

[0016] Wherein, β is a relationship function; ρ is a normalization function; γ is an MLP with two linear layers and a nonlinear layer ReLU; φ, ψ and α are point-by-point feature transformation matrices; X(i) is a set composed of k neighbors of x i ; δ is a position encoding function, and the expression is δ=θ(p i -p j ), p i and p j are the coordinates of 3D points i and j respectively; the encoding function θ is an MLP with two linear layers and a nonlinear layer ReLU.

[0017] Further, the extreme value offset loss function of the extreme value offset deep learning algorithm is as shown in formula (2) and formula (3):

[0018]

[0019]

[0020] where, L extreme is the loss value of the loss function; N is the total number of points; is the true offset of the i-th point to the corresponding extreme point; is the offset predicted by the network for the i-th point; ω i is the dynamic weight of the i-th point.

[0021] Further, in step S3, the adaptive mean shift clustering algorithm uses an adaptive dynamic bandwidth, and the mean shift process includes: first, calculating the mean shift vector of the current center point; second, moving the target point to the end of the mean shift vector; then, taking it as a new starting point, continue to move until the length of the mean shift vector is less than the allowed error; finally, when all points are marked, the iteration ends;

[0022] The calculation of the mean shift vector is as formula (4):

[0023]

[0024] where, p t is the center point; p i is the i-th point within the bandwidth range; n is the number of points within the bandwidth range; g(·) is the derivative of the RBF kernel function; h is the dynamic bandwidth;

[0025] The calculation strategy of the dynamic bandwidth h is as formula (5), and the calculation process includes: first, constructing a KDtree through the offset point cloud, second, calculating the average distance of each point to its k nearest neighbors according to the KDTree; finally, averaging these average distances, and the obtained average neighbor distance is taken as the dynamic bandwidth h;

[0026]

[0027] where, represents the offset point cloud; n is the total number of points; γ j (·) is used to calculate the average distance of each point to its K nearest neighbors. Λ(·) is used to obtain the average neighbor distance as the dynamic bandwidth h.

[0028] Further, in step S4, the gradient change criterion in the ITS point set aggregation algorithm is:

[0029] If the horizontal distance between the vertex of the fragment and the nearest neighbor point of the other cluster is greater than the preset bandwidth, it is considered that the gradient change between the two clusters is violent, and they belong to different single trees respectively and should not be merged;

[0030] If the horizontal distance between the top of the fragment and the nearest neighbor of other cluster is less than the preset bandwidth, it is considered that the gradient change between the two clusters is gentle, they belong to the same single tree, and the two clusters are merged.

[0031] Further, the ITS point set clustering algorithm is as formula (6)-(8):

[0032]

[0033]

[0034]

[0035] Wherein, represents the point set after aggregation, m represents the total number of current sets, and Θ(·) determines whether to merge two sets. xy (P i ,P j ) calculates the horizontal distance between two point sets; is the highest point in the set P i ; is the nearest neighbor of the set P j to the set P i ; represents the total number of points in the set P i ; w represents the grid width; and k represents the distance threshold.

[0036] By adopting the technical scheme, the points are offset to the respective tree top, so as to enhance the distinction between different single trees; then, after the points are offset to the respective tree top, the density of the tree top is significantly increased, and therefore, the mean shift algorithm is adopted to preliminarily segment the single trees; due to the randomness of the deep network, the offset effect of part of the points is poor, and directly adopting the mean shift algorithm for clustering may cause under-segmentation and over-segmentation problems; considering the characteristics that the density of the tree top is significantly increased after extreme value offset, the average adjacent distance is adaptively generated and used as the bandwidth of the mean shift algorithm, so as to cluster the dense points near the tree top into one class, and at the same time, the isolated points with poor offset effect are separately clustered, so as to eliminate the under-segmentation error as much as possible; finally, in order to reduce the over-segmentation error, an ITS point set clustering algorithm based on gradient change criterion is designed to merge the clusters with adjacent neighbors and gentle gradient change, and in general, the present application has the following beneficial effects:

[0037] 1. In view of the problem that the artificial feature extraction capability is incomplete in the traditional ITS segmentation algorithm, an extreme value offset deep learning method is proposed, the PointTransformer deep network is used to automatically extract the forest point cloud features, and the points in the forest point cloud are offset to the vicinity of the corresponding extreme value point, so as to enhance the distinction between adjacent single trees.

[0038] 2. In view of the problem that the mean shift algorithm cannot adaptively determine the bandwidth in the spatial transformed data point cloud, a dynamic bandwidth calculation strategy based on the average nearest neighbor distance is designed, which can automatically determine the bandwidth of the mean shift algorithm without any prior knowledge, and enhances the universality of the mean shift algorithm;

[0039] 3. In view of the over-segmentation problem caused by the randomness of the deep network, considering the characteristics that the crown layer gradient of single trees changes sharply and the crown layer gradient of the same single tree changes gently, an ITS point set aggregation method based on gradient change is designed, which improves the segmentation accuracy of complex forest single trees. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The flowchart of the airborne laser radar single tree segmentation method based on extreme value offset deep learning of the application;

[0041] Figure 2 The figure obtained after each step in the preprocessing process;

[0042] Figure 3 The structure diagram of the PointTransformer network;

[0043] Figure 4 The mean shift process schematic diagram;

[0044] Figure 5 The ITS point aggregation schematic diagram;

[0045] Figure 6 The schematic diagram of four sample plots in the test set;

[0046] Figure 7 The segmentation process and results of part of the subgraph in the test set;

[0047] Figure 8 The three-dimensional segmentation results of the four sample plots in the test set;

[0048] Figure 9 When the training set and the validation set participate in the network training in different offset ways, the loss value and the number of rounds curve; wherein,

[0049] Figure 2 In the figure, (a) point cloud filtering; (b) elevation normalization; (c) sub-sample plot division; (d) point cloud denoising; (e) down-sampling; (f) coordinate normalization;

[0050] Figure 5 In the figure, (a) the point sets of the first gray scale and the second gray scale should not be merged; (b) the point sets of the second gray scale and the third gray scale should be merged;

[0051] Figure 6In the figure, (a) represents sample plot U1; (b) represents sample plot U2; (c) represents sample plot G1; (d) represents sample plot G2;

[0052] Figure 7 In the figure, (a) represents the forest point cloud after data preprocessing; (b) represents the offset point cloud obtained after extreme value offset; (c) represents the labeled offset point cloud obtained after clustering by the adaptive mean shift algorithm; (d) represents the result after mapping back to the smooth forest point cloud; (e) represents the result after point set clustering; (f) represents the final segmentation result after post-processing;

[0053] Figure 8 In the figure, (a) represents sample plot U1; (b) represents sample plot U2; (c) represents sample plot G1; (d) represents sample plot G2;

[0054] Figure 9 In the figure, (a) represents the training set loss value and the number of rounds; (b) represents the validation set loss value and the number of rounds. DETAILED DESCRIPTION

[0055] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments and in conjunction with the accompanying drawings.

[0056] As Figure 1 shown, an airborne laser radar single tree segmentation method based on extreme value offset deep learning includes:

[0057] S1, preprocessing: sequentially performing point cloud filtering, elevation normalization, sub-sample plot division, point cloud denoising, down-sampling and coordinate normalization on the forest point cloud, wherein the forest point cloud is obtained based on airborne laser radar;

[0058] S2, extreme value offset: using an extreme value offset deep learning algorithm, performing spatial transformation on the preprocessed forest point cloud to offset each point to the corresponding tree top point, to obtain an offset point cloud;

[0059] S3, adaptive clustering: using an adaptive mean shift clustering algorithm based on average neighbor distance to cluster the offset point cloud, to divide the offset point cloud set and complete labeling;

[0060] S4, spatial mapping: mapping the clustered and labeled offset point cloud back to the original point cloud space to complete the preliminary segmentation of single trees;

[0061] S5, ITS point set aggregation: merging adjacent crown layers with gentle gradient changes to reduce over-segmentation errors;

[0062] S6, post-processing: after upsampling and coordinate inverse normalization of the result obtained by the point set aggregation, completing the segmentation.

[0063] Wherein, after obtaining the airborne laser radar forest point cloud, the data is preprocessed respectively, and the whole process in step S1 is as shown in Figure 2 The specific steps of data preprocessing are as follows:

[0064] S11, point cloud filtering: using cloth simulation algorithm to filter the point cloud, the point cloud is divided into two categories of ground points and non-ground points;

[0065] S12, elevation normalization: using CloudCompare software to normalize the elevation of the point cloud to remove the influence of terrain undulation on the elevation value of the point cloud data;

[0066] S13, dividing subplots: dividing the plots into subplots with a size of 25m x 25m; of course, the size of the subplots is not limited to this;

[0067] S14, point cloud denoising: removing outliers and branches remaining at the boundary of the subplots by artificial visual denoising;

[0068] S15, downsampling: dividing the xoy plane into grids with a size of 0.5m x 0.5m, and taking the highest point in each grid as the sampling point, and smoothing the sampling points;

[0069] S16, coordinate normalization: after the coordinate normalization operation of the subplots, the data preprocessing is completed.

[0070] Step S2 is based on PointTransformer network, PointTransformer is a deep network based on Transformer, and the introduction of self-attention mechanism makes it have strong feature extraction ability for forest point cloud with complex structure. It consists of three core modules: downsampling module, upsampling module and Transformer module. The PointTransformer network structure is as shown in Figure 3 .

[0071] The function of the downsampling module is to reduce the number of data points and extract deeper feature information, and the working steps include: first, using FPS algorithm to sample the input forest point cloud to obtain the sampled point cloud; then, taking each sampling point as the center point, using kNN graph to find the k nearest neighbors around the center point; finally, each center point and its nearest neighbors form a point set, which is input into the deep network for deep feature extraction;

[0072] The role of the upsampling module is to assign deep-level features to non-sampling points. The working steps include: first, the point cloud composed of center points is subjected to feature extraction through MLP, BN and ReLU; then, the feature of each center point is interpolated to its neighboring point set by using the inverse distance weight interpolation algorithm; finally, the features corresponding to the downsampling module are aggregated to the corresponding upsampling module through a jump connection;

[0073] The role of the Transformer module is to enhance the local semantic perception ability of each point to its surrounding points, which includes two MLPs and a Transformer layer, and the Transformer layer is as shown in formula (1):

[0074]

[0075] wherein, β is a relationship function; ρ is a normalization function; γ is an MLP with two linear layers and a nonlinear layer ReLU; φ, ψ and α are point-by-point feature transformation matrices; X(i) is a set composed of k neighbors of x i ; δ is a position encoding function, and the expression is δ=θ(p i -p j ), p i and p j are the coordinates of 3D points i and j respectively; the encoding function θ is an MLP with two linear layers and a nonlinear layer ReLU.

[0076] The extreme value offset loss function can measure the error between the offset value predicted by the offset network and the true offset value, thereby guiding the update of the offset network. By continuously adjusting the offset network parameters to minimize the loss function, the offset network can gradually improve the accuracy of the predicted offset result and improve the model performance.

[0077] The extreme value offset loss function of the extreme value offset deep learning algorithm is as shown in formula (2) and formula (3):

[0078]

[0079]

[0080] wherein, L extreme is the loss value of the loss function; N is the total number of points; is the true offset of the i-th point to the corresponding extreme value point; is the offset predicted by the network for the i-th point; ω i is the dynamic weight of the i-th point, and the points adjacent to the extreme value point of a single tree are less dependent on the extreme value offset, so they should contribute less to the loss value, and therefore have a smaller weight.

[0081] After the extreme value offset, the scattered points gather towards the corresponding tree top, and the density of the tree top increases significantly, while the density of the non-tree top decreases significantly. According to the density difference, the mean shift algorithm with adaptive dynamic bandwidth is used to cluster the offset point cloud, divide the offset point cloud set, and complete the marking.

[0082] The principle of the mean shift algorithm is to obtain a local optimal solution by using the probability density, and a mean shift process diagram is shown in Figure 4 . Specifically, first, the mean shift vector of the current center point is calculated; second, the target point is moved to the end of the mean shift vector; then, it is taken as a new starting point to continue moving until the length of the mean shift vector is less than the allowed error; finally, when all points are marked, the iteration ends.

[0083] The calculation of the mean shift vector is as formula (4):

[0084]

[0085] Where p t is the center point; p i is the ith point within the bandwidth range; n is the number of points within the bandwidth range; g(·) is the derivative of the RBF kernel function; h is the dynamic bandwidth.

[0086] The calculation strategy of the dynamic bandwidth h is as formula (5), and the calculation process includes: first, the KDtree is constructed by the offset point cloud, and the construction of the KDTree is to arrange the disordered point cloud in order according to a certain order, which facilitates fast and efficient retrieval; second, the average distance of each point to its k nearest neighbors is calculated according to the KDTree; finally, the average of these average distances is obtained as the average nearest neighbor distance as the dynamic bandwidth h.

[0087]

[0088] Wherein, represents the offset point cloud; n is the total number of points; γ j (·) is used to calculate the average distance of each point to its K nearest neighbors. Λ(·) is used to obtain the average nearest neighbor distance as the dynamic bandwidth h.

[0089] After the mean shift clustering on the offset point cloud is completed, all offset points are marked by clustering. Since the order of the points does not change before and after the offset, the class vector after clustering and marking in the offset point cloud is column-pasted with the original point cloud space, so that the offset point cloud after clustering and marking can be mapped back to the original point cloud space, and the preliminary segmentation of the single tree is completed.

[0090] Because the point cloud offset is somewhat random due to the depth migration network, the mean shift algorithm may cluster points belonging to different trees, causing oversegmentation error. To reduce oversegmentation error, the ITS point set aggregation algorithm is designed.

[0091] When merging two clusters, the smaller cluster should be merged into the larger cluster. This is because during extreme value migration, most points successfully migrate to the vicinity of the treetop, while a small portion undergoes invalid migration. Therefore, the larger the cluster, the closer it is to the correct segmentation result. Thus, relatively larger clusters are considered part of the correctly segmented tree, while relatively smaller clusters are considered fragments resulting from invalid migrations.

[0092] In step S4, the gradient change criterion in the ITS point set aggregation algorithm is:

[0093] If the horizontal distance from the vertex of a fragment to its nearest neighbor in another cluster is greater than a preset bandwidth, then the gradient changes between these two clusters are considered drastic, they belong to different individual trees, and should not be merged. Figure 5 Figure (a) in the middle;

[0094] If the horizontal distance from the vertex of a fragment to its nearest neighbor in another cluster is less than a preset bandwidth, then the gradient change between these two clusters is considered gentle, and they belong to the same single tree. Therefore, these two clusters are merged. Figure 5 Figure (b) in the middle;

[0095] The ITS point clustering algorithm is shown in equations (6) to (8);

[0096]

[0097]

[0098]

[0099] in, This represents the set of points after aggregation; m represents the total number of points in the current set; Θ(·) determines whether to merge two sets; dis xy (P i ,P j Calculate the horizontal distance between two point sets; It is set P i The highest point in the middle; It is set P j To set P i The nearest neighbor; In set P i The total number of midpoints; w represents the grid width; k represents the distance threshold; xy is a schematic representation, representing the xoy plane in the coordinate system.

[0100] After the above processing, the result set may contain a small number of noisy point sets. These noisy point sets will be directly merged into the set of nearest neighbor points.

[0101] The following section uses specific experimental data and evaluation metrics to verify the single-tree segmentation method for airborne lidar based on extreme value offset deep learning in the above embodiments.

[0102] To examine the generalization effect of this method on different forest stand structures, two coniferous forest plots in the Blue Ridge Forest region of Washington, USA, and two mixed forest plots in Bretten, Germany, were selected for the experiment. Experimental area 1 was a coniferous forest in the Blue Ridge Forest region of the Capitol National Forest near Olympia, western Washington, USA. The tree species in this area are predominantly coniferous, with geographical coordinates of 122°59′30.84”~123°19′30” and 46°49′55.2”~47°5′11.4”. The main terrain is hilly, with elevations ranging from 260 to 425 m, ground slopes from 0 to 45 degrees, and a point density of 4.86 points / m². Experimental area 2 was a mixed forest in Bretten, Germany. ALS data were acquired by MILANGeoservice GmbH on July 5, 2019, using a RIEGL VQ-780i instrument installed on a Cessna C207 aircraft. The nominal specifications of the ALS instrument are as follows. The accuracy is 20mm within a 250m scanning range, the laser beam divergence is less than 0.25mrad, and the point density is 115.2 points / m2.

[0103] After data preprocessing, to better train and evaluate the network, 211 25m×25m sample plots were randomly divided into training and validation sets in an 8:2 ratio, resulting in 169 sample plots as the training set (containing 1558 trees) and 42 sub-sample plots as the validation set (containing 382 trees). During training, the Point Transformer deep network had 50 epochs and a training time of approximately 2 hours. The initial learning rate was set to 0.001, using the RMSprop optimizer with a smoothing constant alpha of 0.9. The StepLR method with dynamically adjusted learning rates was employed, where the learning rate adjustment factor gamma was set to 0.1.

[0104] The test set consists of four distinct plots, labeled U1, U2, G1, and G2. U1 and U2 are located in the Blue Ridge area of ​​the Capitol National Forest in western Washington, while G1 and G2 are located in a mixed forest near Bruten, Germany. The four plots in the test set are shown below. Figure 6 As shown.

[0105] Refer Canopy Density (RCD) is used to represent the canopy density of different plots. The number of grids containing points is divided by the total number of grids to obtain RCD, as shown in equation (9):

[0106]

[0107] where n is the total number of grids, o i is a flag, if there is a point in grid i, then o i = 1, otherwise o i = 0.

[0108] In U1, U2, G1, G2, the number of single trees is 127, 186, 123, 261 respectively; RCD is 61.58%, 69.09%, 93.25%, 96.40% respectively. Due to the difference in vegetation distribution and different collection equipment in the two regions, the point cloud density difference is large, so the two region plots are selected for method testing, which is representative. The detailed description of the data is shown in Table 1.

[0109] Table 1. Detailed description of the test set.

[0110]

[0111] In order to evaluate the performance of the proposed method, the matching strategy proposed by Kai guang Zhao et al. is used to pair the reference trees (true value) with the estimated trees (predicted value):

[0112]

[0113] where (x i ,y i ,z i ) and (x j ,y j ,z j ) represent the tree top coordinates of the reference tree and the estimated tree respectively. When the two three-dimensional space points representing the tree position information are closest, the estimated tree i is successfully matched with the reference tree j. K is the weight of the height difference (default is 0.5).

[0114] The accuracy (p), recall (r), and F-score (F) are used to evaluate the performance of the airborne LiDAR single tree segmentation method based on extreme offset deep learning:

[0115]

[0116]

[0117]

[0118] where MT represents the one-to-one matching relationship between the reference tree and the estimated tree, i.e. correct segmentation; OE represents the non-matching relationship between some reference trees and estimated trees, i.e. a tree is considered to belong to the branch of other trees and thus is not detected, resulting in under-segmentation; CE represents the one-to-many relationship between the reference tree and the estimated tree, i.e. a tree is segmented into multiple trees, resulting in over-segmentation.

[0119] The segmentation process and results of part of the subgraphs in the test set are shown in Figure 7 different gray scales represent different clusters, wherein (a) represents the forest point cloud after data preprocessing; (b) represents the offset point cloud obtained after extreme value offset; (c) represents the labeled offset point cloud obtained after clustering by the adaptive mean shift algorithm; (d) represents the result after mapping back to the smooth forest point cloud; (e) represents the result after point set clustering; (f) represents the final segmentation result after post-processing.

[0120] The airborne LiDAR single tree segmentation method based on extreme value offset deep learning in the above embodiment is used to segment four sample plots in the test set, and the three-dimensional segmentation results are shown in Figure 8 , wherein (a), (b) represent the sample plots in the Blue Ridge area of the United States, and (c), (d) represent the sample plots in the Brueggenwald area of Germany.

[0121] The ITS segmentation results of the airborne LiDAR single tree segmentation method based on extreme value offset deep learning in the above embodiment in the four sample plots are shown in Table 2, wherein TN is the actual number of tree individuals in the graph; MT is the number of correctly segmented single trees; OE is the number of under-segmented single trees; CE is the number of over-segmented single trees; p, r, F are calculated according to equations (11)-(13).

[0122] Table 2. Accuracy of the proposed method evaluated in four sample plots

[0123]

[0124] 1 Actual number of single trees in the sample plot; 2 Number of correctly segmented single trees; 3 Number of under-segmented single trees; 4 Number of over-segmented single trees.

[0125] In order to verify the necessity of the three core steps in the above embodiment, the following ablation experiments are designed, as shown in Table 3.

[0126] Firstly, the superiority of extreme value offset compared to center offset is explored; secondly, the superiority of dynamic bandwidth strategy compared to fixed bandwidth strategy is explored; finally, the superiority of ITS point set aggregation method compared to HAIS point set aggregation method is explored.

[0127] Table 3. Ablation study for extreme value shift, dynamic bandwidth, and ITS point aggregation

[0128]

[0129] (1) Extreme value shift

[0130] To verify the effectiveness of the extreme value shift, the deep network was trained using the extreme value shift and the center shift, respectively. The loss curves obtained from the training set and the validation set are shown in (a) and (b) of FIG. 6, respectively. It can be seen that when the extreme value shift is used, the final convergence values of the training set and the validation set are the smallest, and the loss values are the smallest, which are 0.064 and 0.059, respectively. When the center shift is used, the final convergence loss values of the training set and the validation set are 0.11 and 0.088, respectively. This shows that the introduced extreme value shift strategy can effectively enhance the robustness of the deep network. Figure 9 In addition, Table 4 compares the influence of using the center shift or the extreme value shift on the segmentation accuracy of the present method. When the extreme value shift is used, the average F-score of the present method is 91%, which is much higher than the average F-score (54%) of the center shift. This shows the effectiveness and superiority of the extreme value shift.

[0131] Table 4. Extreme value shift vs. center shift

[0132]

[0133]

[0134]

[0135] 1 Actual number of individual trees in the sample plot; 2 Number of correctly segmented individuals; 3 Number of under-segmented individual trees; 4 Number of over-segmented individual trees.

[0136] (2) Dynamic bandwidth

[0137] The segmentation results of the airborne laser radar individual tree segmentation method based on the extreme value shift deep learning in the above embodiment using the mean shift algorithm with dynamic bandwidth and the HAIS clustering algorithm with fixed bandwidth are shown in Table 5. After multiple experiments, when the HAIS clustering parameter with fixed bandwidth is set to 0.05, the segmentation result is the best, and the average F-scores of the four sample plots are 0.78, 0.75, 0.78, and 0.70, respectively. When the mean shift algorithm with dynamic bandwidth is used, the average F-scores of the algorithm in the four plots are 0.85, 0.90, 0.84, and 0.84, respectively, which are obviously higher than the HAIS clustering with fixed bandwidth, as shown in Table 5. The results show that the mean shift algorithm with dynamic bandwidth is effective and superior.​

[0138] Table 5. Fixed bandwidth vs dynamic bandwidth

[0139]

[0140] 1 Actual number of individual trees in the plot; 2 Number of correctly segmented individuals; 3 Number of under-segmented individuals; 4 Number of over-segmented individuals.

[0141] (3) ITS point set aggregation

[0142] Table 6 shows that the segmentation accuracy of the proposed method is effectively improved after introducing ITS point set aggregation. In the coniferous forest plots (U1, U2) and mixed forest plots (G1, G2), the average F-score is increased by 13% and 7% respectively compared with the HAIS point set aggregation method.

[0143] Due to the randomness of deep networks, there will be errors when the points are shifted, which will inevitably lead to over-segmentation errors in the mean shift algorithm. The HAIS point set clustering algorithm simply merges clusters with close distances, which can lead to the merging of adjacent clusters of different trees. The ITS point set clustering algorithm considers the gradient features between individuals, which can effectively suppress the merging of adjacent clusters of different trees, thereby reducing over-segmentation errors and improving segmentation accuracy.

[0144] Figure 6 .HAIS point set aggregation vs ITS point set aggregation

[0145]

[0146] 1 Actual number of individual trees in the plot; 2 Number of correctly segmented individuals; 3 Number of under-segmented individuals; 4 Number of over-segmented individuals.

[0147] The following introduces a comparative experiment.

[0148] Several methods are used to test in the four test plots, and the test results are shown in Table 7.

[0149] The average p, r, F of the SHDR method in the four plots reached 0.84, 0.85, 0.84 respectively;

[0150] The average p, r, F of the DK method in the four plots reached 0.66, 0.59, 0.62 respectively;

[0151] The average p, r, F of the improved DK method in the four sample plots reached 0.83, 0.86, 0.85, respectively.

[0152] The average p, r, F of the method in the above embodiment in the four sample plots reached 0.91, 0.88, 0.90, respectively.

[0153] The spatial horizontal distance rule method (SHDR) belongs to the top-down method. This method has high segmentation accuracy, but has large time complexity and is very sensitive to the selection of the horizontal distance threshold parameter, resulting in poor universality.

[0154] The DBSCAN fusion K-means method (DK) belongs to the bottom-up method. The advantage of this method is that it can better segment trees with obvious trunk features. However, due to the fact that the airborne radar scans very sparse trunk points in high tree canopy density forests, the center points found by the DBSCAN algorithm are not accurate, which in turn affects the final segmentation result.

[0155] The improved DBSCAN and K-means method (Improved DK) is an improved strategy that fuses extreme value offset deep learning and the DK method. Experiments show that the introduction of extreme value offset significantly improves the segmentation accuracy of the DK method in high canopy density forests.

[0156] Table 7. Comparison results of single tree segmentation accuracy of SHDR method, DK method, Improved DK method and the method in the four sample plots

[0157]

[0158] 1 The actual number of single trees in the sample plot; 2 The number of correctly segmented single trees; 3 The number of under-segmented single trees; 4 The number of over-segmented single trees.

[0159] The overall accuracy, recall rate and F-score of the method in the four sample plots are higher than those of the comparative methods. In U1 with lower canopy density, the F-score of the method is only slightly higher than that of the ImprovedDK method. However, in actual forest resource survey, the difficulty is usually the forest with high canopy density. Although the F-score of the method in U1 is only slightly higher than that of the ImprovedDK method, the F-score of the method in other sample plots with high canopy density is much higher than that of other comparative algorithms, which shows the superiority of the method in the above embodiment. In addition, the average F-score (85%) of the ImprovedDK method is much higher than the average F-score (68%) of the DK method. This shows that in the ALS point cloud of the high canopy density forest, the extreme value offset can significantly improve the segmentation accuracy of the bottom-up method, and has good scalability.

[0160] To verify the universality and effectiveness of the design method, a coniferous forest sample plot in Blue Ridge, Washington, USA and a mixed forest sample plot near Bruun, Germany were selected as test objects. Among them, the point cloud density of the Blue Ridge coniferous forest is low, and the structure is simple; the point cloud density of the mixed forest near Bruun, Germany is high, and the structure is complex. Selecting the sample plots in these two regions for algorithm testing can effectively verify the universality and effectiveness of the algorithm. The experimental results show that after introducing the adaptive bandwidth strategy in step (3), the segmentation accuracy is better, and it can better adapt to complex scenes and changing environments; the introduction of the point set clustering method in step (5) can effectively reduce the over-segmentation error and improve the segmentation accuracy of the algorithm. In summary, the experimental results effectively verify the universality and effectiveness of the method, and through comparison with other algorithms, the superiority and application potential of the method in segmenting complex forest environments are shown.

[0161] Based on the above ideal embodiments according to the present application, through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the present application. The technical scope of the present application is not limited to the contents in the specification, and must be determined according to the scope of the claims.

Claims

1. An airborne laser radar single tree segmentation method based on extreme value offset deep learning, characterized by comprising: S1, preprocessing: sequentially performing point cloud filtering, height normalization, sub-sample division, point cloud denoising, downsampling and coordinate normalization on the forest point cloud, wherein the forest point cloud is obtained based on airborne laser radar; S2, extreme value offset: using an extreme value offset deep learning algorithm, performing spatial transformation on the preprocessed forest point cloud, offsetting each point to the corresponding tree top point, and obtaining an offset point cloud; S3, adaptive clustering: using an adaptive mean shift clustering algorithm based on average neighborhood distance to cluster the offset point cloud, dividing the offset point cloud set, and completing labeling; S4, spatial mapping: mapping the clustered and labeled offset point cloud back to the original point cloud space to complete the preliminary segmentation of single trees; S5, ITS point set aggregation: merging adjacent crown layers with gentle gradient changes to reduce over-segmentation errors; S6, post-processing: after upsampling and coordinate denormalization, the results obtained by point set aggregation are completed; Step S2 is based on a Point Transformer network, which has three core modules: a downsampling module, an upsampling module and a Transformer module; wherein The function of the downsampling module is to reduce the number of data points while extracting deeper feature information, and the working steps include: first, using the FPS algorithm to sample the input forest point cloud to obtain a sampled point cloud; then, taking each sampling point as a center point, using kNN graph to find k nearest neighbor points around the center point; finally, each center point and its neighbor points form a point set, which is input into a deep network for deep feature extraction; The function of the upsampling module is to distribute deep features to non-sampling points, and the working steps include: first, the point cloud composed of center points is subjected to feature extraction by MLP, BN and ReLU; then, the feature of each center point is interpolated to its neighbor point set by using the inverse distance weight interpolation algorithm; finally, the features corresponding to the downsampling module are aggregated to the corresponding upsampling module through a jump connection; The function of the Transformer module is to enhance the local semantic perception ability of each point to its surrounding points, which includes two MLPs and a Transformer layer, and the Transformer layer is as formula (1): The extreme value offset loss function of the extreme value offset deep learning algorithm is as formula (2) and formula (3): Where β is the relation function; ρ is the normalization function; γ is an MLP with two linear layers and one nonlinear ReLU layer; φ, ψ and α are pointwise feature transformation matrices; X(i) is x i The set of k neighbors; δ is the positional encoding function, expressed as δ = θ(p i -p j ), p i and p j The coordinates of 3D points i and j are respectively; the encoding function θ is an MLP with two linear layers and one nonlinear ReLU layer; In step S3, the adaptive mean shift clustering algorithm uses an adaptive dynamic bandwidth, and the mean shift process includes: first, calculating the mean shift vector of the current center point; second, moving the target point to the end of the mean shift vector; then, taking it as a new starting point, continue to move until the length of the mean shift vector is less than the allowed error; finally, when all points are labeled, the iteration is ended; wherein L extreme is the loss value of the loss function; N is the total number of points; is the true offset of the i-th point to the corresponding extreme point; is the offset predicted by the network for the i-th point; ω i is the dynamic weight of the i-th point; The calculation of the mean shift vector is as formula (4): ​ where p t is the center point; p i is the ith point in the bandwidth range; n is the number of points in the bandwidth range; g(·) is the derivative of the RBF kernel function; and h is the dynamic bandwidth. The calculation strategy of dynamic bandwidth h is as shown in formula (5), and the calculation process comprises the following steps: firstly, constructing a KDtree through offset point cloud; secondly, calculating the average distance of each point to its k nearest neighbors according to the KDTree; and finally, averaging the average distances to obtain the average nearest neighbor distance as the dynamic bandwidth h; where, represents the offset point cloud; n is the total number of points; γ j (·) is used to compute the average distance of each point to its K nearest neighbors. Λ(·) is used to obtain the average nearest neighbor distance as the dynamic bandwidth h.

2. The airborne laser radar single tree segmentation method based on extreme value offset deep learning according to claim 1, wherein, In step S4, the gradient change criterion in the ITS point set aggregation algorithm is: If the horizontal distance between the vertex of the fragment and the nearest neighbor point of the other cluster is greater than the preset bandwidth, it is considered that the gradient change between the two clusters is sharp, and they respectively belong to different single trees and should not be merged; If the horizontal distance between the vertex of the fragment and the nearest neighbor point of the other cluster is less than the preset bandwidth, it is considered that the gradient change between the two clusters is gentle, and they belong to the same single tree, and then the two clusters are merged.

3. The airborne laser radar single tree segmentation method based on extreme value offset deep learning according to claim 2, wherein, The ITS point set clustering algorithm is as shown in formulas (6)-(8). where, represents the aggregated point set; m represents the total number of current sets; Θ(·) determines whether to merge two sets; dis xy (P i , P j ) calculates the horizontal distance between two point sets; is the highest point in the set P i ; is the nearest neighbor point of the set P j to the set P i ; represents the total number of points in the set P i ; w represents the grid width; k represents the distance threshold.

Images