High-canopy-density forest individual tree segmentation method and device, medium and equipment

Abstract

The invention discloses a high canopy density forest single tree segmentation method, device, medium and equipment, based on a gradient directivity map and a multi-scale seed point extraction method, firstly, a digital earth surface model is used for counting directivity of a neighborhood point cloud local gradient vector, the gradient directivity map is generated to replace CHM for crown segmentation, and the method is not affected by terrain; then, preferably using a multi-scale window to extract real tree vertexes from the gradient pointing map, and iteratively screening the real tree vertexes and deleting wrong tree vertexes according to geometric features among tree crowns, thereby effectively inhibiting over-segmentation and under-segmentation phenomena, and remarkably improving the individual tree segmentation precision; and finally, a segmentation method combining water flow path segmentation and post-processing is adopted, so that wrong segmentation results possibly generated by the watershed algorithm on the overlapped crowns in the same height interval are reduced.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing information extraction, and in particular, to a method, device, medium and equipment for single tree segmentation in high-closure forests. Background Art

[0002] Forests play an important role in ecosystem services, providing key services such as carbon cycling, water cycling, energy exchange, and habitats for many species. When using airborne lidar data for detailed forestry surveys, single tree segmentation is a crucial step. The currently commonly used single tree segmentation method is the watershed method, which generates a canopy height model (CHM) from the normalized point cloud data and then performs tree vertex detection and single tree segmentation. The watershed algorithm requires the use of normalized point cloud data, and point cloud normalization depends on a high-precision digital terrain model (DTM). In high-closure forests, airborne lidar is difficult to penetrate the canopy to reach the ground, so it is difficult to accurately depict the terrain and generate a continuous and accurate DTM. At the same time, the traditional watershed algorithm uses the local maximum method to detect tree vertices. In broad-leaved forests, due to the presence of multiple protruding branches in the tree crown, this method is prone to over-segmentation, and in areas where the tree crowns are relatively dense, the boundaries between adjacent tree vertices are blurred, and under-segmentation is likely to occur. Finally, when the classical watershed algorithm performs water expansion at the seed points, the expansion direction occurs in a random order at the same height interval, and this randomness may lead to incorrect results.

[0003] In summary, the problems and deficiencies of the existing technology are as follows:

[0004] (1) In the point cloud of high-closure forests collected by airborne lidar, due to the lack of ground points, the point cloud cannot be accurately normalized.

[0005] (2) The existing local maximum tree vertex detection algorithm is prone to over-segmentation of broad-leaved forests and under-segmentation in areas with dense tree crowns.

[0006] (3) The classical watershed algorithm is prone to incorrect results in the overlapping area of tree crowns at the same height interval. Summary of the Invention

[0007] The present invention provides a method, device, medium and equipment for single tree segmentation in high-closure forests, aiming to at least solve one of the technical problems in the related technologies to a certain extent.

[0008] To this end, the first aspect of the embodiments of the present invention provides a method for single tree segmentation in high-closure forests, including the following steps:

[0009] Step 1, collect the forest point cloud data of the target forest;

[0010] Step 2: After preprocessing the forest point cloud data, classify ground points and non-ground points, and generate a digital surface model of the target forest based on the classified complete point cloud data;

[0011] Step 3: Obtain the neighboring pixels corresponding to each pixel, calculate the local gradient vector of the neighboring pixels and the degree of pointing concentration of the corresponding pixels based on the digital surface model, and generate a gradient pointing map of the target forest;

[0012] Step 4: Extract tree vertices using a preset tree vertex detection algorithm;

[0013] Step 5: Invert the pixel values of the gradient pointing map, simulate the flow of water on each pixel, and classify the watersheds that finally converge on the same pixel into one category to generate a water flow path segmentation result;

[0014] Step 6: Execute the watershed algorithm based on the gradient pointing map and the extracted tree vertices, and merge and cover the water flow path segmentation result with the execution result of the watershed algorithm to generate a single-tree segmentation result of the target forest.

[0015] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which when executed by a processor, implements the above-mentioned single-tree segmentation method for high-closure forests.

[0016] To achieve the above object, a third aspect of the embodiments of the present invention provides a single-tree segmentation device for high-closure forests, including a computer-readable storage medium and a processor, and when the processor executes the computer program on the computer-readable storage medium, the steps of the above-mentioned single-tree segmentation method for high-closure forests are implemented.

[0017] A fourth aspect of the embodiments of the present invention provides a single-tree segmentation device for high-closure forests, including a collection module, a first construction module, a second construction module, an extraction module, a segmentation module, and a merging module,

[0018] The collection module is used to collect forest point cloud data of the target forest;

[0019] The first construction module is used to classify ground points and non-ground points after preprocessing the forest point cloud data, and generate a digital surface model of the target forest based on the classified complete point cloud data;

[0020] The second construction module is used to obtain the neighboring pixels corresponding to each pixel, calculate the local gradient vector of the neighboring pixels and the degree of pointing concentration of the corresponding pixels based on the digital surface model, and generate a gradient pointing map of the target forest;

[0021] The extraction module is used to extract tree vertices by using a preset tree vertex detection algorithm;

[0022] The segmentation module is used to invert the pixel values of the gradient direction map, simulate the flow of water on each pixel, and classify the basins that finally converge on the same pixel into one category, generating a water flow path segmentation result;

[0023] The merging module is used to perform a watershed algorithm based on the gradient direction map and the extracted tree vertices, and merge and cover the water flow path segmentation result with the execution result of the watershed algorithm, generating a single-tree segmentation result of the target forest.

[0024] The beneficial effects of the present invention are as follows: The present invention provides a single-tree segmentation method, device, medium and equipment for high-closure forests. Based on the gradient direction map and the multi-scale seed point extraction method, first, the digital surface model (DSM) is used to statistically analyze the directivity of the local gradient vectors of the neighborhood point cloud and generate a gradient direction map (Gradient Convergence Map, GCM) to replace the CHM for canopy segmentation. This method is not affected by terrain. Then, preferably, windows of multiple scales are used to extract tree vertices from the gradient direction map, and the true tree vertices are iteratively screened according to the geometric features between the canopies, and the false tree vertices are deleted. This method effectively suppresses the over-segmentation and under-segmentation phenomena and significantly improves the single-tree segmentation accuracy. Finally, a segmentation method of water flow path segmentation and post-processing merging is adopted to reduce the possible incorrect segmentation results of the watershed algorithm for overlapping canopies in the same height interval.

[0025] To make the above objects, features and advantages of the invention more obvious and understandable, the following specifically gives preferred embodiments of the present invention and, in conjunction with the accompanying drawings, makes the following detailed description. Description of the Drawings

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0027] Figure 1 is a schematic flowchart of a single-tree segmentation method for high-closure forests provided by an embodiment;

[0028] Figure 2 is a gradient direction map calculated from forest point cloud data in an embodiment;

[0029] Figure 3 is a schematic diagram of tree vertices extracted by a multi-window iteration method in an embodiment;

[0030] Figure 4 It is a schematic diagram of the result of a segmentation algorithm based on the water flow path in an embodiment;

[0031] Figure 5 It is a comparison diagram of the effects based on the method of the present invention and the marker-controlled watershed method in an embodiment;

[0032] Figure 6 It is a schematic structural diagram of a high-closure forest single-tree segmentation device provided by an embodiment;

[0033] Figure 7 It is a schematic structural diagram of a high-closure forest single-tree segmentation device provided by an embodiment. Detailed implementation manners

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0035] It should be noted that if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, and all are within the protection scope of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the flowchart. Furthermore, the terms "first", "second", "third", etc. adopted by the present invention do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0036] Figure 1 It is a schematic flowchart of a high-closure forest single-tree segmentation method provided by Embodiment 1. As Figure 1 shown, the method includes but is not limited to the following steps:

[0037] Step 1, collecting forest point cloud data of the target forest;

[0038] Step 2, classifying ground points and non-ground points after preprocessing the forest point cloud data, and generating a digital surface model of the target forest based on the complete point cloud data after classification;

[0039] Step 3, obtaining the neighborhood pixels corresponding to each pixel, calculating the local gradient vector of the neighborhood pixels and the degree of pointing concentration of the corresponding pixels based on the digital surface model, and generating a gradient pointing map of the target forest;

[0040] Step 4: Extract tree vertices using a preset tree vertex detection algorithm.

[0041] Step 5: Reverse the pixel values of the gradient direction map, simulate the flow of water on each pixel, and classify the basins that finally converge on the same pixel into one category to generate a water flow path segmentation result.

[0042] Step 6: Execute the watershed algorithm based on the gradient direction map and the extracted tree vertices, and merge and cover the water flow path segmentation result with the execution result of the watershed algorithm to generate a single-tree segmentation result of the target forest.

[0043] The above embodiments provide a method for single-tree segmentation of a high-closure forest. The method uses a digital surface model (DSM) to statistically analyze the directivity of local gradient vectors of neighborhood point clouds and generate a gradient convergence map (GCM) to replace the CHM for crown segmentation. This method is not affected by terrain. Then, a segmentation method combining water flow path segmentation and post-processing merging is adopted to reduce the mis-segmentation results that the watershed algorithm may generate for overlapping crowns in the same height interval.

[0044] The above steps are described in detail below through specific embodiments.

[0045] Exemplarily, in one embodiment, the drone can be controlled to fly in a terrain-following mode in an area with large terrain undulations, and high-density point cloud data of the target forest can be obtained using an airborne lidar to fully depict the three-dimensional structure information of the forest canopy. Then, preprocess the point cloud data, such as denoising and point cloud classification, to separate ground points and non-ground points, and establish a digital terrain model (DTM) and a digital surface model (DSM) of the target forest. The digital terrain model DTM represents the bare ground without any objects (such as plants and buildings), and the digital surface model DSM represents the Earth's surface and includes all objects on it.

[0046] In a preferred embodiment, step 2 specifically includes but is not limited to the following steps:

[0047] S201: Remove the noise points in the forest point cloud data using a preset statistical filtering method (SOR) to remove the reflection signals of birds or other flying objects mis-collected by the lidar system.

[0048] S202: Separate the noise-removed forest point cloud data into ground points and non-ground points based on a preset cloth simulation algorithm (CSF). The non-ground points include vegetation points, building points, etc.

[0049] S203. Generate the target resolution of the digital surface model according to the target point density, divide the ground points and the non-ground points based on the target resolution, and construct the digital surface model DSM of the target forest based on the complete point cloud data. The target point density is at least 4 lidar points within each pixel range, so as to ensure the trade-off between empty pixels and retaining sufficient details to describe the characteristics of the forest canopy.

[0050] In the above embodiments, irregular triangular networks (TINs) can be constructed using ground points to represent the local characteristics of the terrain. For sparse ground points, interpolation methods can be used to generate a continuous terrain surface and convert it into a regular grid (raster) format to generate the final DTM model. For the DSM model, the preprocessed point cloud data can be divided into regular grids of a certain resolution, and an elevation value for reflecting the characteristics of the terrain and the top of the ground objects can be assigned to each grid cell. At the same time, for irregularly distributed point cloud data, interpolation methods such as nearest neighbor interpolation, linear interpolation, or inverse distance weighting method can be used to generate a continuous DSM model. The specific method will not be elaborated here.

[0051] Exemplarily, in a preferred embodiment, the step 3 determines the pixels (ground pixels) in the DTM model and the DSM model of the bare land area that are lower than the height threshold, so as to exclude the influence of the bare land when calculating the gradient vector. Then, calculate the gradient vector of the window size of each pixel, divide the gradient vector of the pixel into two perpendicular direction components, and construct convolution kernels to calculate respectively. Specifically, it includes but is not limited to the following steps:

[0052] S301. Construct a convolution kernel, and use the convolution kernel to perform convolution processing on the data of the digital surface model of the target forest in two perpendicular directions to generate the local gradient vector of each pixel and its neighborhood. The convolution kernel and the local gradient vector calculation formulas are as follows:

[0053] K x =[1 0 -1] (1)

[0054]

[0055] G x =DSM×K x (3)

[0056] G y =DSM×K y (4)

[0057] G=(G x ,G y ) (5)

[0058] Where K x represents the convolution kernel for calculating the x-direction component of the gradient vector, Gx Denotes the component of the gradient vector in the x direction, K y Denotes the convolution kernel for calculating the component of the gradient vector in the y direction, G y Denotes the component of the gradient vector in the y direction, and G is the local gradient vector of the pixel.

[0059] S302. Construct a direction vector from the central pixel to the neighboring pixel, and calculate the angular difference between the gradient vector of the neighboring pixel and the direction vector from the central pixel to the neighboring pixel.

[0060] Specifically, to construct a direction vector from the central pixel to the neighboring pixel, it is also divided into components in two mutually perpendicular directions. The gradient vector components from the central pixel to the neighboring pixel are respectively:

[0061]

[0062] D = (D x , D y ) (8)

[0063] where (x a , y a ) is the coordinate of the central pixel, D x is the component of the direction vector from the central pixel to the neighboring pixel in the x direction, (x b , y b ) is the coordinate of the neighboring pixel, D y is the component of the direction vector from the central pixel to the neighboring pixel in the y direction, and D is the direction vector from the central pixel to the neighboring pixel.

[0064] Therefore, the angular difference between the gradient vector of the neighboring pixel and the direction vector from the central pixel to the neighboring pixel is:

[0065]

[0066] where and respectively represent the magnitudes of the direction vector and the local gradient vector.

[0067] Then execute S303. Set weights for neighboring pixels at different distances based on the two-dimensional Gaussian kernel function, calculate the pixel value of each pixel, and generate the gradient pointing map of the target forest according to the pixel value.

[0068] In a specific embodiment, due to the differences in the size of the tree crowns, the pixels closer to the highest point of the tree crown are more likely to be within the crown width range. Therefore, in the preferred embodiment of the present invention, the two-dimensional Gaussian kernel function is selected as the weight, so that the pixels closer to the center point have higher weights, and the pixels farther away from the center point have lower weights. The weight calculation formula is:

[0069]

[0070] Among them, σ is the scale parameter for controlling weight decay. The larger the value of σ, the slower the weight decays with distance.

[0071] The central pixel value can be expressed as:

[0072]

[0073] Among them, θ i,j is the angular difference of the neighboring pixel (i, j), and w i,j is the corresponding distance weight.

[0074] The above algorithm involves three adjustable parameters. The first is the window size for calculating the local gradient vector. The larger this window value, the larger the range represented by the calculated gradient vector. It should be set by comprehensively considering the resolution of the DSM and the size of the forest canopy to capture the detailed features of the tree canopy while avoiding the influence of minor undulations. The second is the neighborhood window size. The larger this value, the larger the neighborhood range when calculating the weighted angular difference. This value should be set to the average tree canopy diameter according to the estimated value. The third is the weight decay parameter σ. The larger this parameter, the higher the weight of the pixels farther from the center point. It should be adjusted according to the actual canopy size of the region.

[0075] Figure 2 is the gradient orientation map calculated according to the collected forest point cloud data in an embodiment, as Figure 2 shown. The higher the pixel value, the more concentrated the local gradient vectors of the neighboring pixels point to this pixel. According to the geometric characteristics of the tree canopy, this value is the highest at the tree vertex and the lowest at the canopy boundary.

[0076] In the embodiments of the present invention, various methods can be used to extract tree vertices. For example, the local maximum method is used to detect tree vertices. Specifically, a square window with a fixed size is moved on each grid to identify whether the CHM value of the grid at the center of the window is the maximum within the window. If it is the maximum, it is considered that the grid may contain a tree vertex, and the center of the grid is extracted as the possible tree vertex position. However, this method is prone to over-segmentation for broad-leaved forests and under-segmentation in areas with dense canopies. Therefore, in the preferred embodiments of the present invention, a multi-scale iterative extraction method is adopted. At least two windows with different scale sizes are used to extract tree vertices from the gradient orientation map or the canopy height model CHM, and the remaining tree vertices are gradually iteratively searched starting from the real tree vertices according to the geometric characteristics between the tree canopies, and the wrong tree vertices are deleted, thereby effectively suppressing the over-segmentation and under-segmentation phenomena and significantly improving the single-tree segmentation accuracy.

[0077] Exemplarily, in a preferred embodiment, step 4 includes but is not limited to the following steps:

[0078] S401. Obtain the gradient direction map of the target forest;

[0079] S402. Use at least two windows of different scales to separately extract the local maxima of the gradient direction map, generating multiple local maxima sets;

[0080] S403. Compare the multiple local maxima sets, determine the repeated points as real tree vertices and the remaining points as candidate points, and establish a real tree vertex set and a candidate point set;

[0081] S404. Identify the geometric features between the tree crowns of the target forest according to the gradient direction map or the digital terrain model of the target forest, iteratively screen out real tree vertices from the candidate point set based on the geometric features between the tree crowns, and update the real tree vertex set and the candidate point set according to the screening results until a preset iteration end condition is reached.

[0082] In one embodiment, the step S404 is specifically as follows:

[0083] S4041. Use all the points in the real tree vertex set and the candidate point set to construct a Delaunay triangulation;

[0084] S4042. Traverse each edge of each triangle, obtain the target candidate points connected by real tree vertices, and determine whether the corresponding edge meets the preset conditions. If so, add the target candidate points to the real tree vertex set; otherwise, remove the target candidate points;

[0085] S4043. Repeat steps S4041 - S4042 until a preset iteration end condition is reached. The preset iteration end condition includes that all the points in the candidate point set have been processed.

[0086] Exemplarily, the step S4042 includes a first preset condition, a second preset condition, and a third preset condition. If the corresponding edge meets any one of the first preset condition, the second preset condition, and the third preset condition, add the target candidate points to the real tree vertex set.

[0087] Specifically, in one embodiment, the first preset condition is as follows: based on the gradient direction map, obtain the ratio of the minimum value to the endpoint value of the connection line, and this ratio is less than a preset ratio. This condition uses the gradient direction map to check the ratio of the minimum value to the endpoint value of the connection line. If the ratio of the minimum value to the endpoint value is less than the preset ratio, then this connection is considered to meet the condition. This condition takes into account that the directionalities of the neighborhood gradient vectors at two canopy boundary points are opposite, so there is a local low value. At the same time, considering the sizes of the two canopies, the endpoint value of the small canopy is smaller, while the endpoint value of the large canopy is larger. Meeting this condition means that the connection line may pass through a boundary (pixel), indicating that this path is a valid connection of real tree vertices.

[0088] In another embodiment, the second preset condition is as follows: based on the digital surface model of the target forest, obtain the angle difference between the vector from the lowest point on the connection line to the two endpoints of the line segment, and this angle difference is less than a preset angle threshold. This condition uses the digital surface model DSM to check whether the angle difference between the vector from the lowest point on the connection line to the two endpoints of the line segment is less than the preset angle threshold. At the same time, it takes into account the distance between the two tree vertices and the possible low points at the boundary. If the angle difference of the connection line is less than this threshold, it indicates that this connection conforms to the geometric relationship between real tree vertices.

[0089] In another embodiment, the third preset condition is as follows: obtain the length of the connection line, and this length of the connection line is greater than a first preset length threshold. This condition checks whether the length of the connection line is greater than the first preset length threshold. This first preset length threshold is generally set as the maximum crown diameter. If the length of the connection line exceeds this threshold, it indicates that the two endpoints of this connection are real tree vertices.

[0090] In another preferred embodiment provided by the present invention, that is, when constructing the Delaunay triangulation in step S401, obtain and remove the edges whose lengths exceed the second preset length threshold, so as to avoid any edge of the Delaunay triangulation result from crossing multiple canopies, and further improve the accuracy of tree vertex extraction.

[0091] Exemplarily, two windows can be used to respectively extract the local maximum values of the gradient direction map or the canopy height model. The window values of these two windows are respectively the minimum canopy diameter and the maximum canopy diameter in the region, so as to take into account both the extraction efficiency and the extraction accuracy of real tree vertices. The final iterative extraction result is as Figure 2 shown.

[0092] In the traditional watershed algorithm, the expansion of water is isotropic. In each tree crown, the water expansion occurs at the same height interval in a random order. This randomness may lead to incorrect results, especially when adjacent tree crowns intersect in the same height range. Exemplarily, a preferred embodiment of the present invention proposes a water flow simulation algorithm based on a gradient direction map and seed points for tree crown segmentation. First, all prominent tree clusters are segmented, and then the tree clusters are merged into the correct single-tree range according to geometric features, avoiding the errors caused by the randomness of water expansion in the watershed. Specifically, it includes but is not limited to the following steps:

[0093] S501, perform a difference operation between the pixel value of each pixel in the gradient direction map and the maximum value in the image to perform pixel value inversion. Specifically, after reading the gradient direction map, to simulate the water flow from high ground to low ground, the pixel values in the image are inverted, that is, by performing a difference operation between the value of each pixel and the maximum value in the image, a new image is generated, such that the high pixel value area corresponds to a lower value.

[0094] Then perform S502, calculate the flow direction of each pixel through a preset eight-neighborhood method, and generate the final convergence point of each pixel. Specifically, when water flow is poured into a pixel, its flow direction is the direction in which the value of the neighborhood pixel drops fastest. By traversing the adjacent pixels of the current pixel, the neighborhood pixel with the minimum value is selected as the next target of the water flow until a local minimum is reached. The flow direction of each pixel is recorded, and the final belonging area of the water flow is determined through path tracing. The present invention assigns a unique label to each local minimum and marks all the pixels on the path as the same area, and then performs S503. Finally, all the pixels with the same final convergence point are divided into a watershed, generating a water flow path segmentation result, that is, all pixels are assigned to their respective watershed areas.

[0095] Finally, execute the watershed algorithm based on the extracted seed points, and merge the initial shape corresponding to the water flow path segmentation result with the shape coverage area obtained by the watershed algorithm to obtain the final segmentation result. Figure 4 is a schematic diagram of the result of the segmentation algorithm based on the above water flow path in an embodiment, where the light gray is the water flow path segmentation result, and the dark gray is the result after merging, as Figure 4 shown. This method obtains all complete convex shapes through water flow path segmentation, and then combines the watershed segmentation result to merge the water flow path segmentation result into a complete tree crown, reducing the segmentation error caused by the randomness of water expansion in the watershed algorithm, and can avoid the segmentation randomness when adjacent tree crowns are in the same height range, improving the segmentation accuracy.

[0096] The algorithm implementation of the present invention was carried out in a highly closed forest in Huzhaoshan Forest Farm, Jingshan City, Jingmen City, Hubei Province. The canopy density of this area is about 0.95, the terrain slope is about 23°, and the main tree species is Liriodendron chinense. To comprehensively evaluate the performance of the single-tree segmentation method proposed by the present invention, the Intersection over Union (IoU) index was used to automatically match the segmentation results in this practice. Specifically, the algorithm segmentation results were matched with the manually labeled crown shapes, and an IoU greater than 50% was set as the threshold for correct matching, that is, when the overlapping area between the segmentation result and the manual annotation exceeded 50%, the tree segmentation was considered correct.

[0097] On this basis, multiple evaluation indicators were further calculated to comprehensively measure the performance of the segmentation algorithm, specifically including:

[0098] Precision (P): Focus on the accuracy of predicting positive samples;

[0099] Recall (R): Focus on the proportion of true positive samples that are correctly identified;

[0100] F1-Score (F1): Balance precision and recall and provide an indicator that comprehensively considers both;

[0101] Accuracy Index (AI): Evaluate the closeness of the algorithm segmentation results to the actual tree situation as a whole, and consider the impact of omitted and mis-segmented trees on the results, which can comprehensively reflect the accuracy of the algorithm in this task.

[0102] The calculation formulas for IoU and each indicator are as follows:

[0103]

[0104] In the formula: A and B respectively represent the areas of two regions, A∩B represents the intersection area of the two regions; A∪B represents the union area of the two regions; TP (True Positives) represents the correct tree vertices; FP (False Positives) represents the incorrect tree vertices; FN (False Negatives) represents the tree vertices that are not detected; N is the total number of single trees manually labeled.

[0105] The results show that the present invention uses non-normalized point cloud to calculate the gradient orientation map, successfully extracts the single-tree crown width range in high canopy density and high slope areas according to the gradient orientation map, and effectively reduces the over-segmentation phenomenon in broad-leaved forests, such as Figure 5 shown in a, 5b, 5c and the under-segmentation phenomenon when the canopy is dense, such as Figure 5d, as shown in 5e. At the same time, the results of quantitative evaluation show that the method of the present invention is superior to the existing marker-controlled watershed algorithm in all evaluation indexes, as shown in Table 1 below.

[0106] Table 1 Comparison results between the method of the present invention and the marker-controlled watershed algorithm

[0107]

[0108] The above embodiments provide a single-tree segmentation method applicable to more types and a wider range, based on airborne lidar data and without using normalized point clouds. It can be used not only in high-closure forests, but also has a wider application range, and improves the accuracy of the existing method, providing a basis for large-scale forestry surveys.

[0109] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0110] The embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned single-tree segmentation method for high-closure forests.

[0111] Figure 6 is a schematic structural diagram of a single-tree segmentation device for high-closure forests provided by an embodiment. As Figure 6 shown, it includes a collection module 100, a first construction module 200, a second construction module 300, an extraction module 400, a segmentation module 500, and a merging module 600.

[0112] The collection module 100 is used to collect forest point cloud data of the target forest.

[0113] The first construction module 200 is used to preprocess the forest point cloud data to separate ground points and generate a digital surface model of the target forest based on the ground points.

[0114] The second construction module 300 is used to obtain the neighborhood pixels corresponding to each pixel, calculate the local gradient vector of the neighborhood pixels and the degree of pointing concentration of the corresponding pixels based on the digital surface model, and generate a gradient pointing map of the target forest.

[0115] The extraction module 400 is used to extract tree vertices by using a preset tree vertex detection algorithm.

[0116] The segmentation module 500 is used to invert the pixel values of the gradient pointing map, simulate the flow of water on each pixel, and classify the watersheds that finally converge on the same pixel into one category to generate a water flow path segmentation result.

[0117] The merging module 600 is configured to perform a watershed algorithm based on the gradient direction map and the extracted tree vertices, and merge and overlay the water flow path segmentation result with the execution result of the watershed algorithm to generate the individual tree segmentation result of the target forest.

[0118] In a preferred embodiment, the second construction module 300 specifically includes:

[0119] A first construction unit, configured to construct a convolution kernel, and perform convolution processing on the data of the digital terrain model of the target forest in two perpendicular directions by using the convolution kernel to generate local gradient vectors of each pixel and its neighborhood;

[0120] A second construction unit, configured to construct a direction vector from a central pixel to a neighborhood pixel, and calculate the angular difference between the neighborhood pixel gradient vector and the direction vector from the central pixel to the neighborhood pixel;

[0121] A gradient direction map generation unit, configured to set weights for neighborhood pixels at different distances based on a two-dimensional Gaussian kernel function, calculate the pixel value of each pixel, and generate the gradient direction map of the target forest according to the pixel value.

[0122] In a preferred embodiment, the extraction module 400 specifically includes:

[0123] An acquisition unit, configured to acquire the gradient direction map of the target forest;

[0124] An extraction unit, configured to extract local maxima of the gradient direction map by using at least two windows of different scales respectively to generate a plurality of local maxima sets;

[0125] A first judgment unit, configured to compare the plurality of local maxima sets, determine the repeated points as real tree vertices and the remaining points as candidate points, and establish a real tree vertex set and a candidate point set;

[0126] An iterative screening unit, configured to identify the geometric features between the tree crowns of the target forest according to the gradient direction map or the digital terrain model of the target forest, iteratively screen out real tree vertices from the candidate point set based on the geometric features between the tree crowns, and update the real tree vertex set and the candidate point set according to the screening result until a preset iteration end condition is reached.

[0127] In a further preferred embodiment, the iterative screening unit includes:

[0128] A third construction unit, configured to construct a Delaunay triangulation by using all points in the real tree vertex set and the candidate point set;

[0129] A second judgment unit is configured to traverse each side of each triangle, obtain target candidate points connected by real tree vertices, and determine whether the corresponding side meets a preset condition. If so, add the target candidate points to the real tree vertex set; otherwise, remove the target candidate points until a preset iteration end condition is reached. The preset iteration end condition includes that all points in the candidate point set have been processed.

[0130] In a preferred embodiment, the segmentation module specifically includes:

[0131] A processing unit is configured to perform a difference operation between the pixel value of each pixel in the gradient direction map and the maximum value in the image to perform pixel value inversion.

[0132] A calculation unit is configured to calculate the flow direction of each pixel through a preset eight-neighborhood method and generate the final convergence point of each pixel.

[0133] A result generation unit is configured to divide all pixels with the same final convergence point into a single watershed and generate a water flow path segmentation result.

[0134] The above embodiments provide a single-tree segmentation device for highly closed forests, which uses a digital surface model (DSM) to statistically analyze the directivity of local gradient vectors of neighborhood point clouds and generate a gradient direction map (Gradient Convergence Map, GCM) to replace the CHM for canopy segmentation. This method is not affected by terrain. Then, a segmentation method combining water flow path segmentation and post-processing merging is adopted to reduce the incorrect segmentation results that the watershed algorithm may generate for overlapping canopies in the same height range.

[0135] An embodiment of the present invention further provides a single-tree segmentation device for highly closed forests, including a computer-readable storage medium and a processor. When the processor executes a computer program on the computer-readable storage medium, the steps of the above-mentioned single-tree segmentation method for highly closed forests are implemented.

[0136] Figure 7 is a schematic structural diagram of a single-tree segmentation device for highly closed forests provided by an embodiment of the present invention. As Figure 7 shown, the single-tree segmentation device 8 for highly closed forests in this embodiment includes: a processor 80, a readable storage medium 81, and a computer program 82 stored in the readable storage medium 81 and executable on the processor 80. When the processor 80 executes the computer program 82, the steps in the above-mentioned method embodiments are implemented, such as Figure 1 the steps shown. Or, when the processor 80 executes the computer program 82, the functions of each module in the above-mentioned device embodiments are implemented, such as Figure 6 the functions of the modules shown.

[0137] Exemplarily, the computer program 82 may be divided into one or more modules. The one or more modules are stored in the readable storage medium 81 and executed by the processor 80 to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 82 in the high-closure forest individual tree segmentation device 8.

[0138] The high-closure forest individual tree segmentation device 8 may include, but is not limited to, a processor 80 and a readable storage medium 81. Those skilled in the art can understand that Figure 7 merely examples of the high-closure forest individual tree segmentation device 8, which do not constitute a limitation on the high-closure forest individual tree segmentation device 8. It may include more or fewer components than those shown in the figure, or combine certain components, or have different components. For example, the high-closure forest individual tree segmentation device may further include a power management module, an arithmetic processing module, input / output devices, a network access device, a bus, etc.

[0139] The so-called processor 80 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] The readable storage medium 81 may be an internal storage unit of the high-closure forest individual tree segmentation device 8, such as a hard disk or memory of the high-closure forest individual tree segmentation device 8. The readable storage medium 81 may also be an external storage device of the high-closure forest individual tree segmentation device 8, such as a plug-in hard disk equipped on the high-closure forest individual tree segmentation device 8, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the readable storage medium 81 may also include both the internal storage unit of the high-closure forest individual tree segmentation device 8 and an external storage device. The readable storage medium 81 is used to store the computer program and other programs and data required by the high-closure forest individual tree segmentation device. The readable storage medium 81 may also be used to temporarily store data that has been output or is to be output.

[0141] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated herein.

[0142] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] Those of ordinary skill in the art can realize that the units and method steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0144] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0147] The present invention is not limited only to what is described in the specification and embodiments. Therefore, for those skilled in the art, additional advantages and modifications can be easily achieved. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to specific details, representative devices and the illustrated examples shown and described here.

Claims

1. A method for segmenting single trees in a high canopy density forest, characterized in that: The following steps are involved: Step 1, collecting forest point cloud data of the target forest; Step 2, classifying the ground points and non-ground points after preprocessing the forest point cloud data, and generating a digital surface model of the target forest based on the classified complete point cloud data; Step 3, obtaining the neighborhood pixels corresponding to each pixel, calculating the local gradient vectors of the neighborhood pixels and the degree of orientation concentration of the corresponding pixels based on the digital surface model, and generating a gradient orientation map of the target forest; Step 4, extracting tree vertices using a preset tree vertex detection algorithm; Step 5, inverting the pixel values ​​of the gradient pointing map, simulating the flow of water on each pixel, and classifying the watersheds that eventually converge on the same pixel into one category, generating a water flow path segmentation result; Step 6: Execute a watershed algorithm based on the gradient pointing graph and the extracted tree vertices, merge and overlay the water flow path segmentation result with the execution result of the watershed algorithm, and generate a single tree segmentation result of the target forest.

2. The method for segmenting single trees in a high canopy density forest according to claim 1, characterized in that: The step 2 is specifically as follows: S201, using a preset statistical filtering method to remove noise points from the forest point cloud data; S202, separating the forest point cloud data after noise removal into ground points and non-ground points based on a preset cloth simulation algorithm; S203, generating a target resolution of the digital surface model according to the target point density, dividing the ground points and the non-ground points based on the target resolution, and constructing a digital surface model of the target forest, wherein the target point density includes at least 4 lidar points within each pixel.

3. The method for segmenting single trees in a high canopy density forest according to claim 1, characterized in that: The step 3 is specifically as follows: S301, constructing a convolution kernel, and using the convolution kernel to perform convolution processing on the data of the digital surface model of the target forest in two vertical directions to generate a local gradient vector of each pixel and its neighborhood; S302, constructing a direction vector from the central pixel to the neighboring pixel, and calculating the angle difference between the local gradient vector of the neighboring pixel and the direction vector from the central pixel to the neighboring pixel; S303, setting weights for neighborhood pixels at different distances based on a two-dimensional Gaussian kernel function, calculating a pixel value of each pixel, and generating a gradient pointing map of the target forest according to the pixel value.

4. The method for segmenting a single tree in a high canopy density forest according to claim 1, characterized in that: In step 4, the tree vertices are extracted based on the gradient pointing graph of the target forest, specifically: S401, obtaining a gradient pointing map of the target forest; S402, using at least two windows of different scales to respectively extract local maxima of the gradient pointing graph to generate multiple local maximum sets; S403, comparing the multiple local maximum value sets, determining the repeated points therein as real tree vertices and the remaining points as candidate points, and establishing a real tree vertex set and a candidate point set; S404, identifying the geometric features between tree crowns of the target forest according to the gradient pointing map or digital surface model of the target forest, and iteratively screening out real tree vertices from the candidate point set based on the geometric features between tree crowns, and updating the real tree vertex set and the candidate point set according to the screening results until a preset iteration end condition is reached.

5. The method for segmenting single trees in a high canopy density forest according to claim 4, characterized in that: The step S404 is specifically as follows: S4041, constructing a Delaunay triangulation using the real tree vertex set and all points in the candidate point set; S4042, traversing each edge of each triangle, obtaining a target candidate point connected by a vertex of the real tree, and determining whether the corresponding edge meets a preset condition, if so, adding the target candidate point to the vertex set of the real tree, otherwise removing the target candidate point; S4043, repeating steps S4041-S4042 until a preset iteration end condition is reached, wherein the preset iteration end condition includes that all points in the candidate point set are processed.

6. The method for segmenting single trees in a high canopy density forest according to claim 5, characterized in that: Step S4042 includes a first preset condition, a second preset condition and a third preset condition. If the corresponding edge meets any one of the first preset condition, the second preset condition and the third preset condition, the target candidate point is added to the real tree vertex set; The first preset condition is: based on the gradient directivity graph, a ratio of a minimum value to an endpoint value of a connecting line is obtained, and the ratio is less than a preset ratio; The second preset condition is: based on the digital surface model of the target forest, the angle difference between the lowest point in the connecting line and the two endpoints of the line segment is obtained, and the angle difference is less than a preset angle threshold; The third preset condition is: obtaining the length of the connection line, and the length of the connection line is greater than a first preset length threshold.

7. The method for segmenting single trees in a high canopy density forest according to claim 1, characterized in that: The step 5 is specifically as follows: S501, performing a difference operation between the pixel value of each pixel in the gradient pointing image and the maximum value in the image to perform pixel value inversion; S502, calculating the flow direction of each pixel by a preset eight-neighborhood method, and generating a final convergence point for each pixel; S503, dividing all pixels with the same final convergence point into one watershed, and generating a water flow path segmentation result.

8. A device for splitting single trees in a high-canopy density forest, based on the method for splitting single trees in a high-canopy density forest according to any one of claims 1 to 7, characterized in that: It includes a collection module, a first construction module, a second construction module, an extraction module, a segmentation module and a merging module. The acquisition module is used to collect forest point cloud data of the target forest; The first construction module is used to classify ground points and non-ground points after preprocessing the forest point cloud data, and generate a digital surface model of the target forest based on the classified complete point cloud data; The second construction module is used to obtain the neighborhood pixels corresponding to each pixel, calculate the local gradient vectors of the neighborhood pixels and the degree of orientation concentration of the corresponding pixels based on the digital surface model, and generate a gradient orientation map of the target forest; The extraction module is used to extract tree vertices using a preset tree vertex detection algorithm; The segmentation module is used to invert the pixel values ​​of the gradient pointing map, simulate the flow of water on each pixel, and classify the watersheds that eventually converge on the same pixel into one category, thereby generating a water flow path segmentation result; The merging module is used to execute the watershed algorithm based on the gradient pointing map and the extracted tree vertices, merge and overlay the water flow path segmentation result with the execution result of the watershed algorithm, and generate a single tree segmentation result of the target forest.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for segmenting individual trees in a high-canopy density forest as described in any one of claims 1 to 7 above is implemented.

10. A high-canopy forest single-tree segmentation device, comprising a computer-readable storage medium and a processor, characterized in that: When the processor executes the computer program on the computer-readable storage medium, the processor implements the steps of the method for segmenting a single tree in a high-canopy density forest as described in any one of claims 1 to 7.

Images