A hierarchical single tree extraction method based on radius expansion

Abstract

The present invention discloses a hierarchical single tree extraction method based on radius expansion, including classifying scenes, extracting tree points therein and performing denoising using a filtering algorithm; determining the projection direction by establishing a local coordinate reference system, finding the positive and negative intersection points of the local maximum smooth difference from the projection points, finding potential tree vertices by setting a height threshold, merging and optimizing the potential tree vertices by Euclidean distance; and performing top-down layer-by-layer segmentation using the relationship between point cloud data and a circle to achieve the extraction of a single tree. The present invention solves the problem that the extraction result is affected by noise interference, scene complexity, data incompleteness and other factors.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and artificial intelligence, and relates to a hierarchical single tree extraction method based on radius expansion. Background Art

[0002] As one of the important elements in outdoor scenes, trees play a very important role in local climate conditions, air quality and ecosystems, and indirectly affect the health and quality of life of residents. Single tree extraction has been widely used in various applications, such as monitoring tree growth and health, navigation and positioning of vehicles and pedestrians, 3D modeling of large-scale outdoor scenes, and urban green volume estimation. Therefore, tree detection and extraction are very necessary. The rapid development of MLS technology has also brought us more abundant tree structure information and provided more methods for tree extraction.

[0003] At present, there are two main ways for scholars at home and abroad to deal with trees: methods based on rasterized data and methods based on point cloud data. The method based on rasterized data usually converts the point cloud into a canopy height model (CHM), and then uses 2D image processing techniques such as template matching, region growing and watershed to extract single trees. Among them, the watershed algorithm is a traditional image segmentation algorithm based on topological theory. By establishing a watershed at the confluence of adjacent catchment basins, it can well achieve the segmentation of a single tree crown. Koch et al. detected local maxima through a fixed window and then used the watershed for tree detection, but the detection results often depend on the window size. Considering that the tree extraction results largely depend on the detection of local maxima, Chen et al. proposed a watershed algorithm based on marker point control, setting the window size according to the crown and tree height, and using the seed points extracted by the dynamic window as markers for the watershed method to achieve more refined single tree segmentation. Mongus et al. detected the tree vertices in the CHM by using a local fitting surface, then used the tree vertices as markers for the watershed algorithm, and reduced the over-segmentation problem by merging regions based on height, area and shape. Li et al. proposed a multi-scale local maximum algorithm to improve the accuracy of local maximum values ​​obtained at different spatial resolutions, thereby segmenting the lidar point cloud into single trees. Therefore, although the segmentation results of the watershed algorithm are relatively stable, it is easy to produce multiple local extreme values, causing over-segmentation. Moreover, in the process of generating CHM, the original point cloud data will be smoothed and removed, and only the surface elevation information will be retained, resulting in the inability to fully utilize the three-dimensional information of the point cloud.

[0004] Methods based on point cloud data mainly rely on information unique to point cloud data, including attribute features, local geometric features, and global features, and combine the structural characteristics of trees to complete detection. Domestic and foreign scholars mainly use two strategies to extract trees: 1) strategies based on clustering classification; 2) strategies based on shape fitting. Morsdorf et al. used a clustering method to complete the segmentation of trees, and used the local maximum obtained by the digital surface model (DSM) as the seed point for k-means clustering to extract single trees, but this method is suitable for evenly distributed point cloud data. Li et al. selected appropriate seed points on the trunk for double growth to extract the trunk and crown parts respectively. Zhong et al. analyzed the horizontal histogram of the octree nodes to detect the trunk and used the improved Ncut method to segment the overlapping crowns. Fan et al. first extracted rod-shaped objects from the scene, then calculated global and local features, and used random forests for classification to extract single trees. The above three methods all need to remove the ground according to the ground detection algorithm before extracting trees, and secondly, when extracting single trees, they need to use the feature differences between trees and other objects to remove interfering objects. However, since the expressive power of these features often depends on the quality of point cloud data and it is difficult to represent the semantic features of the scene as a whole, their application effect in complex scenes is limited. In recent years, deep learning has made breakthrough progress in the field of point cloud. PointNet is the first end-to-end network applied to point cloud data. It solves the permutation invariance and disorder problems of point cloud data by introducing spatial transformation network and maximum pooling respectively. Luo et al. used semantic segmentation network to segment tree points from raw point cloud, and proposed PDE-Net network to predict the direction of each tree cluster pointing to the center of the tree to realize the extraction of single tree. Therefore, the semantic segmentation method based on deep learning effectively overcomes the inherent characteristics of point cloud data such as unstructured and non-uniformity, and provides an effective solution for extracting tree points from raw point cloud. Summary of the invention

[0005] The purpose of the present invention is to provide a hierarchical single tree extraction method based on radius expansion, which solves the problem of inaccurate tree point extraction.

[0006] The technical solution adopted by the present invention is a hierarchical single tree extraction method based on radius expansion, which is specifically implemented according to the following steps:

[0007] Step 1: Use the principal component analysis algorithm PCA to obtain the dimensional features of each point in the original scene, classify the scene by inputting these features into the PointNet network, extract the tree points and use the filtering algorithm to denoise;

[0008] Step 2: For the denoised tree points, the projection direction is determined by establishing a local coordinate reference system, the positive and negative intersection points of the local maximum smooth difference are found from the projection points, the potential tree vertices are found by setting the height threshold, and the potential tree vertices are merged and optimized by Euclidean distance;

[0009] Step 3: After determining the location of the tree vertices, perform preliminary clustering based on distance to obtain the area where the rectangle and circle of each cluster are located, and use the relationship between the point cloud data and the circle to perform top-down layer-by-layer segmentation to achieve the extraction of a single tree.

[0010] The present invention is also characterized in that:

[0011] Step 1 is as follows:

[0012] Step 1.1, dimensional feature extraction, introducing dimensional features into semantic segmentation;

[0013] Step 1.2, tree detection based on semantic segmentation, after obtaining the dimensional features, use the PointNet network to segment the point cloud data of the original scene;

[0014] Step 1.3, segmentation result optimization, extract all tree points based on the semantic segmentation results, and use the pass-through filter and statistical filter in PCL to remove noise points and outliers.

[0015] Step 1.1 is as follows:

[0016] Step 1.1.1, given a point cloud dataset P = {p1, p2, ... p N}, N is the number of point cloud data, given the range of neighborhood points [k min ,k max ], and then the neighborhood k i Initialized to k min ;

[0017] Step 1.1.2, select point p from the 3D point cloud i The local neighborhood points are constructed and the covariance matrix is ​​constructed. The eigenvalues ​​λ1, λ2, and λ3 are calculated according to the PCA algorithm, and λ1≥λ2≥λ3;

[0018] Step 1.1.3, use the eigenvalues ​​to calculate the values ​​of the three different dimensional features,

[0019]

[0020] In formula (1), pro 1D 、pro 2D , and pro 3D They represent the probability that the point belongs to a linear point, a plane point, and a scattered point respectively;

[0021] Step 1.1.4, calculate point p i Shannon entropy value Entropy:

[0022] Entropy=-pro 1D ln(pro 2D )-pro 2D ln(pro 2D )-pro 3D ln(pro 3D ), (2)

[0023] Record the entropy value at this time. The smaller the entropy value, the smaller the information contained in the point and the more single the dimensional feature;

[0024] Step 1.1.5, if k i <k max , go to step 1.1.2, let k i =k i +Δk, Δk is the neighborhood k i The increment of p is 1, and the corresponding p is calculated. i Dimensional features, otherwise, go to step 1.1.6;

[0025] Step 1.1.6: Count the entropy values ​​corresponding to the dimensional features under different neighborhood radii, and take the dimensional feature value with the minimum entropy value as point p i The best eigenvalue of .

[0026] Step 1.2 is as follows:

[0027] Step 1.2.1, combine the spatial coordinates and dimensional features of the point cloud data into six-dimensional data (x, y, z, pro 1D ,pro 2D ,pro 3D ) data is input into the network, and a shared multi-layer perceptron MLP and two T-Nets transform the N*6-dimensional data into N*64-dimensional data that is conducive to segmentation, and obtain the local features of the point cloud data;

[0028] Step 1.2.2, obtain N*1024 high-dimensional data through a shared MLP, and use maxpool to obtain 1*1024 global features;

[0029] In step 1.2.3, the 64-dimensional local features and the 1024-dimensional global features are concatenated to obtain N*1088 data, which is then passed through a shared MLP to obtain an N*2-dimensional matrix representing the probability that each point belongs to a tree point or a non-tree point, completing the scene semantic segmentation and dividing the point cloud data into tree points and non-tree points.

[0030] In step 1.3, the pass filter and statistical filter are used to remove noise points and outliers. Specifically, the pass filter quickly removes outliers by determining the range of the X-axis, Y-axis, and Z-axis of the point cloud data; the statistical filter calculates the average distance from each point to the adjacent points and compares it with the given mean and variance to remove noise points.

[0031] Step 2 is as follows:

[0032] Step 2.1, determine the projection direction according to the local coordinate reference system, use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projection point cluster, and select the plane where the X-axis and Z-axis are located as the projection direction;

[0033] Step 2.2, process the noise points, remove the duplicate points after projection, extract the local maximum based on the denoised data, locate the potential tree vertices according to the local maximum, calculate the Euclidean distance between all potential peak points, sort the distances between the peak points in ascending order, recalculate the distances between the peak point pairs and perform distance evaluation until the distances between all peak points are no less than the distance threshold and the merging is completed.

[0034] Step 2.1 is as follows:

[0035] Step 2.1.1, determine the projection direction according to the local coordinate reference system: Assume that the detected tree point set T = {t i |i=1,2,...,N t},N t is the number of tree points; define a plane model ax+by+cz+d=0, where a=b=d=0, c=1; project the point cloud data in T on a horizontal plane;

[0036] Step 2.1.2, use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projected point cluster, take the direction of the eigenvector v1 corresponding to the largest eigenvalue as the X-axis, the direction of the eigenvector v2 corresponding to the middle eigenvalue as the Y-axis, and the direction of the eigenvector v3 corresponding to the smallest eigenvalue as the Z-axis to establish a local coordinate reference system;

[0037] Step 2.1.3, select the plane where the X-axis and Z-axis are located as the projection direction, project the scene data along the Y-axis, and calculate the centroid of all tree points.

[0038]

[0039] In formula (3), n is the centroid of all projection points;

[0040] The XOZ plane where the centroid n is located is used as the projection plane. The normal vector of the projection plane is v2, and the coordinates are (a, b, c). Let the projected data set be T'={t i'|i=1,2,...N t}, t i ' represents the i-th point t i Projected onto the XOZ plane, for any point t i (t i x ,t i y ,t i z )∈T,t i '(t i ' x ,t i ' y ,t i ' z )∈T', because Same direction as v2, let l be a vector The length of therefore

[0041]

[0042] because And vector The projection length in the v2 direction is a vector The length of l, so Right now

[0043]

[0044] Combining equation (4) and equation (5), we get point t i Projection point t on the XOZ plane i 'The coordinates are:

[0045]

[0046] Step 2.2 is as follows:

[0047] Step 2.2.1, process the noise points and remove the duplicate points after projection: For point t on the projection contour i '(t i ' x ,t i y ,t i ' z ) and t j '(t j ' x ,t j ' y ,t j ' z ), point t j '(t j' x ,t j y ,t j ' z ) represents the point t i '(t i ' x ,t i y ,t i ' z ) non-repeated points, if t i ' x =t j ' x And t i ' z =t j ' z , then only one of the points is retained; for t i ' x =t j ' x Of all the points, only the points with the largest z value are retained, and local maxima are extracted based on the denoised data. The local maxima obtained from the denoised scene are concentrated on the outer contour position of the tree crown;

[0048] Step 2.2.2, locate potential tree vertices according to local maxima: assign an order to all local maxima from the minimum value of the X coordinate to the maximum value of the X coordinate, expressed as M = {m i |i=1,2,...,N m},N m is the number of local maximum points, calculate point m i (m i x ,m i y ,m i z )Difference DM on the Z axis i ,

[0049]

[0050] Step 2.2.3, find the positive and negative change points of the smooth difference to locate the potential tree vertices, first use the sign function to determine the point m i The sign of the smoothed difference,

[0051]

[0052] In formula (8), sign(m i ) is 1, which represents point m i The difference is a positive number, sign(m i ) is 0, which means point m i The difference is zero, sign(mi ) is -1, which means point m i The difference is negative, so when sign(m i )>sign(m i+1 ) is the intersection point, and the potential peak point is found by detecting the point where the Z-axis coordinate exceeds the height threshold;

[0053] Set the tree vertex to be above 1 / 2 of the entire scene height to avoid interference from other data that is obviously not part of the tree vertex. Set the height threshold to,

[0054] h th =(z max -z min )×1 / 2+z min (9)

[0055] In formula (9), z max and z min It is the highest and lowest point of all data on the Z axis, and the height is less than h th All intersections of are removed;

[0056] For a point that meets the above conditions, find two points in its neighborhood, and the point with the largest Z coordinate is the potential tree vertex;

[0057] Step 2.2.4, calculate the Euclidean distance between all potential tree vertices, sort the distances between potential tree vertices in ascending order, and for the closest pair of potential tree vertices, determine whether the distance between them is less than the distance threshold d th If it is less than the threshold, the two potential tree vertices are replaced with their center points, the distance between the peak point pairs is recalculated and the distance evaluation is performed until the distance between all peak points is not less than the distance threshold.

[0058] Step 3 is as follows:

[0059] Step 3.1, for a given tree vertex G = {G i |i=1,2,...,N g}, perform initial clustering based on the neighborhood information of the specified radius to obtain the cluster Clu = {Clu i |i=1,2,...,N g},N g Indicates the number of tree vertices;

[0060] Step 3.2, calculate the initial radius of the circle where the cluster is located R = {R i |i=1,2,...N g} and the center O={O i |i=1,2,N. g .}.;

[0061] Step 3.3, divide the tree points into N according to their height l Layer, each layer of points constitutes a set L = {L i |i=1,2,...N l},N l Indicates the number of layers into which the tree points are divided;

[0062] Step 3.4, let f = 1, based on the initial radius and center of the circle L f After layer data segmentation, go to step 3.2 to update the radius of the current circle center;

[0063] Step 3.5, let f = f + 1, if f < N l , then for L f If the data of the layer is segmented, go to step 3.2 to update the center and radius, and continue to step 3.5; otherwise, directly segment the tree points of the layer until all points are processed to achieve the extraction of single trees.

[0064] The beneficial effects of the present invention are:

[0065] The present invention discloses a top-down single tree extraction method based on radius expansion. By detecting the tree vertices and using them as marker points in the watershed algorithm for top-down expansion, the problem of information loss when directly using the watershed algorithm to convert three-dimensional point cloud data into two-dimensional images can be avoided, and the three-dimensional structural information of the point cloud can be directly used. The present invention can effectively extract single trees, and can also correctly extract connected trees; secondly, the method can solve the problem of over-segmentation and under-segmentation caused by the trees not being connected due to the incompleteness and uneven density of the point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A process diagram of a top-down single tree extraction method based on radius expansion according to the present invention;

[0067] Figure 2 This is the architecture diagram of the PointNet semantic segmentation network of the present invention;

[0068] Figure 3 A schematic diagram of the optimization of tree detection results of the present invention, wherein: Figure 3 (a) is the front view of the tree cluster. Figure 3 (b) is the top view of the tree cluster. Figure 3 (c) is the front view of the optimized tree cluster. Figure 3 (d) is the top view of the optimized tree cluster;

[0069] Figure 4 The schematic diagram of determining the projection direction of the tree point in the present invention, wherein: Figure 4(a) is the horizontal projection of the tree cluster. Figure 4 (b) To establish a local coordinate reference system, Figure 4 (c) is the XOZ plane projection;

[0070] Figure 5 It is a comparative schematic diagram of the tree point denoising before and after the present invention, wherein: Figure 5 (a) is the point cloud data before denoising. Figure 5 (b) is the local maximum of the point cloud data before denoising. Figure 5 (c) is the point cloud data after denoising. Figure 5 (d) is the local maximum of the point cloud data after denoising;

[0071] Figure 6 is the potential tree vertex extraction result in the tree extraction method of the present invention, wherein, Figure 6 (a) Front view of potential tree vertices extracted from point cloud data. Figure 6 (b) Top view of potential tree vertices extracted for point cloud data;

[0072] Figure 7 is the location of the optimal tree vertex in the tree extraction method of the present invention, wherein, Figure 7 (a) is the front view of the optimal tree vertex extracted from point cloud data. Figure 7 (b) Top view of the optimal tree vertex extracted for point cloud data;

[0073] Figure 8 Schematic diagram of hierarchical single tree extraction based on radius expansion in the tree extraction method of the present invention. Figure 8 (a) is the initial clustering based on tree vertices. Figure 8 (b) is the result of initial clustering based on tree vertices. Figure 8 (c) tree point stratification, Figure 8 (d) is the radius expansion method, Figure 8 (e) is a schematic diagram of top-down hierarchical segmentation. Figure 8 (f) is the extraction result of a single tree. DETAILED DESCRIPTION

[0074] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] The present invention provides a hierarchical single tree extraction method based on radius expansion, such as Figure 1 As shown, the specific steps are as follows:

[0076] Step 1: Use the principal component analysis algorithm PCA to obtain the dimensional features of each point in the original scene, classify the scene by inputting these features into the PointNet network, extract the tree points and use the filtering algorithm to denoise;

[0077] Step 1.1, dimensional feature extraction, introducing dimensional features into semantic segmentation;

[0078] Step 1.1.1, given a point cloud dataset P = {p1, p2, ... p N}, N is the number of point cloud data, given the range of neighborhood points [k min ,k max ], and then the neighborhood k i Initialized to k min ;

[0079] Step 1.1.2, select point p from the 3D point cloud i The local neighborhood points are constructed and the covariance matrix is ​​constructed. The eigenvalues ​​λ1, λ2, and λ3 are calculated according to the PCA algorithm, and λ1≥λ2≥λ3;

[0080] Step 1.1.3, use the eigenvalues ​​to calculate the values ​​of the three different dimensional features,

[0081]

[0082] In formula (1), pro 1D 、pro 2D , and pro 3D They represent the probability that the point belongs to a linear point, a plane point, and a scattered point respectively;

[0083] Step 1.1.4, calculate point p i Shannon entropy value Entropy:

[0084] Entropy=-pro 1D ln(pro 2D )-pro 2D ln(pro 2D )-pro 3D ln(pro 3D ), (11)

[0085] Record the entropy value at this time. The smaller the entropy value, the smaller the information contained in the point and the more single the dimensional feature;

[0086] Step 1.1.5, if k i <k max , go to step 1.1.2, let k i =k i +Δk, Δk is the neighborhood k i The increment of p is 1, and the corresponding p is calculated. i Dimensional features, otherwise, go to step 1.1.6;

[0087] Step 1.1.6: Count the entropy values ​​corresponding to the dimensional features under different neighborhood radii, and take the dimensional feature value with the minimum entropy value as point p i The best eigenvalue of .

[0088] Step 1.2, tree detection based on semantic segmentation, after obtaining the dimensional features, such as Figure 2 As shown, the point cloud data of the original scene is segmented using the PointNet network;

[0089] Step 1.2.1, combine the spatial coordinates and dimensional features of the point cloud data into six-dimensional data (x, y, z, pro 1D ,pro 2D ,pro 3D ) data is input into the network, and a shared multi-layer perceptron (MLP) and two T-Nets transform the N*6-dimensional data into N*64-dimensional data that is conducive to segmentation, and obtain the local features of the point cloud data;

[0090] Step 1.2.2, obtain N*1024 high-dimensional data through a shared MLP, and use maxpool to obtain 1*1024 global features;

[0091] In step 1.2.3, the 64-dimensional local features and the 1024-dimensional global features are concatenated to obtain N*1088 data, which is then passed through a shared MLP to obtain an N*2-dimensional matrix representing the probability that each point belongs to a tree point or a non-tree point, completing the scene semantic segmentation and dividing the point cloud data into tree points and non-tree points.

[0092] Step 1.3, such as Figure 3 As shown in the figure, the segmentation result is optimized, all tree points are extracted according to the semantic segmentation result, and the straight-through filter and statistical filter in PCL are used to remove noise points and outliers. The straight-through filter quickly removes outliers by determining the range of the X-axis, Y-axis, and Z-axis of the point cloud data; the statistical filter calculates the average distance from each point to the adjacent points and compares it with the given mean and variance to remove noise points.

[0093] Step 2: For the denoised tree points, the projection direction is determined by establishing a local coordinate reference system, the positive and negative intersection points of the local maximum smooth difference are found from the projection points, the potential tree vertices are found by setting the height threshold, and the potential tree vertices are merged and optimized by Euclidean distance;

[0094] Step 2.1, determine the projection direction according to the local coordinate reference system, use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projection point cluster, and select the plane where the X-axis and Z-axis are located as the projection direction;

[0095] Step 2.1.1, determine the projection direction according to the local coordinate reference system: Assume that the detected tree point set T = {t i |i=1,2,...,N t},N t is the number of tree points; define the plane model ax+by+cz+d=0, where a=b=d=0, c=1; Figure 4 As shown, the point cloud data in T is projected on a horizontal plane;

[0096] Step 2.1.2, use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projected point cluster, take the direction of the eigenvector v1 corresponding to the largest eigenvalue as the X-axis, the direction of the eigenvector v2 corresponding to the middle eigenvalue as the Y-axis, and the direction of the eigenvector v3 corresponding to the smallest eigenvalue as the Z-axis to establish a local coordinate reference system;

[0097] Step 2.1.3, select the plane where the X-axis and Z-axis are located as the projection direction, project the scene data along the Y-axis, and calculate the centroid of all tree points.

[0098]

[0099] In formula (3), n is the centroid of all projection points;

[0100] The XOZ plane where the centroid n is located is used as the projection plane. The normal vector of the projection plane is v2, and the coordinates are (a, b, c). Let the projected data set be T'={t i '|i=1,2,...N t}, t i ' represents the i-th point t i Projected onto the XOZ plane, for any point t i (t i x ,t i y ,t i z )∈T,t i '(t i ' x ,t i ' y ,t i ' z )∈T', because Same direction as v2, let l be a vector The length of therefore

[0101]

[0102] because And vector The projection length in the v2 direction is a vector The length of l, so Right now

[0103]

[0104] Combining equation (4) and equation (5), we get point t i Projection point t on the XOZ plane i 'The coordinates are:

[0105]

[0106] Step 2.2, process the noise points, remove the duplicate points after projection, extract the local maximum based on the denoised data, locate the potential tree vertices according to the local maximum, calculate the Euclidean distance between all potential peak points, sort the distances between the peak points in ascending order, recalculate the distances between the peak point pairs and perform distance evaluation until the distances between all peak points are no less than the distance threshold and the merging is completed.

[0107] Step 2.2.1, as Figure 5 As shown, the noise points are processed and the repeated points after projection are removed: for point t on the projection contour i '(t i ' x ,t i ' y ,t i ' z ) and t j '(t j ' x ,t j ' y ,t j ' z ), point t j '(t j ' x ,t j ' y ,t' j z ) represents the point t i '(t i ' x ,t i ' y ,t i ' z ) non-repeated points, if t i ' x =t j ' x And t i ' z =tj ' z , then only one of the points is retained; for t i ' x =t j ' x Of all the points, only the points with the largest z value are retained, and local maxima are extracted based on the denoised data. The local maxima obtained from the denoised scene are concentrated on the outer contour position of the tree crown;

[0108] Step 2.2.2, locate potential tree vertices according to local maxima: assign an order to all local maxima from the minimum value of the X coordinate to the maximum value of the X coordinate, expressed as M = {m i |i=1,2,...,N m},N m is the number of local maximum points, calculate point m i (m i x ,m i y ,m i z )Difference DM on the Z axis i ,

[0109]

[0110] like Figure 6 The figure shows the front view and top view of the detected potential tree vertices. It can be seen from the figure that the number of detected tree vertices is 8, while the actual number of trees is 6. Therefore, it is necessary to further optimize the potential tree vertices;

[0111] Step 2.2.3, find the positive and negative change points of the smooth difference to locate the potential tree vertices, first use the sign function to determine the point m i The sign of the smoothed difference,

[0112]

[0113] In formula (8), sign(m i ) is 1, which represents point m i The difference is a positive number, sign(m i ) is 0, which means point m i The difference is zero, sign(m i ) is -1, which means point m i The difference is negative, so when sign(m i )>sign(m i+1 ) is the intersection point, and the potential peak point is found by detecting the point where the Z-axis coordinate exceeds the height threshold;

[0114] Set the tree vertex to be above 1 / 2 of the entire scene height to avoid interference from other data that is obviously not part of the tree vertex. Set the height threshold to,

[0115] h th =(z max -z min )×1 / 2+z min (18)

[0116] In formula (9), z max and z min It is the highest and lowest point of all data on the Z axis, and the height is less than h th All intersections of are removed;

[0117] For a point that meets the above conditions, find two points in its neighborhood, and the point with the largest Z coordinate is the potential tree vertex;

[0118] Step 2.2.4, calculate the Euclidean distance between all potential tree vertices, sort the distances between potential tree vertices in ascending order, and for the closest pair of potential tree vertices, determine whether the distance between them is less than the distance threshold d th If it is less than the threshold, the two potential tree vertices are replaced with their center points, the distance between the peak point pairs is recalculated and the distance evaluation is performed until the distance between all peak points is not less than the distance threshold. The tree vertex optimization result is as follows: Figure 7 shown.

[0119] Step 3: After determining the location of the tree vertices, perform preliminary clustering based on distance to obtain the area where the rectangle and circle of each cluster are located, such as Figure 8 As shown, the relationship between point cloud data and circles is used to perform top-down segmentation layer by layer to achieve the extraction of a single tree;

[0120] Step 3.1, for a given tree vertex G = {G i |i=1,2,...,N g}, perform initial clustering based on the neighborhood information of the specified radius to obtain the cluster Clu = {Clu i |i=1,2,...,N g},N g is the number of tree vertices;

[0121] Step 3.2, based on the initial clustering results, find the cluster clu i The maximum value of the x- and y-coordinates of the midpoint (x i max ,y i max ) and the minimum value (x i min ,y imin ), forming the boundary set Bou i (x i max ,y i max ,x i min ,y i min ), the boundary set of all clusters is Bou = {Bou i |i=1,2,...,N g Then calculate the cluster clu according to the boundary value i The radius of the circle R i and the center O i (O i x ,O i y ):

[0122] R i =((x i max -x i min )+(y i max -y i min )) / 4, (19)

[0123]

[0124] Finally, the radius of the circle where each tree vertex is located forms a set R = {R i |i=1,2,...,N g}, the circle centers form a set O = {O i |i=1,2,...,N g}

[0125] Step 3.3, divide all tree points into N according to their height l The number of layers depends on the distribution of trees and the differences in tree species in the scene. l The larger the number of layers, the more accurate the extraction results of individual trees will be, but the processing time will also increase, because the degree of occlusion between trees in the scene is not high, so the selected number of layers N l If there are 15 floors, the height h of each floor is:

[0126] h=(z max -z min ) / N l , (twenty one)

[0127] Sort all the hierarchical segments in descending order to obtain L = {L i |i=1,2,...,Nl}, where L i The number of points in the layer is , start processing the data of each layer in turn, split the tree points in two cases, if L i Points in layer Located in Clu j Boundary Point Bou j Within the range and to the center G j The distance is less than the radius R j , then the point Split into cluster Clu j If the above conditions are not met, the tree points are split according to formula (13).

[0128]

[0129] Step 3.4, let f = 1, based on the initial radius and center of the circle L f After layer data segmentation, go to step 3.2 to update the radius of the current circle center;

[0130] Step 3.5, let f = f + 1, if f < N l , then for L f If the data of the layer is segmented, go to step 3.2 to update the center and radius, and continue to step 3.5; otherwise, directly segment the tree points of the layer until all points are processed to achieve the extraction of single trees.

[0131] Through the above methods, it can be known that the hierarchical single tree extraction method based on radius expansion of the present invention improves the accuracy of single tree extraction by combining deep learning with the watershed algorithm of marker point control. The main work includes: 1) using deep learning methods to detect tree points and non-tree points; 2) proposing a tree vertex positioning method based on projection; 3) designing a single tree extraction method based on radius expansion. The problem that the extraction results are affected by noise interference, scene complexity, and data incompleteness is solved.

Claims

1. A hierarchical single tree extraction method based on radius expansion, characterized in that: The specific steps are as follows: Step 1: Use the principal component analysis algorithm PCA to obtain the dimensional features of each point in the original scene, classify the scene by inputting these features into the PointNet network, extract the tree points and use the filtering algorithm to denoise; The step 1 is specifically as follows: Step 1.1, dimensional feature extraction, introducing dimensional features into semantic segmentation; Step 1.2, tree detection based on semantic segmentation, after obtaining the dimensional features, use the PointNet network to segment the point cloud data of the original scene; Step 1.3, segmentation result optimization, extract all tree points according to the semantic segmentation results, and use the pass filter and statistical filter in PCL to remove noise points and outliers; Step 2: For the denoised tree points, the projection direction is determined by establishing a local coordinate reference system, the positive and negative intersection points of the local maximum smooth difference are found from the projection points, the potential tree vertices are found by setting the height threshold, and the potential tree vertices are merged and optimized by Euclidean distance; The step 2 is specifically as follows: Step 2.1, determine the projection direction according to the local coordinate reference system, use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projection point cluster, and select Axis and The plane where the axis lies is taken as the projection direction; Step 2.1.1, determine the projection direction according to the local coordinate reference system: Assume that the detected tree point set , The number of tree points; define the plane model ,in , ;Will Perform horizontal plane projection on the point cloud data in; Step 2.1.2: Use the PCA algorithm to calculate the three largest eigenvalues ​​and eigenvectors of the projected point cluster, and convert the eigenvector corresponding to the largest eigenvalue into The direction as Axis, eigenvector corresponding to the intermediate eigenvalue The direction as Axis, eigenvector corresponding to the minimum eigenvalue The direction as Axis to establish the local coordinate reference system; Step 2.1.3, select Axis and The plane where the axis is located is used as the projection direction, and the scene data is projected along Axis projection, calculate the centroid of all tree points, (3) In formula (3), is the centroid of all projection points; The centroid Located The plane is used as the projection plane, then the normal vector of the projection plane is , the coordinates are , let the projected data set be , Indicates Points Project to Points on the plane, for any point , , ,because and In the same direction, For vector The length of ,therefore (4) because , and the vector exist The projected length of the direction is a vector Length ,so ,Right now (5) Combining equation (4) and equation (5), we get the point exist Projection point on the plane The coordinates are: (6)； Step 2.2, process the noise points, remove the duplicate points after projection, extract the local maximum value based on the denoised data, locate the potential tree vertices according to the local maximum value, calculate the Euclidean distance between all potential peak points, sort the distances between the peak points in ascending order, recalculate the distances between the peak point pairs and perform distance evaluation until the distances between all peak points are not less than the distance threshold, and the merging is completed; Step 2.2.1, process the noise points and remove the duplicate points after projection: for the points on the projection contour and ,point Representation and point Non-repeating points, if and , then only one of the points is retained; for All points of The point with the largest value, based on the denoised data, extracts the local maximum. The local maximum obtained from the denoised scene is concentrated at the outer contour position of the tree crown; Step 2.2.2, locate potential tree vertices based on local maxima: The minimum value of the coordinates to The maximum value of the coordinate is the order specified by all local maxima, expressed as , is the number of points of local maximum, calculate the point exist Difference of axes , (7)； Step 2.2.3, find the positive and negative change points of the smooth difference to locate the potential tree vertices, first use the sign function to determine the point The sign of the smoothed difference, (8) In formula (8), When 1 represents the point The difference is a positive number. 0 represents the point The difference is zero, When -1 represents a point The difference is negative, so when When is the intersection point, by detecting The potential peak points are found by looking at the points where the axis coordinates exceed the height threshold; Set the tree vertices to be at the entire scene height To avoid interference from other data that obviously does not belong to the tree vertex, the height threshold is set to, (9) In formula (9), and For all data in The highest and lowest points on the axis are less than All intersections of are removed; For a point that meets the above conditions, find two points in its neighborhood and The point with the largest coordinate is the potential tree vertex; Step 2.2.4, calculate the Euclidean distance between all potential tree vertices, sort the distances between potential tree vertices in ascending order, and for the closest pair of potential tree vertices, determine whether the distance between them is less than the distance threshold , if it is less than the threshold, then replace the two potential tree vertices with their center points, recalculate the distance between the peak point pairs and perform distance evaluation until the distance between all peak points is not less than the distance threshold, and the merging ends; Step 3: After determining the location of the tree vertices, perform preliminary clustering based on distance to obtain the area where the rectangle and circle of each cluster are located, and use the relationship between the point cloud data and the circle to perform top-down layer-by-layer segmentation to extract a single tree; The step 3 is specifically as follows: Step 3.1, for a given tree vertex , perform initial clustering based on the neighborhood information of the specified radius to obtain cluster clusters , Ng is the number of tree vertices; Step 3.2, calculate the initial radius of the circle where the cluster is located and the center ; Step 3.3, divide the tree points into Layer, each layer of points constitutes a set , Nl Indicates the number of layers into which the tree points are divided; Step 3.4, let , based on the initial radius and center After layer data segmentation, go to step 3.2 to update the radius of the current circle center; Step 3.5, let ,if , then If the data of the layer is segmented, go to step 3.2 to update the center and radius, and continue to step 3.5; otherwise, directly segment the tree points of the layer until all points are processed to achieve the extraction of single trees.

2. The method for extracting a hierarchical single tree based on radius expansion according to claim 1, characterized in that: The step 1.1 is specifically as follows: Step 1.1.1, given a point cloud dataset , is the number of point cloud data, given the range of neighborhood points , and then the neighborhood Initialize to ; Step 1.1.2, select points from the 3D point cloud Local neighborhood points and construct the covariance matrix, and calculate its eigenvalues ​​according to the PCA algorithm , , , ; Step 1.1.3, use the eigenvalues ​​to calculate the values ​​of the three different dimensional features, (1) In formula (1), , ,as well as They represent the probability that the point belongs to a linear point, a plane point, and a scattered point respectively; Step 1.1.4, calculate the point Shannon entropy Entropy : (2) Record the entropy value at this time. The smaller the entropy value, the smaller the information contained in the point and the more single the dimensional feature; Step 1.1.5, if , go to step 1.1.2, and let , Neighborhood The increment of , the value is 1, calculate the corresponding Dimensional feature value, otherwise, go to step 1.1.6; Step 1.1.6: Count the entropy values ​​corresponding to the dimensional features under different neighborhood radii, and take the dimensional feature value with the smallest entropy value as the point The best eigenvalue of .

3. The method for extracting a hierarchical single tree based on radius expansion according to claim 1, characterized in that: The step 1.2 is specifically as follows: Step 1.2.1, combine the spatial coordinates and dimensional features of the point cloud data into six-dimensional data The data is input into the network, and a shared multi-layer perceptron MLP and two T-Nets transform the N*6-dimensional data into N*64-dimensional data that is conducive to segmentation, and obtain the local features of the point cloud data; Step 1.2.2, obtain N*1024 high-dimensional data through a shared MLP, and use maxpool to obtain 1*1024 global features; In step 1.2.3, the 64-dimensional local features and the 1024-dimensional global features are concatenated to obtain N*1088 data, which is then passed through a shared MLP to obtain an N*2-dimensional matrix representing the probability that each point belongs to a tree point or a non-tree point, completing the scene semantic segmentation and dividing the point cloud data into tree points and non-tree points.

4. The method for extracting a hierarchical single tree based on radius expansion according to claim 1, characterized in that: In step 1.3, the through-filter and statistical filter are used to remove noise points and outliers. Specifically, the through-filter is used to determine the point cloud data. axis, axis, The statistical filter calculates the average distance from each point to the adjacent points and compares it with the given mean and variance to eliminate noise points.

Images