Method for segmenting individual street trees based on MLS point cloud data

Abstract

The present invention proposes a method for segmenting individual street trees based on MLS point cloud data, including steps of obtaining street tree point clouds, clustering street tree point clouds, detecting adhered street trees, and coarsely and finely segmenting adhered street trees to complete the segmentation of individual street trees. The present invention uses a clustering algorithm to segment the street tree point clouds into street tree clusters, then detects adhered street trees in the street tree clusters, and then performs multi-level segmentation from coarse to fine on the street tree clusters containing multiple adhered street trees to obtain individual street tree point clouds; it has a high ability to identify street tree point clouds and the accuracy of segmenting individual street trees, and is applicable to complex urban environments containing various ground objects.

Description

Technical Field

[0001] The present invention relates to the technical field of single tree segmentation of street trees, especially single tree segmentation of street trees based on MLS point cloud data. Specifically, it is a method for segmenting single street trees by first identifying and then segmenting the point cloud data of street trees. Background Art

[0002] As an important part of the urban ecosystem and urban landscape, street trees have important ecological and social service functions. Conducting more in-depth research on street trees is an urgent task for urban management departments and the construction of digital cities. The mobile laser scanning (MLS) system, as an important means for collecting three-dimensional spatial information of the urban near scene, has the ability to quickly obtain the three-dimensional information of street trees on both sides of the road.

[0003] Single tree segmentation of street trees refers to identifying and segmenting individual street trees from MLS point cloud data. It is the primary step in the subsequent research of street trees. Existing methods are mainly divided into two categories:

[0004] Segmenting while identifying. According to the morphological features of street trees and other ground object targets, image / point cloud segmentation methods are used to gradually filter out non-street tree point clouds. Such methods mainly use prior knowledge to formulate relatively rough segmentation / identification features and rules, and have weak recognition ability for street trees, thus reducing the accuracy of single tree segmentation of street trees.

[0005] Identifying first and then segmenting. First, a street tree detector generated by a supervised learning algorithm is used to identify street tree point clouds from MLS point cloud data, and then algorithms such as clustering are used to segment the street tree point clouds into individual street trees. Such methods have high accuracy in identifying street trees, but the subsequent single tree segmentation algorithm for street trees, especially the segmentation algorithm for adhering street trees, needs further research. Summary of the Invention

[0006] The purpose of the present invention is to propose a method for segmenting single street trees by first identifying and then segmenting to solve the segmentation problem of adhering street trees. The method uses a clustering algorithm to segment the street tree point clouds into street tree clusters, then detects the adhering street trees in the street tree clusters, and finally performs a multi-level segmentation from coarse to fine on the street tree clusters containing multiple adhering street trees to obtain the point clouds of single street trees.

[0007] The technical solution of the present invention is as follows:

[0008] The present invention provides a method for segmenting single street trees based on MLS point cloud data. The method includes the following steps: S1. Step of obtaining street tree point clouds: Obtain street tree point cloud data through the MLS point cloud data of the street scanned by mobile laser scanning;

[0009] S2. Steps for clustering street tree point clouds: Use a clustering algorithm to cluster the street tree point clouds and segment them into street tree clusters;

[0010] S3. Steps for detecting adhered street trees: Extract the trunk point clouds and cluster the trunk point clouds for each street tree cluster to detect whether the street tree cluster contains multiple adhered street trees. For the adhered street tree cluster containing multiple adhered street trees, execute S4; for those not containing, complete the single-tree segmentation;

[0011] S4. Steps for rough segmentation of adhered street trees: Roughly segment the adhered street tree cluster into single street trees;

[0012] S5. Steps for fine segmentation of adhered street trees: Perform fine segmentation on the single street trees obtained from the rough segmentation in S4 to complete the single-tree segmentation.

[0013] Furthermore, the specific content of S1 is as follows: Use a street tree detector to identify the street tree point clouds from the MLS point cloud data of the street.

[0014] Furthermore, the clustering algorithm uses the Density-Based Spatial Clustering of Applications with Noise (DBSCAN).

[0015] Furthermore, the specific content of S2 is as follows:

[0016] Taking the (x, y, z) coordinates of the street tree point clouds as the input, use a clustering algorithm to cluster the street tree point clouds and segment them into street tree clusters; where the neighborhood radius ε1 = 0.5m; the minimum number of neighborhood points N1 = C1m1, m1 is the average number of neighborhood points of all street tree points when the neighborhood radius ε1 is 0.5m, and C1 is a constant with a value range of 0.02 - 0.1.

[0017] Furthermore, the specific content of S3 is as follows:

[0018] S3.1. Extract the trunks for each street tree cluster to obtain the minimum point cloud elevation z within the current street tree cluster min , and extract the point clouds within the cluster whose difference between the elevation value and the minimum elevation z min is less than the elevation difference threshold as the trunk point clouds;

[0019] S3.2. Taking the (x, y) coordinates of the trunk point clouds as the input, use a clustering algorithm to cluster the trunk point clouds and segment them into trunk clusters; where the neighborhood radius ε2 = 0.5m; the minimum number of neighborhood points N2 = 1;

[0020] S3.3. Count the number of segmented trunk clusters. If the number is 1, then the street tree cluster only contains a single street tree, and the single-tree segmentation is completed; if it is greater than 1, then the street tree cluster contains multiple adhered street trees, which is an adhered street tree cluster and needs to be further segmented into single street trees, and execute S4.

[0021] Further, the elevation difference threshold in S3.1 is 0.4m.

[0022] Further, S4 is specifically as follows: For each cluster of adhered street trees, the following processing is performed:

[0023] S4.1. Vertical slicing: Calculate the mean of the three-dimensional coordinates of each trunk cluster point cloud. Using the line connecting the aforementioned means of adjacent trunk clusters as the normal vector, perform a vertical slice with a thickness of d on the space between adjacent trunk clusters, where d = 0.01m;

[0024] S4.2. Vertical segmentation: Count the number of street tree points contained in the slice. Using the middle plane of the slice with the fewest points as the segmentation plane, vertically segment the adhered street trees into individual street trees and assign street tree labels.

[0025] Further, S5 is specifically as follows:

[0026] S5.1. Taking the (x, y, z) coordinates of the individual street tree point cloud as the input, use a clustering algorithm to cluster the individual street tree point cloud and segment it into sub-clusters; among them, the neighborhood radius ε3 = 0.2m; the minimum number of neighborhood points N3 = C3m3, where m3 is the average number of neighborhood points of all street tree points when the neighborhood radius ε3 is 0.2m, and C3 is a constant with a value of 0.2 - 0.5;

[0027] S5.2. Taking the sub-cluster with the largest number of points as the main part of the street tree, retain the street tree label assigned in S4.2, and cancel the labels of the remaining sub-clusters.

[0028] S5.3. Using the point cloud of all the assigned street tree labels in the cluster of adhered street trees as the training set, use a k-nearest neighbor classifier to assign labels to the unlabeled point cloud;

[0029] S5.4. Compare the labels of all the points in the cluster of adhered street trees with the labels assigned in S4.2. If they are the same, complete the individual tree segmentation; otherwise, return to S5.1 to perform iterative comparison; during the aforementioned iterative process, compare the labels of all the points in the cluster of adhered street trees with the labels assigned in the previous step of S5.3. If they are the same, complete the individual tree segmentation and end the iteration.

[0030] Further, the number of nearest neighbors k = 11.

[0031] The beneficial effects of the present invention:

[0032] A method for segmenting individual street trees based on MLS point cloud data according to the present invention uses a clustering algorithm to segment the street tree point cloud into street tree clusters, then detects the adhered street trees in the street tree clusters, and then performs multi-level segmentation from coarse to fine on the street tree clusters containing multiple adhered street trees to obtain individual street tree point clouds; it has a high ability to recognize street tree point clouds and the accuracy of segmenting individual street trees, and is applicable to complex urban environments containing various ground objects.

[0033] In the present invention, when segmenting multiple adhered street trees, first use trunk positioning for rough segmentation, and then combine DBSCAN and k-nearest neighbor classifiers for fine segmentation of the tree branch tips, effectively improving the segmentation accuracy.

[0034] Other features and advantages of the present invention will be described in detail in the following specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By describing the exemplary embodiments of the present invention in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present invention will become more obvious. Among them, in the exemplary embodiments of the present invention, the same reference numerals generally represent the same components.

[0036] Figure 1 Shows the flow chart of segmenting individual street trees according to the present invention.

[0037] Figure 2 Shows the schematic diagram of rough segmentation of adhered street trees according to the present invention.

[0038] Figure 3 Shows the schematic diagram of the street MLS point cloud data in the embodiment of the present invention.

[0039] Figure 4 Shows the schematic diagram of the detection and clustering results of street tree point clouds in the embodiment of the present invention.

[0040] Figure 5 Shows the schematic diagram of the rough segmentation results of adhered street trees in the embodiment of the present invention.

[0041] Figure 6 Shows the schematic diagram of the fine segmentation results of adhered street trees in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0042] The following will describe the preferred embodiments of the present invention in more detail with reference to the drawings. Although the preferred embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein.

[0043] As Figure 1 shown, a method for segmenting individual street trees based on MLS point cloud data, the method includes the following steps:

[0044] S1. Steps for obtaining street tree point clouds: Obtain street tree point clouds from the mobile laser scanning (MLS) point cloud data of the street. Specifically, use a street tree detector to identify street tree point clouds from the MLS point cloud data of the street.

[0045] Among them: The detection method of the street tree detector adopts the street tree target recognition method based on vehicle-mounted 2D LiDAR point cloud data, with the patent number ZL201810015134.9; or the street tree point cloud detection method in Weinmann M, Weinmann M, Mallet C, et al. A classification-segmentation framework for the detection of individual trees in dense MMS point cloud data acquired in urban areas [J]. Remote Sensing, 2017, 9(03):277.

[0046] S2. Steps for clustering street tree point clouds: Use a clustering algorithm to cluster the street tree point clouds and segment them into street tree clusters; the clustering algorithm adopts the density-based spatial clustering of applications with noise (DBSCAN). The specific steps of S2 are as follows:

[0047] Take the (x, y, z) coordinates of the street tree point clouds as the input, and use a clustering algorithm to cluster the street tree point clouds and segment them into street tree clusters; among them, the neighborhood radius ε1 = 0.5m; the minimum number of neighborhood points N1 = C1m1, where m1 is the average number of neighborhood points of all street tree points when the neighborhood radius ε1 is 0.5m, and C1 is a constant with a value range of 0.02 - 0.1.

[0048] S3. Steps for detecting adhered street trees: Extract the trunk point clouds and cluster the trunk point clouds for each street tree cluster to detect whether the street tree cluster contains multiple adhered street trees. For the adhered street tree clusters containing multiple adhered street trees, execute S4; for those that do not contain, complete the individual tree segmentation. The specific steps are as follows:

[0049] S3.1. Extract the trunks for each street tree cluster to obtain the minimum point cloud elevation z within the current street tree cluster min , and extract the point clouds within the cluster whose elevation difference from the minimum elevation z min is less than the elevation difference threshold as the trunk point clouds; among them: the elevation difference threshold is 0.4m;

[0050] S3.2. Taking the (x, y) coordinates of the trunk point cloud as input, use a clustering algorithm to cluster the trunk point cloud and segment it into trunk clusters; where the neighborhood radius ε2 = 0.5 m; the minimum number of neighborhood points N2 = 1;

[0051] S3.3. Count the number of segmented trunk clusters. If the number is 1, then this row of street tree clusters contains only a single street tree, and the single-tree segmentation is completed; if it is greater than 1, then this row of street tree clusters contains multiple adhered street trees, which is an adhered street tree cluster and needs to be further segmented into single street trees, and execute S4.

[0052] S4. Coarse segmentation of adhered street trees: Coarsely segment the adhered street tree cluster into single street trees; for each adhered street tree cluster, perform the following processing:

[0053] S4.1. Vertical slicing: Calculate the mean of the three-dimensional coordinates of each trunk cluster point cloud. Taking the line connecting the aforementioned means of adjacent trunk clusters as the normal vector, perform a vertical slice with a thickness of d on the space between adjacent trunk clusters, d = 0.01 m;

[0054] S4.2. Vertical segmentation: Count the number of street tree points contained in the slice. Taking the middle plane of the slice with the fewest points as the segmentation plane, vertically segment the adhered street trees into single street trees and assign street tree labels.

[0055] S5. Fine segmentation of adhered street trees: Perform a fine segmentation on the single street trees obtained from the coarse segmentation in S4, specifically:

[0056] S5.1. Taking the (x, y, z) coordinates of the single street tree point cloud as input, use a clustering algorithm to cluster the single street tree point cloud and segment it into sub-clusters; where the neighborhood radius ε3 = 0.2 m; the minimum number of neighborhood points N3 = C3m3, m3 is the average neighborhood number of all street tree points when the neighborhood radius ε3 is 0.2 m, and C3 is a constant with a value of 0.2 - 0.5;

[0057] S5.2. Take the sub-cluster with the largest number of points as the main part of this street tree, retain the street tree label assigned in S4.2, and cancel the labels of the remaining sub-clusters.

[0058] S5.3. Taking all the point clouds with assigned street tree labels in the adhered street tree cluster as the training set, use a k-nearest neighbor classifier to assign labels to the unlabeled point clouds, and the number of nearest neighbors k = 11;

[0059] S5.4. Compare the labels of all points in the adhered street tree cluster with the labels assigned in S4.2. If they are the same, the single-tree segmentation is completed; otherwise, return to S5.1 to perform iterative comparison; during the aforementioned iterative process, compare the labels of all points in the adhered street tree cluster with the labels assigned in the previous step of S5.3. If they are the same, the single-tree segmentation is completed and the iteration ends.

[0060] When implementing:

[0061] The experiment uses a ZEB-HORIZON mobile handheld scanner to collect point cloud data of a section of road at 32°04'55.1"N and 118°48'58.8"E. Figure 3 As shown, the point cloud is colored using the z coordinate for easy viewing. The main tree species on this road include Cerasus yedoensis, Ginkgo biloba, Celtissinensis, Cinnamomum camphora, etc., with a height range of 4.6-8.2m and a canopy range of 2.1-7.4m. In addition, there are also buildings, lanes, sidewalks, benches, street lights, bicycles, signs, pedestrians, cars, shrubs, grass, flower beds and other features.

[0062] like Figure 3 As shown in the figure, it is the street MLS point cloud data; perform S2 street tree point cloud clustering and S3 adhesion street tree detection, and get the following Figure 4 As shown in Figure 1, the detection and clustering results of the street tree point cloud are shown. There are several clusters of adhered street trees. S4 coarse segmentation of adhered street trees and S5 fine segmentation of adhered street trees are performed. Figure 5 As shown in the figure, it is the rough segmentation result of the adhered roadside trees. There are a few erroneous segmentations at the tips of the branches of the adjacent roadside trees. Figure 6 As shown, this is the result of fine segmentation of the adjoining roadside trees, and the branch tips of the adjacent roadside trees are accurately segmented.

[0063] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for segmenting individual street trees based on MLS point cloud data, characterized in that, The method includes the following steps: S1. Step of obtaining street tree point cloud: Obtain the street tree point cloud data from the MLS point cloud data of the street by mobile laser scanning; S2. Step of clustering street tree point cloud: Use a clustering algorithm to cluster the street tree point cloud and segment it into street tree clusters; S3. Step of detecting adhered street trees: Extract the trunk point cloud and cluster the trunk point cloud for each street tree cluster to detect whether the street tree cluster contains multiple adhered street trees. For the adhered street tree cluster containing multiple adhered street trees, execute S4. For those not containing, complete the individual tree segmentation; S4. Step of roughly segmenting adhered street trees: Roughly segment the adhered street tree cluster into individual street trees; S5. Step of finely segmenting adhered street trees: Finely segment the individual street trees obtained by the rough segmentation in S4 to complete the individual tree segmentation; The specific content of S4 is as follows: For each adhered street tree cluster, perform the following processing: S4.

1. Vertical slicing: Calculate the mean of the three-dimensional coordinates of each trunk cluster point cloud. Take the line connecting the aforementioned means of adjacent trunk clusters as the normal vector, and perform vertical slicing with a thickness of d on the space between adjacent trunk clusters, where d = 0.01 m; S4.

2. Vertical segmentation: Count the number of street tree points contained in the slice. Take the middle plane of the slice with the least number of points as the segmentation plane, vertically segment the adhered street trees into individual street trees, and assign street tree labels; The specific content of S5 is as follows: S5.

1. Take the (x, y, z) coordinates of the individual street tree point cloud as the input, use a clustering algorithm to cluster the individual street tree point cloud, and segment it into sub-clusters; where the neighborhood radius ε3 = 0.2 m; the minimum number of neighborhood points N3 = C3m3, m3 is the average number of neighborhood points of all street tree points when the neighborhood radius ε3 is 0.2 m, and C3 is a constant with a value of 0.2 - 0.5; S5.

2. Take the sub-cluster with the largest number of points as the main part of the street tree, retain the street tree label assigned in S4.2, and cancel the labels of the remaining sub-clusters; S5.

3. Take all the point clouds with assigned street tree labels in the adhered street tree cluster as the training set, and use a k-nearest neighbor classifier to assign labels to the unlabeled point clouds; S5.

4. Compare the labels of all points in the adhered street tree cluster with the labels assigned in S4.

2. If they are the same, complete the individual tree segmentation. Otherwise, return to S5.1 to perform iterative comparison; during the aforementioned iterative process, compare the labels of all points in the adhered street tree cluster with the labels assigned in the previous step of S5.

3. If they are the same, complete the individual tree segmentation and the iteration ends.

2. The method for segmenting individual street trees based on MLS point cloud data according to claim 1, characterized in that, The specific content of S1 is as follows: Use a street tree detector to identify the street tree point cloud from the MLS point cloud data of the street.

3. The method for segmenting individual street trees based on MLS point cloud data according to claim 1, characterized in that, The clustering algorithm uses the density-based spatial clustering of applications with noise algorithm DBSCAN.

4. The method for segmenting individual street trees based on MLS point cloud data according to claim 1, characterized in that, The specific content of S2 is as follows: Take the (x, y, z) coordinates of the street tree point cloud as the input, use a clustering algorithm to cluster the street tree point cloud, and segment it into street tree clusters; where the neighborhood radius ε1 = 0.5 m; the minimum number of neighborhood points N1 = C1m1, m1 is the average number of neighborhood points of all street tree points when the neighborhood radius ε1 is 0.5 m, and C1 is a constant with a value of 0.02 - 0.

1.

5. The method for segmenting individual street trees based on MLS point cloud data according to claim 1, characterized in that, The specific content of S3 is as follows: S3.

1. Extract the tree trunks for each street tree cluster to obtain the minimum point cloud elevation value z within the current street tree cluster min , and extract the point cloud with the difference between the elevation value within the cluster and the minimum elevation value z min less than the elevation difference threshold as the trunk point cloud; S3.

2. Taking the (x, y) coordinates of the trunk point cloud as the input, use a clustering algorithm to cluster the trunk point cloud and divide it into trunk clusters; among them, the neighborhood radius ε2 = 0.5 m; the minimum number of neighborhood points N2 = 1; S3.

3. Count the number of trunk clusters segmented. If the number is 1, it means that the street tree cluster only contains a single street tree, and the single-tree segmentation is completed. If it is greater than 1, it means that the street tree cluster contains multiple adhered street trees, which is an adhered street tree cluster and needs to be further segmented into single street trees, and execute S4.

6. The method for segmenting individual street trees based on MLS point cloud data according to claim 5, characterized in that, In the above S3.1, the elevation difference threshold is 0.4 m.

7. The method for segmenting individual street trees based on MLS point cloud data according to claim 1, characterized in that, The number of nearest neighbors k = 11.

Images