A Mango Tree Point Cloud Instance Segmentation System Based on UAV LiDAR Scanning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA AGRICULTURAL UNIVERSITY
- Filing Date
- 2022-12-29
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technology makes it difficult to accurately separate low-growing mango trees, especially in mango groves where trees are intertwined, making it impossible to effectively separate individual mango trees.
Point cloud data of mango orchards was acquired by UAV LiDAR scanning. Ground points and non-ground points were separated by CSF filtering algorithm, and non-ground points were clustered by DBSCAN algorithm. Individual trees were filtered by threshold and density projection method, and multiple trees were further segmented by K-means algorithm. A relative density projection method was designed based on the characteristics of mango tree canopy to achieve accurate segmentation.
It efficiently separates single and multiple mango trees in a mango grove without requiring uniform tree size or tree counting, solving the problem of separating trees that are stuck together, and is suitable for low-growing mango shrubs.
Smart Images
Figure CN115908820B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a mango tree point cloud instance segmentation system based on UAV lidar scanning. Background Technology
[0002] Existing technology uses mobile laser scanning to acquire point cloud data of urban street trees, then uses morphological filtering to separate ground points from non-ground points. Points larger than 2 meters above the ground are removed to preserve the tree stems. The tree position is determined by the tree stems. Next, the non-ground points are voxelized, and the connectivity of adjacent voxels is analyzed to extract initial tree points. Finally, the shortest path method is used to segment the initial tree points and extract individual trees. Existing technology can only extract tree species with straight and long trunks, and street trees are sparsely planted, generally only two rows. For low-growing mango orchards, since it is impossible to collect tree trunks, this method is not suitable for segmenting mango trees in orchards. Summary of the Invention
[0003] The purpose of this invention is to provide a mango tree point cloud instance segmentation system based on UAV LiDAR scanning. This method can accurately segment individual mango trees and output the number of mango trees in a mango forest, addressing the problem of mango trees sticking together in the forest.
[0004] This invention provides a mango tree point cloud instance segmentation system based on UAV LiDAR scanning, comprising:
[0005] Obtain point cloud data of mango forest;
[0006] The mango forest point cloud data was separated into non-location data and ground point data;
[0007] The non-ground point data is segmented into multiple mango tree data clusters;
[0008] Determine whether each mango tree data cluster is a single tree or multiple trees. If it is a single tree, output 1 directly. If it is multiple trees, determine how many trees it is and then output the number of trees.
[0009] The sum of the judgment results for each mango tree data cluster is output as the total number of mango trees in the mango forest.
[0010] Obtaining point cloud data for mango forests specifically includes:
[0011] Three-dimensional point cloud data of the mango forest was obtained by airborne lidar. The lidar point cloud of the mango forest consists of a series of three-dimensional point coordinates (x, y, z).
[0012] Separating the mango forest point cloud data into non-location data and ground point data specifically includes:
[0013] Ground point data and non-ground point data in the lidar point cloud of a mango orchard were separated using the CSF filtering algorithm:
[0014] Flip the lidar point cloud of the mango orchard;
[0015] By covering the flipped surface with rigid cloth, and analyzing the interaction between the cloth nodes and the corresponding lidar points of the mango forest, the location of the cloth nodes can be determined to generate an approximate surface shape.
[0016] By comparing the distance between the lidar points of the original mango forest and the generated cloth surface, ground point data and non-ground point data can be extracted from the lidar point cloud of the mango forest.
[0017] The specific steps of segmenting non-ground point data into multiple mango tree data clusters include:
[0018] The separated non-location data is clustered using the DBSCAN algorithm. A point is randomly selected, and all points whose distance to this point is less than or equal to the search radius eps are found.
[0019] If the number of data points within eps of the starting point is less than the minimum number of points threshold min_points, then this point is marked as noise. If the number of data points within eps is greater than min_points, then this point is marked as a core sample and assigned a new Mango Tree data cluster label.
[0020] Visit all points within the neighborhood of the given point eps. If they have not yet been assigned a Mango Tree data cluster, assign the newly created Mango Tree data cluster label to them. If they are core samples, visit their neighborhoods in turn until there are no more core samples within the distance of the Mango Tree data cluster eps. Finally, output the Mango Tree data cluster.
[0021] Determine whether each mango tree data cluster is a single tree or multiple trees. If it is a single tree, output 1 directly. Specifically, this includes:
[0022] The DBSCAN clustering results are filtered to select qualified single trees. The filtering condition is: if the span of the mango tree data cluster is less than 5 meters on both the X-axis and Y-axis, it is considered a qualified single tree, and the single tree result is output as 1.
[0023] Determine whether each mango tree data cluster is a single tree or multiple trees. If it is multiple trees, determine the number of trees and output the total number of trees. Specifically, this includes:
[0024] Based on the characteristic of the mango tree canopy protruding in the middle, the data clusters of multiple mango trees are projected onto the XOY plane. The KDTree algorithm is used to calculate the number of points in the radius neighborhood of each point, which is denoted as the density of that point. The radius is set to 1 meter. Based on the density difference, low-density points are filtered out, and the threshold is set to 1440, keeping only high-density points.
[0025] The high-density points filtered out have increased tree spacing compared to the unfiltered points. The DBSCAN algorithm is then used for clustering. The number of clusters obtained from the DBSCAN clustering results is the count of multiple trees in the mango tree data cluster. The output is the number of multiple trees.
[0026] Determine whether each mango tree data cluster is a single tree or multiple trees. If it is multiple trees, determine the number of trees and output the number of trees in the multiple-tree cluster. This can also include further segmentation of the multiple-tree mango tree data clusters using the K-means clustering algorithm.
[0027] The counts of multiple trees in the mango tree data cluster obtained from the DBSCAN clustering results are passed into the Kmeans algorithm, and the counts are selected as the initial centroids.
[0028] Calculate the distance from each sample to each centroid in a multi-tree mango tree data cluster, assign each sample to the mango tree data cluster corresponding to the nearest centroid, and finally divide a mango tree data cluster into multiple mango tree data clusters.
[0029] The K-means algorithm is iteratively trained until the following conditions are met: the change in the position of the centroid is less than a specified threshold (default is 0.0001), or the maximum number of iterations is reached.
[0030] This invention acquires point cloud data through airborne lidar scanning, which is more efficient than ground-based mobile lidar scanning. A bag filter algorithm is used to separate non-ground points from the point cloud data. The DBSCAN algorithm is then used to cluster mango tree data clusters from these non-ground points. A threshold-based method is used to filter out individual trees. Considering the characteristics of the mango tree canopy, a relative density projection method is used to filter out high-density points from the non-individual trees. The DBSCAN algorithm is used again on these high-density points to obtain the tree counts within the mango tree data clusters. These counts are then fed into a K-means algorithm for secondary segmentation of the multi-tree mango tree data clusters, ultimately yielding the segmentation results for all trees. Based on the shape characteristics of the mango tree canopy, a relative density projection method is designed, combining the advantages of two clustering algorithms, resulting in better adaptability to low-growing mango shrubs. Compared to using only the DBSCAN algorithm, this invention solves the problem of segmenting interconnected trees. Compared to existing technologies, this invention segments individual trees without requiring uniform tree size or knowledge of tree counts. Attached Figure Description
[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a flowchart of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention;
[0034] Figure 2 This is the original point cloud image of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention;
[0035] Figure 3 This is a schematic diagram of the CSF algorithm for a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention.
[0036] Figure 4 This invention presents a segmentation system for mango tree point cloud instances based on UAV lidar scanning, which separates non-location and ground point maps.
[0037] Figure 5 This is a DBSCAN clustering result diagram of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention.
[0038] Figure 6 This invention provides a method for filtering out individual tree images from a point cloud instance segmentation system for mango trees based on UAV lidar scanning, as proposed in this invention.
[0039] Figure 7 This is a density diagram of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention.
[0040] Figure 8 The images show the DBSCAN and Kmeans clustering results of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention.
[0041] Figure 9 This is the final extraction result of a mango tree point cloud instance segmentation system based on UAV lidar scanning proposed in this invention. Detailed Implementation
[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0043] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.
[0044] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.
[0045] This invention provides a mango tree point cloud instance segmentation system based on UAV LiDAR scanning, comprising:
[0046] S100, acquire point cloud data of mango forest;
[0047] S200 separates the mango forest point cloud data into non-location data and ground point data;
[0048] S300 divides non-ground point data into multiple mango tree data clusters;
[0049] S400: Determine whether each mango tree data cluster is a single tree or multiple trees. If it is a single tree, output 1 directly. If it is multiple trees, determine how many trees it is and then output the number of trees.
[0050] S500 sums up the judgment results of each mango tree data cluster and outputs the total number of mango trees in the mango forest.
[0051] This invention acquires point cloud data through airborne lidar scanning, which is more efficient than ground-based mobile lidar scanning. A bag filter algorithm is used to separate non-ground points from the point cloud data. The DBSCAN algorithm is then used to cluster mango tree data clusters from these non-ground points. A threshold-based method is used to filter out individual trees. Considering the characteristics of the mango tree canopy, a relative density projection method is used to filter out high-density points from the non-individual trees. The DBSCAN algorithm is used again on these high-density points to obtain the tree counts within the mango tree data clusters. These counts are then fed into a K-means algorithm for secondary segmentation of the multi-tree mango tree data clusters, ultimately yielding the segmentation results for all trees. Based on the shape characteristics of the mango tree canopy, a relative density projection method is designed, combining the advantages of two clustering algorithms, resulting in better adaptability to low-growing mango shrubs. Compared to using only the DBSCAN algorithm, this invention solves the problem of segmenting interconnected trees. Compared to existing technologies, this invention segments individual trees without requiring uniform tree size or knowledge of tree counts.
[0052] S100, acquiring point cloud data of the mango forest specifically includes:
[0053] Three-dimensional point cloud data of the mango forest was obtained by airborne lidar. The lidar point cloud of the mango forest consists of a series of three-dimensional point coordinates (x, y, z).
[0054] For easier demonstration, setting a color gradient based on height (z value) will make it easier to understand. Figure 2 .
[0055] S200 separates mango forest point cloud data into non-location data and ground point data, specifically including:
[0056] Ground point data and non-ground point data in the lidar point cloud of a mango orchard were separated using the CSF filtering algorithm:
[0057] Flip the lidar point cloud of the mango orchard;
[0058] By covering the flipped surface with rigid cloth, and analyzing the interaction between the cloth nodes and the corresponding lidar points of the mango forest, the location of the cloth nodes can be determined to generate an approximate surface shape.
[0059] By comparing the distance between the lidar points of the original mango forest and the generated cloth surface, ground point data and non-ground point data can be extracted from the lidar point cloud of the mango forest.
[0060] CSF, based on cloth simulation, is a computer graphics algorithm used to simulate cloth in computer programs. Figure 3 .
[0061] S300 divides non-ground point data into multiple mango tree data clusters, specifically including:
[0062] The separated non-location data is clustered using the DBSCAN algorithm. A point is randomly selected, and all points whose distance to this point is less than or equal to the search radius eps are found.
[0063] If the number of data points within eps of the starting point is less than the minimum number of points threshold min_points, then this point is marked as noise. If the number of data points within eps is greater than min_points, then this point is marked as a core sample and assigned a new Mango Tree data cluster label.
[0064] Visit all points within the neighborhood of the given point eps. If they have not yet been assigned a Mango Tree data cluster, assign the newly created Mango Tree data cluster label to them. If they are core samples, visit their neighborhoods in turn until there are no more core samples within the distance of the Mango Tree data cluster eps. Finally, output the Mango Tree data cluster.
[0065] The DBSCAN clustering results of this invention are orthophotos Figure 5 As shown, different mango tree data clusters are marked with different colors, and eps and min_points are set to 0.3 (meters) and 35 respectively, resulting in a total of 61 mango tree data clusters.
[0066] S400 checks each mango tree data cluster to determine whether it is a single tree or multiple trees. If it is a single tree, it outputs 1 directly. Specifically, this includes:
[0067] The DBSCAN clustering results are filtered to select qualified single trees. The filtering condition is: if the span of the mango tree data cluster is less than 5 meters on both the X-axis and Y-axis, it is considered a qualified single tree, and the single tree result is output as 1.
[0068] The filtered individual trees are like Figure 6 As shown.
[0069] S400 determines each mango tree data cluster individually, whether it is a single tree or multiple trees. If it is multiple trees, it determines the number of trees and outputs the total number of trees. Specifically, this includes:
[0070] Based on the characteristic of the mango tree canopy protruding in the middle, the data clusters of multiple mango trees are projected onto the XOY plane. The KDTree algorithm is used to calculate the number of points in the radius neighborhood of each point, which is denoted as the density of that point. The radius is set to 1 meter. Based on the density difference, low-density points are filtered out, and the threshold is set to 1440, keeping only high-density points.
[0071] The high-density points filtered out have increased tree spacing compared to the unfiltered points. The DBSCAN algorithm is then used for clustering. The number of clusters obtained from the DBSCAN clustering results is the count of multiple trees in the mango tree data cluster. The output is the number of multiple trees.
[0072] S400 determines whether each mango tree data cluster is a single tree or multiple trees. If it is multiple trees, it determines the number of trees and outputs the number of trees in the multiple-tree cluster. S410 uses the K-means clustering algorithm to further segment the multiple-tree mango tree data clusters.
[0073] S411, take the count of multiple trees in the mango tree data cluster obtained from the DBSCAN clustering results and pass it into the Kmeans algorithm, and select the count as the initial centroid;
[0074] S412, calculate the distance from each sample to each centroid in the multi-tree mango tree data cluster, divide each sample into the mango tree data cluster corresponding to the nearest centroid, and finally divide a mango tree data cluster into multiple mango tree data clusters.
[0075] DBSCAN clustering results and Kmeans clustering results are as follows: Figure 8 As shown.
[0076] Finally, this invention concatenates the K-means clustering results with the DBSCAN clustering results to obtain the final clustering result, such as... Figure 9 As shown, there are a total of 123 trees, and 121 trees were correctly extracted.
[0077] This invention acquires point cloud data through airborne lidar scanning, which is more efficient than ground-based mobile lidar scanning. A bag filter algorithm is used to separate non-ground points from the point cloud data. The DBSCAN algorithm is then used to cluster mango tree data clusters from these non-ground points. A threshold-based method is used to filter out individual trees. Considering the characteristics of the mango tree canopy, a relative density projection method is used to filter out high-density points from the non-individual trees. The DBSCAN algorithm is used again on these high-density points to obtain the tree counts within the mango tree data clusters. These counts are then fed into a K-means algorithm for secondary segmentation of the multi-tree mango tree data clusters, ultimately yielding the segmentation results for all trees. Based on the shape characteristics of the mango tree canopy, a relative density projection method is designed, combining the advantages of two clustering algorithms, resulting in better adaptability to low-growing mango shrubs. Compared to using only the DBSCAN algorithm, this invention solves the problem of segmenting interconnected trees. Compared to existing technologies, this invention segments individual trees without requiring uniform tree size or knowledge of tree counts.
[0078] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A mango tree point cloud instance segmentation system based on unmanned aerial vehicle lidar scanning, characterized in that, include: Obtain point cloud data of mango forest; The mango forest point cloud data was separated into non-location data and ground point data; The non-ground point data is segmented into multiple mango tree data clusters; The separation of mango forest point cloud data into non-location data and ground point data specifically includes: Ground point data and non-ground point data in the lidar point cloud of a mango orchard were separated using the CSF filtering algorithm: Flip the lidar point cloud of the mango orchard; The surface after the flipping is covered with rigid cloth. The location of the cloth nodes is determined by analyzing the interaction between the cloth nodes and the corresponding lidar points of the mango forest to generate an approximate surface shape. By comparing the distance between the lidar points of the original mango forest and the generated cloth surface, ground point data and non-ground point data can be extracted from the lidar point cloud of the mango forest. Determine whether each mango tree data cluster is a single tree or multiple trees. If it is a single tree, output 1 directly. If it is multiple trees, determine how many trees it is and then output the number of trees. The process of determining whether each mango tree data cluster is a single tree or multiple trees, and if it is multiple trees, determining the number of trees and then outputting the number of trees, specifically includes: Based on the characteristic of the mango tree canopy protruding in the middle, the data clusters of multiple mango trees are projected onto the XOY plane. The KDTree algorithm is used to calculate the number of points in the radius neighborhood of each point, which is denoted as the density of that point. The radius is set to 1 meter. Based on the density difference, low-density points are filtered out, and the threshold is set to 1440, keeping only high-density points. The high-density points filtered out have increased tree spacing compared to the unfiltered points. The DBSCAN algorithm is then used for clustering. The number of clusters obtained from the DBSCAN clustering results is the count of multiple trees in the mango tree data cluster. The output is the number of multiple trees.
2. The mango tree point cloud instance segmentation system based on unmanned aerial vehicle laser radar scanning according to claim 1, characterized in that, The acquisition of mango forest point cloud data specifically includes: The mango orchard's three-dimensional point cloud data was obtained using an airborne lidar. The lidar point cloud of the mango orchard consists of a series of three-dimensional point coordinates (x, y, z).
3. The mango tree point cloud instance segmentation system based on unmanned aerial vehicle laser radar scanning according to claim 1, characterized in that, The step of segmenting non-ground point data into multiple mango tree data clusters specifically includes: The separated non-location data is clustered using the DBSCAN algorithm. A point is randomly selected, and all points whose distance to this point is less than or equal to the search radius eps are found. If the number of data points within eps of the starting point is less than the minimum number of points threshold min_points, then this point is marked as noise. If the number of data points within eps is greater than min_points, then this point is marked as a core sample and assigned a new Mango Tree data cluster label. Visit all points within the neighborhood of the given point eps. If they have not yet been assigned a Mango Tree data cluster, assign the newly created Mango Tree data cluster label to them. If they are core samples, visit their neighborhoods in turn until there are no more core samples within the distance of the Mango Tree data cluster eps. Finally, output the Mango Tree data cluster.
4. The mango tree point cloud instance segmentation system based on UAV lidar scanning according to claim 1, characterized in that, The step of determining whether each mango tree data cluster is a single tree or multiple trees, and outputting 1 directly if it is a single tree, specifically includes: The DBSCAN clustering results are filtered to select qualified single trees. The filtering condition is: if the span of the mango tree data cluster is less than 5 meters on both the X-axis and Y-axis, it is considered a qualified single tree, and the single tree result is output as 1.
5. A mango tree point cloud instance segmentation system based on UAV lidar scanning according to claim 4, characterized in that, The step of determining whether each mango tree data cluster is a single tree or multiple trees, and if it is multiple trees, determining the number of trees and outputting the number of trees, also includes using the K-means clustering algorithm to further segment the multiple-tree mango tree data clusters: The counts of multiple trees in the mango tree data cluster obtained from the DBSCAN clustering results are passed into the Kmeans algorithm, and the counts are selected as the initial centroids. Calculate the distance from each sample to each centroid in a multi-tree mango tree data cluster, assign each sample to the mango tree data cluster corresponding to the nearest centroid, and finally divide a mango tree data cluster into multiple mango tree data clusters. The K-means algorithm is iteratively trained until the following conditions are met: the change in the position of the centroid is less than a specified threshold (default is 0.0001), or the maximum number of iterations is reached.