Tree Recognition Method and System Based on Air-Ground Integrated 3D Laser Point Cloud

Abstract

The present invention proposes a tree recognition method and system based on ground-air integrated three-dimensional lidar point cloud, belonging to the field of environmental remote sensing data recognition, including: using an unmanned aerial vehicle to obtain lidar point cloud data within a set forest area, and using a laser scanning device to collect point cloud data within the set forest area; preprocessing the two sets of data to obtain preprocessed lidar point cloud, performing seamless integration of ground-air data on the preprocessed lidar point cloud to obtain fused data; generating a canopy height model based on the watershed segmentation algorithm, and using this model to perform single-tree segmentation on the fused point cloud data; extracting the vector boundaries after single-tree segmentation, assigning a tree ID to each tree, and extracting the single-tree point cloud according to the tree ID; further processing the single-tree point cloud to construct a multi-angle point cloud sample data set; using the multi-angle point cloud sample data set to train a deep learning network model, and using the trained deep learning network model to detect the tree species to be measured.

Description

Technical Field

[0001] The present invention belongs to the field of environmental remote sensing data recognition, and particularly relates to a tree recognition method and system based on ground-air integrated three-dimensional laser point cloud. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] With the intensification of human activities and the impact of global climate change, forest species diversity faces the severe challenge of continuous reduction. At the same time, the existing biodiversity monitoring network is not yet perfect. Therefore, it is particularly important to quickly and accurately monitor large-scale forest resources. Tree species investigation is an important part of forest resource investigation. At present, traditional tree species investigations in China usually require setting up sample plots, which have problems such as high field work intensity, high cost, and long cycle. With the development of a series of emerging technologies such as remote sensing technology and unmanned aerial vehicle technology, the application of lidar LiDAR and unmanned aerial vehicle aerial survey in the forestry field is becoming increasingly widespread, opening up new ways to improve the efficiency and accuracy of forest resource investigation.

[0004] As an advanced active remote sensing technology, lidar measures the distance to the target object accurately by emitting laser beams, thereby obtaining rich spatial information. Compared with traditional optical passive remote sensing, lidar LiDAR data can provide three-dimensional information of the target ground object and accurately extract the vertical information of the forest stand, which has advantages that traditional optical remote sensing cannot match in the forestry application field. Ground-based, backpack and handheld lidars usually operate on the ground under the forest and can obtain the single-tree scale structure parameters inside the forest stand. Among them, handheld lidar can obtain data in a mobile manner, with a flexible operation mode and higher data quality. In recent years, remote sensing sensors have developed towards diversification and lightweight, and the progress of unmanned aerial vehicle technology has brought new opportunities to the remote sensing field. Compared with other remote sensing platforms, unmanned aerial vehicles have low usage costs, flexible takeoff and landing, low flight altitudes, and are more likely to obtain high-precision remote sensing data, making up for the deficiencies in spatial resolution of manned airborne or satellite-borne images.

[0005] Although LiDAR data can obtain the spatial structure information of trees, since the laser light source is in the lower layer of the forest during the collection of ground-based LiDAR data, the occlusion of leaves, branches, etc. and the limitation of the ground three-dimensional laser scanning angle will make it unfavorable for the extraction of the top canopy information by ground-based lidar. Especially in forest stands with a higher canopy density, it will lead to the lack of top canopy spatial structure data, which has an adverse impact on the extraction of tree parameters. Using photography technology can obtain forest canopy information, but problems such as being unable to photograph the front and back landscapes under the forest canopy are likely to occur.

[0006] Traditional tree species identification can be divided into manual discrimination and machine learning discrimination. Manual discrimination mainly relies on visual interpretation based on external morphological characteristics such as the shape of tree leaves and bark texture. It has high recognition accuracy, but high cost and low efficiency. Machine learning-based discrimination mines the hidden laws of macroscopic features to establish a model of tree species and feature information. The evaluation results are objective and the recognition speed is fast, but the model is greatly affected by parameter settings and feature selection. Algorithms based on traditional machine learning are difficult to effectively extract key features manually. Deep learning networks can automatically and intelligently mine the connections between deep features in data and continuously adjust and optimize according to samples, liberating human labor, improving the recognition accuracy, and increasing the universality of the solution. It is widely used in single-tree segmentation and tree species identification research.

[0007] In terms of constructing a point cloud dataset, traditional point cloud processing methods mostly rely on the direct analysis of 3D models. However, in practical applications, the processing complexity of 3D data is relatively high and the consumption of computing resources is huge.

[0008] With the development of technology, there are technical solutions in the existing technology that use the fusion processing of LiDAR data and images collected by drones and use deep learning networks for recognition. For example:

[0009] Li Yiying et al. used the three-dimensional laser point cloud data of ground-air fusion to extract structural features and input them into a deep learning model for tree species identification and classification through feature integration, and further obtained the change of crown width. Zhu Ruoning et al. proposed an identification method based on the ITPNet network model, using hyperspectral images and lidar data to meet the needs of single-tree identification. Li Dan et al. used the radar point cloud data of ground-air fusion for instance segmentation and biomass inversion through a 3D segmentation framework.

[0010] However, the problems still existing in the above technical solutions are: there is a problem of low efficiency in directly identifying trees based on point clouds, and the annotation process of 3D samples is also relatively difficult. In addition, it is usually necessary to perform multi-modal data fusion of point clouds and other sensor data, resulting in a significant increase in the data volume and further increasing the processing complexity. The overlapping and crossing phenomena of trees in the point cloud lead to unsatisfactory individual discrimination effects. At the same time, in the process of ground object semantic recognition, simply relying on the two-dimensional or multi-angle feature information of the point cloud fails to fully mine and utilize the deep features of the point cloud data, thus affecting the recognition accuracy and effect. Summary of the Invention

[0011] To overcome the above deficiencies of the prior art, the present invention provides a tree identification method based on ground-air fusion three-dimensional laser point cloud, which can realize the intelligent identification of tree species semantics (qualitative) at the single-tree scale and the accurate extraction of structural parameters (quantitative) by using the three-dimensional laser point cloud of ground-based - air-based fusion.

[0012] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0013] In the first aspect, a tree recognition method based on ground-air integrated three-dimensional lidar point cloud is disclosed, including:

[0014] Using a drone to obtain lidar point cloud data within a set forest area, and using a laser scanning device to collect point cloud data within the set forest area;

[0015] Preprocessing the two sets of data to obtain preprocessed lidar point cloud, and performing seamless integration of ground-air data on the preprocessed lidar point cloud to obtain integrated data;

[0016] Generating a canopy height model based on the watershed segmentation algorithm, and using this model to perform single-tree segmentation on the integrated point cloud data;

[0017] Extracting the vector boundary after single-tree segmentation, assigning a tree ID to each tree, and extracting the single-tree point cloud based on the tree ID;

[0018] Further processing the single-tree point cloud to construct a multi-angle point cloud sample data set;

[0019] Training a deep learning network model using the multi-angle point cloud sample data set, and using the trained deep learning network model to detect the tree species to be measured.

[0020] As a further technical solution, preprocessing the two sets of data includes denoising, normalizing, removing ground points, and clipping the point cloud to obtain preprocessed lidar point cloud.

[0021] As a further technical solution, when performing seamless integration of ground-air data on the preprocessed lidar point cloud, the iterative closest point algorithm is used, and this algorithm performs seamless integration of ground-air data based on the position and feature matching method.

[0022] As a further technical solution, when performing single-tree segmentation, a variable-sized dynamic window is used to search for local maxima on the canopy height model as the canopy vertices, and using the canopy vertices as markers, the marker-controlled watershed algorithm is used to outline the canopy boundary, and further extract tree height, diameter at breast height, and crown width parameters.

[0023] As a further technical solution, CHM canopy height model = DSM digital surface model - DEM digital elevation model.

[0024] As a further technical solution, the extracted single-tree point cloud is first calibrated for the vertical axis of the tree trunk, and on the basis of hierarchical clustering, the tree trunk detection is completed through a multi-layer merging method.

[0025] Secondly, observe the tree trunk vertically around its axis to form point cloud images at multiple angles, and combine them with the ground measured survey data to construct a single tree image dataset for multi-angle observation.

[0026] As a further technical solution, based on hierarchical clustering, the trunk detection is completed through a multi-layer merging method. The specific process is as follows:

[0027] Divide the tree point cloud P into n layers from bottom to top at a certain distance, and use bottom-up layer-by-layer point cloud clustering to extract the trunk position;

[0028] After obtaining the point cloud Pi of each layer, use the DBSCAN algorithm to perform layer-by-layer clustering to obtain a set, and merge the clustering results of each layer.

[0029] As a further technical solution, the merging process of the clustering results of each layer specifically includes:

[0030] 1) Calculate the centroid and area information of the convex hull formed by the clustering point sets of each layer;

[0031] 2) Take any clustering unit in the first-layer clustering set obtained by hierarchical clustering as each growth starting unit, and calculate the distance between the centroid of the convex hull of each clustering unit and the centroid of the convex hull of the upper layer, as well as the area of the convex hull formed by the projection of each clustering on the horizontal plane starting from this unit;

[0032] 3) Judge whether two clusters belong to the same ground object according to the distance and area of two hierarchical point clusters;

[0033] 3) Repeat the above steps until all layers are traversed, that is, i = n, to obtain the trunk point cloud of a single tree.

[0034] As a further technical solution, when dividing layers, it should be ensured that the number of point clouds of each trunk segment is greater than T minPts , and the segmentation distance should satisfy:

[0035]

[0036] In the formula: T den is the average point cloud density at the trunk; r is the average breast diameter; T minPts is determined according to the point cloud density of the trunk.

[0037] As a further technical solution, the deep learning network model YOLO v11-cls, YOLO v11-cls adopts an improved Backbone and Neck architecture, and introduces C3k2 and C2PSA components;

[0038] The C3k2 component is used for multi-scale feature fusion and can effectively capture the subtle features of trees.

[0039] In a second aspect, a tree recognition system based on ground-air integrated three-dimensional lidar point cloud is disclosed, including:

[0040] A point cloud data acquisition module, configured to: use a drone to acquire lidar point cloud data within a set forest area, and use a laser scanning device to collect point cloud data within the set forest area;

[0041] A fusion module, configured to: preprocess the two sets of data to obtain preprocessed lidar point cloud, and perform seamless ground-air data fusion on the preprocessed lidar point cloud to obtain fused data;

[0042] A single-tree segmentation module, configured to: generate a canopy height model based on the watershed segmentation algorithm, and use this model to perform single-tree segmentation on the fused point cloud data;

[0043] A dataset construction module, configured to: extract the vector boundary after single-tree segmentation, assign a tree ID to each tree, and extract single-tree point cloud according to the tree ID;

[0044] Further process the single-tree point cloud to construct a multi-angle point cloud sample dataset;

[0045] A tree species detection module, configured to: train a deep learning network model using the multi-angle point cloud sample dataset, and use the trained deep learning network model to detect the tree species to be measured.

[0046] The above one or more technical solutions have the following beneficial effects:

[0047] The technical solution of the present invention uses ground-based lidar point cloud data and drone images. Through high-overlap drone images, three-dimensional point cloud information of the tree canopy is generated. The ground-based lidar point cloud data is registered with the point cloud data generated by the drone images for fusion. The fused data not only contains the tree spatial structure information and trunk structure information of the ground-based lidar point cloud data, but also the drone point cloud makes it have a more complete canopy spatial structure information and canopy color information. Then, single-tree segmentation is performed based on the fused point cloud data.

[0048] The technical solution of the present invention integrates new technologies such as lidar scanning, multi-scale object segmentation, and intelligent semantic recognition, improves the apparent defects, structural misalignment and other problems existing in the three-dimensional acquisition of forest tree elements, improves the accuracy of single-tree segmentation and tree species recognition of point clouds, and realizes the integrated acquisition of key semantics such as the appearance, structure, and attributes of forest tree elements.

[0049] The technical solution of the present invention proposes a dimensionality reduction solution based on single-tree point cloud data, aiming to form multi-angle point cloud images through omnidirectional observation around the vertical axis of the tree trunk and generate a high-quality single-tree scale tree species semantic dataset. This method not only retains the structural features of the tree but also enhances the diversity of the dataset, thus providing rich training samples for subsequent deep learning models. Compared with the prior art, a significant advantage of this method is that it can accurately identify individual single trees, avoiding interference from other plants and environmental factors in large areas of forests, thereby improving the accuracy and efficiency of tree species identification and having broad application prospects.

[0050] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0052] Figure 1 is a flowchart of the method of the embodiment of the present invention;

[0053] Figure 2 is a schematic diagram of the effect of 12-angle point cloud images formed by omnidirectional observation of a single tree around the vertical axis of the tree trunk in the embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of the network structure of the ResNet152 + SPP pooling deep learning model described in the tree species semantic recognition of the embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of the network structure of the YOLO v11 deep learning classification model described in the tree species semantic recognition of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0057] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0058] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0059] Embodiment 1

[0060] This embodiment discloses a technical method for realizing semantic intelligent recognition of tree species at the single-tree scale based on the integration of ground-based and air-based 3D laser point clouds, including:

[0061] Step 1: Data acquisition: First, use an unmanned aerial vehicle (UAV) to obtain lidar point cloud data in the study area, and then use a handheld laser scanning device to collect point cloud data in the study area. Through on-site ground surveys, sample trees are obtained, which will be combined with the point cloud data later to form a single-tree sample image dataset, which can be used to train the tree species recognition model and evaluate the accuracy of the model and the precision of structural parameter extraction.

[0062] Step 2: Data preprocessing and data fusion:

[0063] First, perform data synthesis and stitching on the point cloud data of the study area obtained by the UAV flight to obtain a set of air-based point cloud data; perform point cloud calculation on the point cloud data of the study area collected by the handheld laser scanning device to obtain a set of ground-based point cloud data;

[0064] Secondly, preprocess the two sets of data, perform denoising, normalization, ground point removal, and clipping on the point cloud to obtain the preprocessed lidar point cloud;

[0065] For the preprocessed lidar point cloud, use the Iterative Closest Point (ICP) algorithm to perform seamless fusion of ground-air data based on the position and feature matching method.

[0066] 2-1) Denoising: Common noises include high-level gross errors and low-level gross errors. High-level gross errors are usually caused by the influence of low-flying objects during the data acquisition process of the airborne LiDAR system, and the signals reflected by these objects are mistakenly recorded as the reflected signals of the measured targets. Low-level gross errors are extremely low points caused by multipath errors during the measurement process or errors of the laser rangefinder. By selecting appropriate parameters, noise points can be removed to improve data quality.

[0067] Assume that the standard deviation multiple is meanK. The algorithm will search for adjacent points with the specified number of neighborhood points for each point, calculate the average distance D from the point to the adjacent points, calculate the median meanD and standard deviation S of these average distances. If D is greater than the maximum distance MaxD (MaxD = meanD + meanK * S), it is considered a noise point and will be removed. Specific parameter settings: number of neighborhood points = 10; standard deviation S = 5.

[0068] 2-2) Normalization: The process of normalizing the air-based point cloud and the ground-based point cloud according to the ground points involves identifying the ground points, calculating their heights, and subtracting the height of each point in the point cloud from the ground height to achieve height standardization. This operation eliminates height differences, enhances data consistency, and facilitates subsequent analysis.

[0069] Regarding ground point recognition: 1) Sort the elevation coordinates of the point cloud and select the lowest 10% of the points as candidate ground points. 2) Screen the points below the threshold through eight iterations to identify the ground points.

[0070] 2-3) Ground point classification (progressive densification triangulation filtering algorithm):

[0071] 1) Selection of initial seed points. In the point cloud data containing buildings, measure the maximum building size as the grid size to grid the point cloud data, and take the lowest point within the grid as the starting seed point.

[0072] 2) Construct a triangulation network. Use the starting seed points to construct an initial triangulation network.

[0073] 3) Iterative densification process. Traverse all points to be classified, query the triangles into which the horizontal projections of the points fall, calculate the distance d from the point to the triangle and the maximum value of the angles formed by the point to the three vertices of the triangle and the plane where the triangle is located. As shown in the following figure, compare them with the iterative distance and iterative angle respectively. If it is less than the corresponding threshold, determine this point as a ground point and add it to the triangulation network. Repeat this process until all ground points are classified. Finally, extract the point cloud data other than the ground points to achieve the purpose of removing the ground points.

[0074] Specific parameter settings: maximum building size = 20m; maximum terrain slope = 88°; iterative angle = 8°; iterative distance = 1.4m. Compared with the traditional algorithm, the ground seed points of this algorithm are obtained through morphological opening operations; along the buffer zone, simulated ground points participate in the construction of the original digital terrain model based on the progressive triangular irregular network (TIN) to improve the quality of the TIN and avoid the formation of inappropriate triangles; downward densification is carried out before upward densification to improve the ability to densify and process slope changes.

[0075] 2-4) Iterative closest point (ICP) algorithm:

[0076] 1) Sample separately from the point cloud P to be registered in the two sets of data of the obtained empty base point cloud and the ground base point cloud, and then find the point set corresponding to the target point cloud Q, satisfying that the Euclidean distance between each corresponding point is the smallest. Finally, obtain two new point sets, that is, remove the points without corresponding points and the wrong points.

[0077] 2) From the point set found in the first step, calculate the centroid of the corresponding point set according to its coordinate information to provide a reference point for subsequent calculations;

[0078] 3) Obtain the centroid, and calculate the rigid transformation (rotation matrix R and translation vector T) between the source point cloud and the target point cloud by minimizing the error function E(R,T) based on the centroid to make its value the smallest;

[0079]

[0080] Where: P i and Q i are the coordinates of the corresponding points in the source point cloud P and the target point cloud Q respectively, and n is the number of paired points.

[0081] 4) For the points in the source point cloud P, perform a rigid transformation on them using the rotation matrix R and translation vector T obtained in the third step. The new point set obtained after the transformation is named P'. The swapped point set P' is the result of the current iteration and should be closer to the target point cloud Q.

[0082] 5) Calculate the average distance between the corresponding points of the new point set P' and all the points in the target point cloud Q, denoted as ;

[0083]

[0084] Where: P i and Q i are the coordinates of the corresponding points in the source point cloud P and the target point cloud Q respectively, and n is the number of paired points.

[0085] 6) Calculate the average distance according to the above formula, and determine whether the iteration ends. If is greater than the pre-set threshold, then go back to step 2 and continue the calculation until the condition is met; if is less than the pre-set threshold, or the set number of iterations k is satisfied, then it is determined that the algorithm converges and the iteration terminates.

[0086] The advantages of the ICP algorithm are its simplicity, high efficiency, ease of implementation, and the ability to accurately align two sets of three-dimensional point cloud data. Through the iterative optimization process, ICP can effectively find the rigid transformation between point clouds and is widely used in fields such as three-dimensional reconstruction, robot navigation, and object recognition. Its main advantage lies in the low requirement for the initial guess, the ability to handle complex point cloud registration tasks, and its particular suitability for precise local alignment and small-scale point cloud matching.

[0087] Step 3: Individual tree segmentation: Generate a Canopy Height Model (CHM) from the fused point cloud data based on the watershed segmentation algorithm. This model reflects the height variations of the tree canopies. Search for local maxima on the CHM using a dynamic window of variable size as the tree crown vertices, and use these vertices as markers and input them into the watershed algorithm. The watershed algorithm controls the segmentation of the tree crown areas based on these markers, determines the boundaries according to the height variations between different tree crown areas in the CHM and the positions of the "ridges" formed during the simulation of the rising flood in the watershed algorithm, and realizes the precise segmentation of individual trees by extracting the boundaries of each tree crown. Finally, based on the segmentation results, key parameters such as tree height, diameter at breast height, and crown width are further extracted. The advantage of this method lies in its high segmentation accuracy and strong automated processing ability, facilitating the precise extraction of various forestry parameters and reducing the problems of over-segmentation and under-segmentation in traditional methods.

[0088] Regarding the generation of the Canopy Height Model:

[0089] A Digital Surface Model (DSM) is a ground elevation model that includes the heights of surface buildings, bridges, and trees, etc. Interpolation calculations are performed based on the point cloud data (including ground points and non-ground points) to generate the DSM. The specific parameter settings are: Xsize = 2, Ysize = 2, weight = 2, buffer size = 5, interpolation method = inverse distance weight interpolation, radius search = variable radius, distance = 5, number of neighborhood points = 12.

[0090] A Digital Elevation Model (DEM) is a digital simulation of the ground terrain through limited terrain elevation data (i.e., the digital representation of the terrain surface morphology). It is a solid ground model that represents the ground elevation in the form of an ordered numerical array. Interpolation calculations are performed based on the ground point data in the point cloud to generate a continuous DEM surface. The specific parameter settings are: Xsize = 2, Ysize = 2, weight = 2, buffer size = 5, interpolation method = inverse distance weight interpolation, radius search = variable radius, distance = 5, number of neighborhood points = 12.

[0091] CHM (Canopy Height Model) = DSM (Digital Surface Model) - DEM (Digital Elevation Model)

[0092] Step 4: Construction of a multi-angle semantic dataset for single-tree scale tree species: Extract the vector boundary after single-tree segmentation, assign a tree ID to each tree (vector contour), and extract the single-tree point cloud based on the tree ID. First, determine the vertical axis of the tree trunk for the extracted single-tree point cloud. To effectively distinguish the trunk point cloud, on the basis of hierarchical clustering, the trunk detection is completed through a multi-layer merging method. Secondly, observe the point cloud images at 12 angles around the vertical axis of the tree trunk, and combine them with the tree species names in the ground measured survey data to construct a multi-angle observation single-tree image dataset.

[0093] Regarding hierarchical clustering and multi-layer merging for trunk detection:

[0094] Divide the tree point cloud P into n layers from bottom to top at a certain distance, and use bottom-up layer-by-layer point cloud clustering to extract the trunk position. When dividing layers, to effectively merge and detect the trunk later, it should be ensured that the number of point clouds in each trunk segment is greater than T minPts , and the segmentation distance should satisfy:

[0095]

[0096] In the formula: T den is the average point cloud density at the trunk; r is the average breast diameter; T minPts is the minimum point cloud density for trunk clustering, which is determined according to the overall point cloud density of the trunk, and the default value is 5.

[0097]

[0098] In the formula: Z j represents the elevation value of the j-th point.

[0099] After obtaining the point clouds Pi of each layer, use the DBSCAN algorithm to perform layer-by-layer clustering to obtain a set, and merge the clustering results of each layer.

[0100] 1) Calculate the centroid and area information of the convex hull formed by the clustering point sets of each layer.

[0101] The convex hull of the point set is calculated using the Graham scan method. Given the convex hull points Ok (k = 1, 2,..., n) detected by the point set, the coordinates of the convex hull centroid T can be expressed as:

[0102] ,

[0103] where S is the convex hull area, and the calculation formula is S = .

[0104] 2) Take any clustering unit in the first-layer clustering set obtained by hierarchical clustering as each growth starting unit, and start from this unit to calculate the distance D between the centroid of the convex hull of each clustering unit and the centroid of the convex hull of the previous layer, as well as the convex hull area S formed by the projection of each clustering on the horizontal plane.

[0105] The calculation formula for the distance D between the centroids of the convex hulls of two hierarchical point clusters is:

[0106]

[0107] 3) Judge whether two clusters belong to the same ground object according to the distance and area of the two hierarchical point clusters. Repeat the above steps until all hierarchies are traversed, that is, i = n, to obtain the trunk point cloud of a single tree.

[0108] Step 5: Tree species semantic recognition and optimization comparison: Use the multi-angle semantic dataset of single-tree scale tree species to train, optimize, and compare deep learning algorithms (YOLO v11 and ResNet152 + SPP pooling), and finally select YOLO v11 with higher accuracy for tree species recognition.

[0109] YOLO v11-cls: In the tree species recognition task, by adopting a series of improved technical architectures, especially the optimized Backbone and Neck architectures, the model's ability to extract tree features is significantly improved, thus effectively improving the accuracy of tree species detection and its performance in complex environments. Notably, the introduction of the two innovative components C3k2 and C2PSA greatly enhances the model's ability to capture detailed tree features, enabling it to perform well in various practical applications.

[0110] First, the C3k2 component can deeply mine and fuse image information from different scales through multi-scale feature fusion. In the tree species recognition task, various components of a tree, such as leaves, trunks, and crowns, may present different spatial scale features. The C3k2 component enhances the extraction of these multi-scale features, enabling the model to more accurately capture subtle tree features, such as the shape of leaves, the diameter change of trunks, the density and morphology of crowns, etc. Especially in a complex forest environment, the differences between trees are often relatively subtle, and C3k2 can effectively improve the sensitivity to these details, thereby improving the recognition accuracy.

[0111] Secondly, the C2PSA component further optimizes the feature weighting process by introducing an attention mechanism in the Neck part, enhancing the model's ability to focus on important regions. C2PSA performs adaptive weighting in both spatial and channel dimensions, capable of focusing on the most discriminative features in the image. Especially when trees are occluded by the background or other trees, it can effectively distinguish the unique visual features of tree species. In this way, when dealing with complex backgrounds and interference information, the model can improve its recognition ability of the key structures of trees (such as trunks, branches, and leaves), thereby further enhancing the classification accuracy.

[0112] In addition, YOLO v11-cls reduces the number of parameters by 22%, not only significantly reducing the consumption of computing resources but also improving the processing speed, enabling the model to maintain an efficient running state in real-time applications.

[0113] ResNet152 + SPP pooling: In tree species recognition, the deep residual network structure of ResNet152 can deeply mine the multi-level features of trees, providing a richer feature representation. This structure is especially suitable for processing tree images of different varieties and age stages, helping to identify diverse tree species features. Combined with spatial pyramid pooling (SPP), the model can flexibly process input images of various sizes without fixing the input size, avoiding information loss caused by adjusting image sizes. By performing feature pooling at multiple scales, ResNet152 can capture the diversity of trees, including the features of leaves, crowns, and branches, thus enhancing the accuracy and robustness of tree species recognition.

[0114] In the specific implementation process, technical means such as data augmentation, learning rate adjustment, and batch normalization also need to be considered to further improve the performance and stability of the model. In addition, appropriate evaluation metrics (such as accuracy, recall, F1 score, etc.) can be used to evaluate the performance of the model in remote sensing semantic recognition tasks. The specific metrics are as follows:

[0115] Accuracy: Accuracy = (TP + TN) / (TP + TN + FP + FN)

[0116] Recall: Recall = TP / (TP + FN)

[0117] Precision: Precision = TP / (TP + FP)

[0118] F1 Score: F1 Score = 2 * (Precision * Recall) / (Precision + Recall)

[0119] In the formula, TP represents True Positive, that is, the number of positive samples correctly predicted by the model; TN represents True Negative, that is, the number of negative samples correctly predicted by the model; FP represents False Positive, that is, the number of positive samples wrongly predicted by the model; FN represents False Negative, that is, the number of negative samples wrongly predicted by the model.

[0120] In a more detailed implementation example, a tree recognition method based on ground-air integrated three-dimensional lidar point cloud includes:

[0121] S1. Data acquisition: First, the present invention uses a DJI M300 drone equipped with an L1 Zenmuse camera to obtain lidar point cloud data in the study area, and then uses a HC RS10 series handheld laser scanning device to collect point cloud data in the study area. Ground surveys are conducted on more than 2,000 trees to obtain data such as tree species, tree height, diameter at breast height, crown width, longitude and latitude coordinates, etc.

[0122] S2. Data preprocessing and data fusion: The drone flight data is imported into DJI Terra to complete data synthesis and image stitching; the handheld scanning data is processed using CoPre2 software for point cloud calculation; secondly, the two sets of data are preprocessed in LiDAR360 software, and the point cloud is denoised, normalized, ground points removed, and cropped. The preprocessed lidar point cloud is seamlessly fused with ground data using the Iterative Closest Point (ICP) algorithm based on position and feature matching.

[0123] S3. Individual tree segmentation: The fused point cloud data is used to generate a Canopy Height Model (CHM) based on the watershed segmentation algorithm. A variable-sized dynamic window is used to search for local maxima on the canopy height model as the tree crown vertices, and the marker-controlled watershed algorithm is used with the tree crown vertices as markers to outline the tree crown boundaries, and further parameters such as tree height, diameter at breast height, and crown width are extracted.

[0124] S4. Construction of a multi-angle semantic dataset for individual tree species at the individual tree scale: The vector boundaries after individual tree segmentation are extracted, a tree ID is assigned to each tree (vector contour), and the individual tree point cloud is extracted according to the tree ID. The vertical axis of the tree trunk of the extracted individual tree point cloud is determined. To effectively distinguish the tree trunk point cloud, based on hierarchical clustering, the trunk detection is completed through a multi-layer merging method. Secondly, point cloud images at 12 angles are formed by observing the tree trunk vertically around the vertical axis, and combined with the ground survey data to construct a multi-angle observation dataset of individual standing trees.

[0125] S5. Tree species semantic recognition and optimized comparison and selection: Train, optimize, and compare the YOLOv11 model and the ResNet152 model + SPP pooling using a multi-angle point cloud sample dataset.

[0126] YOLO v11-cls training: This network is built using the Pytorch library in Python. 80% of the data is used as the training set, and 20% of the data is used as the test set. Use the YOLO11-cls model configuration file and load the pre-trained weights; pass the defined training parameters into the model to start training.

[0127] ResNet152 model + SPP pooling: This network is built using the Pytorch library in Python. 80% of the data is used as the training set, and 20% of the data is used as the test set. Add a spatial pyramid pooling layer after the last convolutional layer of ResNet152, and configure pooling parameters of different scales (such as 1x1, 2x2, 3x3, etc.) to obtain multi-level features.

[0128] In the example of the present invention, the YOLO11-cls model is finally selected. The evaluation indicators for tree species intelligent recognition are: precision = 0.87, recall = 0.82, F1-score = 0.83, accuracy = 0.91.

[0129] Aiming at the problems of "easy occlusion of the ground base and poor penetration of the airborne base" in forest land scanning, the present invention first collects three-dimensional laser point cloud data of the study area through low-altitude aerial photography by drones and handheld laser scanning equipment, and realizes seamless fusion of ground and airborne platform data through GPS positioning and feature matching, so as to improve the phenotypic and structural integrity of forest tree point cloud elements. Aiming at the limitations of traditional manual tree species surveys and machine learning algorithm recognition, a multi-angle observation single-tree image dataset is constructed based on single-tree point clouds and survey data, and high-precision automatic recognition of tree species semantics at the single-tree scale is realized through training, optimization, and comparison of multiple artificial intelligence algorithms. The method proposed by the present invention can realize the construction of key parameters such as the position, attributes, and structure of surface forest tree elements, and provide technical support for application requirements such as refined forest resource management, biodiversity investigation and protection, and urban livable environment regulation.

[0130] In the technical solution of the present invention, when constructing the dataset, point cloud images at 12 angles are formed by observing all around the vertical axis of the tree trunk, and the three-dimensional point cloud data is reduced to two-dimensional image data to simplify the analysis process and improve the training efficiency of the model, which has important practical significance.

[0131] Example Two

[0132] The objective of this embodiment is to provide a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0133] Embodiment III

[0134] The objective of this embodiment is to provide a computer-readable storage medium.

[0135] A computer-readable storage medium has a computer program stored thereon. When the program is executed by a processor, the steps of the above method are executed.

[0136] Embodiment IV

[0137] The objective of this embodiment is to provide a tree recognition system based on ground-air integrated three-dimensional lidar point cloud, including:

[0138] A point cloud data acquisition module, configured to: use a drone to acquire lidar point cloud data within a set forest area, and use a laser scanning device to collect point cloud data within the set forest area;

[0139] A fusion module, configured to: preprocess the two sets of data to obtain preprocessed lidar point cloud, and perform seamless ground-air data fusion on the preprocessed lidar point cloud to obtain fused data;

[0140] A single-tree segmentation module, configured to: generate a canopy height model based on the watershed segmentation algorithm, and use the model to perform single-tree segmentation on the fused point cloud data;

[0141] A dataset construction module, configured to: extract the vector boundary after single-tree segmentation, assign a tree ID to each tree, and extract the single-tree point cloud according to the tree ID;

[0142] Further process the single-tree point cloud to construct a multi-angle point cloud sample dataset;

[0143] A tree species detection module, configured to: train a deep learning network model using the multi-angle point cloud sample dataset, and use the trained deep learning network model to detect the tree species to be measured.

[0144] Embodiment V

[0145] The objective of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any one of the above embodiments.

[0146] In the devices of the above embodiments, the steps involved correspond to those of the first method embodiment. For specific implementation details, please refer to the relevant description in the first embodiment. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more sets of instructions; it should also be understood to include any medium that can store, encode, or carry a set of instructions for execution by a processor and cause the processor to execute any method of the present invention.

[0147] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented with program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple of them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0148] Although the specific implementation modes of the present invention have been described above in conjunction with the accompanying drawings, this is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A tree recognition method based on ground-air fusion three-dimensional laser point cloud, characterized by: include: Use drones to obtain lidar point cloud data within a set forest area, and use laser scanning equipment to collect point cloud data within a set forest area; The two sets of data are preprocessed to obtain a preprocessed LiDAR point cloud, and the preprocessed LiDAR point cloud is seamlessly fused with ground and air data to obtain fused data; A canopy height model is generated based on the watershed segmentation algorithm, and the fused point cloud data is segmented into individual trees using the model. When performing individual tree segmentation, a dynamic window of variable size is used to search for the local maximum value as the crown vertex on the canopy height model, and the crown vertex is used as a marker. The marker-controlled watershed algorithm is used to outline the crown boundary and extract the tree height, breast diameter and crown width parameters. Extract the vector boundary after the single tree segmentation, assign a tree ID to each tree, and extract the single tree point cloud based on the tree ID; first determine the vertical axis of the trunk of the extracted single tree point cloud, and complete the trunk detection through the multi-layer merging method based on the hierarchical clustering. The specific process is as follows: The tree point cloud P is divided into n layers from bottom to top, and the trunk position is extracted by layer-by-layer point aggregation from bottom to top; After obtaining the point clouds Pi of each layer, the DBSCAN algorithm is used to cluster them layer by layer to obtain a set, and the clustering results of each layer are merged; The single tree point cloud is processed to construct a multi-angle point cloud sample dataset; point cloud images at multiple angles are formed by all-round observation around the vertical axis of the tree trunk, and combined with the ground survey data to construct a multi-angle observation single standing tree image dataset; The deep learning network model is trained using a multi-angle point cloud sample dataset, and the trained deep learning network model is used to detect the tree species to be tested.

2. The tree recognition method based on ground-air fusion three-dimensional laser point cloud as claimed in claim 1 is characterized in that: When seamlessly fusing ground and air data for the preprocessed lidar point cloud, an iterative closest point algorithm is used, which performs seamless fusion of ground and air data based on position and feature matching methods.

3. The tree recognition method based on ground-air fusion three-dimensional laser point cloud as claimed in claim 1 is characterized in that: The clustering results of each layer are merged, including: 1) Calculate the centroid and area information of the convex hull formed by the clustering point sets of each layer; 2) Taking any cluster unit in the first layer of clustering set obtained by hierarchical clustering as each growth starting unit, starting from this unit, calculate the distance between the convex hull centroid of each cluster unit and the convex hull centroid of the previous layer, as well as the convex hull area formed by the projection of each cluster on the horizontal plane; 3) Determine whether the two clusters belong to the same feature based on the distance and area of ​​the two stratified point clusters; 3) Repeat the above steps until all layers are traversed, that is, i=n, ​​and obtain the trunk point cloud of a single tree.

4. A tree recognition system based on ground-air fusion three-dimensional laser point cloud, used to execute the tree recognition method based on ground-air fusion three-dimensional laser point cloud according to any one of claims 1 to 3, characterized in that: include: The point cloud data acquisition module is configured to: use a drone to acquire laser radar point cloud data within a set forest area, and use a laser scanning device to collect point cloud data within the set forest area; The fusion module is configured to: pre-process the two sets of data to obtain a pre-processed lidar point cloud, and seamlessly fuse the ground and air data of the pre-processed lidar point cloud to obtain fused data; The single tree segmentation module is configured to: generate a canopy height model based on a watershed segmentation algorithm, and use the model to perform single tree segmentation on the fused point cloud data; The dataset construction module is configured to: extract the vector boundary after the single tree segmentation, assign a tree ID to each tree, and extract the single tree point cloud based on the tree ID; Process the single tree point cloud to construct a multi-angle point cloud sample dataset; The tree species detection module is configured to: use a multi-angle point cloud sample data set to train a deep learning network model, and use the trained deep learning network model to detect the tree species to be detected.

5. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 3 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 3 are performed.

Images