Transform network-based sparse airborne laser radar data individual tree identification method

Abstract

The invention relates to the technical field of surveying and mapping remote sensing, and discloses a sparse airborne laser radar data single tree identification method based on a Transform network, and the method comprises the following steps: S1, collecting airborne laser radar point cloud data of a forest; s2, preprocessing the laser point cloud data to obtain a single tree instance point cloud data sample; and S3, constructing a double-branch Transform network, training the model by using the training set, selecting an optimal model, and testing. According to the method, a Transformer model of a double-branch structure is designed to process geometric information corresponding to three-dimensional coordinate information of point cloud data and other attribute information such as a scanning angle, an echo number and echo intensity so as to obtain a high-precision classification result.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping remote sensing technology, and in particular to a single tree recognition method for sparse airborne laser radar data based on a Transformer network. Background Art

[0002] As the basis for estimating and extracting forest parameters, tree species identification at the scale of individual trees in forests is an important research direction in the field of forestry remote sensing. It is of great significance to the macro-monitoring and management of forest ecosystems, biodiversity, and terrestrial carbon sinks. Traditional tree species identification at the scale of individual trees in forests mainly relies on field surveys. Although this method has reliable results, the field survey time and labor costs are high, and the working environment is often dangerous. Therefore, the technology of accurately and reliably extracting individual tree species information has become a key link in forestry applications. In recent years, with the continuous iteration and improvement of airborne laser radar scanning technology, the operating cost has been greatly reduced. At the same time, with the characteristics of flexible and efficient operation, it has gradually become the main means of forestry survey. Compared with other remote sensing technologies, airborne laser radar can provide high-resolution three-dimensional structural information, including crown morphology, height, canopy density, etc. These features are very important for tree species identification. Laser radar can penetrate the canopy and obtain information about the surface and different vegetation layers, which helps to fully understand the forest structure.

[0003] At present, the single tree species identification and classification technology of airborne LiDAR point clouds mainly uses traditional machine learning algorithms, such as linear discriminant analysis, support vector machine, or random forest methods. Although these methods are widely used, their generalization ability in different study areas or data set applications remains a huge challenge. In addition, the performance of machine learning classifiers is highly dependent on the feature engineering and selection process, which requires strong expert knowledge to identify and select the most relevant and influential features in the dataset, which greatly limits the wide application of the method. In addition, many current studies are limited to classifying trees as conifers or broadleaf trees, which limits their application in biomass models of specific tree species. Some studies focus on simple classification tasks involving two to four tree species, while the overall accuracy of classification involving six or more tree species is usually less than 80%, and the median accuracy is only 62%. Therefore, in complex forest environments, accurately identifying tree species from airborne laser point clouds remains a huge challenge.

[0004] In recent years, with the rapid development of deep learning and artificial intelligence, the classification and identification of single tree species by building deep learning models has gradually become a new direction in this field. Existing studies have shown that when the number of tree species increases, deep learning models are more accurate and reliable than traditional machine learning models. In 2020, Xi et al. compared the classification effects of 13 machine learning (ML) and deep learning classifiers on nine tree species in complex forests in the study See the forest and the trees: Effective machine and deep learning algorithms for wood filtering and tree species classification from terrestrial laser scanning. The results showed that deep learning methods generally outperformed machine learning methods. In particular, PointNet++ achieved the highest overall accuracy when evaluating voxel-based and point-based deep learning classifiers. In 2023, Fan et al. used PointNet++ combined with data enhancement strategies to classify 11 tree species in the study Tree Species Classification Based on PointNet and Airborne Laser Survey Point Cloud Data Enhancement, achieving an overall accuracy of 91.82%. The study also compared multi-scale sampling and grouping with simplified sampling and grouping methods and found that multi-scale sampling was better for tree species classification in airborne laser point clouds.

[0005] Deep learning methods for tree species classification using ALS data have made significant progress; however, their application in sparse point cloud environments still faces many challenges.

[0006] Many existing frameworks use projection-based data forms, which require rasterizing 3D point cloud data into 2D images, which results in the loss of the most valuable 3D geometric information of laser point cloud data. Some studies have used deep learning models based on 3D point cloud information such as PointNet, PointNet++ or PointCNN to learn the feature spatial distance and relationship between tree species, so as to achieve the purpose of classification and identification of tree species. Such methods are mostly targeted at land-based lidar data with high point cloud density. Globally, most forest inventory projects usually use fixed-wing airborne lidar point cloud data, which has an average density of 30 points / m2 or less, so it is necessary to design a classification model for this type of data. However, there is still a lack of deep learning models for single tree species identification of relatively sparse laser point cloud data scanned from fixed-wing aircraft. In addition, many existing methods based on deep learning frameworks rely on the fusion of multispectral or hyperspectral image data to provide more information for tree species identification and thus improve the accuracy of classification results, which greatly increases the cost of data acquisition and the difficulty of data preprocessing. At the same time, most existing results can only handle 2-4 types of tree species identification. Once the number of tree species exceeds 6, the classifier effect will be significantly reduced.

[0007] Therefore, how to develop more robust and efficient deep learning algorithms to improve the classification accuracy of multiple tree species in sparse point cloud environments and provide more accurate and reliable data support for forest resource management and ecological protection is a key research focus in the future. Summary of the invention

[0008] The purpose of the present invention is to solve the problems existing in the prior art and to propose a single tree recognition method for sparse airborne lidar data based on a Transformer network.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A method for single tree recognition in sparse airborne laser radar data based on a Transformer network comprises the following steps:

[0011] S1: Collect airborne LiDAR point cloud data of forest trees;

[0012] S2: Preprocess the laser point cloud data to obtain a single tree instance point cloud data sample;

[0013] S3: Build a dual-branch Transformer network, train the model using the training set, select the best model, and test it;

[0014] S4: Apply the data preparation in step S2 and the model in S3 to the forest area to be tested.

[0015] Preferably, the specific process of step S1 is: using a fixed-wing or unmanned aerial vehicle laser scanning system to obtain the laser radar scanning point cloud data of the forest area to be measured, and completing conventional laser radar data preprocessing such as data geometry and radiation correction.

[0016] Preferably, the specific process of step S2 is:

[0017] S21: Use cloth simulation filtering to separate ground points and non-ground points in the forest point cloud and obtain forest point cloud data;

[0018] S22: Use the graphics-based cutting tracking algorithm to segment the forest data into individual trees, check the quality of individual trees and optimize them, and obtain individual tree instance point cloud samples;

[0019] S23: manually label each single tree instance with tree species information, prepare a balanced number of training samples for each type of tree with reference to forest types and actual conditions, and divide the final data set into training set, validation set, and test set in a ratio of 7:2:1;

[0020] S24: by counting the number of points in the point cloud of each single tree instance, the number of points at the 90th percentile is selected and the value is rounded to the nearest power of 2 as the model input point number parameter;

[0021] S25: If the number of points contained in a single tree instance is greater than the number of model input points, each single tree instance point is sampled to the input point number through the farthest distance sampling method; if the number of points contained in a single tree instance is less than the number of model input points, the point cloud data of the single tree instance is randomly selected and copied to reach the target input point number;

[0022] S26: normalize the coordinates of each instance data to ensure that the center of the point cloud is at (0,0,0);

[0023] S27: Apply data augmentation techniques to the training data, including random instance rotation, instance scaling, and point perturbations.

[0024] Preferably, in step S25, the farthest distance sampling process is to randomly select a seed point and then use the farthest distance sampling.

[0025] Preferably, in step S26, the intensity information is normalized to [0, 1] to expand the training data set and improve the robustness of the model.

[0026] Preferably, in step S27, the scaling parameters are uniformly distributed in the range of [0.8, 1.25], the rotation parameters are uniformly distributed in the range of [0, 2π], and random point-level perturbations are added, with Gaussian noise having a mean of 0.01 and a variance of 1.0, and an offset ranging from -0.05 to 0.05.

[0027] Preferably, the step S3 is specifically:

[0028] S31: Building a dual-branch Transformer network:

[0029] The main branch network is responsible for generating geometric features, focusing on extracting the feature information of the three-dimensional coordinates (x, y, z), number of echoes, and echo rounds of point cloud data;

[0030] The secondary branch processes other features of the point cloud, including RGB color value, echo intensity, and feature information of scanning angle;

[0031] Both branches start with a multi-layer perceptron (MLP) module to transform the data into high-dimensional features;

[0032] Afterwards, the features of the two branches are fused through the first cross-branch attention module;

[0033] Then, two GLocal (global-local feature extraction) modules are used to capture local context information;

[0034] Afterwards, the features of the two branches are fused through the next cross-branch attention module;

[0035] The fused features will be passed to the next GLocal module of the main branch, while the secondary branch will also directly input the extracted features into the next GLocal module of the secondary branch when fusing the features;

[0036] After the third cross-branch attention module, the convolutional layer is combined with the maximum pooling layer to form a convolutional module;

[0037] The output features of the third stage are integrated with the residual features of the GLocal modules in the first and second stages using a maximum pooling layer. The integrated features are input into the multi-layer perceptron head to perform the tree species classification task.

[0038] S32: Use Pytorch and CUDA to build a GPU-based model training and testing environment, and test the impact of different learning rates, batch sizes, and Transformer module depths on model performance, and select the optimal hyperparameters;

[0039] S33: Fixed hyperparameter settings, compared the detection performance and training and prediction efficiency of different models, and finally selected the best model.

[0040] Preferably, the specific process of step S4 is:

[0041] S41: Prepare a large batch of test data according to steps S21-S26 in the process of step S2;

[0042] S42: Apply the best model selected in step S33 to infer the prepared data, obtain tree species classification information, and use random sampling inspection to control the quality of the final result.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention proposes a latest single tree species classification method adapted to low-density airborne laser scanning radar data, which improves the recognition accuracy of multiple tree species (more than six categories) and has obvious advantages in forestry single tree species recognition. At the same time, the dual-branch Transformer deep learning model of the present invention and the cross-branch attention module therein can not only fully capture the geometric feature information of the three-dimensional point cloud, but also obtain other attribute features at the same time. The GLocal module is used to simultaneously learn the local features and global features of the low-density forestry point cloud (minimum density 10 points / square meter), thereby maximizing the reliability of multi-tree species recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the process of a single tree recognition method for sparse airborne laser radar data based on a Transformer network proposed by the present invention;

[0046] Figure 2 A schematic diagram of the basic construction of a dual-branch Transformer model for a single tree recognition method for sparse airborne laser radar data based on a Transformer network proposed in the present invention;

[0047] Figure 3 Schematic diagram of the cross-branch attention mechanism of the single tree recognition method for sparse airborne lidar data based on the Transformer network proposed in the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] Reference Figure 1-Figure 3 , to achieve fast, accurate and low-cost identification of tree species information at the individual tree level, improve the monitoring efficiency of the forestry department and reduce monitoring costs.

[0050] The technical solution of the invention mainly includes the following steps:

[0051] S1: Collect airborne LiDAR point cloud data of forest trees;

[0052] S2: Preprocess the laser point cloud data to obtain a single tree instance point cloud data sample;

[0053] S3: Build a dual-branch Transformer network, use the training set to train the model, select the best model and test it;

[0054] S4: Apply the data preparation in step S2 and the model in S3 to the forest area to be tested.

[0055] The step S1 is specifically as follows: using a fixed-wing or unmanned aerial vehicle laser scanning system to obtain the laser radar scanning point cloud data of the forest area to be measured, and completing conventional laser radar data preprocessing such as data geometry and radiation correction; the forest area to be measured is not limited to natural, artificially planted or commercial forests, coniferous forests, deciduous forests or mixed forests.

[0056] The step S2 is specifically as follows:

[0057] S21: Use cloth simulation filtering to separate ground points and non-ground points in the forest point cloud and obtain forest point cloud data;

[0058] S22: Use a graphics-based cutting tracking algorithm to segment the forest data into individual tree instances, check the quality of individual tree instances and optimize them, so as to obtain individual tree instance point cloud samples;

[0059] S23: Label the tree species information for each single tree instance by manual labeling, refer to the forest types and actual conditions, prepare a balanced number of training samples for each type of tree as much as possible, and divide the final data set into training set, validation set and test set in a ratio of 7:2:1;

[0060] S24: by counting the number of points in the point cloud of each single tree instance, the number of points at the 90th percentile is selected and the value is rounded to the nearest power of 2 as the model input point number parameter;

[0061] S25: If the number of points contained in a single tree instance is greater than the number of model input points, each single tree instance point is sampled to the input point number through the farthest distance sampling method. The sampling process is to randomly select a seed point and then use the farthest distance sampling (Farthest Point Sampling). If the number of points contained in a single tree instance is less than the number of model input points, the point cloud data of the single tree instance is randomly selected and copied to reach the target input point number;

[0062] S26: Normalize the coordinates of each instance data to ensure that the center of the point cloud is at (0,0,0). In addition, normalize the intensity information to [0,1] to expand the training data set and improve the robustness of the model.

[0063] S27: Apply data augmentation techniques to the training data. This includes random instance rotation, instance scaling, and point perturbations. The scaling parameters are uniformly distributed in the range [0.8, 1.25], while the rotation parameters are uniformly distributed in the range [0, 2π]. Random point-level perturbations are added with Gaussian noise with a mean of 0.01 and a variance of 1.0, and an offset ranging from -0.05 to 0.05.

[0064] The step S3 is specifically as follows:

[0065] S31: The main branch network is responsible for generating geometric features, focusing on extracting the feature information of the three-dimensional coordinates (x, y, z), number of echoes and echo rounds of point cloud data;

[0066] The secondary branch processes other features of the point cloud, including RGB color value, echo intensity, and feature information of scanning angle;

[0067] Both branches start with a multi-layer perceptron (MLP) module to transform the data into high-dimensional features;

[0068] Afterwards, the features of the two branches are fused through the first cross-branch attention module;

[0069] Then, two GLocal (global-local feature extraction) modules are used to capture local context information;

[0070] Afterwards, the features of the two branches are fused through the next cross-branch attention module;

[0071] The fused features will be passed to the next GLocal module of the main branch, while the secondary branch will also directly input the extracted features into the next GLocal module of the secondary branch when fusing the features;

[0072] After the third cross-branch attention module, the convolutional layer is combined with the maximum pooling layer to form a convolutional module;

[0073] The output features of the third stage are integrated with the residual features of the GLocal modules in the first and second stages using a maximum pooling layer. The integrated features are input into the multi-layer perceptron head to perform the tree species classification task.

[0074] S311: Cross-branch Attention Module

[0075] This mechanism allows to fuse features from the primary branch (FP) and the secondary branch (FS), effectively combining geometric and radial features. The working steps of the cross-attention module are as follows:

[0076] Step 1: Fusion of query (Q), key (K) and value (V): During feature fusion, the main branch feature (F P ) and minor branch features (F S) are associated and fused. This fusion allows the integration of geometric and other features.

[0077] Q P =F P W QP , Q S =F S W QS

[0078] K P =F P W KP , K S =F S W KS

[0079] V P =F P W VP , V S =F S W VS

[0080] Step 2: Exchange of key (K) and value (V) elements: In each cross-attention module, F P and F S Its key (K) and value (V) elements are swapped, which then serve as input for the next step.

[0081] Step 3: Generation of attention maps: Using the query (Q) and key (K) elements, attention maps are calculated independently for each branch. These attention maps highlight the relevant information in the feature set of each branch.

[0082]

[0083]

[0084] Step 4 Integration of new features: The value (V) elements are integrated into the attention map to generate new features that capture important information from both domains. This integration enhances the feature extraction process and enables the network to learn meaningful information across various domains (including geometry, radiation, etc.).

[0085] New F P =M P V S +F P

[0086] New F S =M s V P +F S

[0087] Step 5 Residual Connection: The original features of each branch are preserved by creating residual connections. This ensures that the network retains the valuable information from the input features of both branches.

[0088] Step 6: Concatenation of feature maps: Finally, concatenate the feature maps of the two branches to form the output of the cross-attention module. This combined output effectively captures the comprehensive information of geometric, radiation and other features.

[0089] F out =Concat(New F P , New F S )

[0090] S312: Use Weighted Cross Entropy as the loss function to solve the problem of imbalanced tree species and iteratively train the model. The formula is as follows:

[0091]

[0092]

[0093]

[0094] N is the total number of samples, C is the total number of categories, is the normalized weight of class c, y ic is the true label (1 or 0) of sample i in category c, p ic is the probability that the model predicts that sample i belongs to category c. c is the number of samples of category c.

[0095] S32: Use Pytorch and CUDA to build a GPU-based model training and testing environment, and test the impact of different learning rates, batch sizes, and Transformer module depths on model performance, and select the optimal hyperparameters;

[0096] S33: Fixed hyperparameter settings, compared the detection performance and training and prediction efficiency of different models, and finally selected the best model.

[0097] The step S4 is specifically as follows:

[0098] S41: Prepare the large batch of data to be tested according to steps 1-6 in the S2 process.

[0099] S42: Apply the best model selected in S33 to infer the prepared data to obtain tree species classification information. Random sampling inspection can be used to control the quality of the final result.

[0100] Compared with the prior art, the present invention:

[0101] 1. A dual-branch Transformer model was designed to process the geometric information corresponding to the three-dimensional coordinate information of the point cloud data, as well as other attribute information such as scanning angle, echo number, echo intensity, etc. to obtain high-precision classification results;

[0102] 2. A cross-branch attention mechanism module is designed to realize the dual-branch Transformer structure, so as to exchange the feature information of the two branches and efficiently organize and analyze the feature relationship of different global attributes;

[0103] 3. The entire design process of the invention can process single-spectrum or multi-spectral forest area data scanned by fixed-wing and UAVs and achieve single tree species identification results and data products with high accuracy;

[0104] Has the following effects:

[0105] Improved the recognition accuracy of multiple tree species (more than six types), and has obvious advantages in the identification of single tree species in forestry;

[0106] The dual-branch Transformer deep learning model and the cross-branch attention module therein can not only fully capture the geometric feature information of the 3D point cloud, but also obtain other attribute features at the same time;

[0107] The GLocal module is used to simultaneously learn the local and global features of low-density forestry point clouds (minimum density 10 points / square meter), thereby maximizing the reliability of multi-type tree species identification.

[0108] The present invention designs and develops an end-to-end dual-branch model structure and a cross-branch attention mechanism to learn the global and local feature information of sparse point clouds to the greatest extent, thus overcoming the current challenge of low accuracy in tree species classification of sparse point clouds. This method can not only effectively improve the accuracy of single tree species identification in drone laser scanning data, but can also be applied to identify single tree species categories in fixed-wing airborne laser scanning data with a point cloud density as low as 12 points / square meter. It can be perfectly adapted to single-spectrum or multi-spectral lidar scanning data collected by fixed-wing and drones and applied to other algorithms.

[0109] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for single tree recognition in sparse airborne laser radar data based on Transformer network, characterized by: The following steps are involved: S1: Collect airborne LiDAR point cloud data of forest trees; S2: Preprocess the laser point cloud data to obtain a single tree instance point cloud data sample; S3: Build a dual-branch Transformer network, train the model using the training set, select the best model, and test it; S4: Apply the data preparation in step S2 and the model in S3 to the forest area to be tested.

2. According to claim 1, a method for single tree recognition based on sparse airborne laser radar data based on Transformer network is characterized by: The specific process of step S1 is: using a fixed-wing or unmanned aerial vehicle laser scanning system to obtain the laser radar scanning point cloud data of the forest area to be measured, and completing conventional laser radar data preprocessing such as data geometry and radiation correction.

3. According to claim 1, a method for single tree recognition based on sparse airborne laser radar data based on Transformer network is characterized by: The specific process of step S2 is: S21: Use cloth simulation filtering to separate ground points and non-ground points in the forest point cloud and obtain forest point cloud data; S22: Use the graphics-based cutting tracking algorithm to segment the forest data into individual trees, check the quality of individual trees and optimize them, and obtain individual tree instance point cloud samples; S23: manually label each single tree instance with tree species information, prepare a balanced number of training samples for each type of tree with reference to forest types and actual conditions, and divide the final data set into training set, validation set, and test set in a ratio of 7:2:1; S24: by counting the number of points in the point cloud of each single tree instance, the number of points at the 90th percentile is selected and the value is rounded to the nearest power of 2 as the model input point number parameter; S25: If the number of points contained in a single tree instance is greater than the number of model input points, each single tree instance point is sampled to the input point number through the farthest distance sampling method; if the number of points contained in a single tree instance is less than the number of model input points, the point cloud data of the single tree instance is randomly selected and copied to reach the target input point number; S26: normalize the coordinates of each instance data to ensure that the center of the point cloud is at (0,0,0); S27: Apply data augmentation techniques to the training data, including random instance rotation, instance scaling, and point perturbations.

4. The method for single tree recognition based on sparse airborne laser radar data based on Transformer network according to claim 3 is characterized by: In step S25, the farthest distance sampling process is to randomly select a seed point and then use the farthest distance sampling.

5. The method for single tree recognition based on sparse airborne laser radar data based on Transformer network according to claim 3, characterized in that: In step S26, the intensity information is normalized to [0, 1] to expand the training data set and improve the robustness of the model.

6. The method for single tree recognition based on sparse airborne laser radar data based on Transformer network according to claim 3, characterized in that: In step S27, the scaling parameters are uniformly distributed in the range of [0.8, 1.25], the rotation parameters are uniformly distributed in the range of [0, 2π], and a random point-level perturbation is added with Gaussian noise having a mean of 0.01 and a variance of 1.0, and an offset ranging from -0.05 to 0.

05.

7. The method for single tree recognition based on sparse airborne laser radar data based on Transformer network according to claim 3 is characterized by: The step S3 is specifically as follows: S31: Building a dual-branch Transformer network: The main branch network is responsible for generating geometric features, focusing on extracting the feature information of the three-dimensional coordinates (x, y, z), number of echoes, and echo rounds of point cloud data; The secondary branch processes other features of the point cloud, including RGB color value, echo intensity, and feature information of scanning angle; Both branches start with a multi-layer perceptron (MLP) module to transform the data into high-dimensional features; Afterwards, the features of the two branches are fused through the first cross-branch attention module; Then, two GLocal (global-local feature extraction) modules are used to capture local context information; Afterwards, the features of the two branches are fused through the next cross-branch attention module; The fused features will be passed to the next GLocal module of the main branch, while the secondary branch will also directly input the extracted features into the next GLocal module of the secondary branch when fusing the features; After the third cross-branch attention module, the convolutional layer is combined with the maximum pooling layer to form a convolutional module; The output features of the third stage are integrated with the residual features of the GLocal modules in the first and second stages using a maximum pooling layer. The integrated features are input into the multi-layer perceptron head to perform the tree species classification task. S32: Use Pytorch and CUDA to build a GPU-based model training and testing environment, and test the impact of different learning rates, batch sizes, and Transformer module depths on model performance, and select the optimal hyperparameters; S33: Fixed hyperparameter settings, compared the detection performance and training and prediction efficiency of different models, and finally selected the best model.

8. The method for single tree recognition based on sparse airborne laser radar data according to claim 7, characterized in that: The specific process of step S4 is as follows: S41: Prepare a large batch of test data according to steps S21-S26 in the process of step S2; S42: Apply the best model selected in step S33 to infer the prepared data, obtain tree species classification information, and use random sampling inspection to control the quality of the final result.

Images