Airborne laser radar point cloud classification method based on transfer learning

Abstract

The invention discloses an airborne laser radar point cloud classification method based on transfer learning, and relates to the technical field of point cloud data processing, and the method comprises the steps: carrying out the feature extraction of point cloud data, and obtaining the final feature representation of the point cloud data; classifying and marking the point cloud data, and acquiring a category label and a rare category label of each point in the point cloud data; constructing a large sample label classification model, and obtaining a large sample category label in the point cloud data; obtaining a transferable category label corresponding to each rare category label, and constructing a small sample label classification model of each rare category label based on the transferable category label; obtaining a small sample category label of each point in the point cloud data in the current collection period, and constructing a complete category label set of the point cloud data in the collection period according to a large sample category label and the small sample category label in the point cloud data, the problem of model performance caused by unbalanced data volume between the small sample category label and the large sample category label or insufficient data volume of the small sample category label is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data processing, and in particular to an airborne laser radar point cloud classification method based on transfer learning. Background Art

[0002] The prior art CN116403058B "A remote sensing cross-scene multispectral lidar point cloud classification method" first constructs the multispectral lidar point clouds with labeled source scenes and unlabeled target scenes into graph data according to the K nearest neighbor algorithm; then uses a graph convolutional neural network to extract the graph features of the two scenes respectively, and uses a source scene classifier for source scene multispectral lidar point cloud classification, and a target scene classifier for target scene multispectral lidar point cloud classification. A learnable scene-to-scene information transfer module is added for information transfer between the two scenes, and a domain discriminator is used to distinguish whether the point cloud belongs to the source scene or the target scene. Finally, the network parameters are iteratively updated until the model converges.

[0003] The prior art CN116933141A "A multispectral lidar point cloud classification method based on multi-core graph learning" calculates the spatial distance matrix and spectral distance matrix of the multispectral lidar point cloud data respectively. Then, different Gaussian kernels are used to measure the similarity between multispectral lidar point clouds, and multiple base kernel adjacency matrices are calculated. Then, the spatial-spectral distance matrix is ​​used as a high-dimensional feature to expand the feature matrix of the multispectral lidar point cloud. Finally, the multiple kernel adjacency matrices and the expanded feature matrix are input into the improved multi-core graph neural network, and the graph neural network weights and multi-core weights are updated synchronously to train the model.

[0004] With the development of LiDAR technology, airborne LiDAR systems are widely used in terrain mapping, urban planning, disaster assessment and other fields. However, in real-world scenarios, some categories of point cloud data may be far more than other categories, resulting in category imbalance problems that affect the performance of the model. At the same time, airborne LiDAR point cloud data is often expensive and difficult to obtain, resulting in a small data set available for training, which limits the learning ability and generalization performance of the model. Therefore, it is of great significance to develop an efficient airborne LiDAR point cloud classification method. Summary of the invention

[0005] In order to solve the above technical problems, the object of the present invention is to provide an airborne laser radar point cloud classification method based on transfer learning, comprising the following steps: Step s1: setting the collection period, performing feature extraction on the point cloud data collected by the airborne radar terminal in the current collection period, and obtaining the final feature representation of the point cloud data; Step s2: classify and annotate the point cloud data in several historical collection cycles through manual annotation methods, obtain the category label of each point in the point cloud data, and obtain the rare category label; Step s3: Build a large sample label classification model to obtain the large sample category labels of each point in the point cloud data of the current acquisition cycle; Step s4: obtaining transferable category labels corresponding to each rare category label, obtaining source label classification models of each rare category label according to the transferable category labels, and constructing small sample label classification models of each rare category label according to the source label classification models of each rare category label; Step s5: Obtain the small sample category label of each point in the point cloud data of the current acquisition cycle, and construct a complete category label set of the point cloud data of the current acquisition cycle based on the large sample category label and the small sample category label in the point cloud data.

[0006] Furthermore, the acquisition cycle is set, and feature extraction is performed on the point cloud data collected by the airborne radar terminal in the current acquisition cycle. The process of obtaining the final feature representation of the point cloud data includes: The point cloud data is preprocessed in data format, local features are extracted for each point in the point cloud data after the data format preprocessing, the local features of each point are obtained, a deep learning model is constructed, the local features of each point are input into the deep learning model, and the final feature representation of the point cloud data is output through the deep learning model.

[0007] Furthermore, the point cloud data in several historical collection cycles are classified and labeled by manual labeling methods, the category labels of each point in the point cloud data are obtained, and the process of obtaining rare category labels includes: The point cloud data collected by the airborne radar terminal in several historical collection cycles are obtained, the point cloud data in several historical collection cycles are classified and labeled in advance by a manual labeling method, the category label of each point in the point cloud data is obtained, and a statistical analysis is performed on several category labels included in the point cloud data in several historical collection cycles. The total number of each category label in several historical collection cycles is obtained, a training threshold is preset, the total number of each category label is compared with the training threshold, and the category labels whose total number is less than the training threshold are marked as rare category labels.

[0008] Furthermore, the process of obtaining a large sample category label of each point in the point cloud data of the current acquisition period includes: Constructing a large sample label classification model, taking the final feature representation of the point cloud data in several historical acquisition cycles and the category label of each point in the point cloud data as training data, and using the training data to train the large sample label classification model; The final feature representation of the point cloud data of the current acquisition cycle is input into the trained large sample label classification model, the category label of each point in the point cloud data is obtained according to the large sample label classification model, and the category label of each point is marked as the large sample category label.

[0009] Furthermore, the process of obtaining the transferable category labels corresponding to each rare category label includes: Obtain each historical collection period in which the rare category label is located, mark the category labels other than the rare category label in the historical collection period as other category labels, perform statistical analysis on the other category labels, obtain the cumulative number of times each other category label and the rare category label appear together in a historical collection period, filter out the other category label with the highest cumulative number, and mark the other category label as a migratable category label.

[0010] Furthermore, the process of obtaining the source label classification model of each rare category label according to the transferable category label includes: Obtain each historical acquisition cycle in which the migratable category label is located, obtain the final feature representation of the point cloud data in each historical acquisition cycle, use the final feature representation of the point cloud data in each historical acquisition cycle and the category label of each point in the point cloud data as a training set and a test set, build a source label classification model for rare category labels, input the training set into the source label classification model for training until the loss function training is stable, save the model parameters, test the source label classification model through the test set until it meets the preset requirements, and output the model parameters of the source label classification model.

[0011] Furthermore, according to the source label classification model of each rare category label, the process of constructing a small sample label classification model of each rare category label includes: Construct a small sample label classification model for rare category labels, and use the model parameters of the source label classification model for rare category labels as the initialization parameters of the small sample label classification model; The final feature representation of the point cloud data in each historical acquisition cycle where the rare category label is located and the category label of each point in the point cloud data are marked as a positive sample set, and the final feature representation of the point cloud data in each historical acquisition cycle where other category labels are located and the category label of each point in the point cloud data are marked as a negative sample set; Obtain the number of positive sample sets and the number of negative sample sets, and obtain the dynamic sampling number k of the positive sample set and the dynamic sampling number s of the negative sample set according to the number of positive sample sets and the number of negative sample sets; Randomly select k positive sample sets and s negative sample sets as the overall sample set, and iteratively train the small sample label classification model through the overall sample set. In each iterative training, obtain the probability that the classification result of the small sample label classification model for the overall sample set is a positive sample, and adjust the dynamic sampling number of positive sample sets and the dynamic sampling number of negative sample sets in the overall sample set in the next iterative training according to the probability, until the iterative training number of the small sample label classification model reaches the specified iteration number threshold, and output the small sample label classification model of rare category labels that has completed the training.

[0012] Furthermore, the process of obtaining the small sample category labels of each point in the point cloud data of the current acquisition cycle includes: The small sample label classification models of each rare category label are combined into a multi-label rare category classification model, the final feature representation of the point cloud data of the current acquisition cycle is input into the multi-label rare category classification model, the category label of each point in the point cloud data is obtained according to the multi-label rare category classification model, and the category label of each point is marked as a small sample category label.

[0013] Compared with the prior art, this invention solves the model performance problems caused by category imbalance and insufficient data, and improves the classification accuracy and generalization ability through the following aspects: 1. Data preprocessing and feature extraction By performing format preprocessing and local feature extraction on point cloud data, the data quality and feature validity of the input deep learning model are guaranteed, which helps to improve the learning efficiency and accuracy of the model.

[0014] 2. Manual labeling and rare category identification By manually annotating historical data sets and performing statistical analysis on category labels, rare categories are identified, which helps ensure the model's performance on rare categories and avoids classification bias caused by data imbalance.

[0015] 3. Large sample label classification model Constructing a large sample label classification model and using historical data sets for training can make full use of the large amount of existing data and improve the ability to recognize common categories.

[0016] 4. Transfer Learning By using transfer learning technology, the knowledge of large sample category models is transferred to rare categories. By finding transferable category labels and building corresponding source label classification models, the classification performance of rare categories can be improved when the amount of data is limited.

[0017] 5. Small sample label classification model By building a small sample label classification model and combining it with a dynamic sampling strategy, we ensure the model's generalization ability on rare categories, effectively address the problem of small sample categories in the data set, and improve the overall performance of the model.

[0018] 6. Combining multi-label rare category classification models The small sample label classification models of each rare category label are combined to form a multi-label rare category classification model to improve the performance in multi-category classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of an airborne lidar point cloud classification method based on transfer learning in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0021] like Figure 1 As shown, a method for classifying airborne laser radar point cloud based on transfer learning includes the following steps: Step s1: setting the collection period, performing feature extraction on the point cloud data collected by the airborne radar terminal in the current collection period, and obtaining the final feature representation of the point cloud data; Step s2: classify and annotate the point cloud data in several historical collection cycles through manual annotation methods, obtain the category label of each point in the point cloud data, and obtain the rare category label; Step s3: Build a large sample label classification model to obtain the large sample category labels of each point in the point cloud data of the current acquisition cycle; Step s4: obtaining transferable category labels corresponding to each rare category label, obtaining source label classification models of each rare category label according to the transferable category labels, and constructing small sample label classification models of each rare category label according to the source label classification models of each rare category label; Step s5: Obtain the small sample category label of each point in the point cloud data of the current acquisition cycle, and construct a complete category label set of the point cloud data of the current acquisition cycle based on the large sample category label and the small sample category label in the point cloud data.

[0022] It should be further explained that, in the specific implementation process, the acquisition cycle is set, and the feature extraction is performed on the point cloud data collected by the airborne radar terminal in the current acquisition cycle. The process of obtaining the final feature representation of the point cloud data includes: The point cloud data collected by the airborne radar terminal in the current collection cycle is obtained, and the collection cycle duration is 0.5 hours. The point cloud data includes three-dimensional coordinates and reflection intensity. The point cloud data is preprocessed in data format. The data format preprocessing includes removing noise points and outliers through median filtering technology, converting the point cloud data in the original coordinate system into a coordinate system suitable for subsequent processing, such as a geodetic coordinate system, dividing the point cloud data into multiple small areas according to a spatial grid, extracting local features of each point in the point cloud data after the data format preprocessing, and obtaining the local features of each point. The local features include geometric features such as height, density, curvature and normal vector, and constructing a deep learning model. The local features of each point are input into the deep learning model. The deep learning model used in the present invention For the PointNet++ model, the point cloud data is hierarchically clustered, and each layer represents features of different scales. The SetAbstraction module in the PointNet++ model is used to extract higher-level features on each layer, and the features of different levels are directly passed to the deep layer through jump connections, while maintaining multi-level information and reducing information loss. The features of different levels are spliced ​​to form a larger feature vector, retaining the information of each level, and different weights are assigned to the features of different levels. They are fused by weighted summation to emphasize the importance of certain levels. Based on the fused features, a global pooling operation is applied to extract the final global feature representation, which integrates local features, features of different levels, and global features to form the final feature representation.

[0023] It should be further explained that, in the specific implementation process, the point cloud data in several historical collection cycles are classified and labeled by manual labeling methods, the category labels of each point in the point cloud data are obtained, and the process of obtaining rare category labels includes: The point cloud data collected by the airborne radar terminal in several historical collection cycles are obtained, the point cloud data in several historical collection cycles are classified and labeled in advance by a manual labeling method, the category label of each point in the point cloud data is obtained, and a statistical analysis is performed on several category labels included in the point cloud data in several historical collection cycles. The total number of each category label in several historical collection cycles is obtained, a training threshold is preset, the total number of each category label is compared with the training threshold, and the category labels whose total number is less than the training threshold are marked as rare category labels.

[0024] It should be further explained that, in the specific implementation process, the process of constructing a large sample label classification model and obtaining the large sample category labels of each point in the point cloud data of the current acquisition period includes: Constructing a large sample label classification model, taking the final feature representation of the point cloud data in several historical acquisition cycles and the category label of each point in the point cloud data as training data, and using the training data to train the large sample label classification model; The final feature representation of the point cloud data of the current acquisition cycle is input into the trained large sample label classification model, the category label of each point in the point cloud data is obtained according to the large sample label classification model, and the category label of each point is marked as the large sample category label.

[0025] It should be further explained that, in the specific implementation process, the process of obtaining the transferable category labels corresponding to each rare category label includes: Obtain each historical collection period in which the rare category label is located, mark the category labels other than the rare category label in the historical collection period as other category labels, perform statistical analysis on the other category labels, obtain the cumulative number of times each other category label and the rare category label appear together in a historical collection period, filter out the other category label with the highest cumulative number, and mark the other category label as a migratable category label.

[0026] It should be further explained that, in the specific implementation process, the process of obtaining the source label classification model of each rare category label according to the transferable category label includes: Obtain each historical acquisition cycle in which the migratable category label is located, obtain the final feature representation of the point cloud data in each historical acquisition cycle, use the final feature representation of the point cloud data in each historical acquisition cycle and the category label of each point in the point cloud data as a training set and a test set, build a source label classification model for rare category labels, input the training set into the source label classification model for training until the loss function training is stable, save the model parameters, test the source label classification model through the test set until it meets the preset requirements, and output the model parameters of the source label classification model.

[0027] It should be further explained that, in the specific implementation process, the process of constructing the small sample label classification model of each rare category label according to the source label classification model of each rare category label includes: Construct a small sample label classification model for rare category labels, and use the model parameters of the source label classification model for rare category labels as the initialization parameters of the small sample label classification model; The final feature representation of the point cloud data in each historical collection cycle where the rare category label is located and the category label of each point in the point cloud data are marked as a positive sample set, and the final feature representation of the point cloud data in each historical collection cycle where other category labels are located and the category label of each point in the point cloud data are marked as a negative sample set; Obtain the number of positive sample sets and the number of negative sample sets, and obtain the dynamic sampling number k of the positive sample set and the dynamic sampling number s of the negative sample set according to the number of positive sample sets and the number of negative sample sets; Randomly select k positive sample sets and s negative sample sets as the overall sample set, and iteratively train the small sample label classification model through the overall sample set. In each iterative training, obtain the probability that the classification result of the small sample label classification model for the overall sample set is a positive sample, and adjust the dynamic sampling number of positive sample sets and the dynamic sampling number of negative sample sets in the overall sample set in the next iterative training according to the probability, until the iterative training number of the small sample label classification model reaches the specified iteration number threshold, and output the small sample label classification model of rare category labels that has completed the training.

[0028] It should be further explained that, in the specific implementation process, when the probability that the classification result of the small sample label classification model is a positive sample is lower than the preset probability threshold, the sampling number of positive samples is increased, thereby increasing the model's attention to positive samples; when the probability that the classification result is a positive sample is greater than the preset probability threshold, the sampling probability of positive samples is relatively reduced, thereby reducing the model's attention to positive samples, thereby increasing the model's distinguishability of negative samples, and improving the model's classification accuracy and confidence. At the same time, the dynamic sampling number of the positive sample set and the dynamic sampling number of the negative sample set in the overall sample set in the next iterative training are adjusted according to the probability. The calculation formula is: ; ; in, represents the probability threshold, represents the probability that the classification result in the t-th iteration training is a positive sample, represents the dynamic sampling number of the positive sample set in the t-th iteration training, It represents the dynamic sampling number of the negative sample set in the t-th iteration training, round() represents the rounding function, and exp represents the exponent.

[0029] It should be further explained that, in the specific implementation process, the process of obtaining the small sample category label of each point in the point cloud data of the current acquisition cycle includes: The small sample label classification models of each rare category label are combined into a multi-label rare category classification model, the final feature representation of the point cloud data of the current acquisition cycle is input into the multi-label rare category classification model, the category label of each point in the point cloud data is obtained according to the multi-label rare category classification model, and the category label of each point is marked as a small sample category label; Among them, the specific process of combining the small sample label classification models of each rare category label into a multi-label rare category classification model is: Use the layers before the last layer of each small sample label classification model to extract features, because the last layer is usually a classification layer, which is used to output category probabilities. In the layers before the last layer, the model has extracted a lot of comprehensive information about the input data. This information contains multiple levels of features, both low-level and high-level. Using the outputs of these layers as features can retain more detailed information about the input data. At the same time, using the layers before the last layer as features can avoid directly using the output of the classification layer, because the output of the classification layer is usually a highly optimized result for a specific task and is prone to overfitting to the training data, while the features of the penultimate layer (or earlier layers) are usually more general and can be better generalized to unseen data; The features extracted by each small sample label classification model are concatenated and combined, and then a multi-label rare category classification model is constructed based on the multi-layer perception MLP. The model takes the concatenated features as input and selects the cross entropy loss function to train the multi-label rare category classification model. Evaluate the performance of the multi-category classification model on the validation set, using indicators such as accuracy, precision, recall, and F1 score to evaluate the performance of the model and ensure that the model performs well enough on multiple rare categories; Based on the performance on the validation set, further optimize the model, such as adjusting the learning rate, changing the network architecture, etc., and test the final multi-label rare category classification model on an independent test set; Deploy the trained multi-label rare category classification model to the production environment for actual data classification tasks; The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for airborne laser radar point cloud classification based on transfer learning, characterized in that: The following steps are involved: Step s1: setting the collection period, performing feature extraction on the point cloud data collected by the airborne radar terminal in the current collection period, and obtaining the final feature representation of the point cloud data; Step s2: classify and annotate the point cloud data in several historical collection cycles through manual annotation methods, obtain the category label of each point in the point cloud data, and obtain the rare category label; Step s3: Build a large sample label classification model to obtain the large sample category labels of each point in the point cloud data of the current acquisition cycle; Step s4: obtaining transferable category labels corresponding to each rare category label, obtaining source label classification models of each rare category label according to the transferable category labels, and constructing small sample label classification models of each rare category label according to the source label classification models of each rare category label; Step s5: Obtain the small sample category label of each point in the point cloud data of the current acquisition cycle, and construct a complete category label set of the point cloud data of the current acquisition cycle based on the large sample category label and the small sample category label in the point cloud data.

2. The airborne laser radar point cloud classification method based on transfer learning according to claim 1, characterized in that: Set the collection cycle, extract features from the point cloud data collected by the airborne radar terminal in the current collection cycle, and obtain the final feature representation of the point cloud data. The process includes: The point cloud data is preprocessed in data format, local features are extracted for each point in the point cloud data after the data format preprocessing, the local features of each point are obtained, a deep learning model is constructed, the local features of each point are input into the deep learning model, and the final feature representation of the point cloud data is output through the deep learning model.

3. The airborne laser radar point cloud classification method based on transfer learning according to claim 2 is characterized in that: The process of classifying and labeling the point cloud data in several historical collection cycles by manual labeling methods, obtaining the category labels of each point in the point cloud data, and obtaining the rare category labels includes: The point cloud data collected by the airborne radar terminal in several historical collection cycles are obtained, the point cloud data in several historical collection cycles are classified and labeled in advance by a manual labeling method, the category label of each point in the point cloud data is obtained, and a statistical analysis is performed on several category labels included in the point cloud data in several historical collection cycles. The total number of each category label in several historical collection cycles is obtained, a training threshold is preset, the total number of each category label is compared with the training threshold, and the category labels whose total number is less than the training threshold are marked as rare category labels.

4. The airborne laser radar point cloud classification method based on transfer learning according to claim 3 is characterized in that: The process of obtaining a large sample of category labels for each point in the point cloud data of the current acquisition cycle includes: Constructing a large sample label classification model, taking the final feature representation of the point cloud data in several historical acquisition cycles and the category label of each point in the point cloud data as training data, and using the training data to train the large sample label classification model; The final feature representation of the point cloud data of the current acquisition cycle is input into the trained large sample label classification model, the category label of each point in the point cloud data is obtained according to the large sample label classification model, and the category label of each point is marked as the large sample category label.

5. The airborne laser radar point cloud classification method based on transfer learning according to claim 4 is characterized in that: The process of obtaining the transferable category labels corresponding to each rare category label includes: Obtain each historical collection period in which the rare category label is located, mark the category labels other than the rare category label in the historical collection period as other category labels, perform statistical analysis on the other category labels, obtain the cumulative number of times each other category label and the rare category label appear together in a historical collection period, filter out the other category label with the highest cumulative number, and mark the other category label as a migratable category label.

6. The airborne laser radar point cloud classification method based on transfer learning according to claim 5, characterized in that: The process of obtaining the source label classification model of each rare category label according to the transferable category label includes: Obtain each historical acquisition cycle in which the migratable category label is located, obtain the final feature representation of the point cloud data in each historical acquisition cycle, use the final feature representation of the point cloud data in each historical acquisition cycle and the category label of each point in the point cloud data as a training set and a test set, build a source label classification model for rare category labels, input the training set into the source label classification model for training until the loss function training is stable, save the model parameters, test the source label classification model through the test set until it meets the preset requirements, and output the model parameters of the source label classification model.

7. The airborne laser radar point cloud classification method based on transfer learning according to claim 6, characterized in that: According to the source label classification model of each rare category label, the process of constructing a small sample label classification model of each rare category label includes: Construct a small sample label classification model for rare category labels, and use the model parameters of the source label classification model for rare category labels as the initialization parameters of the small sample label classification model; The final feature representation of the point cloud data in each historical collection cycle where the rare category label is located and the category label of each point in the point cloud data are marked as a positive sample set, and the final feature representation of the point cloud data in each historical collection cycle where other category labels are located and the category label of each point in the point cloud data are marked as a negative sample set; Obtain the number of positive sample sets and the number of negative sample sets, and obtain the dynamic sampling number k of the positive sample set and the dynamic sampling number s of the negative sample set according to the number of positive sample sets and the number of negative sample sets; Randomly select k positive sample sets and s negative sample sets as the overall sample set, and iteratively train the small sample label classification model through the overall sample set. In each iterative training, obtain the probability that the classification result of the small sample label classification model for the overall sample set is a positive sample, and adjust the dynamic sampling number of positive sample sets and the dynamic sampling number of negative sample sets in the overall sample set in the next iterative training according to the probability, until the iterative training number of the small sample label classification model reaches the specified iteration number threshold, and output the small sample label classification model of rare category labels that has completed the training.

8. The airborne laser radar point cloud classification method based on transfer learning according to claim 7, characterized in that: The process of obtaining the small sample category labels of each point in the point cloud data of the current acquisition cycle includes: The small sample label classification models of each rare category label are combined into a multi-label rare category classification model, the final feature representation of the point cloud data of the current acquisition cycle is input into the multi-label rare category classification model, the category label of each point in the point cloud data is obtained according to the multi-label rare category classification model, and the category label of each point is marked as a small sample category label.

Images