Cross-domain small sample point cloud terrain classification method and system

Abstract

The invention relates to the technical field of point cloud data processing and application, in particular to a cross-domain small sample point cloud terrain classification method and system, and the method comprises the steps: pre-training a source domain feature extractor and a source domain classifier on a source data set; in the cross-domain transfer learning stage, point-level source domain features and target domain features from the same base class and corresponding prediction results are input into a conditional domain adversarial network to train a target feature extractor and a target classifier of a target domain teacher model, and the target domain features are obtained by using small sample data through the target feature extractor; in a class incremental learning stage in a target domain, a student model in the target domain incrementally learns features of a new class, and incremental feature semantic knowledge distillation is performed between a target extractor of a learned teacher model and a current student model; meanwhile, a semantic adversarial learning module is used for distinguishing the feature representation of the new category and the feature representation of the basic category. According to the method, the precision of point cloud cross-domain transfer learning under the small sample condition is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of point cloud data processing and application technology, and in particular to a cross-domain small sample point cloud object classification method and system. Background Art

[0002] Point cloud semantic segmentation has made rapid progress in recent years, providing new possibilities for terrain mapping and object extraction. Under the supervised training of fully labeled samples, many advanced methods have been proposed to improve the classification performance in complex scenes. However, existing point cloud object classification research focuses on static scenes with similar distributions of training and test samples. When the scene changes, it is difficult for the trained models to adapt to the new differences. Even for models trained in similar scenes, they need to continue learning when encountering rare new categories. At the same time, it is usually not feasible to annotate the entire scene when the target scene is migrated. Therefore, under the needs of a wide range of applications, cross-scene transfer learning has gradually attracted attention. However, the problems of inconsistent categories, domain differences and small samples make it challenging to improve the generalization performance of the model.

[0003] Model fine-tuning is a common migration method to solve the problem of inconsistent labels. However, this relies on a common, large set of basic training samples and a powerful general feature extraction network. Compared with two-dimensional basic models and datasets such as ImageNet, three-dimensional semantic segmentation lacks both aspects. Previous work has confirmed that when there is a large difference between new and old data, fine-tuning may cause the model to forget previously learned general knowledge and be unable to cope with changes in data distribution. Incremental learning updates the model by continuously adding new data to solve the problem of catastrophic forgetting. Many incremental learning methods used in two-dimensional classification tasks use cosine similarity for category comparison, but these methods are difficult to apply to sparse point cloud features. With the addition of new data, the significant data distribution differences between point cloud categories will aggravate the decline in the generalization performance of the model on new categories.

[0004] In order to make the features learned by the model consistent with the data distribution of the target domain, domain adaptation methods reduce the differences between domains by adjusting model parameters or feature representation. However, most works assume that the categories of the source domain and the target domain are consistent, which leads to the model may not be able to migrate to new categories in new scenes when there are significant distribution and semantic differences between different scenes. In particular, the samples of new categories are usually very few compared to general categories. Due to the bias in data sampling, the model is easy to overfit on the small sample data distribution. Semi-supervised learning methods consider using the data distribution of a large number of unlabeled samples to balance sample differences, but in real scenes, domain-specific categories are usually difficult to estimate. Small sample learning uses multi-task sampling to efficiently learn the differences between categories, so that the model learns to learn. However, the imbalance of point cloud data samples and inconsistent features will further aggravate the decline in model performance. Therefore, in order to be able to effectively generalize to new small sample categories, this requires the model to not only remember the old general knowledge, but also dynamically and quickly adapt to the new category data distribution of small samples. In summary, cross-scene migration faces the following difficulties: 1) Existing general small-sample classification methods are difficult to solve the problem of inconsistent categories; 2) The small number of samples of scene-specific object categories makes it difficult for the model to learn discriminative features; 3) The sampled data distribution bias and cross-scene offset will affect the generalization performance of the model. Summary of the invention

[0005] The purpose of this invention is to solve the problem that the current deep learning model has insufficient applicability to different target scenarios with only a small number of labels, and to propose a cross-domain small sample point cloud object classification method and system, which effectively improves the accuracy of cross-domain point cloud transfer learning under small sample conditions.

[0006] To achieve the above purpose, the technical solution adopted is:

[0007] A cross-domain small sample point cloud object classification method, including:

[0008] Pre-train the source domain feature extractor and source domain classifier on the source dataset;

[0009] In the cross-domain transfer learning stage, the point-level source domain features, target domain features and corresponding prediction results from the same base class are input into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, where the target domain features are obtained by the target feature extractor using small sample data;

[0010] In the incremental category learning stage within the target domain, the student model in the target domain incrementally learns the features of new categories, performs incremental feature semantic knowledge distillation between the target extractor of the learned teacher model and the current student model, and uses the semantic adversarial learning module to distinguish the feature representation of new categories from that of base categories.

[0011] According to the cross-domain small sample point cloud object classification method of the present invention, further, the conditional domain adversarial network aims to minimize the prediction error of the domain discriminator for source domain features and target domain features with the same labels, so that the domain discriminator cannot distinguish the feature representations generated across domains; since the prediction results contain potential information, the joint distribution of the features and the prediction results is used as the input of the domain discriminator D.

[0012] According to the cross-domain small sample point cloud object classification method of the present invention, further, the joint variable h=(f,g)=(G f (X),G c (G f (X))) replaces the feature input of the domain discriminator, where G f and G c Represent the feature extractor and classifier respectively, X represents the input data; the joint distribution of features and prediction results modeled using random multilinear mapping is as follows:

[0013]

[0014] in, and is a random matrix, and d< <d f ×d g , ⊙ is the XOR operation.

[0015] According to the cross-domain small sample point cloud object classification method of the present invention, further, the adversarial loss formula of the conditional domain adversarial network is as follows:

[0016]

[0017] in, and denote the target feature extractor and target classifier of the teacher model, respectively, and h s is the source domain joint variable, h t is the target domain joint variable, and Represents the expectation of data distribution in the source domain and the target domain; in this stage, only the target feature extractor, target classifier, and domain discriminator of the target domain teacher model are optimized:

[0018]

[0019] in, Represents the base class data of the target dataset. represents the target feature extractor parameters and the target classifier parameters, the superscript s represents the source domain, and the superscript T represents the target domain.

[0020] According to the cross-domain small sample point cloud object classification method of the present invention, further, since a small number of target samples have labels, a supervised cross entropy loss function is calculated:

[0021]

[0022] Among them, C base represents the number of base classes in the target dataset, p i represents the predicted category label of the i-th point, q i Represents the true category label of the i-th point.

[0023] According to the cross-domain small sample point cloud object classification method of the present invention, further, the total loss of cross-domain transfer learning is calculated as follows:

[0024]

[0025] Among them, θ d is the domain discriminator parameter, Y t base is the true category label of the base class data of the target dataset, and λ1 is the weighting factor.

[0026] According to the cross-domain small sample point cloud object classification method of the present invention, further, the target feature extractor parameters of the teacher model are used and the target classifier parameters Initialize the student model and define the semantic loss of the student model for the base category and the new category:

[0027]

[0028] Among them, C base +C novel is the total number of categories, p i represents the predicted category label of the i-th point, and qi represents the true category label of the i-th point.

[0029] According to the cross-domain small sample point cloud object classification method of the present invention, further, the incremental feature semantic knowledge distillation includes:

[0030] First, the local structural information in the teacher model is transferred to all input points in the student model through feature distillation:

[0031]

[0032] Feature distillation is achieved by minimizing the embedding f (X) and the student embedding network G′ f (X) is achieved by the mean square error between the local features extracted;

[0033] Then the KL divergence loss between the teacher model and the student model is minimized to model the implicit inter-category information between the underlying categories:

[0034]

[0035] Among them, logits base',c and logits base,c are the log odds of the cth base class in the student classifier and the teacher classifier, respectively. T represents the distillation temperature. t is a scalar used for control. β represents the hyperparameter that controls the influence of temperature T on KL divergence.

[0036] According to the cross-domain small sample point cloud object classification method of the present invention, further, the adversarial loss formula of the semantic adversarial learning module is as follows:

[0037]

[0038] in, and represents the expected distribution of base class and new class target data, Ds represents the semantic discriminator, G′ f (X) represents the target feature extractor of the student model;

[0039] The total loss of the final intra-domain category incremental learning is as follows:

[0040]

[0041] Among them, λ2, λ3, λ4, and λ5 are all weighting factors.

[0042] Furthermore, the present invention also proposes a cross-domain small sample point cloud object classification system, comprising:

[0043] The source domain pre-training module is used to pre-train the source domain feature extractor and source domain classifier on the source dataset;

[0044] A cross-domain learning module is used to input point-level source domain features, target domain features and corresponding prediction results from the same base class into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, wherein the target domain features are obtained by the target feature extractor using small sample data;

[0045] The incremental learning module is used to incrementally learn the features of new categories in the student model of the target domain, perform incremental feature semantic knowledge distillation between the target extractor of the learned teacher model and the current student model, and use the semantic adversarial learning module to distinguish the feature representation of new categories from that of basic categories.

[0046] The beneficial effects achieved by adopting the above technical solution are:

[0047] In order to solve the problems of insufficient airborne lidar point cloud data for different target scenes and poor cross-scene migration and generalization capabilities of existing methods, the present invention proposes a cross-domain small sample point cloud object classification method. Unlike previous methods that are limited to a small number of samples in the same field or domain adaptation methods that are limited to the same semantic labels, the proposed method proposes two strategies, cross-domain transfer learning and intra-domain incremental learning, for domain-shared categories and domain-specific categories, respectively, which are suitable for a wider range of application scenarios. Experimental results show that this method has proven its effectiveness in three groups of cross-domain migration tasks of small sample airborne lidar point clouds, and is superior to the most advanced small sample learning methods, especially showing good generalization capabilities on those domain-specific minority categories. Specifically, the present invention has the following advantages:

[0048] (1) This paper proposes a solution that combines cross-domain transfer learning with category incremental learning to solve the 3D cross-domain transfer problem. By decomposing the task into two learning stages from a category perspective, the proposed framework can simultaneously address the challenges of few-shot learning and domain adaptation.

[0049] (2) This paper proposes a cross-domain transfer learning method based on subject categories. In order to generalize common features to different target domains, a conditional domain adversarial network is introduced in the fine-tuning process of the main categories. The use of aligned semantic labels enables dynamic adaptation in different scenarios without the need for a large amount of labeled data.

[0050] (3) This paper proposes an intra-domain category incremental learning method. In order to learn domain-specific category features, this method can not only retain basic features but also continuously learn new category features. Combined with small sample learning, the proposed multi-level distillation constraints and semantic adversarial learning module can further improve the generalization performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0052] Figure 1 1 is a flow chart of a method for classifying small sample point cloud objects across domains according to an embodiment of the present invention;

[0053] Figure 2 is a schematic diagram of an intra-domain category incremental learning process according to an embodiment of the present invention;

[0054] Figure 3 (a)-(d) are the experimental results of Proto-3D, attMPTI, Thr-MPRNet and Fine-tuning methods on the segmentation of the DALES target dataset; Figure 3 (e) is the experimental result of the method of the present invention on the segmentation of the DALES target data set; Figure 3 (f) is the standard segmentation result of the DALES target dataset;

[0055] Figure 4 (a)-(d) are the experimental results of Proto-3D, attMPTI, Thr-MPRNet and Fine-tuning methods on the ISPRS target dataset segmentation; Figure 4 (e) is the experimental result of the segmentation of the ISPRS target data set by the method of the present invention; Figure 4 (f) is the standard segmentation result of the ISPRS target dataset;

[0056] Figure 5 (a)-(d) are the experimental results of Proto-3D, attMPTI, Thr-MPRNet and Fine-tuning methods on the segmentation of H3D target dataset; Figure 5 (e) is the experimental result of the segmentation of H3D target data set by the method of the present invention; Figure 5 (f) is the standard segmentation result of the H3D target dataset. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0058] This embodiment discloses a cross-domain small sample point cloud object classification method, such as Figure 1 As shown, it contains the following contents:

[0059] Step S101: pre-train a source domain feature extractor and a source domain classifier on a source data set.

[0060] Given a source dataset Ds containing N points and m classes and their corresponding labels, we train the source domain feature extractor on the annotated source data. and source domain classifier Get the parameters of the source embedding network

[0061] Step S102: In the cross-domain transfer learning stage, the point-level source domain features, target domain features and corresponding prediction results from the same base class are input into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, wherein the target domain features are obtained by the target extractor using small sample data. Cross-domain transfer learning is performed using the source domain features and target domain features of the common category to reduce domain differences.

[0062] Through the key representations in the dataset, the main categories can reflect the distribution of the entire data. When there are only a few labeled samples, fine-tuning the model pre-trained on the main categories of the source can obtain target-specific features, but it may also lead to overfitting. Therefore, by paying attention to the changes in the main categories, the distribution shift between domains can be indicated in order to obtain more general performance on a small number of samples. Based on this, a cross-domain learning module is proposed to alleviate the inconsistency of domain distribution during fine-tuning.

[0063] Specifically, we first use the learned parameters Initialize the target extractor Next, adversarial learning and domain adaptation are integrated in the proposed cross-domain learning module, whose goal is to minimize the prediction error of the domain discriminator for source domain features and target domain features with the same label, so that the domain discriminator cannot distinguish the feature representations generated across domains. However, domain adaptation of embeddings alone is not enough, because the prediction results may also contain potential information. Therefore, inspired by the conditional domain adversarial network, the domain discriminator D is conditioned on the joint distribution of features and predictions. Specifically, the joint variable h = (f, g) = (G f (X),G c (G f (X))) replaces the feature input of the domain discriminator, where G f and G c Respectively represent feature extractors and classifiers (including source domains and target domains), and X represents input data (including source datasets and target dataset base class data). In order to model joint distribution, random multilinear mapping is used to alleviate the computational burden brought by high-dimensional point cloud features:

[0064]

[0065] in, and is a random matrix, and d< <d f ×d g , ⊙ is an XOR operation, using element-wise product to accurately approximate the multilinear mapping. The adversarial loss formula of the conditional domain adversarial network is as follows:

[0066]

[0067] in, and denote the target feature extractor and target classifier of the teacher model, respectively, and h s is the source domain joint variable, h t is the target domain joint variable, and Represents the expectation of the data distribution of the source domain and the target domain. Since the general embedding of the main categories is learned from the well-annotated source dataset, only the target feature extractor, target classifier and domain discriminator are optimized at this stage:

[0068]

[0069] in, Represents the base class data of the target dataset. represents the target feature extractor parameters and the target classifier parameters, the superscript s represents the source domain, and the superscript T represents the target domain.

[0070] Since a small number of target samples have labels, a supervised cross entropy loss function can be calculated:

[0071]

[0072] Among them, C base represents the number of base classes in the target dataset, p i represents the predicted category label of the i-th point, q i Represents the true category label of the i-th point.

[0073] Finally, the total loss of cross-domain transfer learning is calculated as follows:

[0074]

[0075] Among them, θ d is the domain discriminator parameter, Y t base is the true category label of the base class data of the target dataset, and λ1 is the weighting factor.

[0076] Step S103, incrementally learn the features of new categories in the target domain, while ensuring that the previously learned base class features are not forgotten. Specifically, based on the target teacher model and the new class data of the target domain, incremental feature semantic knowledge distillation is performed between the target extractor of the previously learned teacher model and the current student model, and a semantic adversarial learning module is used to prevent the confusion between the new class features and the base class features, such as Figure 2 shown.

[0077] In addition to the differences in domain distribution, each target dataset has its own unique feature categories. For the migration task, these categories can be regarded as new categories that are different from the previous base categories. Since the samples of the new categories are also limitedly annotated, the proposed incremental learning phase in the domain needs to focus on learning new semantics while not forgetting the base class features. Based on this, the scheme in this case considers two aspects at the same time: 1) expanding the existing feature space for the new category, but the new category features cannot be confused with the old category; 2) under the condition of small samples, do not overfit to the new category, nor forget the old category. Therefore, the distillation method is used to retain previous knowledge. At the same time, in order to avoid overfitting on the old category, a semantic discriminator that is adversarial to the feature extractor is used to better distinguish between new and old categories, thereby improving the model's ability to continuously learn new knowledge. Distillation constraints, adversarial training, and small sample supervision are used together to enhance the continuous learning of new knowledge.

[0078] In order to dynamically learn domain-specific categories, this embodiment proposes adding new dimensions to the classifier to accommodate new categories. During the learning process of the base category, the classifier is separated from the feature extractor. In order to independently expand the output layer of the classifier, the number of nodes with randomly initialized weights is increased. Each node in the output layer corresponds to a category label, so the total number of nodes matches the base category and the new category. Subsequently, the feature extractor and the classifiers of all categories are jointly trained to better generalize to the new category. Using the target feature extractor obtained in step S102 and target classifier Initialize the student model and define the semantic loss of the student model for the base category and the new category:

[0079]

[0080] Among them, C base +C novel is the total number of categories, p i represents the predicted category label of the i-th point, q i Represents the true category label of the i-th point.

[0081] Given that the feature space will inevitably change to adapt to the new semantics, multi-level knowledge distillation is used to prevent the catastrophic forgetting of basic features. Specifically, knowledge distillation is performed between the teacher model and the student model in the previous stage.

[0082] In the incremental learning 3D semantic segmentation task, the spatial consistency between new categories and base categories is crucial. First, local information in the point cloud is used to avoid segmentation discontinuities. By continuously aggregating neighborhood features, DGCNN extracts point-by-point local information through graph convolution. Therefore, by performing feature distillation, the local structural information in the teacher model is transferred to all input points in the student model:

[0083]

[0084] Feature distillation is achieved by minimizing the embedding f (X) and the student embedding network G f It is achieved by calculating the mean square error between the local features extracted from ′(X).

[0085] The predicted log-odds reflect the class confidence of each point, and the difference in probability distribution between the base categories can also indicate implicit inter-category information. Therefore, in addition to local feature distillation, this embodiment also uses the probability information of the base categories obtained in the classifier output to retain previous knowledge. The output log-odds of the teacher classifier are used as soft labels to guide the student model to learn the base categories. The implicit inter-category information between the base categories is simulated by minimizing the Kullback-Leibler (KL) divergence loss between the teacher and student outputs:

[0086]

[0087] Among them, logits base',c and logits base,c are the log odds of the cth base class in the student classifier and the teacher classifier, respectively. T represents the distillation temperature. t is a scalar used for control. β represents the hyperparameter that controls the influence of temperature T on KL divergence.

[0088] Even when new semantics are involved while retaining the base features, there is still the problem of mutual interference in the old feature space because no unique features are learned. Therefore, to further prevent feature confusion, a semantic adversarial learning module is proposed to isolate the feature representations of new categories and base categories.

[0089] Specifically, given a target input whose semantic labels include base categories and new categories, a semantic discriminator will be used to compete with the feature extractor to distinguish between base categories and new categories. f (X) Learn shared features through adversarial loss to achieve a unified feature space. The adversarial loss formula is as follows:

[0090]

[0091] in, and represents the expected distribution of base class and new class target data, Ds represents the semantic discriminator, G f ′(X) represents the target feature extractor of the student model.

[0092] The total loss of the final intra-domain category incremental learning is as follows:

[0093]

[0094] Among them, λ2, λ3, λ4, and λ5 are all weighting factors.

[0095] Corresponding to the above method, this embodiment also proposes a cross-domain small sample point cloud object classification system, including:

[0096] The source domain pre-training module is used to pre-train the source domain feature extractor and source domain classifier on the source dataset.

[0097] The cross-domain learning module is used to input point-level source domain features, target domain features and corresponding prediction results from the same base class into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, where the target domain features are obtained through the target feature extractor using small sample data.

[0098] The incremental learning module is used to incrementally learn the features of new categories in the student model of the target domain, perform incremental feature semantic knowledge distillation between the target extractor of the learned teacher model and the current student model, and use the semantic adversarial learning module to distinguish the feature representation of new categories from that of basic categories.

[0099] In order to evaluate the cross-scene transfer performance of the proposed method on airborne LiDAR point clouds, knowledge is transferred from well-annotated photogrammetric point cloud datasets. Specifically, four different datasets are utilized for experiments. The Cambridge Area (CAM) in the photogrammetric point cloud SensatUrban dataset is used as the source dataset. Three airborne LiDAR point cloud datasets with different densities and scales, including DALES, ISPRS, and H3D, are used as target datasets. The detailed dataset comparison is shown in Table 1, where the base categories in each dataset are highlighted in bold.

[0100] Table 1 Detailed information of experimental dataset

[0101]

[0102]

[0103] The large-scale ALS and photogrammetric point clouds are first divided into 50×50 meter blocks, each containing 4096 points, and their coordinates are normalized before feature learning. Then, in each stage, the feature embedding network extracts 448-dimensional features for classification. Each classifier contains two multi-layer perceptrons and an activation function, where the number of final nodes corresponds to the number of semantics for each task. During cross-domain transfer learning and intra-domain incremental learning, the Adam optimizer is used with a learning rate of 0.001 and a weight decay of 0.0001, and the framework is trained for 400 and 200 cycles, respectively. The method is evaluated on the photogrammetric CAM source dataset and three ALS point cloud target datasets, using the mean intersection over union (mIoU) and overall accuracy (OA) metrics. All experiments are performed on a single 16G GeForce RTX 3090 GPU.

[0104] Test verification and comparison results:

[0105] In order to evaluate the effectiveness of the proposed method in semantic segmentation of small-sample airborne lidar point clouds, it is compared with some state-of-the-art learning methods. Specifically, the comparison methods are selected in line with the goals of this study: First, the pre-training fine-tuning method, in which a small number of samples are jointly used to train a fine-tuned feature extractor and a new classifier capable of classifying base classes and new classes. In order to compare with meta-learning methods that only use a small number of samples for cross-category learning, a 3D meta-learning baseline Proto-3D based on ProtoNet is first designed, which serves as the basis for meta-learning in 2D tasks. In addition, comparisons are also made with the latest 3D semantic segmentation methods attMPTI and Thr-MPRNet.

[0106] Figures 3 to 5 , Tables 2-4 give the comparison results on the three target datasets. It can be observed that the proposed method outperforms other methods in all experiments. Compared with the highest scores obtained by other methods, the proposed method improves the overall accuracy by approximately 5.69%, 1.94%, and 0.62% on the DALES, ISPRS, and H3D datasets, respectively. This demonstrates that it is able to learn more transferable features between different domains. In addition, meta-learning-based methods generally achieve higher accuracy than fine-tuning methods, except on the H3D dataset. Although 4,096 points are sampled per block, the high-density point cloud of the DALES dataset allows for more detailed extraction of local features, leading to higher classification accuracy. In comparison, the point density is only 4-7 points / m 2The sparse ISPRS dataset has a significant difference between its small amount of annotated data and the test area, resulting in low accuracy. The lower average accuracy of the H3D dataset is attributed to its overly fine-grained category annotations, where there is a lack of significant inter-category differences between the 11 categories. This can be seen from the low classification accuracy of categories such as Veh, Shrub, Soil, and Chim. In addition, the classification results of each category show that the performance of the proposed method is more consistent with that of the fine-tuning method. However, the accuracy of the meta-learning method always fluctuates between different categories.

[0107] Table 2 Quantitative comparison results on the DALES dataset (%)

[0108]

[0109] Table 3 Quantitative comparison results on ISPRS dataset (%)

[0110]

[0111] Table 4 Quantitative comparison results on H3D dataset (%)

[0112]

[0113]

[0114] Aiming at the problem of insufficient airborne laser radar point cloud data for different target scenes and poor cross-scene migration and generalization ability of existing methods, this paper proposes a cross-domain small sample point cloud object classification method. Different from the previous small sample methods limited to the same field or the domain adaptation methods limited to the same semantic labels, the proposed method proposes two strategies, cross-domain migration learning and intra-domain incremental learning, for domain-shared categories and domain-specific categories, respectively, solving these two problems at the same time and being applicable to a wider range of application scenarios. Experimental results show that the proposed method has proven its effectiveness in three sets of small sample airborne laser radar point cloud cross-domain migration tasks, and outperforms the most advanced small sample learning methods, especially showing good generalization ability on those domain-specific minority categories.

[0115] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0116] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0117] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0118] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cross-domain small sample point cloud object classification method, characterized in that: Include: Pre-train the source domain feature extractor and source domain classifier on the source dataset; In the cross-domain transfer learning stage, the point-level source domain features, target domain features and corresponding prediction results from the same base class are input into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, where the target domain features are obtained by the target feature extractor using small sample data; In the incremental category learning stage within the target domain, the student model in the target domain incrementally learns the features of new categories, performs incremental feature semantic knowledge distillation between the target extractor of the learned teacher model and the current student model, and uses the semantic adversarial learning module to distinguish the feature representation of new categories from that of base categories.

2. The cross-domain small sample point cloud object classification method according to claim 1 is characterized in that: The goal of the conditional domain adversarial network is to minimize the prediction error of the domain discriminator for source domain features and target domain features with the same labels, so that the domain discriminator cannot distinguish the feature representations generated across domains; since the prediction results contain potential information, the joint distribution of features and prediction results is used as the input of the domain discriminator D.

3. The cross-domain small sample point cloud object classification method according to claim 2 is characterized in that: Use the joint variable h = (f, g) = (G f (X),G c (G f (X))) replaces the feature input of the domain discriminator, where G f and G c Represent the feature extractor and classifier respectively, X represents the input data; the joint distribution of features and prediction results modeled using random multilinear mapping is as follows: in, and is a random matrix, and d<<d f ×d g , ⊙ is the XOR operation.

4. The cross-domain small sample point cloud object classification method according to claim 3 is characterized in that: The adversarial loss formula of the conditional domain adversarial network is as follows: in, and denote the target feature extractor and target classifier of the teacher model, respectively, and h s is the source domain joint variable, h t is the target domain joint variable, and Represents the expectation of data distribution in the source domain and the target domain; in this stage, only the target feature extractor, target classifier, and domain discriminator of the target domain teacher model are optimized: in, Represents the base class data of the target dataset. represents the target feature extractor parameters and the target classifier parameters, the superscript s represents the source domain, and the superscript T represents the target domain.

5. The cross-domain small sample point cloud object classification method according to claim 4 is characterized in that: Since a small number of target samples have labels, a supervised cross entropy loss function is calculated: Among them, C base represents the number of base classes in the target dataset, p i represents the predicted category label of the i-th point, q i Represents the true category label of the i-th point.

6. The cross-domain small sample point cloud object classification method according to claim 5 is characterized in that: The total loss of cross-domain transfer learning is calculated as follows: Among them, θ d is the domain discriminator parameter, Y t base is the true category label of the base class data of the target dataset, and λ1 is the weighting factor.

7. The cross-domain small sample point cloud object classification method according to claim 4 is characterized in that: Utilize the target feature extractor parameters of the teacher model and the target classifier parameters Initialize the student model and define the semantic loss of the student model for the base category and the new category: Among them, C base +C novel is the total number of categories, p i represents the predicted category label of the i-th point, q i Represents the true category label of the i-th point.

8. The cross-domain small sample point cloud object classification method according to claim 7 is characterized in that: Incremental feature semantic knowledge distillation includes: First, the local structural information in the teacher model is transferred to all input points in the student model through feature distillation: Feature distillation is achieved by minimizing the embedding f (X) and the student embedding network G f It is achieved by the mean square error between the local features extracted from ′(X); Then the KL divergence loss between the teacher model and the student model is minimized to model the implicit inter-category information between the underlying categories: Among them, logits base',c and logits base,c are the log odds of the cth base class in the student classifier and the teacher classifier, respectively. T represents the distillation temperature. t is a scalar used for control. β represents the hyperparameter that controls the influence of temperature T on KL divergence.

9. The cross-domain small sample point cloud object classification method according to claim 8, characterized in that: The adversarial loss formula of the semantic adversarial learning module is as follows: in, and represents the expected distribution of base class and new class target data, Ds represents the semantic discriminator, G′ f (X) represents the target feature extractor of the student model; The total loss of the final intra-domain category incremental learning is as follows: Among them, λ2, λ3, λ4, and λ5 are all weighting factors.

10. A cross-domain small sample point cloud object classification system, characterized in that: Include: The source domain pre-training module is used to pre-train the source domain feature extractor and source domain classifier on the source dataset; A cross-domain learning module is used to input point-level source domain features, target domain features and corresponding prediction results from the same base class into the conditional domain adversarial network to train the target feature extractor and target classifier of the target domain teacher model, wherein the target domain features are obtained by the target feature extractor using small sample data; The incremental learning module is used to incrementally learn the features of new categories in the student model of the target domain, perform incremental feature semantic knowledge distillation between the target extractor of the learned teacher model and the current student model, and use the semantic adversarial learning module to distinguish the feature representation of new categories from that of basic categories.

Images