Remote sensing image classification method based on multi-classifier adversarial active transfer learning

Abstract

This invention provides a remote sensing image classification method based on multi-classifier adversarial active transfer learning, comprising: acquiring a hyperspectral remote sensing image to be classified; inputting the hyperspectral remote sensing image to be classified into a classification model, and the classification model outputting a classification result; the classification model is trained based on source domain data including multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data including only multiple sample hyperspectral remote sensing images. During the training process of the classification model, the training network structure includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process includes a transfer learning phase and an active learning phase. This invention can improve the classification accuracy of unsupervised transfer strategies, solve the problem of poor compatibility between traditional active learning and transfer learning in terms of point selection strategies, and ultimately effectively improve the classification accuracy of hyperspectral images.

Description

Technical Field

[0001] This invention relates to the field of pattern recognition technology, and in particular to a remote sensing image classification method based on multi-classifier adversarial active transfer learning. Background Technology

[0002] Hyperspectral image classification has always been a research hotspot in the field of remote sensing. It encompasses the spectral and spatial information of ground features. The type and development of ground features cause variations in the values ​​of remote sensing image data. By analyzing the patterns of change in remote sensing image data, this information can be used to classify and identify ground features. Current hyperspectral remote sensing data can simultaneously acquire information from dozens or even hundreds of spectral bands. This rich spectral information greatly improves the ability to identify and distinguish various types of ground features. Its application technologies are also constantly improving, and it has already been applied in many fields, such as geological exploration, geological mapping, marine remote sensing, atmospheric environment, agricultural monitoring, soil surveys, water quality monitoring, and battlefield reconnaissance.

[0003] Unsupervised domain adaptation, as a key research area in transfer learning, has received widespread attention. However, in unsupervised transfer learning, due to the lack of true label data in the target domain for most classification tasks, a portion of the data will always be misclassified. Unsupervised domain adaptation still lags far behind supervised learning in terms of classification accuracy. Active learning to query and incorporate true label data in the target domain can significantly improve the model's classification performance. Therefore, some research has focused on combining active learning and transfer learning to maximize the enhancement of unsupervised domain adaptation models.

[0004] Traditional active transfer learning initially employed a two-stage training approach: alternating between domain adaptation and active learning querying labeled data points. However, this was merely a simple combination. Traditional active learning may be unsuitable for transfer learning scenarios, as these query strategies may fail to select information-rich data points within the target domain under domain adaptation. Traditional active learning strategies still present challenges in transfer learning situations. Therefore, the combination of active learning and transfer learning needs to be compatible, ensuring that the information-rich data points queried by active learning can maximize the classification performance of transfer learning.

[0005] In summary, traditional active learning strategies suffer from incompatibility with transfer learning. Traditional active learning selects the most useful data from a large dataset of unlabeled target domain data for training through query strategies, which is usually successful in single-domain scenarios. However, since current domain adaptation efforts cannot fully align the distributions of the source and target domains, these traditional standards may not be applicable to transfer learning scenarios. Traditional active learning query strategies may not be able to select data points with high information content in the target domain under domain adaptation.

[0006] In summary, how to effectively improve the accuracy of unsupervised classification and find an active learning algorithm that matches the transfer strategy remains a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] This invention provides a remote sensing image classification method based on multi-classifier adversarial active transfer learning, which solves the problems of low accuracy of unsupervised classification in the prior art and the lack of active learning algorithms that match the transfer strategy.

[0008] This invention provides a remote sensing image classification method based on multi-classifier adversarial active transfer learning, comprising:

[0009] Acquire hyperspectral remote sensing images to be classified;

[0010] The hyperspectral remote sensing image to be classified is input into the classification model, and the classification model outputs the classification result.

[0011] The classification model is trained on source domain data, which includes multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data, which includes only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process of the classification model includes a transfer learning stage and an active learning stage.

[0012] According to the present invention, a remote sensing image classification method based on multi-classifier adversarial active transfer learning is provided, wherein the training process of the transfer learning stage is as follows:

[0013] First, the source domain data and the target domain data are input into the training network for adversarial transfer learning. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn the knowledge of the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned, as well as the domain adaptation of the source domain data and the target domain data.

[0014] According to the present invention, a remote sensing image classification method based on multi-classifier adversarial active transfer learning is provided, wherein the learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn knowledge of the source domain data, and the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned. The implementation method is as follows:

[0015] The first total loss function for training the network is determined by minimizing the classification cross-entropy loss function of the first classifier C1 and the second classifier C2, and minimizing the overall and maximum mean difference loss of the source domain data and target domain data after passing through the feature extractor G.

[0016] According to the remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the present invention, the first total loss function f1 is expressed by the following formula:

[0017] f1=min(Loss_C1_Xs+Loss_C2_Xs+Loss_MMD)

[0018]

[0019]

[0020]

[0021] Where, x i Represents the i-th data point, y i Let Ni represent the label of the i-th data point, Ns and Nt represent the number of data points in the source and target domains, respectively, and S be the softmax function. It is a feature mapping function that maps the features extracted by the feature extractor G to a high-dimensional Hilbert space. Then, in the high-dimensional Hilbert space, constraints are applied to the overall maximum mean difference and the maximum mean difference of each category. C is the number of data categories, and N is the number of categories. s c and N t c These are the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. N s λ represents the number of data points in the source domain data. Since the target domain does not have true labels, the classification results obtained by the classifier trained on the source domain data are used as pseudo-labels in the calculation. λ is the trade-off parameter for the maximum mean difference loss of each class. y () represents the cross-entropy loss function.

[0022] According to the present invention, a remote sensing image classification method based on multi-classifier adversarial active transfer learning is provided, wherein the entity relationships include semantic relationships, temporal relationships, and spatial relationships.

[0023] According to the remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the present invention, the domain adaptation of the source domain data and the target domain data is implemented as follows:

[0024] With the parameters of the feature extractor G fixed, the target domain data X t After data transformation, the data is input into the first classifier C1 and the second classifier C2. Then, the difference between the classification results of the two classifiers is maximized. Next, the first classifier C1 and the second classifier C2 are fixed, and the difference between the classification results of the two classifiers is minimized to determine the second total loss function of the training network.

[0025] According to the remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the present invention, the second total loss function f2 is expressed by the following formula:

[0026]

[0027]

[0028] Where, θ G θ C1 and θ C2 These are the network parameters of the feature extractor G, the first classifier C1, and the second classifier C2, respectively; d is the probability distribution difference function; N t The number of data points in the target domain data.

[0029] According to the present invention, a remote sensing image classification method based on multi-classifier adversarial active transfer learning is provided, wherein the training process of the active learning phase is as follows:

[0030] Active learning query strategies still select target domain data that are inconsistently classified by multi-classifiers;

[0031] The inconsistent target domain data is manually labeled, and then added to the source domain data for training in the next iteration.

[0032] According to the present invention, a remote sensing image classification method based on multi-classifier adversarial active transfer learning is provided. The active learning query strategy selects target domain data that are still inconsistently classified by the multi-classifier. Specifically, it includes:

[0033] From the inconsistent data points in the target domain data, select the top N data points by weighted summation of information content and diversity values ​​as the inconsistent target domain data that cannot be migrated, where N is a positive integer.

[0034] This invention also provides a remote sensing image classification device based on multi-classifier adversarial active transfer learning, comprising:

[0035] The acquisition unit is used to acquire hyperspectral remote sensing images to be classified.

[0036] The classification unit is used to input the hyperspectral remote sensing image to be classified into the classification model, and the classification model outputs the classification result.

[0037] The classification model is trained on source domain data, which includes multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data, which includes only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process of the classification model includes a transfer learning stage and an active learning stage.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the remote sensing image classification method based on multi-classifier adversarial active transfer learning as described above.

[0039] This invention provides a remote sensing image classification method based on multi-classifier adversarial active transfer learning. The method involves acquiring a hyperspectral remote sensing image to be classified; inputting the image into a classification model; and outputting a classification result. The classification model is trained on source domain data including multiple sample hyperspectral remote sensing images and corresponding category labels, and target domain data including only multiple sample hyperspectral remote sensing images. The training network structure includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process includes both a transfer learning phase and an active learning phase. This invention improves the classification accuracy of unsupervised transfer learning strategies and explores active learning methods that match these strategies, addressing the poor compatibility of traditional active learning with transfer learning in terms of point selection strategies. Ultimately, this effectively improves the classification accuracy of hyperspectral images. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the remote sensing image classification method based on adversarial active transfer learning using a multi-classifier provided by the present invention.

[0042] Figure 2 A flowchart of the hyperspectral remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the present invention;

[0043] Figure 3 The overall network framework diagram of the classification model provided by this invention;

[0044] Figure 4 This invention provides a multi-classifier adversarial transfer learning network framework diagram;

[0045] Figure 5 The multi-classifier adversarial active learning network framework diagram provided by this invention;

[0046] Figure 6 A schematic diagram of the structure of the remote sensing image classification device based on multi-classifier adversarial active transfer learning provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] Existing technologies suffer from problems such as poor compatibility between traditional active learning and transfer learning strategies in terms of point selection, and low unsupervised classification accuracy. The following section combines... Figure 1 This invention describes a remote sensing image classification method based on multi-classifier adversarial active transfer learning. Figure 1 The flowchart of the remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the present invention is shown below. Figure 1 As shown, the method includes:

[0050] Step 110: Obtain the hyperspectral remote sensing image to be classified.

[0051] Specifically, since the classification model provided by this invention is a classification model based on unsupervised transfer learning, the hyperspectral remote sensing images to be classified are extracted from the target domain data participating in the training of the classification model. The target domain data contains only a large number of sample hyperspectral remote sensing images without category result labels. The target domain data gradually obtains labels in multiple rounds of adversarial transfer learning and participates in training.

[0052] Step 120: Input the hyperspectral remote sensing image to be classified into the classification model, and the classification model outputs the classification result;

[0053] The classification model is trained on source domain data, which includes multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data, which includes only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process of the classification model includes a transfer learning stage and an active learning stage.

[0054] Specifically, the classification model is obtained by training the network based on source domain data and target domain data. The source domain data includes a large number of sample hyperspectral remote sensing images as well as the category labels corresponding to each sample hyperspectral remote sensing image. The target domain data only includes a large number of sample hyperspectral remote sensing images. The training objective of the classification model is that the trained classification model can accurately classify the hyperspectral remote sensing images in the target domain data. The training method provided by this invention is based on a multi-classifier adversarial transfer learning network. Therefore, the network structure in the training process of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. The output of the feature extractor G is connected to the input of the first classifier C1, and the output of the feature extractor G is also connected to the input of the second classifier C2.

[0055] During the training of the classification model, a multi-classifier adversarial transfer learning network is constructed and combined with active learning. Utilizing the adversarial approach, while multiple classifiers classify the target domain data inconsistently, the generator's transformation of the target domain data ensures consistent classification results across multiple classifiers. This continuous adversarial process gradually brings the target domain distribution closer to the source domain distribution. Based on this, the maximum mean difference is used to measure inter-domain and intra-class differences, aligning the distributions of the two domains. Furthermore, active learning selects inconsistent data points from the target domain data; these data points are labeled by experts and added to the training, further improving classification performance.

[0056] This invention provides a remote sensing image classification method based on multi-classifier adversarial active transfer learning. The method involves acquiring a hyperspectral remote sensing image to be classified; inputting the image into a classification model; and the model outputting a classification result. The classification model is trained on source domain data (including multiple sample hyperspectral remote sensing images and corresponding category labels) and target domain data (including only multiple sample hyperspectral remote sensing images). The training network structure includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process includes both a transfer learning phase and an active learning phase. This invention improves the classification accuracy of unsupervised transfer learning strategies and explores active learning methods that match these strategies, addressing the poor compatibility of traditional active learning with transfer learning in terms of point selection strategies. Ultimately, this effectively improves the classification accuracy of hyperspectral images.

[0057] Based on the above embodiments, the training process of the transfer learning phase in this method is as follows:

[0058] First, the source domain data and the target domain data are input into the training network for adversarial transfer learning. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn the knowledge of the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned, as well as the domain adaptation of the source domain data and the target domain data.

[0059] Specifically, in the transfer learning stage, the source domain data is first input into the network to ensure the network correctly classifies it. Next, two classifiers that maximize the difference in classification on the source domain data are constructed using an adversarial approach, and then the target domain data is input. The classifier parameters are then fixed, and the target domain data is transformed by changing the parameters of the feature extractor. This transformation ensures the transformed target domain data is consistently classified by the classifiers, thus achieving domain adaptation between the source and target domain data. Consistent classification means that multiple classifiers produce the same classification result for a given data point. The domain adaptation process described above applies to the overall source and target domains, but overall alignment does not necessarily mean alignment between classes. Furthermore, the overall maximum mean difference (MMD) and class MMD are incorporated to measure the overall and class distribution differences between the source and target domain data. Joint domain adaptation is then performed using these two MMD losses to align the overall and class distributions of the source and target domain data.

[0060] Based on the above embodiments, in this method, the learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn knowledge from the source domain data, and the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned as follows:

[0061] The first total loss function for training the network is determined by minimizing the classification cross-entropy loss function of the first classifier C1 and the second classifier C2, and minimizing the overall and maximum mean difference loss of the source domain data and target domain data after passing through the feature extractor G.

[0062] Specifically, firstly, the source domain data X s and target domain data X t The data is fed into the network for adversarial transfer learning. For two classifiers, the source domain data Xs is first transformed by the feature extractor G and then input into the two classifiers. The classification cross-entropy loss function of the two first classifiers C1 and C2 is minimized to ensure that the two classifiers can learn the source domain knowledge. However, this method's domain adaptation does not consider explicit distribution alignment between the source and target domains. Furthermore, when it is expected that the features learned by the target domain are as far away from the classification boundary of the source domain as possible, the lack of class guidance may lead to negative transfer. Therefore, for adversarial transfer in classifier adversarial learning, adding the maximum mean difference (MMD) alignment approach to the deep network helps extract domain-invariant features. Combined with the robust classification decision boundary learned in adversarial learning, this can better promote transfer.

[0063] Based on the above embodiments, in this method, the first total loss function f1 is expressed by the following formula:

[0064] f1=min(Loss_C1_Xs+Loss_C2_Xs+Loss_MMD)

[0065]

[0066]

[0067]

[0068] Where, x i Represents the i-th data point, y i N represents the label of the i-th data point. s and N t Let S and d represent the number of data points in the source and target domains, respectively. S is the softmax function, and d is a function that measures the difference in probability distributions. It is a feature mapping function that maps the features extracted by the feature extractor G to a high-dimensional Hilbert space. Then, in the high-dimensional Hilbert space, constraints are applied to the overall maximum mean difference and the maximum mean difference of each category. C is the number of data categories, and N is the number of categories. s c and N t c These are the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. N sλ represents the number of data points in the source domain data. Since the target domain does not have true labels, the classification results obtained by the classifier trained on the source domain data are used as pseudo-labels in the calculation. λ is the trade-off parameter for the maximum mean difference loss of each class. y () represents the cross-entropy loss function.

[0069] Specifically, the cross-entropy loss of the first classifier C1 on the source domain data transformed by the feature extractor G and the cross-entropy loss of the second classifier C2 on the source domain data transformed by the feature extractor G are formulated as follows:

[0070]

[0071]

[0072] The formulas for the overall and class-specific maximum mean difference loss of the source and target domain data after transformation by feature extractor G are as follows:

[0073]

[0074] Where, x i Represents the i-th data point, y i N represents the label of the i-th data point. s and N t Let S represent the number of data points in the source and target domains, respectively, and S be the softmax function. It is a feature mapping function that maps the features extracted by the feature extractor G to a high-dimensional Hilbert space. Then, in the high-dimensional Hilbert space, constraints are applied to the overall maximum mean difference and the maximum mean difference of each category. C is the number of data categories, and N is the number of categories. s c and N t c These are the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. N s λ represents the number of data points in the source domain data. Since the target domain does not have true labels, the classification results obtained by the classifier trained on the source domain data are used as pseudo-labels in the calculation. λ is the trade-off parameter for the maximum mean difference loss of each class. y () represents the cross-entropy loss function.

[0075] Based on the above embodiments, the domain adaptation of the source domain data and the target domain data in this method is implemented as follows:

[0076] With the parameters of the feature extractor G fixed, the target domain data X tAfter data transformation, the data is input into the first classifier C1 and the second classifier C2. Then, the difference between the classification results of the two classifiers is maximized. Next, the first classifier C1 and the second classifier C2 are fixed, and the difference between the classification results of the two classifiers is minimized to determine the second total loss function of the training network.

[0077] Specifically, for the target domain data X t For example, if the parameters of the feature extractor G are fixed, X t After data transformation, the data is input into two classifiers, C1 and C2. The goal is to maximize the difference in classification results between the two classifiers to obtain different decision boundaries. Based on the above steps, with the parameters of the two classifiers C1 and C2 fixed, the aim is to minimize the difference in classification results between the two classifiers for the transformed target domain data X. t The goal is to transform the target domain data X to achieve consistent classification results, considering the differences in classification outcomes between the two classifiers. The aim is to transform the target domain data X. t By moving away from the classifier's decision boundary, the target domain distribution can be made as similar as possible to the source domain distribution.

[0078] Based on the above embodiments, in this method, the second total loss function f2 is expressed by the following formula:

[0079]

[0080]

[0081] Where, θ G θ C1 and θ C2 These are the network parameters of the feature extractor G, the first classifier C1, and the second classifier C2, respectively; d is the probability distribution difference function; N t The number of data points in the target domain data.

[0082] Specifically, the second overall loss function of the multi-classifier adversarial network is:

[0083]

[0084] Where, θ G θ C1 and θ C2These are the network parameters for the feature extractor G, the first classifier C1, and the second classifier C2, respectively. First, the first term of this formula is minimized to ensure correct classification of the source domain data while minimizing the overall and class distribution differences between the source and target domains through maximizing the mean difference. Next, a maximum-minimum constraint is applied to the second term of the formula. Loss_Dis measures the difference in classification results between the two classifiers for the target domain; a larger loss results in a smaller difference. This adversarial network first fixes the parameters of the feature extractor G to minimize the loss, aiming to adjust the parameters of the two classifiers to increase their difference, thereby ensuring that the classification boundaries described by the two classifiers are as inconsistent as possible while correctly classifying the source domain data. Then, the classifier parameters are fixed to maximize the loss, aiming to adjust the feature extractor parameters so that the features learned from the target domain data are gradually classified consistently by the two classifiers, thus achieving domain transfer.

[0085] Based on the above embodiments, the training process in the active learning phase of this method is as follows:

[0086] Active learning query strategies still select target domain data that are inconsistently classified by multi-classifiers;

[0087] The inconsistent target domain data is manually labeled, and then added to the source domain data for training in the next iteration.

[0088] Specifically, during the active learning phase, while the network is adapting to the domain, some target domain data may still be misclassified after adaptation. Therefore, the active learning query strategy selects target domain data that cannot be transferred. Furthermore, this actively learned target domain data is manually labeled and then used to retrain the network along with the existing training data.

[0089] Based on the above embodiments, in this method, the active learning query strategy selects target domain data that are still inconsistently classified by the multi-classifier, specifically including:

[0090] From the inconsistent data points in the target domain data, select the top N data points by weighted summation of information content and diversity values ​​as the inconsistent target domain data that cannot be migrated, where N is a positive integer.

[0091] Specifically, in the aforementioned multi-classifier adversarial transfer learning network, the first step is to minimize the source domain cross-entropy loss and target domain classification inconsistency loss constraints of the multi-classifier. This ensures that the multi-classifier can correctly classify the source domain data while misclassifying the target domain data. This constructed multi-classifier exhibits greater inconsistency with the target domain data after transformation by the feature extractor. Next, domain adaptation is performed, i.e., maximizing the target domain classification inconsistency loss to ensure that the target domain data, after transformation by the feature extractor, moves away from the decision boundary of the source domain. At this point, Qc(x1) is calculated for each misclassified target domain data point, and data points with higher calculated values ​​are selected and, through expert labeling, are included in the next training iteration along with the source domain data. The formula is as follows:

[0092] Q c (x)=C1(G(x))*C2(G(x))

[0093] Where x represents data points with inconsistent classification in the target domain, C1 and C2 are classifiers, and G is a feature extractor.

[0094] Active learning query strategies not only need to select data points with high information content, but also need to ensure the diversity of selected data points to avoid selecting data belonging to the same category. To address these two issues, data points are clustered regardless of whether they are classifying similarly or inconsistently, and then sample points are selected from the clusters. The density peak clustering algorithm, based on density space and capable of discovering clusters of arbitrary shapes, incorporates two core ideas: first, the density of sample points at the cluster center should be higher than the density of their surrounding neighbors; second, the cluster centers should be relatively far apart. Therefore, two values ​​need to be calculated for each data point: density ρ and distance δ from a specific sample. The calculation formula is as follows:

[0095]

[0096]

[0097]

[0098] Where, d c To cut off the distance, d ij Let be the distance between data point i and data point j, sign be the sign function, and density ρ be the distance between other inconsistent data points and this data point whose distance is less than d. c The number of data points. Active learning, based on the selection of inconsistent data points, selects data points with high density values ​​but belonging to different clusters to ensure that the selected data points have high information content and diversity.

[0099] The overall multi-criteria active learning query strategy is as follows:

[0100] Q1(x1)=λ1Q c (x1)+λ2Q d (x1)

[0101] Q2(x2)=Q d (x2)

[0102] In this framework, Q1(x1) represents the query strategy for inconsistent data classified by the multi-classifier, where x1 represents inconsistent data classified by the multi-classifier, and λ1 and λ2 are weight parameters, with λ1 + λ2 = 1. Q2(x2) represents the query strategy for consistent data classified by the multi-classifier, where x2 represents inconsistent data classified by the multi-classifier. Based on the above, the source domain data is weighted. For weighting, if the actively labeled data point's category is inconsistent with the surrounding source domain data points, the weight of the source domain data is divided by 2. If the actively labeled data point's category is consistent with the surrounding source domain data points, the weight of the source domain data is multiplied by 2. By querying both inconsistent and consistent data, the domain adaptation of inconsistent data can be further enhanced, while also mitigating the target domain misclassification problem caused by negative transfer.

[0103] Based on the above embodiments, the present invention provides a hyperspectral remote sensing image classification method based on multi-classifier adversarial active transfer learning. Figure 2 The flowchart of the hyperspectral remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by this invention is as follows: Figure 2 As shown, this method consists of two parts: transfer learning and active learning, which are performed iteratively and alternately. The specific process of the algorithm is as follows:

[0104] Firstly, regarding transfer learning, this invention achieves domain adaptation through a multi-classifier adversarial approach. Figure 3 The overall network framework diagram of the classification model provided by this invention is as follows: Figure 3 As shown:

[0105] (1) First, transfer the source domain data X s and target domain data X t The data is fed into the network for adversarial transfer learning.

[0106] (2) For the two classifiers, firstly, the source domain data X s The data is transformed by feature extractor G and then input into two classifiers. The cross-entropy loss functions of classifiers C1 and C2 are minimized to ensure that both classifiers can learn the source domain knowledge. The formulas for the cross-entropy loss of classifier C1 with respect to the source domain data transformed by feature extractor G and the cross-entropy loss of classifier C2 with respect to the source domain data transformed by feature extractor G are as follows:

[0107]

[0108]

[0109] (3) For the target domain data Xt, with the parameters of the feature extractor G fixed, Xt is transformed and then input into two classifiers C1 and C2. The goal is to maximize the difference in classification results between the two classifiers to obtain different decision boundaries. The difference loss in classification results between classifiers C1 and C2 for the target domain data Xt after transformation by the feature extractor G is:

[0110]

[0111] (4) Based on the third step, fix the parameters of the two classifiers C1 and C2, and minimize the difference in the classification results of the two classifiers on the target domain data Xt after data transformation. It is expected that the features learned by the feature extractor from the target domain will make the classification results of the two classifiers consistent. The purpose is to transform the target domain data Xt away from the decision boundary of the classifier, so as to make the distribution of the target domain as similar as possible to the distribution of the source domain.

[0112] By changing the parameters of the feature extractor and minimizing the loss function in the third step, G effectively migrates the target domain data.

[0113] (5) Iterate and optimize the above steps to complete the training and domain adaptation is completed;

[0114] (6) Figure 4 The multi-classifier adversarial transfer learning network framework diagram provided by this invention is as follows: Figure 4 As shown, the domain adaptation method does not consider explicit distribution alignment between the source and target domains. Furthermore, when it aims for the features learned in the target domain to be as far away from the classification boundary of the source domain as possible, it lacks class guidance, potentially leading to negative transfer. Therefore, for adversarial transfer in classifier adversarial systems, adding the maximum mean difference (MMD) alignment approach to the deep network helps extract domain-invariant features. Simultaneously, combining this with the robust classification decision boundary learned in adversarial classifier adversarial systems can better promote transfer. The overall and class-specific maximum mean difference loss formulas for the source and target domain data after transformation by the feature extractor G are as follows:

[0115]

[0116] Where, x i Represents the i-th data point, y i N represents the label of the i-th data point. s and N tThese represent the number of data points in the source and target domains, respectively. S is the softmax function, d is the probability distribution difference function, and is the feature mapping function, which maps the features extracted by the feature extractor to a high-dimensional Hilbert space. Then, in the high-dimensional Hilbert space, constraints are applied to the overall MMD and the MMD for each category. C is the number of data categories, and N... s c and N t c These represent the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. Since the target domain does not have a true label, the classification result obtained by the classifier trained on the source domain data is used as a pseudo-label in the calculation. λ is a trade-off parameter for the maximum mean difference loss across all categories.

[0117] (7) The overall loss function of the multi-classifier adversarial network is:

[0118]

[0119] Where, θ G θ C1 and θ C2 These are the network parameters of the feature extractor G, the classifier C1, and the classifier C2, respectively. First, the first term of this formula is minimized to ensure correct classification of the source domain data while minimizing the overall and class distribution differences between the source and target domains through maximizing the mean difference. Next, a maximum-minimum constraint is applied to the second term of the formula. Loss_Dis measures the difference in classification results between the two classifiers for the target domain; a larger loss results in a smaller difference. This adversarial network first fixes the parameters of the feature extractor G to minimize the loss, aiming to adjust the parameters of the two classifiers to increase their difference, thereby ensuring that the classification boundaries described by the two classifiers are as inconsistent as possible while correctly classifying the source domain data. Then, the classifier parameters are fixed to maximize the loss, aiming to adjust the feature extractor parameters so that the features learned from the target domain data are gradually classified consistently by the two classifiers, thus achieving domain transfer.

[0120] (8) Figure 5 The multi-classifier adversarial active learning network framework diagram provided by this invention is as follows: Figure 5As shown, in the aforementioned multi-classifier adversarial transfer learning network, the first step is to minimize the source domain cross-entropy loss and target domain classification inconsistency loss constraints of the multi-classifier. This ensures that the multi-classifier can correctly classify the source domain data while misclassifying the target domain data. This constructed multi-classifier exhibits greater inconsistency for the target domain data after transformation by the feature extractor. Next, domain adaptation is performed, which involves maximizing the target domain classification inconsistency loss to ensure that the target domain data, after transformation by the feature extractor, moves away from the decision boundary of the source domain. Then, Qc(x1) is calculated for each misclassified target domain data point. Data points with higher calculated values ​​are identified and, through expert labeling, are included in the next training iteration along with the source domain data. The formula is as follows:

[0121] Q c (x)=C1(G(x))*C2(G(x))

[0122] Where x represents data points with inconsistent classification in the target domain, C1 and C2 are classifiers, and G is a feature extractor.

[0123] (9) The active learning query strategy not only needs to select data points with a large amount of information, but also needs to ensure that the selected data points are as diverse as possible to avoid the situation where the selected data belong to the same category. To address these two issues, the data is clustered for both consistent and inconsistent data points, and then sample points are selected from the clusters. For the density peak clustering algorithm, this algorithm is based on density space and can find clusters of arbitrary shapes. It includes two core ideas: first, the density of the sample point at the center of the cluster should be higher than the density of its surrounding neighboring points; second, the cluster centers should be far apart. Therefore, two values ​​need to be calculated for each data point: density ρ and distance δ from a specific sample. The calculation formula is as follows:

[0124]

[0125]

[0126]

[0127] d c To cut off the distance, d ij Let be the distance between data point i and data point j, sign be the sign function, and density ρ be the distance between other inconsistent data points and this data point whose distance is less than d. c The number of data points. Active learning, based on the selection of inconsistent data points, selects data points with high density values ​​but belonging to different clusters to ensure that the selected data points have high information content and diversity.

[0128] The overall multi-criteria active learning query strategy is as follows:

[0129] Q1(x1)=λ1Q c (x1)+λ2Q d (x1)

[0130] Q2(x2)=Q d (x2)

[0131] In this framework, Q1(x1) represents the query strategy for inconsistent data classified by the multi-classifier, where x1 represents inconsistent data classified by the multi-classifier, and λ1 and λ2 are weight parameters, with λ1 + λ2 = 1. Q2(x2) represents the query strategy for consistent data classified by the multi-classifier, where x2 represents inconsistent data classified by the multi-classifier. Based on the above, the source domain data is weighted. For weighting, if the actively labeled data point's category is inconsistent with the surrounding source domain data points, the weight of the source domain data is divided by 2. If the actively labeled data point's category is consistent with the surrounding source domain data points, the weight of the source domain data is multiplied by 2. By querying both inconsistent and consistent data, the domain adaptation of inconsistent data can be further enhanced, while also mitigating the target domain misclassification problem caused by negative transfer.

[0132] (10) Through continuous iteration, the target domain data and source domain data can achieve domain alignment and class alignment after data transformation by the feature extractor G. However, inconsistency still exists between the two classifiers, which is unavoidable. Such data points are of great significance in multi-classifier inconsistency adversarial networks, containing a large amount of information. If they are actively learned and queried and labeled by experts before being fed into the network for training, the classification performance of the network in the target domain will be improved.

[0133] Specifically, the training process for the classification model is as follows:

[0134] The remote sensing images used in this invention refer to hyperspectral remote sensing images taken by remote sensing satellites at high altitudes, with multiple sets of images captured at different times and from different shooting angles. The same ground feature may exhibit different spectral characteristics in different sets of images due to variations in lighting, shooting angle, climate, and season. This invention is based on learning the characteristics of a particular ground feature and identifying it at other times or from other shooting angles.

[0135] The datasets used in this invention include the BOT dataset and the DFC dataset. The BOT dataset consists of three images acquired by NASA's EO-1 satellite in Botswana in May, June, and July 2001. The original images acquired by the EO-1 sensor contain 242 bands with a spectral resolution of 30 megapixels, covering the 400-2500nm portion of the spectrum at 10nm intervals. After removing uncalibrated and noise bands (including water absorption bands) from the original images, BOT contains only 145 bands from the original images, including [10-55, 82-97, 102-119, 134-164, 187-220]. These three images contain nine classes. The dataset was acquired by the ITRES CASI-1500 sensor in the urban area surrounding the University of Houston campus in Houston, Texas, and was provided in the 2013 IEEE GRSS Data Fusion Competition. The data size is 349*1905, including 144 bands from 364nm to 1046nm, with a spectral resolution of 10nm.

[0136] Three target domain data classification prediction experiments were conducted on the BOT dataset: BOT56, BOT65, and BOT75. BOT56 represents source domain data from May and target domain data from June. BOT65 represents source domain data from June and target domain data from May. BOT75 represents source domain data from July and target domain data from May. In these three experiments, each query strategy selected 50 data points, querying 2 data points per attempt, for a total of 25 queries. Two target domain data classification prediction experiments were conducted on the DFC dataset: DFC_BS and DFC_SB. DFC_BS represents source domain data from non-shaded areas and target domain data from shaded areas. DFC_SB represents source domain data from shaded areas and target domain data from non-shaded areas. In these two experiments, each query strategy selected 100 data points, querying 2 data points per attempt, for a total of 50 queries. The next section will introduce the comparison algorithms used in the experiments. One set of hyperspectral remote sensing image data is used as the source domain data, and another set of hyperspectral remote sensing images is used as the target domain data. Both are input into the network for training.

[0137] Specifically, in the transfer learning stage, the source domain data first enters the generator, where the data undergoes a high-dimensional transformation: the original 1*145 dimension pixels are mapped to 1*250 dimension pixels, and then fed into two classifiers.

[0138] Two classifiers, C1 and C2, classify the source domain data. Each pixel will be classified into a 1*C classification result vector, where C is the number of classes. The classifiers perform a backpropagation to minimize the cross-entropy loss, thereby constraining C1 and C2 to correctly classify the source domain data.

[0139] Next, a multi-classifier adversarial process is implemented. Based on the classification results of the two classifiers C1 and C2 on the target domain data, the difference between their classification results is calculated, and the classification difference loss is obtained. Backpropagation is then performed in accordance with minimizing the cross-entropy loss and maximizing the classification difference loss, so that the two classifiers C1 and C2 can classify the source domain data correctly while maximizing the classification difference of the target domain data.

[0140] The network parameters of two classifiers, C1 and C2, are fixed for the next stage of the adversarial process. The generator transforms the target domain data, and the difference in classification results between the two classifiers C1 and C2 is calculated. This difference is represented as a single value, calculated by determining the classification result for each pixel. Backpropagation is then performed to minimize the classification difference loss, ensuring that the classification difference between the target domain data and the two classifiers C1 and C2 is as small as possible after the generator's transformation. This achieves domain adaptation, i.e., aligning the distributions of the source and target domain data.

[0141] The domain adaptation mentioned above is carried out in an adversarial manner. However, multi-classifier adversarial methods only pay attention to the overall domain adaptation and do not focus on the domain adaptation of each class. Therefore, the maximum mean difference loss is added. By calculating the difference between the source domain data and the target domain data by class, the distribution of target domain data and source domain data in each class can be further mapped.

[0142] During the active learning phase, the total number of target domain data points selected is much smaller than the number of target domain data points. Each time, two data points are selected from the target domain.

[0143] In the multi-classifier adversarial domain adaptation algorithm during the transfer learning phase, it cannot guarantee that all target domain data will be consistently classified by the two classifiers C1 and C2. Therefore, target domain data that still cannot be consistently classified is manually labeled. The active learning query strategy involves querying target domain data that are inconsistently classified. After labeling these target domain data, they are reintroduced into the transfer learning phase for retraining. Furthermore, this invention uses random selection to ensure data diversity in the selection of the active learning query strategy.

[0144] The remote sensing image classification device based on adversarial active transfer learning based on multi-classifiers provided by the present invention will be described below. The remote sensing image classification device based on adversarial active transfer learning based on multi-classifiers described below can be referred to in correspondence with the remote sensing image classification method based on adversarial active transfer learning based on multi-classifiers described above.

[0145] Figure 6 This is a schematic diagram of the remote sensing image classification device based on multi-classifier adversarial active transfer learning provided by the present invention, as shown below. Figure 6As shown, the device includes an acquisition unit 610 and a classification unit 620, wherein,

[0146] The acquisition unit 610 is used to acquire the hyperspectral remote sensing image to be classified;

[0147] The classification unit 620 is used to input the hyperspectral remote sensing image to be classified into the classification model, and the classification model outputs the classification result.

[0148] The classification model is trained on source domain data, which includes multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data, which includes only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a classifier C1, and a classifier C2. Each iteration of the training process includes a transfer learning stage and an active learning stage.

[0149] This invention provides a remote sensing image classification device based on multi-classifier adversarial active transfer learning. The device acquires a hyperspectral remote sensing image to be classified; inputs the image into a classification model, which outputs a classification result. The classification model is trained on source domain data including multiple sample hyperspectral remote sensing images and corresponding category labels, and target domain data including only multiple sample hyperspectral remote sensing images. The training network structure includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process includes a transfer learning phase and an active learning phase. This invention improves the classification accuracy of unsupervised transfer learning strategies and studies active learning methods that match these strategies, addressing the poor compatibility of traditional active learning with transfer learning in point selection strategies. Ultimately, it effectively improves the classification accuracy of hyperspectral images.

[0150] Based on the above embodiments, the training process of the transfer learning phase in this device is as follows:

[0151] First, the source domain data and the target domain data are input into the training network for adversarial transfer learning. The learning objective is that the feature extractor G, classifier C1, and classifier C2 can all learn the knowledge of the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned, as well as the domain adaptation of the source domain data and the target domain data.

[0152] Based on the above embodiments, in this device, the learning objective is that feature extractor G, classifier C1, and classifier C2 can all learn knowledge from the source domain data, and the overall distribution of the source domain data and target domain data and the distribution of each classification result are aligned as follows:

[0153] The first total loss function for training the network is determined by minimizing the cross-entropy loss function of classifiers C1 and C2, and minimizing the overall and class-specific maximum mean difference loss of the source and target domain data after passing through the feature extractor G.

[0154] Based on the above embodiments, in this device, the first total loss function f1 is expressed by the following formula:

[0155] f1=min(Loss_C1_Xs+Loss_C2_Xs+Loss_MMD)

[0156]

[0157]

[0158]

[0159] Where, x i Represents the i-th data point, y i N represents the label of the i-th data point. s and N t Let S and d represent the number of data points in the source and target domains, respectively. S is the softmax function, and d is a function that measures the difference in probability distributions. It is a feature mapping function that maps the features extracted by the feature extractor G to a high-dimensional Hilbert space. Then, in the high-dimensional Hilbert space, constraints are applied to the overall maximum mean difference and the maximum mean difference of each category. C is the number of data categories, and N is the number of categories. s c and N t c These are the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. N s λ represents the number of data points in the source domain data. Since the target domain does not have true labels, the classification results obtained by the classifier trained on the source domain data are used as pseudo-labels in the calculation. λ is the trade-off parameter for the maximum mean difference loss of each class. y () represents the cross-entropy loss function.

[0160] Based on the above embodiments, the domain adaptation of the source domain data and the target domain data in this device is implemented as follows:

[0161] With the parameters of the feature extractor G fixed, the target domain data X t After data transformation, the data is input into classifiers C1 and C2. Then, the difference between the classification results of the two classifiers is maximized. Next, classifiers C1 and C2 are fixed, and the difference between the classification results of the two classifiers is minimized to determine the second total loss function of the training network.

[0162] Based on the above embodiments, in this device, the second total loss function f2 is expressed by the following formula:

[0163]

[0164]

[0165] Where, θ G θ C1 and θ C2 These are the network parameters of the feature extractor G, the classifier C1, and the classifier C2, respectively; d is the probability distribution difference function; and N... t The number of data points in the target domain data.

[0166] Based on the above embodiments, the training process in the active learning phase of this device is as follows:

[0167] Active learning query strategies still select target domain data that are inconsistently classified by multi-classifiers;

[0168] The inconsistent target domain data is manually labeled, and then added to the source domain data for training in the next iteration.

[0169] Based on the above embodiments, in this device, the active learning query strategy selects target domain data that are still inconsistently classified by the multi-classifier, specifically including:

[0170] From the inconsistent data points in the target domain data, select the top N data points by weighted summation of information content and diversity values ​​as the inconsistent target domain data that cannot be migrated, where N is a positive integer.

[0171] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communication interface 720, a memory 730, and a communication bus 740. The processor 710, communication interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a remote sensing image classification method based on multi-classifier adversarial active transfer learning. This method includes: acquiring a hyperspectral remote sensing image to be classified; inputting the hyperspectral remote sensing image to be classified into a classification model, and the classification model outputting a classification result. The classification model is trained based on source domain data including multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data including only multiple sample hyperspectral remote sensing images. During the training of the classification model, the training network structure includes a feature extractor G, a classifier C1, and a classifier C2. Each iteration of the training process includes a transfer learning phase and an active learning phase.

[0172] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the remote sensing image classification method based on multi-classifier adversarial active transfer learning provided by the above methods, the method comprising: acquiring a hyperspectral remote sensing image to be classified; inputting the hyperspectral remote sensing image to be classified into a classification model, the classification model outputting a classification result; wherein the classification model is trained based on source domain data including multiple sample hyperspectral remote sensing images and corresponding category result labels and target domain data including only multiple sample hyperspectral remote sensing images, the training network structure of the classification model during training includes a feature extractor G, a classifier C1 and a classifier C2, and each iteration of the training process of the classification model includes a transfer learning stage and an active learning stage.

[0174] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned remote sensing image classification methods based on adversarial active transfer learning using multiple classifiers. The method includes: acquiring a hyperspectral remote sensing image to be classified; inputting the hyperspectral remote sensing image to be classified into a classification model, and having the classification model output a classification result; wherein the classification model is trained based on source domain data including multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data including only multiple sample hyperspectral remote sensing images. During the training of the classification model, the training network structure includes a feature extractor G, a classifier C1, and a classifier C2. Each iteration of the training process includes a transfer learning phase and an active learning phase.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image classification method based on multi-classifier adversarial active transfer learning, characterized in that, include: Acquire hyperspectral remote sensing images to be classified; The hyperspectral remote sensing image to be classified is input into the classification model, and the classification model outputs the classification result. The classification model is trained on source domain data consisting of multiple sample hyperspectral remote sensing images and corresponding category labels, and target domain data consisting of only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process includes a transfer learning phase and an active learning phase. The training process in the transfer learning phase is as follows: First, the source domain data and the target domain data are input into the training network for adversarial transfer learning. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn the knowledge of the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned, as well as the domain adaptation of the source domain data and the target domain data. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn knowledge from the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned. The implementation method is as follows: The first total loss function for training the network is determined by minimizing the classification cross-entropy loss function of the first classifier C1 and the second classifier C2, and minimizing the overall and maximum mean difference loss of the source domain data and target domain data after passing through the feature extractor G. The domain adaptation of the source domain data and target domain data is implemented as follows: With the parameters of the feature extractor G fixed, the target domain data X t After data transformation, the data is input into the first classifier C1 and the second classifier C2. Then, the difference between the classification results of the two classifiers is maximized. Next, the first classifier C1 and the second classifier C2 are fixed, and the difference between the classification results of the two classifiers is minimized to determine the second total loss function of the training network. The training process in the active learning phase is as follows: Active learning query strategies still select target domain data that are inconsistently classified by multi-classifiers; The inconsistent target domain data is manually labeled, and then added to the source domain data for training in the next iteration. Active learning query strategies select target domain data that are still inconsistently classified by multi-classifiers, specifically including: From the inconsistent data points in the target domain data, the top N data points, obtained by weighted summation of information content and diversity values, are selected as the inconsistent target domain data that cannot be migrated, where N is a positive integer; Specifically, in the aforementioned multi-classifier adversarial transfer learning network, the first step is to minimize the source domain cross-entropy loss and target domain classification inconsistency loss of the multi-classifier, ensuring that the multi-classifier correctly classifies the source domain data while misclassifying the target domain data. Next, domain adaptation is performed, i.e., by maximizing the target domain classification inconsistency loss, the target domain data, after transformation by the feature extractor, is moved away from the decision boundary of the source domain. At this point, Qc(x) for each misclassified target domain data point is calculated, as shown in the following formula: in, These are data points with inconsistent classification in the target domain. C1 and C2 are classifiers, and G is a feature extractor. For each data point, two values ​​are calculated: density ρ and distance δ from a specific sample. The calculation formula is as follows: in, To cut off the distance, Let be the distance between data point i and data point j, sign be the sign function, and density ρ be the distance between other inconsistent data points and this data point when the distance is less than 1. The number of; The overall multi-criteria active learning query strategy is as follows: Q1(x1) is the query strategy in inconsistent data classified by the multi-classifier, where x1 is the inconsistent data classified by the multi-classifier, λ1 and λ2 are weight parameters, and λ1+λ2=1. Active learning selects data points with high density values ​​but belonging to different clusters on the basis of selecting inconsistent data points, so as to ensure that the selected data points have high information content and diversity.

2. The remote sensing image classification method based on multi-classifier adversarial active transfer learning according to claim 1, characterized in that, First total loss function This can be expressed by the following formula: in, Represents the i-th data point. N represents the label of the i-th data point. s and N t These represent the number of data points in the source and target domains, respectively. S For the softmax function, It is a feature mapping function that maps the features extracted by the feature extractor G to a high-dimensional Hilbert space, and then applies constraints on the overall maximum mean difference and the maximum mean differences of various categories in the high-dimensional Hilbert space. C For the number of data categories, N s c and N t c These are the number of data points in category c of the source domain data and the number of data points in category c of the target domain data, respectively. This represents the number of data points in the source domain data. Since the target domain does not have true labels, the classification results obtained by the classifier trained on the source domain data are used as pseudo-labels in the calculation. λ is a trade-off parameter for the maximum mean difference loss across all classes. () represents the cross-entropy loss function.

3. The remote sensing image classification method based on adversarial active transfer learning using a multi-classifier as described in claim 1, characterized in that, Second total loss function This can be expressed by the following formula: in, θ G , θ C1 and θ C2 These are the network parameters of the feature extractor G, the first classifier C1, and the second classifier C2, respectively. d It is a function that measures the difference in probability distributions. The number of data points in the target domain data.

4. A remote sensing image classification device based on multi-classifier adversarial active transfer learning, characterized in that, include: The acquisition unit is used to acquire hyperspectral remote sensing images to be classified. The classification unit is used to input the hyperspectral remote sensing image to be classified into the classification model, and the classification model outputs the classification result. The classification model is trained based on source domain data including multiple sample hyperspectral remote sensing images and corresponding category result labels, and target domain data including only multiple sample hyperspectral remote sensing images. The training network structure of the classification model includes a feature extractor G, a first classifier C1, and a second classifier C2. Each iteration of the training process of the classification model includes a transfer learning stage and an active learning stage. The training process in the transfer learning phase is as follows: First, the source domain data and the target domain data are input into the training network for adversarial transfer learning. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn the knowledge of the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned, as well as the domain adaptation of the source domain data and the target domain data. The learning objective is that the feature extractor G, the first classifier C1, and the second classifier C2 can all learn knowledge from the source domain data, and that the overall distribution of the source domain data and the target domain data and the distribution of each classification result are aligned. The implementation method is as follows: The first total loss function for training the network is determined by minimizing the classification cross-entropy loss function of the first classifier C1 and the second classifier C2, and minimizing the overall and maximum mean difference loss of the source domain data and target domain data after passing through the feature extractor G. The domain adaptation of the source domain data and target domain data is implemented as follows: With the parameters of the feature extractor G fixed, the target domain data X t After data transformation, the data is input into the first classifier C1 and the second classifier C2. Then, the difference between the classification results of the two classifiers is maximized. Next, the first classifier C1 and the second classifier C2 are fixed, and the difference between the classification results of the two classifiers is minimized to determine the second total loss function of the training network. The training process in the active learning phase is as follows: Active learning query strategies still select target domain data that are inconsistently classified by multi-classifiers; The inconsistent target domain data is manually labeled, and then added to the source domain data for training in the next iteration. Active learning query strategies select target domain data that are still inconsistently classified by multi-classifiers, specifically including: From the inconsistent data points in the target domain data, the top N data points, obtained by weighted summation of information content and diversity values, are selected as the inconsistent target domain data that cannot be migrated, where N is a positive integer; Specifically, in the aforementioned multi-classifier adversarial transfer learning network, the first step is to minimize the source domain cross-entropy loss and target domain classification inconsistency loss of the multi-classifier, ensuring that the multi-classifier correctly classifies the source domain data while misclassifying the target domain data. Next, domain adaptation is performed, i.e., by maximizing the target domain classification inconsistency loss, the target domain data, after transformation by the feature extractor, is moved away from the decision boundary of the source domain. At this point, Qc(x) for each misclassified target domain data point is calculated, as shown in the following formula: in, These are data points with inconsistent classification in the target domain. C1 and C2 are classifiers, and G is a feature extractor. For each data point, two values ​​are calculated: density ρ and distance δ from a specific sample. The calculation formula is as follows: in, To cut off the distance, Let be the distance between data point i and data point j, sign be the sign function, and density ρ be the distance between other inconsistent data points and this data point when the distance is less than 1. The number of; The overall multi-criteria active learning query strategy is as follows: Q1(x1) is the query strategy in inconsistent data classified by the multi-classifier, where x1 is the inconsistent data classified by the multi-classifier, λ1 and λ2 are weight parameters, and λ1+λ2=1. Active learning selects data points with high density values ​​but belonging to different clusters on the basis of selecting inconsistent data points, so as to ensure that the selected data points have high information content and diversity.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the remote sensing image classification method based on multi-classifier adversarial active transfer learning as described in any one of claims 1 to 3.

Images