Graph migration-based hyperspectral remote sensing image ground feature classification method, medium and equipment

Abstract

The invention discloses a hyperspectral remote sensing image ground feature classification method based on graph migration, a medium and equipment, and relates to the technical field of ground feature classification, and the method comprises the steps: obtaining a source domain data set Xs, a target domain data set Xt and a source domain data label; inputting Xs and Xt into a feature extractor to obtain Zs and Zt; inputting the Zs into a reconstructor to reconstruct the Xs, reconstructing the Xt by the reconstructor for optimizing the reconstruction error of the Xs, and obtaining the weight of the Xt according to the reconstruction error of the Xt; based on the weight, a discriminator carries out domain judgment on Zs and Zt, and the discriminator and the feature extractor carry out domain confrontation training; selecting a feature Zt'of which the weight is greater than a threshold value, and outputting a classification probability P't of the Zt 'by a classifier; calculating an inter-domain graph adjacency matrix according to the Zt ', the Zs, the classification probability P't and a source domain label to realize circulation consistency; through iteration, knowledge migration is realized through domain confrontation and loop consistency, unknown classes are identified through a classifier, and known class data are classified. The method not only can classify known classes, but also can identify unknown classes.

Description

Technical Field

[0001] The present invention relates to the technical field of ground object classification, and in particular to a method, medium and equipment for ground object classification of hyperspectral remote sensing images based on image migration. Background Art

[0002] With the continuous growth in the number of remote sensing satellites and the significant reduction in revisit cycles, we are able to obtain abundant hyperspectral image data. However, in the field of hyperspectral image classification, a significant challenge has emerged: it is very difficult to label samples, which consumes a lot of manpower, material resources and financial resources. Therefore, under the condition that labeled samples are scarce or even completely missing, realizing the classification of hyperspectral images has become a key research direction. To solve this problem, an effective method is to "borrow" the knowledge of existing relevant data. For example, when we have a hyperspectral image with rich labels, we can use this existing knowledge to classify new images that lack label information. If the labeled data of the old image is directly used as the training set and the new image is used as the test data for classification, the effect is not good. In hyperspectral images collected in different time periods or different geographical areas, the spectral characteristics of the objects may change. This phenomenon leads to differences in the actual distribution of training data and test data.

[0003] In order to make it possible to "borrow" knowledge, domain adaptation methods have entered the public eye. Domain adaptation methods are a type of transfer learning algorithm, and their main function is to transfer existing knowledge. In domain adaptation, data with a large number of annotations is called source domain data, and data without annotations is called target domain data. There is a connection between the objects in the source domain and the target domain, but they are not the same. In hyperspectral image classification, feature-based domain adaptation methods are commonly used. This type of method hopes to find a new space for data mapping, in which the features of the data in the two domains have strong similarities, so as to achieve the purpose of knowledge transfer. However, most feature-based methods are proposed under closed set conditions and do not take into account more practical scenarios.

[0004] The closed set condition means that the data in the two domains share a labeling space. However, in practical applications, the model will encounter new categories of data that are not in the training set. In order to enable the model to cope with more flexible scenarios, scholars have proposed a domain adaptation algorithm based on open sets. In the open set, the categories shared by the source domain and the target domain are called known categories, and the categories private to the target domain are called unknown categories. The trained model can not only accurately classify the known category data, but also effectively identify the unknown category data. However, there are currently few open set domain adaptation algorithms applied in the field of hyperspectral image classification, and most of them are aimed at natural images, fault detection and other fields. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that hyperspectral image classification under closed set conditions is not applicable to new category data that may appear in actual scenes, and propose a hyperspectral remote sensing image object classification method based on graph migration, comprising the following steps:

[0006] S1. Obtain hyperspectral remote sensing images and select source domain dataset X from them s and the target domain dataset X t , and obtain the source domain data label;

[0007] S2. Use feature extractor G f Extract features from the two domains and obtain the source domain feature Z s and the target domain feature Z t ;

[0008] S3, Z s Input reconstructor G r Get X s Reconstructed data Utilizing X s The reconstruction error optimization G r , Z t Enter the optimized G r Get X t Reconstructed data According to X t The reconstruction error of the target domain data is obtained;

[0009] S4, based on the weight of the target domain data, using the discriminator G d Judge Z s and Z t Belongs to the source domain or the target domain, the discriminator G d With feature extractor G f Conduct domain adversarial training;

[0010] S5. Set the weight threshold of the target domain data and select M1 target domain data features Z whose weight is greater than the threshold t ′, using classifier G c Output Z t ′’s classification probability P′ t ;

[0011] S6. Use Z t ′、Z s , source domain data label, classification probability P′ t Compute the inter-domain graph adjacency matrix to achieve cycle consistency;

[0012] S7, through continuous iteration, based on domain confrontation and cycle consistency, knowledge transfer is achieved, through classifier G c Identify unknown class data and classify known class data.

[0013] Furthermore, X s The expression of the reconstruction error is:

[0014]

[0015] Among them, L r Represents X s The reconstruction error is , N represents the number of source domain data, represents the i-th source domain data, Represents the i-th reconstructed source domain data.

[0016] Furthermore, the weight of the target domain data is expressed as:

[0017]

[0018] Among them, W i represents the weight of the i-th target domain data, represents the i-th target domain data, represents the i-th reconstructed target domain data, m represents X t The minimum value of the reconstruction error, M represents the number of target domain data.

[0019] Furthermore, the domain adversarial loss is expressed as:

[0020]

[0021] Among them, L w_domain represents the domain adversarial loss, N represents the number of source domain data features, M represents the number of target domain data features, Denotes the discriminator G d The source domain loss, Denotes the discriminator G d The target domain loss is Represents the i-th source domain feature After the discriminator G d The judgment result of represents the domain label of the i-th source domain feature, Represents the jth target domain feature After the discriminator G d The judgment result of Represents the domain label of the jth target domain feature. The domain label value of the source domain is 0, and the domain label value of the target domain is 1.

[0022] Furthermore, the loop connection probability of cycle consistency is expressed as:

[0023]

[0024] in, Represents the i-th row and j-th column element of the cyclic connection probability matrix; Indicates P st The element in the i-th row and k-th column in represents the connection probability of the i-th source domain data and the k-th target domain data; Indicates P ts The element in the kth row and jth column in represents the connection probability of the kth target domain data and the jth source domain data; P st and P ts represents the normalized inter-domain graph adjacency matrix;

[0025] P st and P ts By using different domain-to-domain graph adjacency matrices W st and W ts Normalized to get;

[0026] Z s ∈N×D and Z t The dot product calculation of the transpose of ′∈M1×D is obtained st ∈N×M1:

[0027]

[0028] By classifier G c Get Z t ′’s classification probability P′ t , according to the source domain data X s =[X s1 ,X s2 ,...,X sN ]The label of each data, from P′ t Select the corresponding category column value as the column vector of the column where the corresponding source domain data is located, and splice it into a matrix W ts , N is the number of source domain data.

[0029] Furthermore, the constraint loss for cycle consistency is:

[0030] L cycle =β1L 循环连接约束 +β2L 同等约束连接

[0031] Among them, L cycle represents the constraint loss for cycle consistency, L 循环连接约束 represents the loop connection constraint loss, L 同等约束连接 represents the equal constraint connection loss, β1 and β2 represent the weights of the loop connection constraint loss and the equal constraint connection loss respectively;

[0032]

[0033] Among them, T ijrepresents the element at row i and column j in the indicator matrix;

[0034]

[0035] in, represents the number of source domain data belonging to the cth category, represents the i-th source domain feature, represents the jth source domain feature, Ω c Indicates the cth feature type;

[0036]

[0037] in, Indicates P st The element in the i-th row and j-th column represents the connection probability of the i-th source domain data and the j-th target domain data.

[0038] Furthermore, the classifier G c Using Z s and source domain data label Y s One-hot encoding training to improve G c The classification ability of source domain data, classifier G c The classification loss for source domain data is:

[0039]

[0040] Among them, L s Represents the classifier G c The classification loss of source domain data, N represents the number of source domain data, represents the one-hot encoding of the i-th source domain data label, Represents G c For the i-th source domain data feature The classification results;

[0041] Enter Z t , W and the one-hot encoding of the unknown class label, so that G c With the ability to effectively identify unknown classes, the classifier G c The recognition loss for unknown classes is:

[0042]

[0043] Among them, L wt Represents the classifier G c The recognition loss for unknown classes, M represents the amount of target domain data, W i represents the weight of the i-th target domain data, Y unk-onehot represents the one-hot encoding of the unknown class label, Represents G cFor the i-th target domain data feature The recognition result.

[0044] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for classifying objects in hyperspectral remote sensing images is implemented.

[0045] The present invention also proposes an electronic device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the above-mentioned hyperspectral remote sensing image object classification method.

[0046] The beneficial effects brought by the technical solution provided by the present invention are:

[0047] The present invention makes full use of the relationship between source domain data and target domain data, designs a weight calculation method based on a reconstructor, and realizes the separation of known classes and unknown classes. Cycle consistency is introduced into the open set scenario, and combined with weighted domain adversarial methods to jointly reduce the distribution differences of known classes in the two domains. Separation and domain adaptation help each other. The smaller the distribution difference, the more accurate the weights generated by the model, the more obvious the separation of known classes and unknown classes, and more known class data participate in domain adaptation, which in turn promotes the reduction of inter-domain differences. Finally, the classifier is trained using the two-domain data and weights so that the classifier not only has the ability to classify known classes, but also has the ability to identify unknown classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a method for classifying objects in a hyperspectral remote sensing image according to an embodiment of the present invention;

[0049] Figure 2 It is a block diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0051] Related terms:

[0052] 1. Source domain: The source domain refers to the dataset or field where the model is pre-trained. The source domain is a field with a large amount of labeled data and the task has been fully solved. The source domain can be a task, a field, or a dataset, depending on the specific application scenario.

[0053] 2. Target domain: The target domain is a new domain that hopes to use the knowledge of the source domain to improve its performance. In the target domain, there are usually fewer labeled samples to learn from, so the model needs to use transfer learning to utilize the knowledge and features learned in the source domain in order to achieve good performance on the target task. The target domain can be a different task, a different domain, or a different dataset.

[0054] There may be differences in data distribution, feature space, or task objectives between the source and target domains.

[0055] 3. Closed-set domain adaptation is a sub-problem in transfer learning, assuming that all categories in the source and target domains are known and identical. This means that when performing domain adaptation, there are no new categories that appear or disappear between the source and target domains.

[0056] 4. Open-set domain adaptation: The source domain and the target domain may only share a portion of the same categories, and the target domain may contain categories that are completely irrelevant to the source domain.

[0057] 5. Domain confrontation: It can also be called domain adaptation. It is an important branch of transfer learning, which is used to eliminate the differences in feature distribution between different domains. Its purpose is to map the data in the source domain and the target domain with different distributions to the same feature space, and find a certain metric to make their "distance" in this space as close as possible. Then, the classifier trained on the source domain (with labels) can be directly used to classify the target domain data. The domain adaptation network structure mainly consists of three parts:

[0058] Feature extractor: 1) Map and mix source domain samples and target domain samples so that the domain discriminator cannot distinguish which domain the data comes from; 2) Extract the features required for the subsequent network to complete the task so that the label predictor can distinguish the category of the data from the source domain.

[0059] Label predictor: classify the data and try to assign the correct label as much as possible.

[0060] Domain discriminator: classifies the data in the feature space and tries to distinguish which domain the data comes from.

[0061] Embodiment 1: The flowchart of the method for classifying objects in hyperspectral remote sensing images according to the embodiment of the present invention is as follows Figure 1 , specifically including the following steps:

[0062] S1. Obtain hyperspectral remote sensing images and select source domain dataset X from them s and the target domain dataset X t , and obtain the source domain data label.

[0063] Among them, X s∈N×D is N labeled source domain data, the dimension of each data is D, X s The corresponding label set is Y s ∈N×1.X t ∈M×D is the target domain data with M dimensions. The source domain and target domain data have C types of features in common, expressed as Ω=[Ω1…Ω C ],Ω i is the category data, i = 1…C; the private feature type of the target domain is represented by Ω C+1 represent.

[0064] S2. Use feature extractor G f Extract features from the two domains and obtain the source domain feature Z s , target domain feature Z t .

[0065] S3, Z s Input reconstructor G r Get X s Reconstructed data Utilizing X s The reconstruction error optimization G r , Z t Enter the optimized G r Get X t Reconstructed data According to X t The reconstruction error of is used to obtain the weight of the target domain data. The known class data is given a larger weight, and the unknown class data is given a smaller weight.

[0066] X s The expression of the reconstruction error is:

[0067]

[0068] Among them, L r Represents X s The reconstruction error is , N represents the number of source domain data, represents the i-th source domain data, Represents the i-th reconstructed source domain data.

[0069] The weight of the target domain data is expressed as:

[0070]

[0071] Among them, W i represents the weight of the i-th target domain data, represents the i-th target domain data, represents the i-th reconstructed target domain data, m represents X t The minimum value of the reconstruction error, M represents the number of target domain data.

[0072] S4, based on the weight of the target domain data, using the discriminator G d Judge Z s and Z t Belongs to the source domain or the target domain, the discriminator G d With feature extractor G f Conduct domain adversarial training.

[0073] Define the domain label d, the domain label value of the source domain data is 0 (d = 0), and the domain label value of the target domain data is 1 (d = 1). In adversarial domain adaptation, G f It is expected that the extracted features can confuse the discriminator G d , so that it outputs the wrong domain label; G d It tries its best to determine whether the feature belongs to the source domain or the target domain and outputs the correct domain label. f We hope to maximize the domain classification loss, G d Try to minimize the domain classification loss. Through this maximization-minimization adversarial training, the difference in the overall distribution of known class data in the two domains is reduced.

[0074] Among them, the weight W of the target domain data plays the role of an influencing factor. The larger the value of W, the deeper the target domain data participates in the domain confrontation; otherwise, it is smaller.

[0075] The domain adversarial loss is expressed as:

[0076]

[0077] Among them, L w_domain represents the domain adversarial loss, N represents the number of source domain data features, M represents the number of target domain data features, Denotes the discriminator G d The source domain loss, Denotes the discriminator G d The target domain loss is Represents the i-th source domain feature After the discriminator G d The judgment result of represents the domain label of the i-th source domain feature, Represents the jth target domain feature After the discriminator G d The judgment result of Represents the domain label of the jth target domain feature. The domain label value of the source domain data is 0 (d=0), and the domain label value of the target domain data is 1 (d=1).

[0078] S5. Set the weight threshold of the target domain data and select M1 target domain data features Z′ whose weight is greater than the threshold t, using the classifier G c Output Z′ t The classification probability P′ t . Classifier G c Output classification probability P′ t ∈M1×C.

[0079] In the preferred embodiment of the present invention, the weight threshold TH is set to 0.8. When the weight of the target domain data is greater than the threshold, it is considered that it is likely to belong to the known class. In this way, the data set X′ is selected. t ∈M1×D as known class data.

[0080] Classifier G c Using Z s and source domain data label Y s One-hot encoding training to improve G c The classification ability of source domain data, classifier G c The classification loss for source domain data is:

[0081]

[0082] Among them, L s Represents the classifier G c The classification loss of source domain data, N represents the number of source domain data, represents the one-hot encoding of the i-th source domain data label, Represents G c For the i-th source domain data feature The classification results.

[0083] Enter Z t , W and the one-hot encoding of the unknown class label, so that G c With the ability to effectively identify unknown classes, the classifier G c The recognition loss for unknown classes is:

[0084]

[0085] Among them, L wt Represents the classifier G c The recognition loss for unknown classes, M represents the amount of target domain data, W i represents the weight of the i-th target domain data, Y unk-onehot represents the one-hot encoding of the unknown class label, Represents G c For the i-th target domain data feature The recognition result.

[0086] S6. Use Z t ′、Z s , source domain data label, classification probability P′t Compute the inter-domain graph adjacency matrix and achieve cycle consistency.

[0087] The principle of cycle consistency is to extract inter-domain similarity features by constraining the probability of cyclic connection. The connection probability is determined by the adjacency matrix W between two different inter-domain graphs. st and W ts Calculated, any element in the matrix represents the similarity between two data in the two domains.

[0088] The loop connection probability of cycle consistency is expressed as:

[0089]

[0090] in, Represents the i-th row and j-th column element of the cyclic connection probability matrix; Indicates P st The element in the i-th row and k-th column in represents the connection probability of the i-th source domain data and the k-th target domain data; Indicates P ts The element in the kth row and jth column in represents the connection probability of the kth target domain data and the jth source domain data; P st and P ts represents the normalized inter-domain graph adjacency matrix.

[0091] P st and P ts By computing the inter-domain graph adjacency matrix W st and W ts Perform normalization operation to obtain;

[0092] Z s ∈N×D and Z t The dot product calculation of the transpose of ′∈M1×D is obtained st ∈N×M1:

[0093]

[0094] By classifier G c Get Z t ′’s classification probability P′ t , according to the source domain data X s =[X s1 ,X s2 ,...,X sN ]The label of each data, from P′ t Select the corresponding category column value as the column vector of the column where the corresponding source domain data is located, and splice it into a matrix W ts , N is the number of source domain data.

[0095] P′ tIt is a matrix of M1×C, where M1 represents the number of target domain data selected according to the weight W and the threshold, C represents the number of categories, and the category is represented as Ω=[Ω1…Ω C ], Ω1 represents P′ t The category of column 1, Ω C Represents P′ t Category in column C, P′ t The i-th row and j-th column in represents the probability that the i-th data in M1 belongs to the j-th category. According to the labels of N source domain data, N is the matrix W ts The number of columns, if the category of the i-th source domain data is Ω j (j=1,2,...,C), P′ t The M1 element in the jth column of W ts The element of the i-th column, the obtained W ts It is an M1×N matrix.

[0096] S7, through continuous iteration, based on domain confrontation and cycle consistency, knowledge transfer is achieved, through classifier G c Identify unknown class data and classify known class data.

[0097] The constraint loss for cycle consistency is:

[0098] L cycle =β1L 循环连接约束 +β2L 同等约束连接

[0099] Among them, L cycle represents the constraint loss for cycle consistency, L 循环连接约束 represents the loop connection constraint loss, L 同等约束连接 represents the equal constraint connection loss, β1 and β2 represent the weights of the loop connection constraint loss and the equal constraint connection loss respectively;

[0100]

[0101] Among them, T ij Represents the element at the i-th row and j-th column in the indicator matrix.

[0102]

[0103] in, represents the number of source domain data belonging to the cth category, represents the i-th source domain feature, represents the jth source domain feature, Ω c Indicates the cth feature type.

[0104] Since the source domain data is connected to the data with the greatest similarity in the target domain, there is an extreme case where all the data of the cth category in the source domain are connected to the same data. This will result in the inability to extract the similarity features between domains. To solve this problem, the equal connection constraint is used to make each data in the target domain connected to the source domain data with the same probability.

[0105]

[0106] in, Indicates P st The element in the i-th row and j-th column represents the connection probability of the i-th source domain data and the j-th target domain data.

[0107] The classifier trained as above classifies the known class data of the target domain and identifies the unknown class data.

[0108] In an exemplary embodiment, a computer-readable storage medium is included, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned hyperspectral remote sensing image ground object classification method is implemented.

[0109] See also Figure 2 In an exemplary embodiment, an electronic device is also included, including at least one processor, at least one memory, and at least one communication bus.

[0110] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through a communication bus to execute the above-mentioned hyperspectral remote sensing image ground object classification method.

[0111] In order to demonstrate the effect of the method proposed in the present invention, actual multi-temporal hyperspectral remote sensing data is taken as an example for illustration.

[0112] The BOT data used consists of three hyperspectral remote sensing images, which were collected by the Hyperion sensor carried by the EO-1 observation satellite in Region 1 and Region 2 in May, June, and July 2001. The Region 2 data covers 242 bands, with a spectral coverage from 357 nanometers to 2576 nanometers, a spectral resolution of 10 nanometers, and a spatial resolution of 30×30㎡. After eliminating uncalibrated bands, noise bands, and spectrally overlapping bands, a total of 145 bands were used for the study. The study area involves two different ecosystems, wetlands and highlands, and 9 different land feature categories. These land features are:

[0113] Category 1: Water;

[0114] Category 2: Floodplain;

[0115] Category 3: Riverbank;

[0116] Category 4: areas with burnt trees;

[0117] Category 5: Islands;

[0118] Category 6: Forest land;

[0119] Category 7: Tropical savanna vegetation;

[0120] Category 8: Short broadleaf trees;

[0121] Category 9: Bare soil.

[0122] For the three groups of data in region 2, one group is selected as the source domain data, and the first six categories of the source domain data are used as known categories. Another group is selected as the target domain data, and the last three categories of the target domain are used as private categories. For example, "June-July" means that the source domain data is the BOT image in June, and the target domain data is the BOT image in July. Therefore, there are a total of six groups of experimental data, namely "May-June", "June-May", "May-July", "July-May", "June-July", and "July-June".

[0123] Five indicators are used to conduct a comprehensive evaluation of the present invention. ①OS*: Average accuracy of known classes, which describes the average level of recognition accuracy of known classes. ②OS: Average accuracy of each class, which describes the average level of recognition accuracy of each class. ③UNK: Describes the accuracy of unknown class recognition. ④OA: Overall accuracy, which describes the proportion of correctly predicted samples in the total number of samples. ⑤HOS: An indicator used to evaluate the performance of open set classification models. A model that has poor recognition ability for unknown classes but strong prediction ability for known classes still has a high OS value obtained from the experiment. With the increase of known class data, this phenomenon will become more obvious. In order to solve this problem, HOS is proposed as an indicator to comprehensively evaluate the performance of the model.

[0124]

[0125] The method of the present invention is compared with other open set domain adaptation algorithms, which are Open set domain adaptation by backpropagation (OSBP), Adversarial Network With Multiple Classifiers for Open Set Domain Adaptation (DAMC), and Dual adversarial network for cross-domain open set fault diagnosis (DACD). In addition, in order to highlight that the open set algorithm is more suitable for processing open set scenarios than the closed set algorithm, two closed set algorithms are added as comparison algorithms, which are: Domain-adversarial training of neural networks (DANN) and Maximum classifier discrepancy for unsupervised domain adaptation (MCD). Table 1 gives five indicators of the method of the present invention and the five comparison methods on the region 2 data.

[0126] Table 1

[0127]

[0128]

[0129]

[0130] The following conclusions can be drawn through analysis: (1) The recognition ability of the model trained by the closed set domain adaptation algorithm for unknown classes is 0%, which proves that the open set domain adaptation algorithm is more suitable for processing open set scenes; (2) Compared with the open set comparison algorithm, the method proposed in the present invention performs best in all indicators and has stronger known class classification ability. And the ability to recognize unknown classes, the accuracy rate is above 95%, and the highest can reach 99%; (3) In data with more serious spectral drift, such as "June-May" and "July-May", the values ​​of OS, OS* and OA of the three methods of OSBP, DAMC and DACD are all very low, and the method of the present invention still performs well, and the lowest values ​​of the three indicators exceed 76%, 73% and 77% respectively, far exceeding the comparison algorithm.

[0131] The hyperspectral remote sensing image open set classification method of the base graph migration network proposed in the present invention is suitable for solving the hyperspectral image classification problem in the unlabeled open set scenario. Make full use of the relationship between the source domain data and the target domain data, design a weight module based on reconstruction, and realize the separation of known classes and unknown classes. Introduce cycle consistency into the open set scenario, and combine it with weighted domain adversarial to jointly reduce the distribution difference of known classes in the two domains. Separation and domain adaptation help each other. The smaller the distribution difference, the more accurate the weight generated by the model, the more obvious the separation of known classes and unknown classes, and more real known class data participate in domain adaptation, which promotes the reduction of inter-domain differences. Finally, the classifier is trained using the target domain data and weights to enable the classifier to have the ability to identify unknown classes.

[0132] Currently, only one type of experimental results of hyperspectral remote sensing data is given, and the present invention can also be applied to other hyperspectral data.

[0133] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hyperspectral remote sensing image object classification method based on graph migration, characterized in that: The following steps are involved: S1. Obtain hyperspectral remote sensing images and select source domain dataset X from them s and the target domain dataset X t , and obtain the source domain data label; S2. Use feature extractor G f Extract features from the two domains and obtain the source domain feature Z s and the target domain feature Z t ; S3, Z s Input reconstructor G r Get X s Reconstructed data Utilizing X s The reconstruction error optimization G r , Z t Enter the optimized G r Get X t Reconstructed data According to X t The reconstruction error of the target domain data is obtained; S4, based on the weight of the target domain data, using the discriminator G d Judge Z s and Z t Belongs to the source domain or the target domain, the discriminator G d With feature extractor G f Conduct domain adversarial training; S5. Set the weight threshold of the target domain data and select M1 target domain data features Z′ whose weight is greater than the threshold t , using the classifier G c Output Z′ t The classification probability P′ t ; S6, through Z′ t , Z s , source domain data label, classification probability P′ t Compute the inter-domain graph adjacency matrix to achieve cycle consistency; S7, through continuous iteration, based on domain confrontation and cycle consistency, knowledge transfer is achieved, through classifier G c Identify unknown class data and classify known class data.

2. The hyperspectral remote sensing image object classification method based on graph migration according to claim 1 is characterized in that: X s The expression of the reconstruction error is: Among them, L r Represents X s The reconstruction error is , N represents the number of source domain data, represents the i-th source domain data, Represents the i-th reconstructed source domain data.

3. The hyperspectral remote sensing image object classification method based on graph migration according to claim 1 is characterized in that: The weight of the target domain data is expressed as: Among them, W i represents the weight of the i-th target domain data, represents the i-th target domain data, represents the i-th reconstructed target domain data, m represents X t The minimum value of the reconstruction error, M represents the number of target domain data.

4. The hyperspectral remote sensing image object classification method based on graph migration according to claim 1 is characterized in that: The domain adversarial loss is expressed as: Among them, L w_domain represents the domain adversarial loss, N represents the number of source domain data features, M represents the number of target domain data features, Denotes the discriminator G d The source domain loss, Denotes the discriminator G d The target domain loss is Represents the i-th source domain feature After the discriminator G d The judgment result of represents the domain label of the i-th source domain feature, Represents the jth target domain feature After the discriminator G d The judgment result of Represents the domain label of the j-th target domain feature.

5. The hyperspectral remote sensing image object classification method based on graph migration according to claim 1 is characterized in that: The loop connection probability of cycle consistency is expressed as: in, Represents the i-th row and j-th column element of the cyclic connection probability matrix; Indicates P st The element in the i-th row and k-th column in represents the connection probability of the i-th source domain data and the k-th target domain data; Indicates P ts The element in the kth row and jth column in represents the connection probability of the kth target domain data and the jth source domain data; P st and P ts represents the normalized inter-domain graph adjacency matrix; P st and P ts By using different domain-to-domain graph adjacency matrices W st and W ts Normalized to get; Z s ∈N×D and Z′ t ∈M1×D transpose and dot product calculation to get W st ∈N×M1: By classifier G c Get Z′ t The classification probability P′ t , according to the source domain data X s =[X s1 ,X s2 ,...,X sN ]The label of each data, from P′ t Select the corresponding category column value as the column vector of the column where the corresponding source domain data is located, and splice it into a matrix W ts , N is the number of source domain data.

6. The hyperspectral remote sensing image object classification method based on graph migration according to claim 5 is characterized in that: The constraint loss for cycle consistency is: L cycle =β1L 循环连接约束 +β2L 同等约束连接 Among them, L cycle represents the constraint loss for cycle consistency, L 循环连接约束 represents the loop connection constraint loss, L 同等约束连接 represents the equal constraint connection loss, β1 and β2 represent the weights of the loop connection constraint loss and the equal constraint connection loss respectively; Among them, T ij represents the element at row i and column j in the indicator matrix; in, represents the number of source domain data belonging to the cth category, represents the i-th source domain feature, represents the jth source domain feature, Ω c Indicates the cth feature type; in, Indicates P st The element in the i-th row and j-th column represents the connection probability of the i-th source domain data and the j-th target domain data.

7. The method for classifying objects in hyperspectral remote sensing images based on graph migration according to claim 1, characterized in that: Classifier G c Using Z s and source domain data label Y s One-hot encoding training to improve G c The classification ability of source domain data, classifier G c The classification loss for source domain data is: Among them, L s Represents the classifier G c The classification loss of source domain data, N represents the number of source domain data, represents the one-hot encoding of the i-th source domain data label, Represents G c For the i-th source domain data feature The classification results; Enter Z t , W and the one-hot encoding of the unknown class label, so that G c With the ability to effectively identify unknown classes, the classifier G c The recognition loss for unknown classes is: Among them, L wt Represents the classifier G c The recognition loss for unknown classes, M represents the amount of target domain data, and W i represents the weight of the i-th target domain data, Y unk-onehot represents the one-hot encoding of the unknown class label, Represents G c For the i-th target domain data feature The recognition result.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the method according to any one of claims 1 to 7.

Images