Hyperspectral image classification method based on causal element decoupling learning

Abstract

The invention discloses a hyperspectral image classification method based on causal element decoupling learning. The method comprises the following steps: firstly, by simulating a field generalization scene, helping a model to summarize causal meta-knowledge with invariant domains from the field generalization scene, and ensuring the strong generalization ability of the model to an unseen target domain; secondly, causal element decoupling learning is provided, a causal element knowledge induction framework based on distributed decoupling, virtual domain generation and invariant-generalization risk minimization is established, and a new normal form is provided for generalization in the hyperspectral field. Through the invariant causal effect loss, hyperspectral images of the same category are restrained to have invariant causal effects in different fields, accurate capture of real causal association between data and labels is achieved, and therefore the classification performance of a model in a field generalization task with invisible target domains is effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of pattern recognition, and in particular relates to a hyperspectral image classification method based on causal element decoupling learning. Background Art

[0002] Hyperspectral image classification is an important technology in the field of remote sensing. By analyzing and processing hyperspectral images, rich spectral information can be extracted from them, thereby achieving accurate classification of objects. Compared with traditional multispectral images, hyperspectral images have higher resolution in the spectral dimension, which makes them have broad application prospects in agriculture, forestry, environmental monitoring, urban planning and other fields. However, due to the characteristics of hyperspectral images such as large data volume, high dimension and high noise, their classification tasks face huge challenges. In response to these problems, researchers have proposed a variety of hyperspectral image classification methods, mainly including classification methods based on spectral features, classification methods based on spatial features and classification methods combining spectral-spatial features.

[0003] Although the existing hyperspectral image classification methods have improved the classification accuracy to a certain extent, these methods also have many shortcomings. For example, the methods based on spectral features are easily interfered by noise and it is difficult to effectively utilize spatial information; the methods based on spatial features are limited in the use of spectral information. In addition, many methods perform poorly in cross-domain generalization ability, that is, when the distribution of training data and test data is inconsistent, the classification performance will be significantly reduced. These shortcomings limit the effectiveness and universality of hyperspectral image classification methods in practical applications. In view of this, a new method based on causal reasoning mechanism and knowledge induction mechanism is particularly important. This method can better utilize the essential connection between data and improve the cross-domain generalization ability of the model through causal meta-decoupling learning. The causal reasoning mechanism can identify and utilize the causal relationship in the data, while the knowledge induction mechanism helps to summarize data features applicable to different domains from the existing data, so that the model has good adaptability and generalization ability between different fields.

[0004] However, there are still some problems to be solved under the causal meta-decoupling learning framework. First, how to effectively decouple the causal and non-causal relationships in the data distribution to accurately capture the essential connection of the data is still a challenge. Second, although the causal meta-knowledge induction framework can improve the generalization ability through steps such as distribution decoupling, virtual domain generation, and invariant-generalization risk minimization, in practical applications, the implementation effect of these steps may be affected by data complexity and inter-domain differences. For example, distribution decoupling may be difficult to achieve an efficient and accurate decoupling process on complex data sets, and the authenticity and effectiveness of the generation effect of virtual domain generation in high-dimensional data also need to be further verified. In addition, although the application of the invariant causal effect loss can constrain hyperspectral images of the same category to have an invariant causal effect in different fields, the effectiveness and stability of achieving this in diverse hyperspectral image data still need to be further verified and optimized. Summary of the invention

[0005] Purpose of the invention: In view of the problems existing in the above-mentioned background technology, the present invention provides a hyperspectral image classification method based on causal meta-decoupling learning, which decouples causal meta-knowledge and establishes a causal meta-knowledge induction framework based on distribution decoupling, virtual domain generation, and invariance-generalization risk minimization, constrains HSI of the same category to have an invariant causal effect in different fields, and realizes the accurate capture of the true causal relationship between data and labels.

[0006] Invention content: To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a hyperspectral image classification method based on causal element decoupling learning, comprising the following steps:

[0007] Segment the hyperspectral image HSI to obtain HIS patch samples, input the HIS patch samples into the feature extractor to obtain spatial-spectral features;

[0008] Bottleneck VAE is used to reconstruct the spatial-spectral features. In distribution learning, B-VAE is used to decouple the spatial-spectral features into class distribution and domain distribution. The forward information bottleneck is used to capture key class information, the reverse information bottleneck is used to capture class-independent domain information, and finally the decoder is used to reconstruct the features to ensure distribution modeling capabilities.

[0009] For the target domain data, after extracting the features, the class encoder in the trained B-VAE is used to obtain the class distribution, and then the class features are obtained from the class distribution, and the target domain is classified based on the obtained class features. Further, the feature extractor includes a spectral encoder and a spatial spectrum encoder connected in series;

[0010] The HIS patch sample is input into the feature extractor to obtain the space-spectrum feature; specifically comprising the following steps:

[0011] Step 1.1, segment the HIS patch in the spectral dimension to form a spectral sequence;

[0012] Step 1.2, input the spectral sequence into the spectral embedding module to obtain spectral features;

[0013] Step 1.3, segment the spectral features into multiple spatial regions in terms of spatial dimension, add a learnable class token to each of the divided spatial regions, and obtain a spatial sequence containing the class token;

[0014] In step 1.4, the spatial sequence containing the class token is input into the spatial embedding module for spatial feature extraction. The class token is the output of the spatial embedding module, and the class token contains spatial-spectral features.

[0015] Further, the bottleneck VAE includes a class encoder, a domain encoder and a decoder;

[0016] Input the spatial-spectral feature into the class encoder to obtain the class distribution, and sample the class latent variable Z from the class distribution c ;

[0017] Input the spatial-spectral feature into the domain encoder to obtain the domain distribution, and sample the domain latent variable Z from the domain distribution d ;

[0018] The class latent variable Z c and domain latent variable Z d After splicing, the data is input into the decoder to reconstruct the spatial-spectral features.

[0019] Furthermore, we can use Jensens inequality to express the lower bound of the evidence for bottleneck VAE.

[0020] Furthermore, considering that the class distribution should contain as little domain-specific information as possible and as much class-identifying information as possible, the following forward information bottleneck is constructed:

[0021]

[0022] Among them, I(Z c ,Y) represents the mutual information between the class latent variable and the label, represents the mutual information between the class latent variable and the space-spectral feature, and β is the weight factor.

[0023] According to the mutual information upper bound and mutual information lower bound, we can get the lower bound of the forward information bottleneck.

[0024] In addition, in the present invention, the final forward information bottleneck lower bound can be obtained by using a class encoder to learn the posterior distribution for approximation.

[0025] Considering that the domain distribution should contain as little class identification information as possible, and on the other hand, it should be ensured to have a strong correlation with the input to prevent it from learning trivial solutions that are irrelevant to the input, the following reverse information bottleneck is constructed:

[0026]

[0027] in, represents the mutual information between the domain latent variables and the space-spectral features, I(Z d ,Y) represents the mutual information between the domain latent variable and the label, and χ is the weight factor.

[0028] According to the upper bound and lower bound of mutual information, we can get the lower bound of the reverse information bottleneck

[0029] In addition, in the present invention, the domain encoder is used to learn the posterior distribution for approximation, and the final reverse information bottleneck lower bound can be obtained.

[0030] Furthermore, the forward-inverse information bottleneck requires the calculation of the mutual information between variables, however, it is difficult to directly calculate the mutual information between variables. Therefore, the upper and lower bounds of the mutual information are given to provide a basis for the optimization of the forward-inverse information bottleneck.

[0031] Furthermore, we sample class latent variables from the class distribution, perturb the domain distribution to generate virtual domain latent variables, and then generate virtual domain features through B-VAE decoding. We use virtual domain features to construct domain generalization meta-tasks, including meta-training tasks of multiple virtual domains and meta-verification tasks of a single virtual domain, to learn domain invariance and verify generalization performance; specifically:

[0032] First, a batch of samples are randomly sampled from the source domain, and the corresponding spatial-spectral features are obtained using the feature extractor. Then, the spatial-spectral features are input into the class encoder of B-VAE to obtain the class distribution And sample from the class distribution to get the class latent variable Among them, μ c and are the mean and variance of the class distribution respectively. Next, the spatial-spectral features are input into the domain encoder of B-VAE to obtain the domain distribution And perturb the domain distribution to obtain the virtual domain distribution Among them, μ v and are the mean and variance of the virtual domain distribution, respectively.

[0033] Next, the virtual domain latent variables are sampled from the virtual domain distribution After that, the class hidden variables and virtual domain hidden variables are concatenated and input into the decoder of B-VAE θ (·), generating virtual domain features

[0034] Then, the above process is repeated until K groups of virtual domain features are obtained.

[0035] Finally, K-1 groups of virtual domain features are randomly selected as virtual source domains to construct meta-training tasks to simulate possible domain distribution changes in the real world. The remaining 1 group of virtual domain features is selected as a virtual target domain to construct a meta-validation task to simulate an unseen target domain in the real world.

[0036] Furthermore, a two-layer optimization mechanism is customized for the domain generalization meta-task. In the inner layer of the invariant risk minimization process, the domain-invariant causal meta-knowledge is guided to be learned. In the outer layer of the generalization risk minimization process, the generalization ability of the causal meta-knowledge to the unseen target domain needs to be strengthened to ensure its applicability in the domain generalization task; specifically:

[0037] Input each virtual domain feature in the meta-training task into the class encoder to obtain the class latent variable z c , and z is randomly replaced by random vectors sampled from a Gaussian distribution in each dimension of e. c Conduct interventions;

[0038] Then, z c The category predictions obtained after intervention in each dimension are integrated to obtain the causal effect vector π;

[0039] Next, the causal effect vectors corresponding to the same type of virtual source domain features in the meta-training task are regarded as positive sample pairs, and the causal effect vectors corresponding to the different types of virtual domain features are regarded as negative sample pairs. The causal effect loss is used to minimize the difference between the positive sample pairs, while maximizing the difference between the positive and negative sample pairs to improve the intra-class consistency and inter-class difference of the class latent variables corresponding to each virtual domain feature.

[0040] Next, the causal effect loss and the constant risk minimization are combined to design the constant causal effect loss;

[0041] Finally, the class encoder is pseudo-updated based on the invariant causal effect loss.

[0042] Furthermore, the generalization risk loss L of the pseudo-updated model in the virtual target domain in the meta-validation task is calculated GR ;

[0043] Then, the meta-gradient G is calculated based on the invariant risk loss and the generalization risk loss M ;

[0044] Finally, based on the meta-gradients, the original class encoder parameters are meta-updated.

[0045] Beneficial effects: First, the present invention inputs each virtual domain feature in the meta-training task into the class encoder to obtain the class latent variable zc , and by adding z c The dimensions of z are randomly replaced with random vectors sampled from a Gaussian distribution. c Then c The category predictions obtained after intervention in each dimension are integrated to obtain the causal effect vector. Afterwards, the causal effect vectors corresponding to the same virtual source domain features in the meta-training task can be regarded as positive sample pairs, and the causal effect vectors corresponding to the different virtual domain features can be regarded as negative sample pairs. The causal effect loss is used to minimize the difference between the positive sample pairs, while maximizing the difference between the positive and negative sample pairs to improve the intra-class consistency and inter-class difference of the class latent variables corresponding to each virtual domain feature.

[0046] The present invention establishes a causal intervention mechanism based on front-gate adjustment based on the structural causal model, and achieves accurate capture of the true causal effect between data and labels by cutting off the false statistical associations between non-causal factors and labels.

[0047] Then, the domain distribution and class distribution of HSI are decoupled, and a virtual domain is generated by perturbing the domain distribution to simulate the potential complex domain distribution changes in the real world, providing a data basis for domain generalization scenario simulation.

[0048] Next, a causal meta-decoupling learning framework based on causal reasoning mechanism and knowledge induction mechanism was customized for domain generalization tasks. Specifically, a bottleneck VAE was proposed, which uses the forward-inverse information bottleneck to separate the tightly coupled domain distribution and class distribution in HSI, and simulates the potential domain distribution changes of HSI in the real world by perturbing the domain distribution.

[0049] Finally, virtual domain generation is performed based on the powerful data generation capability of VAE to provide a data basis for causal meta-knowledge induction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a principle block diagram of the method of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The hyperspectral image classification method based on decoupling learning provided by the present invention has the following specific principles: Figure 1 As shown, the following steps are included:

[0053] Step 1, task construction;

[0054] First, the hyperspectral image HSI is segmented to obtain HIS patch samples, and the HIS patch samples are input into a feature extractor. The feature extractor is a feature extractor pre-trained in the source domain and is used to map the high-dimensional HSI patch to a low-dimensional feature space. The feature extractor includes a spectral encoder and a spatial spectrum encoder connected in series. The spectral encoder is used to decompose the HSI patch into a spectral sequence and capture the dependency between spectral bands in the HIS patch; the spatial encoder is used to decompose the HSI patch into a spatial sequence and further capture the spatial dependency between pixels in the HIS patch to obtain the spatial-spectral feature. The specific steps are as follows:

[0055] Step 1.1: segment the hyperspectral image (HSI) patch in the spectral dimension to form a spectral sequence, specifically:

[0056] The HSI patch is divided into a series of spectral bands in order of wavelength to form a spectral sequence. Each spectral band represents the spectral information at different wavelengths, thus constructing a multi-dimensional representation of the hyperspectral data.

[0057] Step 1.2: Input the spectral sequence into the spectral embedding module to capture the dependencies between spectral bands. The specific steps are as follows:

[0058] The role of the spectrum embedding module is to capture the dependencies between spectral bands and process and extract spectral information through a complex neural network model, so that the characteristics of the spectral sequence can be updated and enhanced. This step ensures that the spectral features at different wavelengths can influence each other, thereby better reflecting the spectral characteristics of the material.

[0059] Step 1.3, split the spectral characteristics in the spatial dimension and add class tokens to obtain a spatial sequence containing class tokens, specifically:

[0060] The HSI patch is divided into several small spatial regions, each of which contains a certain number of pixels. A learnable class token is added to each spatial region to obtain a spatial sequence containing class tokens. The introduction of class tokens is intended to integrate the global spatial-spectral information of the HSI patch, acting as a representative of global information, and helping the model better understand and utilize the overall spatial-spectral characteristics.

[0061] Step 1.4, input the spatial sequence containing the class token into the spatial embedding module. The main function of the spatial embedding module is to capture the spatial dependencies between different pixels. By updating the spatial sequence containing the class token, the method of the present invention can understand the mutual relationship and spatial distribution characteristics between pixels in the spatial neighborhood, thereby improving the expression ability of spatial features.

[0062] In step 1.5, the class token in the updated spatial sequence in the spatial embedding module is used as the output. The class token already contains the integrated spatial-spectral features and represents the global features of the hyperspectral image HIS patch. In this way, the model can not only capture the complex relationship between spectral bands, but also understand the spatial dependency between pixels, thereby obtaining a more comprehensive and accurate spatial-spectral feature expression.

[0063] Step 2, distributed learning;

[0064] In distribution learning, B-VAE is used to decouple spatial-spectral features into class distribution and domain distribution. The key class information is captured through the forward information bottleneck, and the class-independent domain information is captured through the reverse information bottleneck. Finally, the features are reconstructed through the decoder to ensure the distribution modeling capability.

[0065] Part I: Evidence Lower Bounds for Bottleneck VAEs

[0066] The bottleneck variational autoencoder includes a class encoder, a domain encoder and a decoder;

[0067] Use the class encoder to model the class distribution of the empty-spectral features and sample the class latent variable Z from the class distribution c ;

[0068] Use the domain encoder to model the domain distribution of the spatial-spectral features and sample the domain latent variable Z from the domain distribution d ;

[0069] The class latent variable Z c and domain latent variable Z d After splicing, the data is input into the decoder to reconstruct the spatial-spectral features.

[0070] The evidence lower bound is a measure of how close the variational distribution is to the true posterior distribution. It is defined based on the KL divergence, which measures the difference between two probability distributions. In variational inference, we hope to minimize the KL divergence between the variational distribution and the posterior distribution, which is equivalent to maximizing the evidence lower bound;

[0071] Next, we can use Jensens inequality to express the lower bound of the evidence for the bottleneck VAE.

[0072] The lower bound of evidence is:

[0073]

[0074] Among them, D KL (·) represents KL divergence, encoder-like Domain Encoder Ψ and Φ are the class encoder and domain encoder parameters respectively, and the class latent variable z c , domain hidden variable z d , decoder θ is the decoder parameter, Represents the space-spectrum characteristics.

[0075] In the lower bound of evidence, To reconstruct the term, in order to optimize this term, the decoder is used based on the class latent variable z c and domain hidden variable z d right Reconstruction is performed to ensure the distribution modeling ability of the bottleneck-variational autoencoder B-VAE. The goal of the term is to minimize the domain posterior distribution and domain prior distribution p(z d ), in the invention, the domain prior distribution p(z d ) is set to N(0,I). The goal of the term is to minimize the class posterior distribution and class prior distribution p(z c ) is the KL divergence between .

[0076] The forward-inverse information bottleneck requires the calculation of the mutual information between variables, however, it is difficult to directly calculate the mutual information between variables. Therefore, the upper and lower bounds of the mutual information given in the present invention provide a basis for the optimization of the forward-inverse information bottleneck.

[0077] For variables A and B, the mutual information between them is expressed as:

[0078] I(A,B)=∫∫p(a,b)logp(b|a)dadb-∫p(b)logp(b)db (2)

[0079] Among them, p(a,b) is the joint probability distribution of A and B, p(a) and p(b) are the marginal probability distributions of A and B respectively. Since p(b) is difficult to calculate directly, r(b) is introduced as a variational approximation of p(b). KL (p(b),r(b))≥0, we can get ∫p(b)logp(b)db≥∫p(b)logr(b)db. Therefore, we can get the upper bound of the mutual information I(A,B) between A and B:

[0080]

[0081] The lower bound of the mutual information between A and B is:

[0082]

[0083] Among them, H(B) is the entropy of variable B, which represents the uncertainty of B; H(B|A) is the conditional entropy of B given A, which represents the uncertainty of B under the condition that A is known.

[0084] Part II: Forward-Reverse Information Bottleneck

[0085] Considering that the class distribution should contain as little domain-specific information as possible and as much class-identifying information as possible, this application constructs the following forward information bottleneck:

[0086]

[0087] Among them, I(z c ,Y) represents the mutual information between the class latent variable and the label, represents the mutual information between the class latent variable and the space-spectral feature, and β is the weight factor.

[0088] It is difficult to directly calculate the mutual information between variables. Therefore, the upper bound of the mutual information between the class latent variable and the label and the lower bound of the mutual information between the class latent variable and the space-spectrum feature are used to provide a basis for optimizing the positive information bottleneck.

[0089] Among them, by replacing A in formula (4) with class latent variables and replacing B in formula (4) with space-spectral features, the lower bound of the mutual information between class latent variables and space-spectral features can be calculated;

[0090] Among them, by replacing A in formula (3) with the class latent variable and replacing B in formula (3) with the label, the upper bound of the mutual information between the class latent variable and the label can be calculated.

[0091] According to the lower bound of the mutual information between the class latent variable and the label and the upper bound of the mutual information between the class latent variable and the space-spectral feature, the lower bound of the forward information bottleneck FIB can be obtained:

[0092]

[0093] Since the first term H(Y) of the lower bound is the entropy of the label, which is a constant here, it can be ignored in the optimization process. In addition, in the present invention, the class encoder is used to learn the posterior distribution right Approximate and use the empirical distribution approximate

[0094] Among them, Ψ is the class encoder parameter, and nare the observed n-th spatial-spectral features and their corresponding labels. Therefore, the final forward information bottleneck lower bound can be obtained:

[0095]

[0096] Among them, N is the number of sample samples, and C represents the label category set.

[0097] Class latent variables are sampled from the class distribution, and the mutual information between class latent variables and space-spectral features is minimized using the forward information bottleneck, while the mutual information between class latent variables and labels is maximized, forcing the model to capture key class identification information in space-spectral features.

[0098] Considering that the domain distribution should contain as little class identification information as possible, and on the other hand, it should be ensured to have a strong correlation with the input to prevent it from learning trivial solutions that are irrelevant to the input, the following reverse information bottleneck is constructed:

[0099]

[0100] in, represents the mutual information between the domain latent variables and the space-spectral features, I(Z d ,Y) represents the mutual information between the domain latent variable and the label, and χ is the weight factor.

[0101] Therefore, according to the upper bound of the mutual information between the domain latent variables and the spatial-spectral features and the lower bound of the mutual information between the domain latent variables and the labels, a basis is provided for the optimization of the reverse information bottleneck.

[0102] Among them, by replacing A in formula (3) with domain latent variables and replacing B in formula (3) with spatial-spectral features, the upper bound of the mutual information between domain latent variables and spatial-spectral features can be calculated;

[0103] Among them, by replacing A in formula (4) with the domain latent variable and replacing B in formula (4) with the label, the lower bound of the mutual information between the domain latent variable and the label can be calculated.

[0104] The lower bound of the reverse information bottleneck RIB is expressed as:

[0105]

[0106] Where Γ is the domain decoder parameter. The domain decoder reconstructs the spatial-spectral features using domain latent variables.

[0107] Since the first term of this lower bound H(Z d ) can be regarded as a constant, so it can be ignored in the optimization process. In this paper, the domain encoder is used to learn the posterior distribution right Approximate and use the empirical distribution approximate Among them, Φ is the domain encoder parameter. Therefore, the final reverse information bottleneck lower bound can be obtained:

[0108]

[0109] Class latent variables are sampled from the domain distribution, and the mutual information between the domain distribution and the spatial-spectral features is maximized using the inverse information bottleneck, while minimizing the mutual information between the domain distribution and the label, to ensure that the domain distribution accurately captures the class-independent domain-specific information in the spatial-spectral features.

[0110] Step 3: Meta-task construction.

[0111] First, class latent variables are sampled from the class distribution. Then, the domain distribution is perturbed to obtain a virtual domain distribution, and virtual domain latent variables are sampled from the virtual domain distribution. Next, the class latent variables and virtual domain latent variables are input into the decoder of B-VAE to generate virtual domain features. After that, the domain generalization meta-task is obtained through virtual domain generation. The domain generalization meta-task includes a meta-training task and a meta-validation task. Among them, the meta-training task contains multiple virtual domains to help the model learn domain-invariant causal knowledge from them, and the meta-validation task contains a virtual domain to verify the generalization performance of the model in unseen domains.

[0112] Specifically, a batch of samples are randomly sampled from the source domain, and the corresponding spatial-spectral features are obtained using the feature extractor. Then, the spatial-spectral features are input into the class encoder of B-VAE to obtain the class distribution And sample from the class distribution to get the class latent variable Among them, μ c and are the mean and variance of the class distribution respectively. Next, the spatial-spectral features are input into the domain encoder of B-VAE to obtain the domain distribution And perturb the domain distribution to obtain the virtual domain distribution Among them, μ v and are the mean and variance of the virtual domain distribution, respectively.

[0113] Next, the virtual domain latent variables are sampled from the virtual domain distribution After that, the class hidden variables and virtual domain hidden variables are concatenated and input into the decoder of B-VAE θ (·), generating virtual domain features

[0114] Then, the above process is repeated until K groups of virtual domain features are obtained.

[0115] Finally, K-1 groups of virtual domain features are randomly selected as virtual source domains to construct meta-training tasks to simulate possible domain distribution changes in the real world. The remaining 1 group of virtual domain features is selected as a virtual target domain to construct a meta-validation task to simulate an unseen target domain in the real world.

[0116] Step 4: Double-layer optimization.

[0117] A two-layer optimization mechanism is customized for the domain generalization meta-task. In the inner layer of the invariant risk minimization process, the domain-invariant causal meta-knowledge is guided to be learned. In the outer layer of the generalization risk minimization process, the generalization ability of the causal meta-knowledge to the unseen target domain needs to be strengthened to ensure its applicability in the domain generalization task.

[0118] Specifically, the features of each virtual domain in the meta-training task are input into the class encoder of the bottleneck VAE to obtain the class latent variable z c , and by transforming the class latent variable z c Each dimension of is randomly replaced by a random vector sampled from a Gaussian distribution, and the class latent variable z c Conduct interventions;

[0119] Then, the class latent variable z c The category predictions obtained after intervention in each dimension are integrated to obtain the causal effect vector π;

[0120] Next, the causal effect vectors corresponding to the same type of virtual source domain features in the meta-training task are regarded as positive sample pairs, and the causal effect vectors corresponding to the different types of virtual domain features are regarded as negative sample pairs. The causal effect loss is used to minimize the difference between the positive sample pairs, while maximizing the difference between the positive and negative sample pairs to improve the intra-class consistency and inter-class difference of the class latent variables corresponding to each virtual domain feature.

[0121] Next, the causal effect loss and the constant risk minimization are combined to design the constant causal effect loss;

[0122]

[0123] in, It represents the causal effect loss in the kth virtual source domain when the selection range of positive and negative samples is all virtual source domain features. It indicates the causal effect loss of the model in the kth virtual source domain when the selection range of positive and negative samples is the kth virtual source domain.

[0124] Finally, the class encoder is pseudo-updated based on the invariant causal effect loss.

[0125] Calculate the generalization risk loss L of the pseudo-updated model in the virtual target domain in the meta-validation task GR ;

[0126] Then, based on the constant risk loss and the generalized risk loss L GR Calculate the meta-gradient G M ;

[0127] Finally, based on the meta-gradient G M , meta-update the original class encoder parameters.

Claims

1. A hyperspectral image classification method, characterized in that: The steps include: Segment the hyperspectral image HSI to obtain HIS patch samples, input the HIS patch samples into the feature extractor to obtain spatial-spectral features; Bottleneck VAE is used to reconstruct the spatial-spectral features.

2. A hyperspectral image classification method according to claim 1, characterized in that: The feature extractor includes a spectral encoder and a spatial spectrum encoder connected in series; The HIS patch sample is input into the feature extractor to obtain the spatial-spectral feature; The specific steps include: Step 1.1, segment the HIS patch in the spectral dimension to form a spectral sequence; Step 1.2, input the spectral sequence into the spectral embedding module to obtain spectral features; Step 1.3, segment the spectral features into multiple spatial regions in terms of spatial dimension, add a learnable class token to each of the divided spatial regions, and obtain a spatial sequence containing the class token; In step 1.4, the spatial sequence containing the class token is input into the spatial embedding module, and the spatial dependency between pixels is captured to update the spatial sequence. The class token in the updated spatial sequence is used as the output of the spatial embedding module, and the class token contains spatial-spectral features.

3. A hyperspectral image classification method according to claim 1, characterized in that: The bottleneck VAE includes a class encoder, a domain encoder and a decoder; Input the spatial-spectral feature into the class encoder to obtain the class distribution, and sample the class latent variable Z from the class distribution c ; Input the spatial-spectral feature into the domain encoder to obtain the domain distribution, and sample the domain latent variable Z from the domain distribution d ; The class latent variable Z c and domain latent variable Z d After splicing, the data is input into the decoder to reconstruct the spatial-spectral features.

4. The hyperspectral image classification method according to claim 1, characterized in that: The evidence lower bound for the bottleneck VAE is expressed as: Among them, D KL (·) represents KL divergence, encoder-like Domain Encoder Ψ is the class encoder parameter, Φ is the domain encoder parameter, and the class latent variable z c , domain hidden variable z d , decoder q is the decoder parameter, Represents the space-spectrum characteristics.

5. The hyperspectral image classification method according to claim 2, characterized in that: Constructing a positive information bottleneck: Among them, I(z c ,Y) represents the mutual information between the class latent variable and the label, represents the mutual information between the class latent variable and the space-spectral feature, and β is the weight factor.

6. A hyperspectral image classification method according to claim 2, characterized in that: Constructing a reverse information bottleneck: in, represents the mutual information between the domain latent variables and the space-spectral features, I(z d ,Y) represents the mutual information between the domain latent variable and the label, and χ is the weight factor.

7. A hyperspectral image classification method according to claim 5 or 6, characterized in that: According to the mutual information I(z c ,Y), and the mutual information between class latent variables and space-spectral features Upper bound, calculate the lower bound of the forward information bottleneck FIB; Mutual information between domain latent variables and spatial-spectral features The upper bound of , and the mutual information I(z d ,Y) Lower bound, calculate the lower bound of the reverse information bottleneck RIB.

8. A hyperspectral image classification method according to claim 7, characterized in that: The upper bound of the mutual information between A and B is expressed as: Where p(a,b) is the joint probability distribution of A and B, p(b|a) represents the probability distribution of B given A, r(b) is the variational approximation of p(b), and p(b) is the marginal probability distribution of B; The class latent variable z c Replace A in formula (2), replace B in formula (2) with label Y, and calculate the mutual information I(z c ,Y)’s upper bound; Use domain hidden variable z d Replace A in equation (2), space-spectral feature Replace B in equation (2); calculate the mutual information between the domain latent variable and the space-spectral feature The upper bound of The lower bound of the mutual information between A and B is: Among them, H(B) is the entropy of variable B, which represents the uncertainty of B; H(B|A) is the conditional entropy of B given A, which represents the uncertainty of B under the condition that A is known; Class hidden variable z c Replace A in equation (3), space-spectral feature Replace B in formula (3) and calculate the mutual information between the class latent variable and the space-spectral feature The Nether, Use domain hidden variable z d Replace A in equation (3), replace B in equation (3) with label y, and calculate the lower bound of the mutual information between the domain latent variable and the label.

9. A hyperspectral image classification method according to claim 8, characterized in that: The lower bound of the forward information bottleneck FIB is: The lower bound of the reverse information bottleneck RIB is expressed as:

10. A hyperspectral image classification method according to claim 8, characterized in that: Randomly sample a batch of samples from the source domain and use the feature extractor to obtain the corresponding spatial-spectral features; Then, the spatial-spectral features are fed into the class encoder of the bottleneck VAE to obtain the class distribution And sample from the class distribution to get the class latent variable Among them, μ c and are the mean and variance of the class distribution respectively; Next, the spatial-spectral features are input into the domain encoder of B-VAE to obtain the domain distribution And perturb the domain distribution to obtain the virtual domain distribution Among them, μ v and are the mean and variance of the virtual domain distribution, respectively; Next, distribute from the virtual domain The virtual domain latent variables are obtained by sampling Afterwards, the class hidden variable z c and virtual domain latent variables Splice and input into B-VAE decoder q (·), generating virtual domain features Then, repeat the above process until K groups of virtual domain features are obtained; Finally, K-1 groups of virtual domain features are randomly selected as the virtual source domain to construct the meta-training task, and the remaining 1 group of virtual domain features is selected as the virtual target domain to construct the meta-verification task. Input each virtual domain feature in the meta-training task into the class encoder to obtain the class latent variable z c , and by adding z c The dimensions of z are randomly replaced with random vectors sampled from a Gaussian distribution. c Conduct interventions; Then, z c The category predictions obtained after intervention in each dimension are integrated to obtain the causal effect vector π; then, the causal effect vectors corresponding to the same virtual source domain features in the meta-training task are regarded as positive sample pairs, and the causal effect vectors corresponding to the different virtual domain features are regarded as negative sample pairs. The causal effect loss is used to minimize the difference between the positive sample pairs, while maximizing the difference between the positive and negative sample pairs to improve the intra-class consistency and inter-class difference of the class latent variables corresponding to each virtual domain feature; Next, the causal effect loss and the constant risk minimization are combined to design the constant causal effect loss; Next, the class encoder is pseudo-updated based on the invariant causal effect loss; Next; calculate the generalization risk loss of the pseudo-updated model in the virtual target domain in the meta-validation task; Then, the meta-gradient is calculated based on the invariant risk loss and the generalized risk loss; Finally, based on the meta-gradients, the original class encoder parameters are meta-updated.

Images