A cross-scene hyperspectral image classification method

Abstract

This application relates to a cross-scene hyperspectral image classification method. The method includes: constructing a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transform network, a supervised contrastive learning module, a domain-independent directed acyclic graph (DAG) learning module, and a prototype contrastive learning module; constructing a total loss function based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent DAG learning loss, and prototype contrastive learning loss; training the cross-scene hyperspectral image classification model using the total loss function; and classifying the cross-scene hyperspectral image to be classified based on the trained cross-scene hyperspectral image classification model. This method can improve the performance of cross-scene hyperspectral image classification.

Description

Technical Field

[0001] This application relates to the field of image classification technology, and in particular to a cross-scene hyperspectral image classification method. Background Technology

[0002] Hyperspectral remote sensing technology has attracted much attention due to its wide range of applications in mineral exploration, ecological protection, agricultural management, and wetland monitoring. Hyperspectral sensors can measure the radiant flux reflected by the ground and atmosphere across hundreds of narrow and continuous spectral intervals in the visible and infrared reflective regions. The reflectivity of each pixel can be estimated based on the ratio of the radiant flux reflected at each pixel to the incident radiant flux on the pixel surface. The resulting hyperspectral images contain rich spatial and spectral information, and their reflectivity in different bands can reflect the inherent chemical composition characteristics of materials, providing more detailed features than traditional RGB and multispectral images.

[0003] However, in practical applications of land cover classification using hyperspectral images, the spectral reflectance of the same land cover category can differ in different scenes due to temporal and geographical differences between study areas, as well as the inevitable influence of sensor nonlinearity, season, and weather during hyperspectral image acquisition. This leads to spectral drift, causing models trained only on the source domain to experience a sharp performance drop when faced with new scenes they have never seen before.

[0004] To mitigate the impact of spectral and spatial distribution shifts between source and target scenes on model performance, domain adaptation techniques are widely employed. Domain adaptation typically learns cross-domain shared knowledge by aligning feature distributions between the source and target domains in the feature space, thereby achieving hyperspectral image classification on the target scene. In recent years, unsupervised domain adaptation based on supervised contrastive learning (SCLUDA) has reduced inter-domain differences through domain similarity loss based on single classification; generative domain adaptation based on class alignment and class balance (CCGDA) employs a novel class alignment domain adversarial loss function to address the challenge of cross-domain distribution differences in cross-scene hyperspectral image classification; and the Topology and Semantic Information Transfer Network (TSTnet) utilizes graph optimization to align topological relationships between different land cover categories and aligns the distributions between the source and target domains through maximum mean difference (MMD). While these methods have shown some effectiveness in mitigating distribution differences between the source and target domains, they require access to target domain data during training, and the model knowledge is specific to the target scene. When faced with new scenarios, it is still necessary to retrain the model by combining the source domain and the new target scenario data, which greatly limits its practical application.

[0005] In contrast, domain generalization techniques aim to address the limitations of domain adaptation methods. Neighborhood generalization does not rely on target domain data, focusing instead on learning cross-domain invariant class semantic information from source domain data, enabling the model to perform well in target domains unseen during training. The core idea of ​​this approach is to build a robust model capable of handling distributional differences across various scenarios. The application of domain generalization in cross-scene hyperspectral image classification effectively avoids the need to retrain the model when facing new scenes, thereby improving the model's practicality and adaptability. Hyperspectral cross-scene classification methods based on domain generalization primarily enhance the model's robustness in unseen domains by expanding the diverse distribution of the source domain. For example, the Single Source Domain Extension Network (SDEnet) for cross-scene hyperspectral image classification uses contrastive adversarial learning to enable a generator containing a semantic encoder and a morphological encoder to generate an extended domain with spatial and spectral variations, from which the discriminator learns class-domain invariant representations; the Locally Linear Unbiased Randomization Network (LLURnet) models feature distributions through 3D convolution and combines style randomization layers, locally linear embedding aggregated resampling, and intra-class comparison penalty terms to generate an extended domain to reduce domain gaps; the Spatial-Spectral Enhancement and Causal Constraint Network creates a generator to simulate the spectral and spatial biases of the target domain, enabling the causal contribution discriminator to extract "potential" causal invariance; the generalization network based on sublinear sample generation designs a random environmental noise to simulate the differences between different domains and generates new samples by adding noise to the linear part; the Spectral-Spanish Adversarial Multi-Domain Synthesis Network (S2AMSnet) uses a two-step min-max strategy to enable two structurally identical but parameter-dissimilar spectral-spatial domain generation networks to generate generated domains with diverse spatial and spectral fusion features, and constrains the spectral-spatial domain generation network through a multi-scale mutual information regularization network to ensure that the essential class semantic information of the generated domain does not deviate from the source domain.

[0006] However, these methods that improve the robustness of models in unseen domains by expanding the diversity of the source domain often overlook the detrimental effects of domain-specific features and easily fitted spurious features on model generalization. Furthermore, adversarial learning encounters instability and model collapse issues in practice. Domain-specific features and spurious features typically lead to models performing well in the source domain but poorly generalizing in the target domain. This is because these features differ significantly between the source and target domains, and the model is prone to overfitting to these features during training, resulting in poor performance in the target domain. Therefore, effectively mining domain-invariant features and pruning unstable domain-specific and spurious features is crucial for improving the cross-scene classification performance of hyperspectral images. Summary of the Invention

[0007] Therefore, it is necessary to provide a cross-scene hyperspectral image classification method that can improve the cross-scene classification performance of hyperspectral images in response to the above-mentioned technical problems.

[0008] A cross-scene hyperspectral image classification method, the method comprising:

[0009] Obtain the source domain hyperspectral dataset; the source domain hyperspectral dataset includes multiple source domain hyperspectral images; construct a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral patch low-frequency transformation network, a supervised contrastive learning module, a domain-independent directed acyclic graph learning module, and a prototype contrastive learning module;

[0010] The source domain hyperspectral image is sampled to obtain the original spectral block. The low-frequency transformation branch in the spectral block low-frequency transformation network perturbs the original spectral block with low frequency through the spectral block low-frequency transformation layer and extracts the spectral block features after low-frequency perturbation. The original spectral block features are learned in the frequency domain space of the frequency domain filtering branch. By maintaining the consistency of prediction between the spectral block features after low-frequency perturbation and the original spectral block features, the spectral block low-frequency transformation network learns the domain-independent features.

[0011] In the domain-independent directed acyclic graph (DAG) learning module, a structural causal model is obtained by modeling based on domain-independent features and corresponding class labels. The structural causal model is represented as a DAG, and the adjacency matrix of the DAG is learned from the domain-independent features and class labels according to the causal factor reconstruction method to set the generation function of the causal factors. The domain-independent features are compressed into multiple class condition prototype vectors and the causal factors are reconstructed. The reconstructed causal factors are then used to generate the reconstructed domain-independent features and class labels.

[0012] A total loss function is constructed based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss. The total loss function is used to train a cross-scene hyperspectral image classification model, and the trained cross-scene hyperspectral image classification model is used to classify the cross-scene hyperspectral images to be classified.

[0013] The aforementioned cross-scene hyperspectral image classification method, as described in this application, constructs a cross-scene hyperspectral image classification model. This model includes a spectral patch low-frequency transform network, a supervised contrastive learning module, a domain-independent directed acyclic graph (DAG) learning module, and a prototype contrastive learning module. Unlike previous methods, the domain-independent spectral patch low-frequency transform network, through a joint frequency domain filtering branch and a low-frequency transform branch, focuses more on domain-invariant features. Since the low-frequency components of the spectral patch contain more domain-specific spectral reflectance information, the joint frequency domain filtering branch and low-frequency transform branch effectively reduce the influence of domain-specific features, focusing on mining domain-invariant features from the data. Furthermore, it achieves generalization to unseen domains by pruning unstable domain-specific features and easily fitted spurious features. Simultaneously, by constructing a domain-independent DAG, the model can learn to identify domain-invariant features through causal structure learning and prune domain-specific features and spurious features that are detrimental to model generalization. To learn domain-independent directed acyclic graphs (DAGs), a progressive contrastive learning framework was designed. This framework optimizes the model through progressive supervised contrastive learning and prototype contrastive learning, thereby obtaining more representative class-conditional prototypes from which to search for domain-independent DAGs. Finally, the classifier extracts domain-independent features from the target domain hyperspectral data based on the joint spectral patch low-frequency transform network and the domain-independent DAG learned from the source domain hyperspectral data, achieving cross-scene hyperspectral image classification. Attached Figure Description

[0014] Figure 1 This is a flowchart of a cross-scene hyperspectral image classification method in one embodiment;

[0015] Figure 2 This is a schematic diagram of the framework of a cross-scene hyperspectral image classification model in one embodiment;

[0016] Figure 3 This is a schematic diagram of the frequency domain filtering branch network structure in one embodiment;

[0017] Figure 4 This is a schematic diagram of the low-frequency transform branch network structure in another embodiment.

[0018] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] In one embodiment, such as Figure 1 As shown, a cross-scene hyperspectral image classification method is provided, including the following steps:

[0021] Step 102: Obtain the source domain hyperspectral dataset; the source domain hyperspectral dataset includes multiple source domain hyperspectral images; construct a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrastive learning module, a domain-independent directed acyclic graph learning module, and a prototype contrastive learning module.

[0022] For a single-source-domain generalization problem, the goal is to use only a single source domain. The target domain not seen during training is obtained from the trained model. The results are good. In particular, this application considers the K-classification cross-scene land cover classification problem. The source domain hyperspectral dataset uses... It means that among them and Let P and B represent the spectral patches constructed from the source domain hyperspectral image and their corresponding labels, respectively. P×P is the spatial size of the spectral patch, and B is the number of bands in each spectral patch. It is worth noting that... and They share the same tag space.

[0023] Previous hyperspectral cross-scene classification methods aimed at generalizing to single-source domains focused on improving the model's robustness to unseen domains by expanding the diversity of source domain distributions. However, these methods ignored the detrimental effects of domain-specific features and easily fitted spurious features on model generalization. In contrast, DSPLTnet focuses on mining domain-invariant features from the data and pruning unstable domain-specific features and easily fitted spurious features to achieve generalization to unseen domains. Figure 2As shown, the cross-scene hyperspectral image classification model (DSPLTnet) includes a spectral patch low-frequency transform network, a supervised contrastive learning module, a domain-independent directed acyclic graph (DAG) learning module, and a prototype contrastive learning module. Since the low-frequency components of spectral patches contain more domain-specific spectral reflectance information, DSPLTnet, by combining the frequency domain filtering branch and the low-frequency transform branch, makes the spectral patch low-frequency transform network more focused on domain-invariant features. To further mine domain-invariant features from source domain hyperspectral data and reduce the adverse effects of domain-specific features and spurious features in source domain data that are easier for the network to fit, DSPLTnet constructs a domain-invariant DAG, enabling the model to learn and identify domain-invariant features through causal structure learning and to prune domain-specific features and spurious features that are detrimental to model generalization. To learn the domain-invariant DAG, this application optimizes the model through a progressive contrastive learning framework to obtain more representative class-conditional prototypes, from which a domain-invariant DAG is searched. Finally, the classifier, in conjunction with the low-frequency transform network of spectral blocks and the domain-independent DAG learned from the source domain hyperspectral data, extracts domain-independent features from the target domain hyperspectral data to achieve cross-scene hyperspectral image classification.

[0024] Step 104: Sample the source domain hyperspectral image to obtain the original spectral block; in the low-frequency transformation branch of the spectral block low-frequency transformation network, perform low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extract the spectral block features after low-frequency perturbation; learn the original spectral block features in the frequency domain space of the frequency domain filtering branch; and learn the domain-independent features by maintaining the prediction consistency between the spectral block features after low-frequency perturbation and the original spectral block features.

[0025] Each band of a hyperspectral image can reflect the reflectivity of electromagnetic waves of different frequencies to a specific material. The low-frequency components of each band can reflect the overall reflectivity information of the same land cover within a spectral patch for a specific area. Therefore, the low-frequency components in a spectral patch are considered as area-specific information of the land cover. To extract domain-invariant information from source-domain hyperspectral images for model generalization, a low-frequency transform network for spectral patches, based on frequency, was constructed to initially extract domain-invariant information from source-domain hyperspectral data. Specifically, the low-frequency transform network for spectral patches is designed to include a frequency-domain filtering branch and a low-frequency transform branch. Compared to the frequency-domain filtering branch, the low-frequency transform branch only adds a low-frequency transform layer for spectral patches without learnable parameters; the other layers are consistent with the frequency-domain filtering branch and share parameters. Spectral patches are sampled from source-domain hyperspectral images. First, after embedding the spectral block, it passes through a low-frequency transform branch and a frequency domain filtering branch. The low-frequency transform branch perturbs the original spectral block with a low-frequency transform layer and extracts the features of the perturbated spectral block. The frequency domain filtering branch learns the features of the original spectral block in the frequency domain. This application achieves consistency between the model's predictions of the perturbated spectral block features and the original spectral block features by enabling the spectral block low-frequency transform network to learn domain-invariant features. The low-frequency transform branch applies the spectral block low-frequency transform layer to the original spectral block, perturbing it with low frequencies and further extracting the perturbated spectral block features. Simultaneously, the frequency domain filtering branch focuses on learning the features of the original spectral block in the frequency domain. To enable the model to learn domain-invariant features, a mechanism is designed to ensure that the model maintains consistency in its predictions between the perturbated spectral block features and the original spectral block features.

[0026] The network structure of the frequency domain filtering branch is as follows: Figure 3 As shown, it mainly includes layer normalization, fast Fourier transform, learnable filters, inverse fast Fourier transform, and an MLP composed of linear layers. For the fast Fourier transform layer, given the input spectral patch, the embedded features are... Where H, W, and D represent the height, width, and number of channels of the spectral block embedding feature, respectively. The Fast Fourier Transform layer first embeds the spectral block feature... Transforming to the frequency domain, it can be formalized as follows:

[0027]

[0028] Among them, i 2 =-1, where m and n represent spatial frequencies, with the default setting shifting low-frequency components to the frequency center. Then, the frequency domain characteristics of the spectral patch are learned in the frequency domain space through learnable filter layers. This process can be represented as...

[0029]

[0030] in, The frequency domain characteristics of the spectral block, For a learnable filter, ⊙ denotes an element-wise multiplication operation. Secondly, the frequency domain characteristics of the spectral patch. Through the inverse Fourier transform layer The frequency domain features are transformed into the spatial domain, and finally the spectral block features are obtained through MLP.

[0031] The network structure of the low-frequency transform branch is as follows: Figure 4As shown, compared to the frequency domain filtering branch, only an additional low-frequency transform layer without learnable parameters is added between the fast Fourier transform layer and the learnable filtering layer. The low-frequency components in the spectral block represent the reflectivity information of the main land cover materials in that spectral block to electromagnetic waves of different frequencies, and can be considered as specific neighborhood information of that type of land cover. This application uses the low-frequency transform layer to perform low-frequency perturbation on the spectral block embedding features, so as to enable the low-frequency transform network to learn preliminary neighborhood-invariant features from it.

[0032] Specifically, for the low-frequency transform layer of spectral patches, given the embedding features of a batch of spectral patches in the source domain... Where b is the number of samples in a batch. The embedding features for each spectral block are obtained first through a Fast Fourier Transform layer to obtain the frequency domain features of the embedding features for a batch of spectral blocks. Then through a binary mask The low-frequency components can be obtained by formalizing them as follows:

[0033]

[0034] Among them, binary mask The central region is 1, and the other regions are 0. It can be represented as

[0035]

[0036] in, γ Used to distinguish between low-frequency components and high-frequency components.

[0037] For the low-frequency components of a batch of spectral block data obtained The distribution of each element in the low-frequency components of different samples in a batch is modeled as a multivariate Gaussian distribution. The mean of the Gaussian distribution is the original element value of the low-frequency component of the i-th spectral block at position (m, n, d), and the variance can be calculated from the element values ​​of the low-frequency components of different samples in a batch at position (m, n, d). This process can be formalized as follows:

[0038]

[0039] Then, the values ​​of low-frequency elements in the frequency domain features of each spectral block are resampled from the estimated distribution. This process can be represented as follows:

[0040]

[0041] Where β∈[0,1] is the perturbation parameter used to control the perturbation intensity. Finally, the original high-frequency components are combined with the resampled low-frequency components to obtain the spectral patch features in the frequency domain after low-frequency perturbation. (For spectral patches sampled from source region hyperspectral images) First, let x i In terms of spatial dimension, it is divided into n sub-spectral blocks of size q×q×B {x i1 x i2 , ..., x in Then, each sub-spectral block is linearly projected into a d-dimensional eigenvector using a learnable weight matrix.

[0042] Step 106: In the domain-independent directed acyclic graph learning module, a structural causal model is obtained by modeling based on domain-independent features and corresponding category labels. The structural causal model is represented as a directed acyclic graph. The adjacency matrix of the directed acyclic graph is learned from the domain-independent features and category labels using the causal factor reconstruction method to set the generation function of the causal factors. The domain-independent features are compressed into multiple class condition prototype vectors and then the causal factors are reconstructed. The reconstructed causal factors are used to generate the reconstructed domain-independent features and category labels.

[0043] Although the low-frequency transform network of spectral blocks can initially extract neighborhood-independent features from source domain hyperspectral images, the model still faces the risk of being affected by domain-specific features and spurious features that are easier for the network to fit. To enable the model to generalize by mining stable, category-corresponding neighborhood-invariant features from source domain hyperspectral images and to reduce the influence of domain-specific features and spurious features that are easier for the network to fit in source domain hyperspectral images, this application learns the feature relationships of a neighborhood-independent directed acyclic graph structure from the low-frequency transform data of spectral blocks. This allows the model to extract stable, category-corresponding neighborhood-independent features from hyperspectral data, achieving single-source domain generalization.

[0044] Specifically, let This represents the feature element output of the low-frequency transform branch in the spectral block low-frequency transform network, where y is the category label. To simplify the notation, S is used. i Representing characteristic elements And the tag y. This represents the entire set of causal factors. This application utilizes the principle of directed acyclic graph construction, first defining a structural causal model on the entire set of causal factors, namely...

[0045]

[0046] Among them, Pa i This indicates the relationship between low-frequency transform feature elements and low-frequency transform domain data. The set of parent nodes related to the direct cause of label y. The structural causal model v can be represented by a directed acyclic graph G = (S, E), where E = {(s...} i s j )} is containing, for example, s i→s j The causal marginal relationship. Therefore, the i-th factor s i It can be generated by function s i =g i (s) can be directly inferred.

[0047] Since the low-frequency transform features output by the low-frequency transform branch of the spectral block low-frequency transform network differ in different epochs for the same spectral block, the domain-independent directed acyclic graph G is defined as a DAG constructed from the low-frequency transform features of the spectral block. The common structure of G. If G is the same as all G i If the common structure matches, then G will discard domain-specific factors. The initial directed edge, and The association is identified as a correct causal direction. Strict acyclic constraints in domain-independent DAGs naturally avoid spurious factors. The impact of this is considered. To obtain a domain-independent DAG structure from source domain hyperspectral data, the adjacency matrix of the DAG is learned from the low-frequency transform features and labels of the spectral blocks using a causal factor reconstruction method in each training epoch. To search for a domain-independent DAG structure. Specifically, assume s is a vector formed by concatenating feature vectors and labels, i.e., s = [z...]. l [y], This application is based on the entire factor set Inferring arbitrary factors s i First, row vector A is used. i Masking non-parent elements and generating data using the data generation function g i Mapping the parent node to the i-th factor, the generating function used for causal factor reconstruction is:

[0048]

[0049] in, It is a learnable weight matrix used to map parent nodes in a DAG to classification logistic values, A i It is an indicator matrix that assigns parent nodes to child nodes.

[0050] However, to obtain a domain-independent DAG structure in the source domain hyperspectral data, it is necessary to traverse the entire training data to extract the latent factors of each sample for optimization, which results in high computational costs. Therefore, this application maintains K class conditional prototypes. By compressing the low-frequency transform features of the source domain spectral block into K representative class-conditional prototype vectors, and updating the class-conditional prototypes using momentum, this can be formalized as follows:

[0051]

[0052] Where θ∈(0,1) is the momentum parameter controlling the prototype update. It is the low-frequency transformation feature of the spectral block corresponding to the category label.

[0053] Specifically, e A⊙A The total causal effect among characteristic elements, including direct causal effects and effects indirectly transmitted through other variables, is represented by the positivity of the (i, j) elements of A raised to the power of n, indicating the existence of a path s of length n. i →…→s j The derivation of this idea is as follows:

[0054]

[0055] Among them, [e A⊙A ] i，j This is analogous to the total causal effect. [e] A⊙A ] t+1，1：t Row t+1 corresponds to the target variable, while columns 1:t correspond to the feature element set. A⊙A ] t+1，1：t This represents the total causal effect of each feature element on the target variable y. Indicates the low-frequency transformation characteristics of the spectral block The invariant features of the target label y contain all direct and indirect causal features.

[0056] Then, by reconstructing the causal factor vector represented by the class condition prototype, the search for the DAG can be achieved, which can be represented as follows:

[0057]

[0058] Where L cos (x1, x2) := 1-cos(x1, x2) is the cosine embedding loss, used to measure the similarity between the reconstructed data and the original data. L ce It is the cross-entropy loss function, used to measure the correspondence between reconstructed data and labels.

[0059] Secondly, to ensure the directed graph is acyclic and mitigate the impact of spurious factors on the model, DSPLTnet minimizes the trace of the adjacency matrix exponent to guarantee graph acyclicity, thereby eliminating the influence of spurious factors that easily lead to network fitting. This can be formalized as follows:

[0060] Tr(e A⊙A )-(t+1)=0 (12)

[0061] The constraint of formula (10) restricts a node in a directed graph from returning to itself even after an infinite number of steps.

[0062] The ultimate learning objective of DAG is then...

[0063]

[0064] stTr(e A☉A )-(t+1)=0

[0065] Where vec(·) is the vectorization operation of the matrix, and |vec(A)|1 is a constraint term used to force the DAG to tend towards sparsity.

[0066] Step 108: Construct a total loss function based on the pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss; train the cross-scene hyperspectral image classification model using the total loss function, and classify the cross-scene hyperspectral images to be classified based on the trained cross-scene hyperspectral image classification model.

[0067] To learn domain-independent DAGs from class-conditional prototypes, ensuring the representativeness of the class-conditional prototypes is crucial. This application designs a progressive contrastive learning framework to obtain more representative class-conditional prototypes. First, this application uses supervised contrastive learning to enable a spectral patch low-frequency transform network to initially extract domain-invariant features. This ensures that the class-conditional prototypes constructed from the features obtained by the spectral patch low-frequency transform network contain domain-invariant feature elements, thereby reducing the difficulty of searching for domain-independent DAGs. Specifically, for a batch of spectral patch embedding features... Frequency domain filter branch f ori Extract the original frequency domain features of each spectral block, and perform a low-frequency transform branch f. LPT Extracting low-frequency transform features from the spectral blocks after low-frequency transform, and learning class-domain-invariant representations from the original frequency domain features and low-frequency transform features through supervised ratio learning, can be formalized as follows:

[0068]

[0069] in, It is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and These represent the positive and negative sample index sets, respectively. Features obtained from the same spectral block in the original frequency domain features and low-frequency transform features, as well as features belonging to the same category, are considered elements in the positive sample set. Through optimization... It enables the low-frequency transform network of spectral blocks to extract shared representations from the original frequency domain features and low-frequency transform features of the same spectral block, and makes the representation distance of spectral blocks of the same category closer, and the representation distance of spectral blocks of different categories farther.

[0070] Then, through prototype contrastive learning, the invariant features of each spectral patch are aligned with its corresponding class prototype in the embedding space, making the class-conditional prototype constructed from the invariant features of each sample more representative. This process can be represented as follows:

[0071]

[0072] in, The prototype set of positive samples is defined as having invariant features. The class-conditional prototype vector corresponding to the category, p′ represents the invariant feature. Other class conditional prototype vectors that do not correspond to the category.

[0073] The optimization objective of the cross-scene hyperspectral image classification model consists of four parts, namely prediction consistency loss. DAG construction loss Supervised contrastive learning loss Loss compared to prototype Where prediction consistency loss is defined as

[0074]

[0075] in, The weights are for the classifier. The prediction consistency loss is used to ensure that the model can make the same predictions for the original frequency domain features and low-frequency transform features obtained from the same spectral block, prompting the model to make predictions based on the domain-invariant features in the samples.

[0076] The overall optimization objective can be formalized as follows:

[0077]

[0078] Where θ is the low-frequency transform network of the spectral block, and g i and the parameter set of the classifier, λ DAG and λ CL These are the hyperparameters used to balance the DAG reconstruction loss and the progressive contrastive learning.

[0079] When using the trained cross-scene hyperspectral image classification model for image classification, the hyperspectral image to be classified is input into the model, and the corresponding image category is output.

[0080] In the aforementioned cross-scene hyperspectral image classification method, this application constructs a cross-scene hyperspectral image classification model. This model includes a spectral patch low-frequency transform network, a supervised contrastive learning module, a domain-independent directed acyclic graph (DAG) learning module, and a prototype contrastive learning module. Unlike previous methods, the domain-independent spectral patch low-frequency transform network, through a joint frequency domain filtering branch and a low-frequency transform branch, focuses more on domain-invariant features. Since the low-frequency components of the spectral patch contain more domain-specific spectral reflectance information, the joint frequency domain filtering branch and low-frequency transform branch effectively reduce the influence of domain-specific features, focusing on mining domain-invariant features from the data. It also achieves generalization to unseen domains by pruning unstable domain-specific features and easily fitted spurious features. Simultaneously, by constructing a domain-independent DAG, the model can learn to identify domain-invariant features through causal structure learning and prune domain-specific features and spurious features that are detrimental to model generalization. To learn domain-independent directed acyclic graphs (DAGs), a progressive contrastive learning framework was designed. This framework optimizes the model through progressive supervised contrastive learning and prototype contrastive learning, thereby obtaining more representative class-conditional prototypes from which to search for domain-independent DAGs. Finally, the classifier extracts domain-independent features from the target domain hyperspectral data based on the joint spectral patch low-frequency transform network and the domain-independent DAG learned from the source domain hyperspectral data, achieving cross-scene hyperspectral image classification.

[0081] In one embodiment, the frequency domain filtering branch includes a normalization layer, a fast Fourier transform layer, a learnable filter layer, an inverse fast Fourier transform layer, and a linear layer; learning the original spectral patch features in the frequency domain space of the frequency domain filtering branch includes:

[0082] After the original spectral block is passed through a normalization layer, the embedding features of the original spectral block are transformed into the frequency domain space using a fast Fourier transform layer. Then, the frequency domain features of the spectral block are learned in the frequency domain space through a learnable filter layer. The frequency domain features are then transformed into the spatial domain through an inverse fast Fourier transform layer. Finally, the output is obtained through a linear layer to obtain the original spectral block features.

[0083] In one embodiment, a fast Fourier transform layer is used to transform the embedding features of the original spectral block into the frequency domain space, including:

[0084] The process of transforming the embedding features of the original spectral block into the frequency domain using a Fast Fourier Transform layer is as follows:

[0085]

[0086] in, Represents the original spectral block features, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block features, respectively. 2 =-1, where m and n represent spatial frequencies.

[0087] In one embodiment, the low-frequency transform branch perturbs the original spectral block with low frequencies through the spectral block low-frequency transform layer and extracts the features of the perturbated spectral block, including:

[0088] The embedding features of the original spectral block are transformed into the frequency domain space using a fast Fourier transform layer to obtain the frequency domain features;

[0089] In the low-frequency transform layer of the spectral block, the frequency domain features are binary-masked to obtain the low-frequency components; the distribution of each element in the low-frequency components of different samples in a batch is modeled as a multivariate Gaussian distribution; the values ​​of the low-frequency elements in the frequency domain features of each spectral block are resampled from the multivariate Gaussian distribution; the original high-frequency components are combined with the resampled low-frequency components to obtain the spectral block features perturbed by low frequency in the frequency domain space.

[0090] In one embodiment, the distribution of each element in the low-frequency component of different samples in a batch is modeled as a multivariate Gaussian distribution.

[0091]

[0092] in, This represents the original spectral block features, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block features, respectively.

[0093] In one embodiment, the original high-frequency component is combined with the resampled low-frequency component to obtain a spectral block characteristic of low-frequency perturbation in the frequency domain space.

[0094]

[0095] Where β∈[0,1] is a disturbance parameter used to control the disturbance intensity.

[0096] In one embodiment, the generation function of the causal factor is set, including:

[0097] Set the generation function of the causal factor as

[0098]

[0099] Where s = [z l ,y] represents the characteristic element z l and category label y, It is a learnable weight matrix used to map parent nodes in a directed acyclic graph to classification logistic values, A iIt is the indicator matrix that assigns parent nodes to child nodes, i.e., the adjacency matrix of a directed acyclic graph, where i represents the causal factor index.

[0100] In one embodiment, causal factors are reconstructed after domain-independent features are compressed into multiple class-conditional prototype vectors, including:

[0101] The process of reconstructing causal factors after compressing domain-independent features into multiple class-conditional prototype vectors is represented as follows:

[0102]

[0103] Among them, L cos (x1, x2) := 1-cos(x1, x2) is the cosine embedding loss, L ce It is the cross-entropy loss function, P i It is a conditional prototype, e A⊙A This represents the total causal effect among characteristic elements, including direct causal effects and effects indirectly transmitted through other variables. This represents the adjacency matrix of a directed acyclic graph.

[0104] In one embodiment, a total loss function is constructed based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss, including:

[0105] The total loss function is constructed based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss.

[0106]

[0107]

[0108] stTr(e A☉A )-(t+1)=0

[0109]

[0110] in, This represents the loss of prediction consistency. This represents the domain-independent learning loss for directed acyclic graphs. This indicates the learning loss in supervised comparison. This represents the prototype-contrast learning loss. It is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and These represent the positive sample index set and the negative sample index set, respectively. The prototype set of positive samples is defined as having invariant features. The class-conditional prototype vector corresponding to the category, p′ represents the invariant feature. Other class conditional prototype vectors that do not correspond to the category, θ is the low-frequency transform network of the spectral block, g i Let λ be the set of parameters for the classifier. DAG and λ CL These are the hyperparameters used to balance the domain-independent directed acyclic graph learning loss and the asymptotic contrastive learning loss, respectively. The weights of the classifier are vec(·), which is a matrix vectorization operation. |vec(A)|1 is a constraint term used to force the directed acyclic graph to tend towards sparsity. A⊙A This represents the total causal effect among characteristic elements, including direct causal effects and effects indirectly transmitted through other variables. Represents the adjacency matrix of a directed acyclic graph. This indicates the characteristics of the original spectral block.

[0111] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0112] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a cross-scene hyperspectral image classification method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0113] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A cross-scene hyperspectral image classification method, characterized in that, The method includes: Obtain a source domain hyperspectral dataset; the source domain hyperspectral dataset includes multiple source domain hyperspectral images; construct a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral patch low-frequency transformation network, a supervised contrastive learning module, a domain-independent directed acyclic graph learning module, and a prototype contrastive learning module; The source domain hyperspectral image is sampled to obtain the original spectral block; the low-frequency transformation branch in the spectral block low-frequency transformation network perturbs the original spectral block with low frequency through the spectral block low-frequency transformation layer and extracts the spectral block features after low-frequency perturbation; the original spectral block features are learned in the frequency domain space of the frequency domain filtering branch; by maintaining the prediction consistency between the spectral block features after low-frequency perturbation and the original spectral block features, the spectral block low-frequency transformation network learns the domain-independent features; In the domain-independent directed acyclic graph learning module, a structural causal model is obtained by modeling based on the domain-independent features and corresponding category labels. The structural causal model is represented as a directed acyclic graph. The adjacency matrix of the directed acyclic graph is learned from the domain-independent features and category labels using the causal factor reconstruction method to set the generation function of the causal factors. The domain-independent features are compressed into multiple class condition prototype vectors and then the causal factors are reconstructed. The reconstructed causal factors are used to generate the reconstructed domain-independent features and category labels. A total loss function is constructed based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss. The cross-scene hyperspectral image classification model is trained using the total loss function, and the cross-scene hyperspectral image to be classified is classified based on the trained cross-scene hyperspectral image classification model.

2. The method according to claim 1, characterized in that, The frequency domain filtering branch includes a normalization layer, a fast Fourier transform layer, a learnable filter layer, an inverse fast Fourier transform layer, and a linear layer. The original spectral patch features are learned in the frequency domain space of the frequency domain filtering branch, including: After the original spectral block is passed through a normalization layer, the embedding features of the original spectral block are transformed into the frequency domain space using a fast Fourier transform layer. Then, the frequency domain features of the spectral block are learned in the frequency domain space through a learnable filter layer. The frequency domain features are then transformed into the spatial domain through an inverse fast Fourier transform layer. Finally, the output is obtained through a linear layer to obtain the original spectral block features.

3. The method according to claim 2, characterized in that, The embedding features of the original spectral block are transformed into the frequency domain using a Fast Fourier Transform layer, including: The process of transforming the embedding features of the original spectral block into the frequency domain using a Fast Fourier Transform layer is as follows: in, Represents the original spectral block features, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block features, respectively. 2 =-1, where m and n represent spatial frequencies.

4. The method according to claim 1, characterized in that, The low-frequency transform branch perturbs the original spectral block with low frequencies through the spectral block low-frequency transform layer and extracts the features of the perturbated spectral block, including: The embedding features of the original spectral block are transformed into the frequency domain space using a fast Fourier transform layer to obtain the frequency domain features; The frequency domain features are binary-masked in the low-frequency transform layer of the spectral block to obtain low-frequency components; the distribution of each element in the low-frequency components of different samples in a batch is modeled as a multivariate Gaussian distribution; the values ​​of low-frequency elements in the frequency domain features of each spectral block are resampled from the multivariate Gaussian distribution; the original high-frequency components are combined with the resampled low-frequency components to obtain the spectral block features perturbed by low frequency in the frequency domain space.

5. The method according to claim 4, characterized in that, The distribution of each element in the low-frequency component of different samples in a batch is modeled as a multivariate Gaussian distribution as follows: in, This represents the original spectral block features, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block features, respectively.

6. The method according to claim 4, characterized in that, The original high-frequency components are combined with the resampled low-frequency components to obtain the spectral patch features in the frequency domain after low-frequency perturbation, including: The original high-frequency components are combined with the resampled low-frequency components to obtain the spectral block characteristics in the frequency domain after low-frequency perturbation. Where β∈[0,1] is a disturbance parameter used to control the disturbance intensity.

7. The method according to claim 1, characterized in that, Set the generation function for causal factors, including: Set the generation function of the causal factor as Where s = [z l ,y] represents the characteristic element z l and category labels It is a learnable weight matrix used to map parent nodes in a directed acyclic graph to classification logistic values, A i It is the indicator matrix that assigns parent nodes to child nodes, i.e., the adjacency matrix of a directed acyclic graph, where i represents the causal factor index.

8. The method according to claim 1, characterized in that, After compressing the domain-independent features into multiple class-conditional prototype vectors, causal factors are reconstructed, including: The process of reconstructing causal factors after compressing the domain-independent features into multiple class-conditional prototype vectors is represented as follows: Among them, L cos (x1, x2) := 1-cos(x1, x2) is the cosine embedding loss, L ce It is the cross-entropy loss function, p i It is a conditional prototype, e A⊙A This represents the total causal effect among characteristic elements, including direct causal effects and effects indirectly transmitted through other variables. Represents the adjacency matrix of a directed acyclic graph. It is a learnable weight matrix used to map parent nodes in a directed acyclic graph to classification logistic values.

9. The method according to claim 8, characterized in that, The total loss function is constructed based on pre-defined prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss, including: The total loss function is constructed based on pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss. stTr(e A☉A )-(t+1)=0 in, This represents the loss of prediction consistency. This represents the domain-independent learning loss for directed acyclic graphs. This indicates the learning loss in supervised comparison. This represents the prototype-contrast learning loss. It is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and These represent the positive sample index set and the negative sample index set, respectively. The prototype set of positive samples is defined as having invariant features. The class-conditional prototype vector corresponding to the category, p′ represents the invariant feature. Other class conditional prototype vectors that do not correspond to the category, θ is the low-frequency transform network of the spectral block, g i Let λ be the set of parameters for the classifier. DAG and λ CL These are the hyperparameters used to balance the domain-independent directed acyclic graph learning loss and the asymptotic contrastive learning loss, respectively. The weights of the classifier are vec(·), which is a matrix vectorization operation. |vec(A)|1 is a constraint term used to force the directed acyclic graph to tend towards sparsity. A⊙A This represents the total causal effect among characteristic elements, including direct causal effects and effects indirectly transmitted through other variables. Represents the adjacency matrix of a directed acyclic graph. This indicates the characteristics of the original spectral block.

Images