Cross-scene hyperspectral image classification method

Abstract

The invention relates to a cross-scene hyperspectral image classification method. The method comprises the following steps: constructing a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model comprises a spectral block low-frequency transformation network, a supervision comparative learning module, a domain-independent directed acyclic graph learning module and a prototype comparative learning module; constructing a total loss function according to preset prediction consistency loss, supervision contrast learning loss, field-independent directed acyclic graph learning loss and prototype contrast learning loss; and training the cross-scene hyperspectral image classification model by using the total loss function, and classifying cross-scene hyperspectral images to be classified according to the trained cross-scene hyperspectral image classification model. By adopting the method, the cross-scene classification performance of the hyperspectral image can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image classification, and in particular to a cross-scene hyperspectral image classification method. Background Art

[0002] Hyperspectral remote sensing technology has attracted much attention due to its wide application in mineral exploration, ecological protection, agricultural management, wetland monitoring and other fields. Hyperspectral sensors can measure the radiation flux reflected by the ground and the atmosphere in hundreds of narrow and continuous spectral intervals in the visible light and infrared reflection parts. The reflectivity of each pixel can be estimated based on the ratio of the radiation flux reflected on each pixel to the incident radiation flux on the pixel surface. The resulting hyperspectral image has rich spatial and spectral information. Its reflectivity in different bands can reflect the inherent chemical composition characteristics of the substance and can provide more detailed features than traditional RGB and multispectral images.

[0003] However, in the actual application of land cover classification using hyperspectral images, due to the temporal and geographical differences between the study areas, and the inevitable influence of sensor nonlinearity, seasons and weather factors in the acquisition of hyperspectral images, the spectral reflectance of the same land cover category in different scenes will be different, leading to the problem of spectral drift. This makes the performance of the model trained only on the source domain drop sharply when facing new scenes that the model has not seen.

[0004] In order to reduce the impact of spectral and spatial distribution offsets between source and target domains on model performance, domain adaptation techniques are widely used. Domain adaptation usually aligns the feature distribution between the source and target domains in the feature space, thereby learning cross-domain shared knowledge and achieving hyperspectral image classification on the target scene. In recent years, the supervised contrastive learning-based unsupervised domain adaptation (SCLUDA) method reduces inter-domain differences by domain similarity loss based on single classification; the class alignment and class balance-based generative domain adaptation (CCGDA) method adopts a new class alignment domain adversarial loss function to try to solve the challenge of cross-domain distribution differences in cross-scene hyperspectral image classification; the topological structure and semantic information transfer network (TSTnet) uses graph optimization to transfer and align the topological relationship between different land cover categories, and aligns the distribution between the source and target domains through the maximum mean difference (MMD). Although these methods have some effect in alleviating the distribution difference between the source and target domains, they need to access the target domain data during training, and the model knowledge is specific to the target scene. When faced with new scenarios, the model still needs to be retrained by combining the source domain with the new target scene data, which greatly limits its practical application.

[0005] In contrast, domain generalization techniques aim to address the limitations of domain adaptation methods. Neighborhood generalization does not rely on target domain data, but focuses on learning cross-domain invariant class-domain semantic information from source domain data, so that the model can perform well on target domains that have not been seen during training. The core idea of ​​this method is to build a robust model that can cope with distribution differences in different scenes. The application of domain generalization in cross-scene hyperspectral image classification can effectively avoid the need to retrain the model when facing new scenes, thereby improving the practicality and adaptability of the model. Hyperspectral cross-scene classification methods based on domain generalization mainly improve the robustness of the model in unseen domains by expanding the diverse distribution of the source domain. For example, the Single Source Domain Extension Network (SDEnet) for cross-scene hyperspectral image classification uses contrastive adversarial learning to enable the generator containing the semantic encoder and the morphological encoder to generate an extended domain with spatial and spectral variations and enable the discriminator to learn class domain invariant representations from it; the Local Linear Unbiased Randomization Network (LLURnet) models feature distribution through 3D convolutions and combines style random layers, local linear embedding aggregation resampling and intra-class comparison penalties to generate an extended domain to reduce the domain gap; the spatial-spectral enhancement and causal constraint network creates a generator to simulate the spectral and spatial deviations of the target domain so that the causal contribution discriminator can extract "latent" causal invariance from it; the generalization network based on split-linear sample generation designs a random environmental noise to simulate the differences between different domains, and generates new samples by adding noise to the linear part; the spectral-spatial adversarial multi-domain synthesis network (S2AMSnet) uses a two-step minimum-maximum strategy to enable two spectral-spatial domain generation networks with the same structure but no shared parameters to generate a generation domain with diversified spatial and spectral fusion features, and constrains the spectral-spatial domain generation network through a multi-scale mutual information regularization network so that the essential class semantic information of the generated domain does not deviate from the source domain.

[0006] However, these methods that improve the robustness of models on unseen domains by expanding the diverse distribution of the source domain often ignore the adverse effects of domain-specific features and easily fitted spurious features on model generalization, and adversarial learning encounters problems of instability and model collapse in practice. Domain-specific features and spurious features usually lead to models that perform well on the source domain but have poor generalization ability on the target domain. This is because these features differ significantly between the source and target domains, and the model tends to overfit these features during training, resulting in poor performance on the target domain. Therefore, how to effectively mine domain-invariant features and prune unstable domain-specific features and spurious features becomes the key to improving the cross-scene classification performance of hyperspectral images. Summary of the invention

[0007] Based on this, it is necessary to provide a cross-scene hyperspectral image classification method that can improve the cross-scene classification performance of hyperspectral images in response to the above technical problems.

[0008] A cross-scene hyperspectral image classification method, the method comprising:

[0009] Obtain a source domain hyperspectral dataset; the source domain hyperspectral dataset includes multiple source domain hyperspectral images; construct a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module;

[0010] The source domain hyperspectral image is sampled to obtain the original spectral block; the low-frequency transformation branch in the spectral block low-frequency transformation network performs low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extracts the spectral block features after the low-frequency perturbation, and learns the original spectral block features in the frequency domain space of the frequency domain filtering branch, and the spectral block low-frequency transformation network learns domain-independent features by maintaining the consistency of prediction between the spectral block features after the low-frequency perturbation and the original spectral block features;

[0011] In the domain-independent directed acyclic graph learning module, modeling is performed based on domain-independent features and corresponding category labels to obtain a structural causal model; the structural causal model is represented as a directed acyclic graph, and the adjacency matrix of the directed acyclic graph is learned from the domain-independent features and category labels according to the causal factor reconstruction method to set the generating function of the causal factor; the domain-independent features are compressed into multiple class conditional prototype vectors and then the causal factors are reconstructed, and the reconstructed domain-independent features and category labels are generated using the reconstructed causal factors;

[0012] A total loss function is constructed based on the pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss and prototype contrastive learning loss. The cross-scene hyperspectral image classification model is trained using the total loss function, and the cross-scene hyperspectral images to be classified are classified according to the trained cross-scene hyperspectral image classification model.

[0013] The above-mentioned cross-scene hyperspectral image classification method, the present application constructs a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module. By setting a domain-independent spectral block low-frequency transformation network, unlike previous methods, the spectral block low-frequency transformation network combines frequency domain filtering branches and low-frequency transformation branches to make the spectral block low-frequency transformation network pay more attention to domain-invariant features. Since the low-frequency component of the spectral block contains more domain-specific ground object spectral reflectance information, the combined frequency domain filtering branch and the low-frequency transformation branch can effectively reduce the influence of domain-specific features, focus on mining domain-invariant features from the data, and achieve generalization of unseen domains by pruning unstable domain-specific features and easily fitted false features. At the same time, by constructing a domain-independent directed acyclic graph, the model can identify domain-invariant features through causal structure learning, and prune domain-specific features and false features that are not conducive to model generalization. At the same time, in order to learn a domain-independent directed acyclic graph, a progressive contrastive learning framework is designed. Through progressive supervised contrastive learning and prototype contrastive learning optimization models, a more representative class conditional prototype is obtained to search for domain-independent directed acyclic graphs. Finally, the classifier extracts domain-independent features in the target domain hyperspectral data based on the joint spectral block low-frequency transform network and the domain-independent directed acyclic graph learned from the source domain hyperspectral data, realizing cross-scene hyperspectral image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a cross-scene hyperspectral image classification method in one embodiment;

[0015] Figure 2 A schematic diagram of a framework of a cross-scene hyperspectral image classification model in one embodiment;

[0016] Figure 3 A schematic diagram of a frequency domain filtering branch network structure in one embodiment;

[0017] Figure 4 A schematic diagram of a low-frequency conversion branch network structure in another embodiment

[0018] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] In one embodiment, Figure 1 As shown, a cross-scene hyperspectral image classification method is provided, comprising the following steps:

[0021] Step 102, obtaining a source domain hyperspectral dataset; the source domain hyperspectral dataset includes multiple source domain hyperspectral images; constructing a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module.

[0022] For a single-source domain generalization problem, the goal is to use only a single source domain The trained model is trained on target domains that were not seen during training In particular, this application considers the cross-scene object classification problem of K classification. The source domain hyperspectral dataset is used Indicates that and They represent the spectral blocks constructed from the source domain hyperspectral image and their corresponding labels, P×P is the spatial size of the spectral block, and B is the number of bands in each spectral block. It is worth noting that and share the same label space.

[0023] Previous hyperspectral cross-scene classification methods for single-source domain generalization focused on improving the robustness of the model to unseen domains by expanding the diverse distribution of the source domain. However, these methods ignore the adverse effects of domain-specific features and easier-to-fit spurious features on model generalization. In contrast, DSPLTnet focuses on mining domain-invariant features from data and pruning unstable domain-specific features and easier-to-fit spurious features to achieve generalization to unseen domains. Figure 2As shown, the cross-scene hyperspectral image classification model (DSPLTnet) includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module. Since the low-frequency component of the spectral block contains more domain-specific spectral reflectance information of the ground object, DSPLTnet combines the frequency domain filtering branch and the low-frequency transformation branch to make the spectral block low-frequency transformation network pay more attention to the domain-invariant features. In order to further mine domain-invariant features from the source domain hyperspectral data and reduce the adverse effects of domain-specific features and false features in the source domain data that make it easier for the network to fit on the model, DSPLTnet constructs a domain-independent directed acyclic graph to enable the model to identify domain-invariant features through causal structure learning and prune domain-specific features and false features that are not conducive to model generalization. In order to learn a domain-independent directed acyclic graph, this application optimizes the model through a constructed progressive contrast learning framework to obtain a more representative class-conditional prototype, and then searches for a domain-independent directed acyclic graph from it. Finally, the classifier combines the spectral block low-frequency transform network with the domain-independent DAG learned from the source domain hyperspectral data to extract domain-independent features of the target domain hyperspectral data to achieve cross-scene hyperspectral image classification.

[0024] Step 104, sampling the source domain hyperspectral image to obtain original spectral blocks; the low-frequency transformation branch in the spectral block low-frequency transformation network performs low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extracts the spectral block features after the low-frequency perturbation, and learns the original spectral block features in the frequency domain space of the frequency domain filtering branch, and the spectral block low-frequency transformation network learns domain-independent features by maintaining the consistency of prediction for the spectral block features after the low-frequency perturbation and the original spectral block features.

[0025] Each band of a hyperspectral image can reflect the reflectivity of electromagnetic waves of different frequencies to a specific material. The low-frequency component of each band can reflect the overall reflectivity information of the same feature in a spectral block to a specific field. Therefore, the low-frequency component in the spectral block is regarded as the field-specific information of the feature. In order to mine domain-invariant information from source domain hyperspectral images for model generalization, a spectral block low-frequency transform network based on frequency is constructed to preliminarily mine domain-invariant information in source domain hyperspectral data. Specifically, the spectral block low-frequency transform network is designed to include a frequency domain filtering branch and a low-frequency transform branch. Compared with the frequency domain filtering branch, the low-frequency transform branch only has an additional spectral block low-frequency transform layer without learnable parameters. The other layers are consistent with the frequency domain filtering branch and share parameters. Spectral blocks sampled from source domain hyperspectral images First, after the spectral block is embedded, it passes through the low-frequency transformation branch and the frequency domain filtering branch respectively. The low-frequency transformation branch performs low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extracts the features of the spectral block after the low-frequency perturbation. The frequency domain filtering branch learns the features of the original spectral block in the frequency domain space. This application enables the model to maintain the consistency of prediction for the features of the spectral block after the low-frequency perturbation and the features of the original spectral block so that the spectral block low-frequency transformation network can learn domain-invariant features from it. The low-frequency transformation branch achieves low-frequency perturbation by applying the spectral block low-frequency transformation layer to the original spectral block, and further extracts the features of the spectral block affected by the low-frequency perturbation. At the same time, the frequency domain filtering branch focuses on learning the features of the original spectral block in the frequency domain space. In order to enable the model to learn domain-invariant features from it, a mechanism is designed to ensure that the model maintains consistency in prediction between the features of the spectral block after the low-frequency perturbation and the features of the original spectral block.

[0026] The network structure of the frequency domain filtering branch is as follows Figure 3 As shown in Figure 1, it mainly includes layer normalization, fast Fourier transform, learnable filter, inverse fast Fourier transform and MLP composed of linear layers. For the fast Fourier transform layer, given the input spectrum block embedding feature Where H, W and D are the height, width and number of channels of the spectral block embedding feature respectively. The fast Fourier transform layer first embeds the spectral block into the feature Transformed into the frequency domain, it can be formalized as

[0027]

[0028] Among them, i 2 = -1, m and n represent spatial frequencies, and the low-frequency components are moved to the center of the frequency by default. Then the frequency domain features of the spectral block are learned in the frequency domain space through the learnable filter layer. This process can be expressed as

[0029]

[0030] in, is the frequency domain feature of the spectral block, is a learnable filter, and ⊙ represents an element-by-element multiplication operation. Secondly, the frequency domain features of the spectral block Through the inverse Fourier transform layer The frequency domain features are converted to the spatial domain, and finally the features of the spectral block are obtained through MLP.

[0031] The network structure of the low-frequency transform branch is as follows Figure 4As shown, compared with the frequency domain filtering branch, there is only one more spectral block low-frequency transformation layer without learnable parameters between the fast Fourier transform layer and the learnable filter layer. The low-frequency component in the spectral block represents the reflectivity information of the main ground object material in the spectral block to electromagnetic waves of different frequencies, which can be considered as the specific domain information of this type of ground object. The present application performs low-frequency perturbation on the spectral block embedded features through the spectral block low-frequency transformation layer to prompt the spectral block low-frequency transformation network to learn preliminary domain-invariant features therefrom.

[0032] Specifically, for the spectral block low-frequency transformation layer, given a batch of spectral blocks in the source domain, the embedding features Where b is the number of samples in a batch, The embedded features of each spectral block. First, the frequency domain features of a batch of spectral block embedding features are obtained through the fast Fourier transform layer. Then through the binary mask The low-frequency component can be formalized as

[0033]

[0034] Among them, the binary mask The center area is 1, and the rest of the area is 0. It can be expressed as

[0035]

[0036] in, γ Used to distinguish low-frequency components from high-frequency components.

[0037] For the low-frequency component of a batch of spectral block data obtained Each element distribution in the low-frequency component of different samples in a batch is modeled as a multivariate Gaussian distribution. The mean of the Gaussian distribution is the original element value of the low-frequency component of the i-th spectral block at the position (m, n, d), and the variance can be calculated by the element value of the low-frequency component of different samples in a batch at the position (m, n, d). The process can be formalized as

[0038]

[0039] Then the values ​​of the low-frequency elements in the frequency domain features of each spectral block are resampled from the estimated distribution. This process can be expressed as

[0040]

[0041] Among them, β∈[0,1] is the perturbation parameter used to control the perturbation intensity. Finally, the original high-frequency component is combined with the resampled low-frequency component to obtain the spectral block features after low-frequency perturbation in the frequency domain space. (For spectral blocks sampled from source domain hyperspectral images First, x i The spatial dimension is divided into n sub-spectral blocks of size q×q×B {x i1 , x i2 , …, x in}, and then each sub-spectral block is linearly projected into a d-dimensional feature vector through a learnable weight matrix. )

[0042] Step 106, modeling is performed in a domain-independent directed acyclic graph learning module based on the domain-independent features and the corresponding category labels to obtain a structural causal model; the structural causal model is represented as a directed acyclic graph, and the adjacency matrix of the directed acyclic graph is learned from the domain-independent features and category labels according to the causal factor reconstruction method to set the generating function of the causal factors; the domain-independent features are compressed into multiple class conditional prototype vectors and then the causal factors are reconstructed, and the reconstructed domain-independent features and category labels are generated using the reconstructed causal factors.

[0043] Although the spectral block low-frequency transformation network can initially extract domain-independent features from the source domain hyperspectral image, the model still has the risk of being affected by domain-specific features and false features that are more likely to make the network fit. In order to enable the model to mine stable domain-invariant features corresponding to the category from the source domain hyperspectral image for generalization and reduce the impact of the model on the domain-specific features and false features that are more likely to make the network fit in the source domain hyperspectral image, the present application learns the feature relationship of the domain-independent directed acyclic graph structure from the spectral block low-frequency transformation data, so that the model can extract stable domain-independent features corresponding to the category from the hyperspectral data to achieve single source domain generalization.

[0044] Specifically, It represents the characteristic element of the low-frequency transform branch output in the spectral block low-frequency transform network, y is the category label, and S is used to simplify the symbolic representation. i Representing characteristic elements and label y. Represents the entire set of causal factors. This application uses the principle of directed acyclic graph construction to first define a structural causal model on the entire set of causal factors s, namely

[0045]

[0046] Among them, Pa i Indicates the low-frequency transform feature element in the low-frequency transform domain data The set of parent nodes related to the direct cause of label y. The structural causal model v can be represented by a directed acyclic graph G = (S, E), where E = {(s i ,s j )} is a string containing, for example, i→s j Therefore, the ith factor s i The function s can be generated i =g i (s) can be directly inferred.

[0047] Since the low-frequency transformation features output by the low-frequency transformation branch of the spectrum block low-frequency transformation network are different for the same spectrum block in different epochs, the domain-independent directed acyclic graph G is defined as a DAG constructed from the low-frequency transformation features of the spectrum block: If G and all G i If the common structure matches, G will discard domain-specific factors The directed edge at the beginning and The strict acyclic constraints in the domain-independent DAG can naturally avoid spurious factors. In order to obtain the domain-independent DAG structure from the source domain hyperspectral data, the adjacency matrix of the DAG is learned from the low-frequency transformation features and labels of the spectral blocks according to the causal factor reconstruction method in each training epoch. To search for a domain-independent DAG structure. Specifically, assume that s is a vector consisting of a feature vector and a label, that is, s = [z l , y], this application is based on the entire factor set Infer any factor s i , first use the row vector A i Shield non-parent node elements and generate function g through data i Mapping the parent node to the i-th factor, the generating function for causal factor reconstruction is

[0048]

[0049] in, is a learnable weight matrix that maps parent nodes in the DAG to categorical logical values, A i is the indicator matrix that assigns parent nodes to child nodes.

[0050] However, in order to obtain the domain-independent DAG structure in the source domain hyperspectral data, it is necessary to traverse the entire training data to extract the latent factors of each sample for optimization, which leads to high computational cost. The low-frequency transformation features of the source domain spectral block are compressed into K representative class-conditional prototype vectors. The class-conditional prototypes are updated by momentum, which can be formalized as

[0051]

[0052] Among them, θ∈(0,1) is the momentum parameter for controlling the prototype update, It is the low-frequency transformation feature of the spectral block corresponding to the category label.

[0053] Special, A⊙A It represents the total causal effect between feature elements, including direct causal effects and effects indirectly transmitted through other variables. According to the positivity of the (i, j) element of A to the nth power, it indicates that there is a path s of length n. i →…→s j The idea is derived, that is,

[0054]

[0055] Among them, [e A⊙A ] i，j Analogous to total causal effect. A⊙A ] t+1，1：t The t+1 rows in correspond to the target variables, and the 1:t columns correspond to the set of feature elements, [e A⊙A ] t+1，1：t represents the total causal effect of each feature element on the target variable y, Represents the low-frequency transformation characteristics of the spectral block The invariant features of y contain all direct and indirect causal features about the target label y.

[0056] Then, the causal factor vector represented by the class condition prototype is reconstructed to realize the search of DAG, which can be expressed as

[0057]

[0058] Where L cos (x1, x2): = 1-cos(x1, x2) is the cosine embedding loss, which is used to measure the similarity between the reconstructed data and the original data. ce It is the cross entropy loss function, which is used to measure the correspondence between the reconstructed data and the label.

[0059] Secondly, in order to ensure that the directed graph is acyclic to reduce the impact of false factors on the model, DSPLTnet minimizes the trace of the adjacency matrix index to ensure the acyclicity of the graph, thereby eliminating the influence of false factors that easily make the network fit, which can be formalized as

[0060] Tr(e A⊙A )-(t+1)=0 (12)

[0061] The constraint of formula (10) restricts the nodes in the directed graph from returning to themselves even after infinite steps.

[0062] Then the ultimate learning goal of DAG is

[0063]

[0064] stTr(e A☉A )-(t+1)=0

[0065] where vec(·) is a vectorized operation on the matrix and |vec(A)|1 is a constraint term used to force the DAG to be sparse.

[0066] Step 108, constructing a total loss function according to the preset prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss and prototype contrastive learning loss; using the total loss function to train the cross-scene hyperspectral image classification model, and classifying the cross-scene hyperspectral images to be classified according to the trained cross-scene hyperspectral image classification model.

[0067] In order to learn a domain-independent DAG from a class-conditional prototype, it is crucial to ensure the representativeness of the class-conditional prototype. This application designs a progressive contrastive learning framework to obtain a more representative class-conditional prototype. First, this application uses supervised contrastive learning to enable the spectral block low-frequency transformation network to initially extract domain-invariant features, so that the class-conditional prototype constructed by the features obtained by the spectral block low-frequency transformation network contains domain-invariant feature elements, thereby reducing the difficulty of searching for domain-independent DAGs therefrom. Specifically, for a batch of spectral block embedding features Frequency domain filtering branch f ori Extract the original frequency domain features of each spectral block, and the low-frequency transform branch f LPT Extract the low-frequency transformation features of the low-frequency transformed spectral block, and learn the class domain invariant representation from the original frequency domain features and low-frequency transformation features through supervised ratio learning, which can be formalized as

[0068]

[0069] in, is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and Represent the positive sample index set and the negative sample index set respectively. The features obtained from the original frequency domain features and low-frequency transformation features that belong to the same spectral block and the features that belong to the same category are regarded as elements in the positive sample set. The spectral block low-frequency transformation network can extract shared representations from the original frequency domain features and low-frequency transformation features belonging to the same spectral block, and can make the representation distance of spectral blocks belonging to the same category closer and the representation distance of spectral blocks of different categories farther.

[0070] Then, prototype contrast learning is used to align the invariant features of each spectral block with the prototype of the category to which it belongs in the embedding space, making the class conditional prototype constructed by the invariant features of each sample more representative. This process can be expressed as

[0071]

[0072] in, is the positive sample prototype set, and the positive sample is defined as The class condition prototype vector corresponding to the category, p′ represents the invariant feature The conditional prototype vectors of other classes whose categories do not correspond.

[0073] The optimization objectives of the cross-scene hyperspectral image classification model include four parts in total, namely, prediction consistency loss DAG construction loss Supervised Contrastive Learning Loss Compared with the prototype loss where the prediction consistency loss is defined as

[0074]

[0075] in, is the weight of the classifier. The prediction consistency loss is used to enable the model to obtain the same prediction results for the original frequency domain features and low-frequency transformation features obtained from the same spectral block, prompting the model to make predictions based on the domain invariant features in the sample.

[0076] The overall optimization goal can be formalized as

[0077]

[0078] Among them, θ is the spectral block low-frequency transformation network, g i and the set of classifier parameters, λ DAG and λ CL are the hyperparameters used to balance the DAG reconstruction loss and progressive contrastive learning, respectively.

[0079] When the trained cross-scene hyperspectral image classification model is used for image classification, the hyperspectral image to be classified is input into the model, and the corresponding output image category can be obtained.

[0080] In the above-mentioned cross-scene hyperspectral image classification method, the present application constructs a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module. By setting a domain-independent spectral block low-frequency transformation network, unlike previous methods, the spectral block low-frequency transformation network combines frequency domain filtering branches and low-frequency transformation branches to make the spectral block low-frequency transformation network pay more attention to domain-invariant features. Since the low-frequency component of the spectral block contains more domain-specific ground object spectral reflectance information, the combined frequency domain filtering branch and the low-frequency transformation branch can effectively reduce the influence of domain-specific features, focus on mining domain-invariant features from the data, and achieve generalization of unseen domains by pruning unstable domain-specific features and easily fitted false features. At the same time, by constructing a domain-independent directed acyclic graph, the model can identify domain-invariant features through causal structure learning, and prune domain-specific features and false features that are not conducive to model generalization. At the same time, in order to learn a domain-independent directed acyclic graph, a progressive contrastive learning framework is designed. Through progressive supervised contrastive learning and prototype contrastive learning optimization models, a more representative class conditional prototype is obtained to search for domain-independent directed acyclic graphs. Finally, the classifier extracts domain-independent features in the target domain hyperspectral data based on the joint spectral block low-frequency transform network and the domain-independent directed acyclic graph learned from the source domain hyperspectral data, realizing cross-scene hyperspectral image classification.

[0081] In one embodiment, the frequency domain filtering branch includes a normalization layer, a fast Fourier transform layer, a learnable filter layer, an inverse fast Fourier transform layer and a linear layer; learning the original spectrum block features in the frequency domain space of the frequency domain filtering branch includes:

[0082] After the original spectral block passes through the normalization layer, the fast Fourier transform layer is used to transform the embedded features of the original spectral block into the frequency domain space, and then the frequency domain features of the spectral block are learned in the frequency domain space through the learnable filter layer. The frequency domain features are then converted to the spatial domain through the inverse fast Fourier transform layer, and finally output through the linear layer to obtain the original spectral block features.

[0083] In one embodiment, the embedded features of the original spectrum block are transformed into the frequency domain space using a fast Fourier transform layer, including:

[0084] The process of transforming the embedded features of the original spectral block into the frequency domain space using the fast Fourier transform layer is as follows:

[0085]

[0086] in, represents the original spectral block feature, where H, W and D are the height, width and number of channels of the embedded feature of the original spectral block feature, respectively, and i 2 =-1, m and n represent spatial frequencies.

[0087] In one embodiment, the low-frequency transformation branch performs low-frequency perturbation on the original spectrum block through the spectrum block low-frequency transformation layer and extracts the features of the spectrum block after the low-frequency perturbation, including:

[0088] The embedded features of the original spectral block are transformed into the frequency domain space using the fast Fourier transform layer to obtain the frequency domain features;

[0089] In the low-frequency transformation layer of the spectral block, the frequency domain features are binary masked to obtain the low-frequency components; the distribution of each element in the low-frequency components of different samples in a batch is modeled as a multivariate Gaussian distribution; the values ​​of the low-frequency elements in the frequency domain features of each spectral block are resampled from the multivariate Gaussian distribution; the original high-frequency components are combined with the resampled low-frequency components to obtain the spectral block features that have undergone low-frequency perturbations in the frequency domain space.

[0090] In one embodiment, each element distribution in the low frequency component of different samples in a batch is modeled as a multivariate Gaussian distribution

[0091]

[0092] in, represents the original spectral block feature, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block feature, respectively.

[0093] In one embodiment, the original high-frequency component is combined with the resampled low-frequency component to obtain a spectrum block feature after low-frequency perturbation in the frequency domain space:

[0094]

[0095] Among them, β∈[0, 1] is the disturbance parameter used to control the disturbance intensity.

[0096] In one embodiment, setting a generation function of a causal factor includes:

[0097] Set the generating function of the causal factor to

[0098]

[0099] Where s = [z l , y] represents the characteristic element z l and class label y, is a learnable weight matrix that maps parent nodes in a directed acyclic graph to categorical logical values, A iIt is the indicator matrix that assigns parent nodes to child nodes, that is, the adjacency matrix of the directed acyclic graph, and i represents the sequence number of the causal factor.

[0100] In one embodiment, domain-independent features are compressed into multiple class-conditional prototype vectors and then causal factors are reconstructed, including:

[0101] The process of compressing domain-independent features into multiple class-conditional prototype vectors and reconstructing causal factors is expressed as

[0102]

[0103] Among them, L cos (x1, x2): = 1-cos(x1, x2) is the cosine embedding loss, L ce is the cross entropy loss function, P i is a class conditional prototype, e A⊙A Represents the total causal effect between characteristic elements, including direct causal effects and effects indirectly transmitted through other variables, Represents the adjacency matrix of a directed acyclic graph.

[0104] In one embodiment, a total loss function is constructed based on preset prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss, including:

[0105] The total loss function is constructed based on the pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss and prototype contrastive learning loss:

[0106]

[0107]

[0108] stTr(e A☉A )-(t+1)=0

[0109]

[0110] in, represents the prediction consistency loss, represents the domain-independent directed acyclic graph learning loss, represents the supervised contrastive learning loss, represents the prototype contrastive learning loss, is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and Represent the positive sample index set and the negative sample index set respectively, is the positive sample prototype set, and the positive sample is defined as The class condition prototype vector corresponding to the category, p′ represents the invariant feature The conditional prototype vectors of other classes that do not correspond to the category, θ is the spectral block low-frequency transformation network, g i is the parameter set of the classifier, λ DAG and λ CL are the hyperparameters used to balance domain-independent DAG learning loss and progressive contrastive learning, is the weight of the classifier, vec(·) is the vectorization operation of the matrix, |vec(A)|1 is a constraint term used to force the directed acyclic graph to tend to sparsity, and e A⊙A Represents the total causal effect between characteristic elements, including direct causal effects and effects indirectly transmitted through other variables, represents the adjacency matrix of a directed acyclic graph, Represents the original spectral block features.

[0111] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0112] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cross-scene hyperspectral image classification method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0113] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0115] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A cross-scene hyperspectral image classification method, characterized in that: The method comprises: Acquire a source domain hyperspectral dataset; the source domain hyperspectral dataset includes a plurality of source domain hyperspectral images; construct a cross-scene hyperspectral image classification model; the cross-scene hyperspectral image classification model includes a spectral block low-frequency transformation network, a supervised contrast learning module, a domain-independent directed acyclic graph learning module and a prototype contrast learning module; The source domain hyperspectral image is sampled to obtain an original spectral block; the low-frequency transformation branch in the spectral block low-frequency transformation network performs low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extracts the spectral block features after the low-frequency perturbation, and learns the original spectral block features in the frequency domain space of the frequency domain filtering branch, and the spectral block low-frequency transformation network learns domain-independent features by maintaining the consistency of prediction for the spectral block features after the low-frequency perturbation and the original spectral block features; In the domain-independent directed acyclic graph learning module, modeling is performed according to the domain-independent features and corresponding category labels to obtain a structural causal model; the structural causal model is represented as a directed acyclic graph, and the adjacency matrix of the directed acyclic graph is learned from the domain-independent features and category labels according to the causal factor reconstruction method to set the generating function of the causal factor; the domain-independent features are compressed into multiple class conditional prototype vectors and then the causal factors are reconstructed, and the reconstructed domain-independent features and category labels are generated using the reconstructed causal factors; A total loss function is constructed according to the preset prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss and prototype contrastive learning loss; the cross-scene hyperspectral image classification model is trained using the total loss function, and the cross-scene hyperspectral images to be classified are classified according to the trained cross-scene hyperspectral image classification model.

2. The method according to claim 1, characterized in that: The frequency domain filtering branch includes a normalization layer, a fast Fourier transform layer, a learnable filter layer, an inverse fast Fourier transform layer and a linear layer; The original spectral block features are learned in the frequency domain space of the frequency domain filtering branch, including: After the original spectral block passes through the normalization layer, the fast Fourier transform layer is used to transform the embedded features of the original spectral block into the frequency domain space, and then the frequency domain features of the spectral block are learned in the frequency domain space through the learnable filter layer. The frequency domain features are then converted to the spatial domain through the inverse fast Fourier transform layer, and finally output through the linear layer to obtain the original spectral block features.

3. The method according to claim 1, characterized in that The embedded features of the original spectral block are transformed into the frequency domain space using the Fast Fourier Transform layer, including: The process of transforming the embedded features of the original spectral block into the frequency domain space using the fast Fourier transform layer is as follows: in, represents the original spectral block feature, where H, W and D are the height, width and number of channels of the embedded feature of the original spectral block feature, respectively, and i 2 =-1, m and n represent spatial frequencies.

4. The method according to claim 1, characterized in that: The low-frequency transformation branch performs low-frequency perturbation on the original spectral block through the spectral block low-frequency transformation layer and extracts the features of the spectral block after low-frequency perturbation, including: The embedded features of the original spectral block are transformed into the frequency domain space using the fast Fourier transform layer to obtain the frequency domain features; The frequency domain features are binary-masked at the spectral block low-frequency transformation layer to obtain low-frequency components; the distribution of each element in the low-frequency components of different samples in a batch is modeled as a multivariate Gaussian distribution; the values ​​of the low-frequency elements in the frequency domain features of each spectral block are resampled from the multivariate Gaussian distribution; the original high-frequency components are combined with the resampled low-frequency components to obtain the spectral block features that have undergone low-frequency perturbations in the frequency domain space.

5. The method according to claim 4, characterized in that Model each element distribution in the low-frequency component of different samples in a batch as a multivariate Gaussian distribution in, represents the original spectral block feature, where H, W, and D are the height, width, and number of channels of the embedded features of the original spectral block feature, respectively.

6. The method according to claim 4, characterized in that The original high-frequency component is combined with the resampled low-frequency component to obtain the spectral block features after low-frequency perturbation in the frequency domain space, including: The original high-frequency component is combined with the resampled low-frequency component to obtain the spectral block characteristics after low-frequency perturbation in the frequency domain space: Among them, β∈[0, 1] is the disturbance parameter used to control the disturbance intensity.

7. The method according to claim 1, characterized in that Set the generating function of the causal factor, including: Set the generating function of the causal factor to Where s = [z l , y] represents the characteristic element z l and class label y, is a learnable weight matrix that maps parent nodes in a directed acyclic graph to categorical logical values, A i It is the indicator matrix that assigns parent nodes to child nodes, that is, the adjacency matrix of the directed acyclic graph, and i represents the sequence number of the causal factor.

8. The method according to claim 1, characterized in that The domain-independent features are compressed into multiple class-conditional prototype vectors and then the causal factors are reconstructed, including: The process of compressing the domain-independent features into multiple class-conditional prototype vectors and then reconstructing the causal factor is expressed as Among them, L cos (x1, x2): = 1-cos(x1, x2) is the cosine embedding loss, L ce is the cross entropy loss function, p i is a class conditional prototype, e A⊙A Represents the total causal effect between characteristic elements, including direct causal effects and effects indirectly transmitted through other variables, Represents the adjacency matrix of a directed acyclic graph.

9. The method according to claim 8, characterized in that The total loss function is constructed based on the pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss, and prototype contrastive learning loss, including: The total loss function is constructed based on the pre-set prediction consistency loss, supervised contrastive learning loss, domain-independent directed acyclic graph learning loss and prototype contrastive learning loss: stTr(e A☉A )-(t+1)=0 in, represents the prediction consistency loss, represents the domain-independent directed acyclic graph learning loss, represents the supervised contrastive learning loss, represents the prototype contrastive learning loss, is a scalar temperature parameter, |ε(i)| is the number of positive samples, and ε(i) and Represent the positive sample index set and the negative sample index set respectively, is the positive sample prototype set, and the positive sample is defined as The class condition prototype vector corresponding to the category, p′ represents the invariant feature The conditional prototype vectors of other classes that do not correspond to the category, θ is the spectral block low-frequency transformation network, g i is the parameter set of the classifier, λ DAG and λ CL are the hyperparameters used to balance domain-independent DAG learning loss and progressive contrastive learning, is the weight of the classifier, vec(·) is the vectorization operation of the matrix, |vec(A)|1 is the constraint term used to force the directed acyclic graph to tend to sparsity, and e A⊙A Represents the total causal effect between characteristic elements, including direct causal effects and effects indirectly transmitted through other variables, represents the adjacency matrix of a directed acyclic graph, Represents the original spectral block features.

Images