Hyperspectral image domain generalization classification method based on dynamic enhancement and double constraints

Abstract

The invention discloses a hyperspectral image domain generalization classification method based on dynamic enhancement and double constraints, and aims to solve the problem of inter-domain difference in cross-scene hyperspectral image classification. According to the method, diversified samples are generated through a dynamic style enhancement module, meanwhile, semantic factors are used for controlling style randomization, and it is ensured that key semantic information of a source domain is reserved while diversity of the generated samples is kept; an auxiliary domain synthesis strategy guided by a category feature manifold is introduced, an auxiliary sample conforming to actual category distribution is generated, and the generalization ability of the model is further improved; in a discriminator, a dual strategy of confrontation consistency constraint and structure consistency constraint is adopted, the discrimination capability is enhanced by supervising comparative confrontation learning, intra-class compactness and inter-class separability are ensured, a potential structure of a generated sample is constrained, and generation of an ambiguous sample is avoided. The method shows excellent classification performance on cross-time-phase, cross-region and cross-sensor data sets, is suitable for the fields of ecological environment monitoring, ground coverage investigation and the like, and has wide application prospects.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image processing technology, and in particular to a hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints. Background Art

[0002] Hyperspectral imaging technology can capture hundreds of consecutive spectral bands, providing rich and detailed spectral information, and is widely used in fields such as ecological environment monitoring and land cover surveys. With the increasing ability to acquire hyperspectral data, the need for cross-scene classification is becoming increasingly urgent. However, due to the distribution differences between the source and target domains, existing classification models often perform poorly in unseen target domains. Domain generalization, as an important cross-scene classification technique, aims to train models using only source domain data and directly generalize to other target domains, avoiding dependence on target domain data.

[0003] Currently, most mainstream domain generalization methods use uncontrolled data generation methods to expand diverse samples, resulting in the loss of key semantic information and the generation of data that may contain interfering information. Summary of the Invention

[0004] To address the problems of existing technologies, this invention proposes a hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints. The method generates diverse samples through a dynamic enhancement module while preserving key semantic information, and optimizes the learning of domain-invariant knowledge through dual constraints. Specifically, it includes the following steps:

[0005] Two hyperspectral images are read and used as source domain data and target domain data, respectively;

[0006] The network model is designed by first establishing a generator, which includes a dynamic enhancement module. This module performs linear mapping for dimensionality reduction and feature map extraction on the source domain data, calculates feature statistics to obtain semantic style-enhanced samples, and finally outputs generated domain samples. The generator employs an auxiliary domain synthesis strategy guided by category feature manifolds to generate auxiliary domain samples Z. a Then, a dual-constraint discriminator is established, which includes a feature extractor f, a classification head C, and a feature projection P.

[0007] Calculate the total loss of the generator and the dual-constraint discriminator, and iteratively train the generator and the dual-constraint discriminator through backpropagation to make them continuously compete with each other until they reach a balance.

[0008] Classification is performed using the test dataset. During the testing phase, the target domain image input is processed by a feature extractor f trained on the source domain to extract neighborhood-invariant features, and the classification head directly outputs the classification result.

[0009] Furthermore, let the source domain data be... The corresponding category label is Where i represents the data index, x i Representing sample data, y i The dynamic enhancement module performs linear mapping dimensionality reduction on a certain size of source domain data, and performs 3D spatial-spectral coding on the mapped features to extract local features, obtaining a feature map. Based on the feature map category information y... sem Semantic factor β is extracted through a fully connected layer F. sem =F(y sem ), randomly select 2 instances x and y from a batch Dynamic style randomization is achieved by combining semantic factors and style transfer. Semantic style-enhanced samples are obtained by calculating feature statistics σ(x) and μ(x). Generative domain samples are obtained by performing 3D spatial-spectral decoding and linear deconvolution operations on the semantic style-enhanced samples.

[0010] Furthermore, the dynamic enhancement module employs an auxiliary domain synthesis strategy guided by category feature manifolds to generate auxiliary domain samples Z. a Time: Assume source domain sample Z s and generating domain samples Z g The feature representation of category l is denoted as l. s and l g Construct a joint feature set T l =[l s ,l g ], through the weighted similarity kernel function M l (T l ) Calculate the category feature manifold to capture the manifold structure of samples within a category, and obtain l by locally uniform downsampling the feature set. ε Define the transition weight ω l Auxiliary domain samples of category l are generated by adaptive weighting based on the category feature manifold. Finally, the auxiliary domain sample Z is synthesized. a .

[0011] Furthermore, the joint samples of source domain data, generated domain samples, and auxiliary domain samples are input into the feature extractor f to extract deep spatial-spectral features; classification prediction is then performed on the three samples, and the classification loss is calculated using cross-entropy.

[0012]

[0013] where X g To generate domain data, X aFor auxiliary domain data, p represents the predicted probability, and N represents the number of samples in the source domain. Feature projection p maps the output feature vector to the feature space for dual-constraint training. Supervised contrastive adversarial learning is used to overcome the consistency constraint, constructing positive and negative sample pairs, and the inter-class contrastive loss L... stru Intra-class contrast loss L adv Comparison of loss L SCL Construct structural consistency constraints L stru .

[0014] Furthermore, the total loss L of the generator G =L GD +λ2L adv +λ3L stru The total loss of the dual-constraint discriminator is L. D =L SD +L GD +L AD +λ2L adv +λ1L SCL λ1, λ2, and λ3 are hyperparameters used to balance the contrastive learning and structural consistency constraint loss. The generator and the dual-constraint discriminator are iteratively trained through backpropagation to continuously challenge each other until they reach a balance.

[0015] Furthermore, the semantic style enhancement samples are generated as follows:

[0016]

[0017] Where γ sem and λ sem These are two instances x and x randomly selected from a batch. The characteristic statistics are specifically defined as follows:

[0018]

[0019] Where, β sem For semantic factors, a random mixing ratio is generated for different samples, and μ(x) and σ(x) are the mean and standard deviation of the instance feature cube.

[0020] Furthermore, the auxiliary domain samples of category l The specific calculation formula is as follows:

[0021]

[0022] Z s Z represents the source domain sample. g Represents the generated domain sample, ω l This indicates overweighting, specifically defined as:

[0023]

[0024] Where l represents the category, l s and l g The feature label representing category l is calculated as follows:

[0025]

[0026] The samples obtained from each category are finally synthesized into an auxiliary domain sample Z. a .

[0027] By employing the aforementioned technical solutions, this invention provides a hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints. This method, based on a symmetric encoder-decoder structure, introduces an adjustable style randomization mechanism in the space-spectral encoding stage, solving the problem of key semantic information loss during data generation and generating more diverse samples. To further enhance the model's learning ability, this method synthesizes reliable auxiliary domain samples by capturing the manifold structure of category features, thus providing richer training data. Furthermore, in the supervised contrastive adversarial training of the discriminator, this invention introduces structural consistency constraints, solving the intra-class consistency and structural consistency problems between source and generated domain samples, thereby effectively improving the model's generalization ability. This method can serve as an effective means for cross-scene classification of hyperspectral images, possessing significant application value in complex scene classification tasks across regions, time phases, and sensors. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A detailed technical flowchart of a hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints provided by the present invention;

[0030] Figures 2a-2d The images in this invention are true-color composite images and ground reference information images from the Houston dataset. Figures 2a-2b Images of the Houston 2013 (Hou-13) scene. Figures 2c-2d Images of the Houston 2018 (Hou-18) scene;

[0031] Figure 3 The image shows the classification result for the target domain "Houston 2018" in the Houston dataset.

[0032] Figure 4 This is a schematic diagram of the dynamic style enhancement process in this invention;

[0033] Figure 5 This is a schematic diagram of the supervised contrastive adversarial training process with dual constraints in this invention;

[0034] Figures 6a-6d The Pavia dataset in this invention consists of true-color composite images and ground reference information images, wherein... Figures 6a-6b Image of a scene from the University of Pavia (PaviaU). Figures 6c-6d Image of the Pavia Center (PaviaC) scene;

[0035] Figure 7 The image shows the classification result for the target domain PaviaC in the Pavia dataset;

[0036] Figures 8a-8d The images in this invention are true-color composite images and ground reference information images from the Yancheng dataset. Figures 8a-8b For GF_YC scene images, Figures 8c-8d The scene image is ZY_YC;

[0037] Figure 9 This is the classification result image for the target domain ZY_YC in the Yancheng dataset. Detailed Implementation

[0038] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:

[0039] like Figure 1 The hyperspectral image domain generalization classification method shown includes the following steps:

[0040] Assuming source domain data is defined Corresponding category tags Where d represents the dimension of the data and N represents the number of samples in the source domain.

[0041] Step 1: Dynamic style enhancement to generate domain samples;

[0042] Specifically, the hyperspectral image of the source domain is input into the dynamic style enhancement module, assuming the source domain image is X. S ,in H and W represent the height and width of the image, respectively, and C is the number of spectral bands. Next, dimensionality reduction is performed by applying a 1×1 linear mapping to the input hyperspectral image to preserve its low-dimensional structure and semantic information. The feature representation after dimensionality reduction is as follows: Where D represents the feature dimension after dimensionality reduction. Local features at the spatial and spectral levels are extracted using 3D convolutional layers to obtain the feature map. Based on feature map category information y sem The semantic factor β, containing semantic information, is extracted through a fully connected layer-max pooling-fully connected-max pooling structure F(·). sem The calculation method is as follows:

[0043] β sem =F(y sem (1)

[0044] Randomly select 2 instances x and And calculate the characteristic statistics:

[0045]

[0046] The mean and standard deviation are calculated as follows:

[0047]

[0048] Semantic style enhancement samples are obtained using the following formula:

[0049]

[0050] After dynamic style randomization, the feature samples are restored to the input dimension by 3D spatial-spectral decoding and linear deconvolution operations, and the final generated domain samples are output.

[0051] Step 2: Guided by the category feature manifold, synthesize auxiliary domain samples;

[0052] Specifically, first, the popular structure of category features is calculated, assuming the source domain sample Z... s and generating domain samples Z g The feature representation of category l is denoted as l. s and l g Construct a joint feature set T l =[l s ,l g The category feature manifold M is calculated using a weighted similarity kernel function. l (T l The calculation formula is as follows:

[0053]

[0054] Where σ represents the bandwidth parameter of the Gaussian kernel.

[0055] Local uniform downsampling of the feature set yields l ε For synthesis, based on the category feature manifold, auxiliary domain samples of category l are generated through adaptive weighting. The specific calculation formula is as follows:

[0056]

[0057] Where, ω l The transition weight is represented by the following formula:

[0058]

[0059] Finally, the samples obtained from each category are combined into an auxiliary domain sample Z. a .

[0060] Step 3: Perform generative adversarial training based on a discriminator with dual constraints;

[0061] In inter-class contrastive learning, training is completed by constructing positive and negative sample pairs. Features belonging to the same class in the source domain, generator domain, and auxiliary domain are used as positive samples, and features belonging to different classes are used as negative samples. The formula for inter-class contrastive loss is as follows:

[0062]

[0063] In intra-class contrastive learning, for the same class, samples of that class from the source domain are used as positive samples, and samples of that class from the generator domain or auxiliary domain are randomly selected as negative samples. The intra-class contrastive loss formula is as follows:

[0064]

[0065] The underlying structure of the generated samples is constrained to avoid sample distortion or the introduction of interfering information. The loss formula for structural consistency constraints is as follows:

[0066]

[0067] Where f(·) represents the extracted latent structural features, and ||·|| is used to measure the structural difference between the source domain and the generator domain.

[0068] Step 4: The trained feature extractor;

[0069] Specifically, the feature extractor consists of convolutional layers, pooling layers, and fully connected layers. During training, the feature extractor learns domain-invariant feature representations to reduce the distribution differences between the source and target domains. Using the trained feature extractor, hyperspectral image features are extracted from both the source and target domains, where the extracted feature representations are... This makes it easier to directly migrate to the target domain to complete the classification.

[0070] Step 5: Target domain classification;

[0071] X t Input the trained feature extractor to extract features The extracted features are classified using a classification head, which outputs a classification result map of the target domain. The classification head consists of a fully connected layer and a softmax function, outputting the class probability distribution for each pixel.

[0072]

[0073] Among them, w k and b k The weights and biases of the classification heads are K, where K is the number of categories.

[0074] Real hyperspectral data experiment

[0075] The following section will use three sets of publicly available cross-scene hyperspectral image datasets with different differences to analyze and evaluate the application effect of the hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints provided by this invention.

[0076] 1. Dataset and parameter settings

[0077] (1) Houston dataset

[0078] The first set of cross-temporal Houston datasets, including the Houston 2013 (Hou-13) and Houston 2018 (Hou-18) scenes, was collected by the National Airborne Laser Mapping Center in different years in the University of Houston and surrounding areas. The Hou-13 data has a spatial resolution of 2.5m, a size of 349×1905 pixels, contains 144 spectral channels, and a total of 2530 samples. The Hou-18 data has a spatial resolution of 1m, contains 48 bands, and a total of 53200 samples. Both datasets have a wavelength range of 0.38–1.05 μm. In the experiment, a 209×955 overlapping region was selected, seven common categories were chosen, and 48 bands were extracted from Hou-13 to match Hou-18. (See Figure 2).

[0079] (2) Pavia dataset

[0080] The second set is the cross-regional Pavia dataset, including two scenes: the University of Pavia (PaviaU) and the Pavia Center (PaviaC), both captured by the Reflection Optical Systems Imaging Spectroradiometer (ROSIS) in Pavia, northern Italy. The spatial resolution is 1.3 m, and the wavelength range is 0.43–0.86 μm. The PaviaU dataset is 610 × 340 pixels in size, containing 103 spectral channels and 39,332 samples. The PaviaC dataset contains 102 channels, is 1096 × 715 pixels in size, and contains 39,355 samples. To ensure the same number of bands, the last band of the PaviaU dataset was removed, and seven common categories were selected. (See Figure 6.)

[0081] (3) Yancheng dataset

[0082] The third group is the cross-sensor Yancheng dataset (Sun et al., 2023), including two datasets, GF_YC and ZY_YC, acquired by AHSI from the Gaofen-5 (GF-5) and Ziyuan-1 02D (ZY1-02D) satellites, respectively. The image sizes are 1175×585 and 1398×942, containing 5475 and 8219 samples, respectively. In the experiment, 147 bands and 7 common categories were selected. (See Figure 8.)

[0083] 2. Experimental Evaluation Indicators

[0084] (1) Overall Accuracy (OA)

[0085] The definition of overall accuracy (OA) is as follows:

[0086]

[0087] Where Y i N represents the number of pixels in the classification result that are labeled as class i and also belong to class i in the ground reference image; i This represents the total number of samples of class i in the ground reference image.

[0088] (2) Class-dependent Accuracy (CA)

[0089] The definition of Category Precision (CA) is as follows:

[0090] CA = Y i / N i (2)

[0091] (3) Kappa coefficient (κstatistic)

[0092] The Kappa coefficient is a method for quantitatively evaluating the consistency between a remote sensing classification image and a ground reference image. It reflects the overall classification error of the image and is defined as follows:

[0093]

[0094] in This represents the number of pixels that should belong to class i but were misclassified as other classes in the classification results; This represents the number of pixels that should belong to other categories but were misclassified as the i-th category in the classification results.

[0095] 3. Analysis and Evaluation of Experimental Results

[0096] This invention provides a hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints (hereinafter referred to as "DSEDC"). 2 The results of the experiment using three sets of real hyperspectral image data are shown in Table 1-3, and the corresponding classification result images are attached. Figure 3 , 7 As shown in Figure 9.

[0097] This experiment employs a domain generalization method to achieve cross-scene classification. A dynamic style enhancement module is designed, and an auxiliary domain synthesis strategy guided by category feature manifolds is introduced. A domain-invariant discriminator with double consistency constraints is also designed. Based on the classification results, the following conclusions can be drawn:

[0098] On the Houston 18 and PaviaC datasets, compared to domain adaptation methods (such as TSTnet), single-source domain generalization methods (such as DSEDC) show superior performance. 2 The better performance of the -net method indicates that the single-source domain generalization method can still achieve excellent classification performance without accessing the target domain data.

[0099] On the target domain ZY_YC dataset, the domain adaptation method TSTnet outperforms the single-source domain generalization methods PDEN and LLURnet, indicating that domain adaptation methods and single-source domain generalization methods each have their advantages in different scenarios.

[0100] Compared to other single-source domain generalization methods, the hyperspectral image domain generalization classification method with dynamic enhancement and dual constraints provided in this invention achieves higher overall accuracy, indicating that its dynamic enhancement module and dual-constraint training can effectively extract neighborhood-invariant representations, improve classification accuracy, and effectively solve the neighborhood offset problem between the source and target domains.

[0101] Table 1. Classification results of the Hou-18 dataset (80% selected as training samples)

[0102]

[0103]

[0104] Table 2. Classification results of the PaviaC dataset (50% selected as training samples)

[0105]

[0106] Table 3. Classification results of the ZY_YC dataset (80% selected as training samples)

[0107]

[0108] This invention successfully addresses the inter-domain discrepancy problem in cross-scene hyperspectral image classification through dynamic enhancement and dual-constraint design, significantly improving the model's adaptability to unknown scenarios. Experimental results demonstrate that this method exhibits excellent classification performance on multiple public datasets across time, regions, and sensors, maintaining high classification accuracy even when target domain data is scarce or has significant distribution differences, fully validating its robustness and generalization ability across different scenarios. The dynamic style enhancement module, by introducing semantic factors, achieves a balance between style diversity and semantic preservation, effectively avoiding the semantic information loss problem caused by excessive randomization in traditional methods, ensuring the diversity and semantic consistency of generated samples. Furthermore, the auxiliary domain synthesis strategy guided by category feature manifolds generates more representative training samples by capturing the latent distribution structure of category features, further enhancing the model's adaptability to complex scenarios. The dual consistency constraint, through the synergistic effect of adversarial consistency and structural consistency, not only ensures the intra-class compactness and inter-class separability of generated samples but also avoids the generation of ambiguous samples, thereby improving the model's classification accuracy and stability. In summary, this invention has not only verified its effectiveness on multiple datasets, but also provides an efficient and reliable solution for cross-scene hyperspectral image classification, with broad application prospects, especially in fields such as ecological environment monitoring and land cover surveys, where it can significantly improve the efficiency and accuracy of classification tasks.

[0109] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints, characterized in that... include: Two hyperspectral images are read and used as source domain data and target domain data, respectively; The network model is designed by first establishing a generator, which includes a dynamic enhancement module. This module performs linear mapping for dimensionality reduction and feature map extraction on the source domain data, calculates feature statistics to obtain semantic style-enhanced samples, and finally outputs generated domain samples. The generator employs an auxiliary domain synthesis strategy guided by category feature manifolds to generate auxiliary domain samples Z. a ; A dual-constraint discriminator is then established, which includes a feature extractor f, a classification head C, and a feature projection P; Calculate the total loss of the generator and the dual-constraint discriminator, and iteratively train the generator and the dual-constraint discriminator through backpropagation to make them continuously compete with each other until they reach a balance. Classification is performed using the test dataset. During the testing phase, the target domain image input is processed by a feature extractor f trained on the source domain to extract neighborhood-invariant features, and the classification head directly outputs the classification result.

2. The hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints according to claim 1, characterized in that: Let the source domain data be... The corresponding category label is Where i represents the data index, x i Representing sample data, y i The dynamic enhancement module performs linear mapping dimensionality reduction on a certain size of source domain data, and performs 3D spatial-spectral coding on the mapped features to extract local features, obtaining a feature map. Based on the feature map category information y... sem Semantic factor β is extracted through the fully connected layer F. sem =F(y sem ), randomly select 2 instances x and y from a batch Dynamic style randomization is achieved by combining semantic factors and style transfer. Semantic style-enhanced samples are obtained by calculating feature statistics σ(x) and μ(x). Generative domain samples are obtained by performing 3D spatial-spectral decoding and linear deconvolution operations on the semantic style-enhanced samples.

3. The hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints according to claim 1, characterized in that: The dynamic enhancement module uses a category feature manifold-guided auxiliary domain synthesis strategy to generate auxiliary domain samples Z. a Time: Assume source domain sample Z s and generating domain samples Z g The feature representation of category l is denoted as l. s and l g Construct a joint feature set T l =[l s ,l g ], through the weighted similarity kernel function M l (T l ) Calculate the category feature manifold to capture the manifold structure of samples within a category, and obtain l by locally uniform downsampling the feature set. ε Define the transition weight ω l Based on the category feature manifold, auxiliary domain samples of category l are generated through adaptive weighting. Finally, the auxiliary domain sample Z is synthesized. a .

4. The hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints according to claim 1, characterized in that: The joint sample of source domain data, generated domain samples, and auxiliary domain samples is input into the feature extractor f to extract deep spatial-spectral features; classification prediction is performed on the three samples, and the classification loss is calculated using cross-entropy. Where X g To generate domain data, X a For auxiliary domain data, p represents the predicted probability, and N represents the number of samples in the source domain. Feature projection p maps the output feature vector to the feature space for dual-constraint training. Supervised contrastive adversarial learning is used to overcome the consistency constraint, constructing positive and negative sample pairs, and the inter-class contrastive loss L... stru Intra-class contrast loss L adv Comparison of loss L SCL Construct structural consistency constraints L stru .

5. The hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints according to claim 4, characterized in that: The total loss L of the generator G =L GD +λ2L adv +λ3L stru The total loss of the dual-constraint discriminator is L. D =L SD +L GD +L AD +λ2L adv +λ1L SCL λ1, λ2, and λ3 are hyperparameters used to balance the contrastive learning and structural consistency constraint loss. The generator and the dual-constraint discriminator are iteratively trained through backpropagation to continuously challenge each other until they reach a balance.

6. The hyperspectral image domain generalization classification method based on dynamic enhancement and dual constraints according to claim 2, characterized in that: The semantic style enhancement samples are generated in the following way: Where γ sem and λ sem These are two instances x and x randomly selected from a batch. The characteristic statistics are specifically defined as follows: Where, β sem For semantic factors, a random mixing ratio is generated for different samples, and μ(x) and σ(x) are the mean and standard deviation of the instance feature cube.

7. The method according to claim 1, characterized in that: The auxiliary domain samples of category l The specific calculation formula is as follows: Z s Z represents the source domain sample. g Represents the generated domain sample, ω l This indicates overweighting, specifically defined as: Where l represents the category, l s and l g The feature label representing category l is calculated as follows: The samples obtained from each category are finally synthesized into an auxiliary domain sample Z. a .

Images