A hyperspectral image domain generalization classification method, system, device and medium

Abstract

This invention relates to the field of image classification technology, and discloses a hyperspectral image domain generalization classification method, system, device, and medium. The method includes: acquiring several hyperspectral images of different land cover types; acquiring the mean and variance of each hyperspectral image in the channel dimension, and randomly shuffling them; determining the spectral variation parameters of the hyperspectral images based on the mean and variance before and after random shuffling to generate corresponding spectral variation images; separating the center and background of each hyperspectral image to obtain their respective center and background images, and randomly shuffling them; generating corresponding spatial variation images by combining the shuffled center and background images of each hyperspectral image; training a hyperspectral image classification model using an enhanced hyperspectral image obtained by fusing several corresponding spectral variation images and spatial variation images, and then classifying land cover types in the hyperspectral images to be classified, thereby improving the domain generalization performance of the hyperspectral image classification model.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a hyperspectral image domain generalized classification method, system, device and medium. Background Technology

[0002] With the rapid development of remote sensing technology, hyperspectral remote sensing image classification technology plays an important role in fields such as ecological environment protection, precision agricultural management, and disaster early warning and monitoring. Existing traditional supervised methods generally assume that the training data (source domain) and test data (target domain) have the same distribution. However, in reality, due to the influence of factors such as differences in imaging sensors, environmental conditions, time phase, and region, there is often a significant data distribution shift between the source domain and the target domain. This results in a significant drop in classification performance when the model trained in the source domain is directly applied to the target domain.

[0003] To address the aforementioned issues, current research has explored both domain adaptation and domain generalization. The former typically assumes that training and test samples come from the same probability distribution, and predicts the output of the test samples by designing corresponding models and discrimination criteria, resulting in low efficiency. Furthermore, in most current learning scenarios, the probability distributions of training and test samples are different. The latter aims to build a model using only source domain data, enabling it to directly generalize to the unknown target domain. While multi-source domain methods can improve diversity by integrating data from multiple source domains, the amount of source domain data is often limited in real-world scenarios, making it difficult to cover all possible variations in the target domain. Single-source domain methods typically rely on random generation to expand the data, but this randomness easily overlooks some inherent differences, making it difficult for the generated samples to truly reflect the spectral-spatial variation patterns of the target domain.

[0004] Therefore, existing hyperspectral image classification models have poor domain generalization performance. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral image domain generalization classification method, system, device and medium, which can solve the problem of poor domain generalization performance of existing hyperspectral image classification models.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a hyperspectral image domain generalization classification method, comprising the following steps:

[0007] Acquire hyperspectral images of several different land cover types;

[0008] The mean and variance of each hyperspectral image in the channel dimension are obtained, and the mean and variance of each hyperspectral image are randomly shuffled. Based on the mean and variance of each hyperspectral image before and after random shuffling, the spectral variation parameters of the hyperspectral image are determined, so as to generate the corresponding spectral variation image through the spectral variation parameters.

[0009] The center and background of each hyperspectral image are separated to obtain their respective center and background images. The center and background images of each hyperspectral image are then randomly shuffled. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated.

[0010] The corresponding spectral variation images and spatial variation images are fused to obtain several enhanced hyperspectral images. A hyperspectral image classification model is then trained using these enhanced hyperspectral images to classify land features from the hyperspectral images to be classified.

[0011] Optionally, separating the center and background of each hyperspectral image to obtain their respective center and background images includes:

[0012] For each hyperspectral image, the mean value of the center pixel of the hyperspectral image is obtained to obtain the center feature, and the mean value of the surrounding pixels of the hyperspectral image is obtained to obtain the edge feature;

[0013] The difference between each pixel in the feature image corresponding to the center feature and each pixel in the hyperspectral image is calculated respectively to obtain a first matrix reflecting the similarity between each pixel in the hyperspectral image and the center feature, and a second matrix reflecting the similarity between each pixel in the hyperspectral image and the background feature.

[0014] After subtracting the second matrix from the first matrix, the positive results in the new matrix are used as the background mask, and the negative results are used as the center mask. The hyperspectral image is then multiplied by the background mask and the center mask respectively to obtain the background image and the center image.

[0015] Optionally, the random shuffling of the center and background images of each hyperspectral image, and the generation of a corresponding spatial variation image by combining the center and background images of each shuffled hyperspectral image, includes:

[0016] The mean values ​​of all pixels in the center image and the background image are obtained separately, and the absolute value of the difference between the two is taken; the absolute value is used to characterize the separation quality of the center image and the background image of each hyperspectral image;

[0017] The center and background images of several hyperspectral images with separation quality greater than a preset threshold are randomly shuffled, and the corresponding spatial variation image is generated by combining the center and background images of each shuffled hyperspectral image.

[0018] Optionally, the spectral variation image is obtained through the following steps:

[0019] For each hyperspectral image, the two-dimensional and three-dimensional spatial features of the hyperspectral image are obtained to obtain the two-dimensional spectral image and the three-dimensional spectral image;

[0020] The two-dimensional spectral image is processed by a residual structure consisting of multiple linear layers plus ReLU activation layers. The processed two-dimensional spectral image is then fused with the unprocessed two-dimensional spectral image and processed again by a linear layer plus ReLU activation layer to obtain the target two-dimensional spectral image.

[0021] The three-dimensional spectral image is processed by a structure with a convolutional layer and a ReLU activation layer, and then the processed three-dimensional spectral image is fused with the target two-dimensional spectral image to obtain a fused image.

[0022] Based on the mean and variance of the processed 3D spectral image in the channel dimension corresponding to each hyperspectral image, a first spectral variation image corresponding to each processed 3D spectral image is generated.

[0023] Based on the mean and variance of the fusion image corresponding to each hyperspectral image in the channel dimension, a second spectral variation image is generated for each fusion image;

[0024] The corresponding first and second spectral variation images are fused to obtain the spectral variation image corresponding to each hyperspectral image.

[0025] Optionally, the fusion of the corresponding spectral variation image and spatial variation image to obtain several enhanced hyperspectral images includes:

[0026] Based on the variance of hyperspectral images of the same type of land cover in the channel dimension among several hyperspectral images, the weight of each channel of the hyperspectral image of each land cover type is determined; whereby the weight is used to characterize the stability and contribution of the hyperspectral image channel to the hyperspectral image classification.

[0027] Based on the weight of each channel of the hyperspectral image, each hyperspectral image and its corresponding spectral variation image are fused to obtain the corresponding spectral enhancement fused image;

[0028] The spectral enhancement fusion image and spatial variation image corresponding to each hyperspectral image are fused to obtain several enhanced hyperspectral images.

[0029] Optionally, the hyperspectral image classification model is trained using a generative adversarial network;

[0030] The generator of the generative adversarial network is used to generate an enhanced hyperspectral image corresponding to each hyperspectral image;

[0031] The discriminator in a generative adversarial network includes a feature extractor, a residual classifier, and a master classifier;

[0032] The feature extractor is used to extract the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image, and obtain the difference vector between the original features and the enhanced features as the residual features;

[0033] The residual classifier is used to determine whether the residual features come from hyperspectral images or enhanced hyperspectral images;

[0034] The master classifier is used to classify ground features in hyperspectral images based on enhanced features.

[0035] Optionally, the feature extractor consists of two layers. The first layer uses convolutional kernels to perform channel dimensionality reduction on the hyperspectral image and the enhanced hyperspectral image respectively, and performs max pooling operation after nonlinear activation. The second layer uses convolutional kernels to perform channel dimensionality increase on the hyperspectral image processed by the first layer and the enhanced hyperspectral image respectively, and performs max pooling operation after nonlinear activation, so as to obtain the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image.

[0036] Embodiments of the present invention also provide a hyperspectral image domain generalization classification system, comprising:

[0037] The image acquisition module is used to acquire hyperspectral images of several different land cover types;

[0038] The first image variation module is used to obtain the mean and variance of each hyperspectral image in the channel dimension, and randomly shuffle the mean and variance of each hyperspectral image. Based on the mean and variance of each hyperspectral image before and after random shuffling, the spectral variation parameters of the hyperspectral image are determined, so as to generate the corresponding spectral variation image through the spectral variation parameters.

[0039] The second image variation module is used to separate the center and background of each hyperspectral image to obtain their respective center and background images, and randomly shuffle the center and background images of each hyperspectral image. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated.

[0040] The image classification module is used to fuse the corresponding spectral variation images and spatial variation images to obtain several enhanced hyperspectral images. The hyperspectral image classification model is then trained using these enhanced hyperspectral images to classify land features from the hyperspectral images to be classified.

[0041] Embodiments of the present invention also provide a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described hyperspectral image domain generalization classification method.

[0042] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described hyperspectral image domain generalization classification method.

[0043] The hyperspectral image domain generalization classification method provided by this invention has at least the following beneficial effects:

[0044] For hyperspectral image samples used in training a hyperspectral image classification model, this invention generates corresponding spectral variant images and spatial variant images by performing spectral and spatial variants on the hyperspectral images. This joint expansion of the hyperspectral images from both spectral and spatial dimensions can simulate both the spectral feature variations of hyperspectral images in different scenarios and the geometric deformation and object distribution of hyperspectral images in the spatial domain, realistically reflecting their diversity in different scenarios. This effectively compensates for the shortcomings of existing methods in generating diverse hyperspectral image data. Based on this, a hyperspectral image classification model can be trained, which can improve the domain generalization performance of the hyperspectral image classification model. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0046] Figure 1 This is a flowchart illustrating a hyperspectral image domain generalization classification method provided by the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0048] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] One embodiment of the present invention relates to a hyperspectral image domain generalization classification method. The specific process of the hyperspectral image domain generalization classification method in this embodiment can be as follows: Figure 1 As shown, it includes:

[0050] Step 101: Obtain hyperspectral images of several different land cover types.

[0051] Step 102: Obtain the mean and variance of each hyperspectral image in the channel dimension, and randomly shuffle the mean and variance of each hyperspectral image. Based on the mean and variance of each hyperspectral image before and after random shuffling, determine the spectral variation parameters of the hyperspectral image, so as to generate the corresponding spectral variation image through the spectral variation parameters.

[0052] Step 103: Separate the center and background of each hyperspectral image to obtain their respective center and background images, and randomly shuffle the center and background images of each hyperspectral image. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated.

[0053] Step 104: The corresponding spectral variation image and spatial variation image are fused to obtain several enhanced hyperspectral images. A hyperspectral image classification model is then trained using these enhanced hyperspectral images to classify land features from the hyperspectral images to be classified.

[0054] The implementation details of the hyperspectral image domain generalization classification method in this embodiment are described below. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0055] In step 101, the dataset used in this embodiment is obtained. The data comes from the Pavia University (source domain) and Pavia Centre (target domain) hyperspectral remote sensing datasets. This dataset contains 102 spectral bands with a spatial resolution of 1.3 meters, covering seven types of ground features, including asphalt pavement, grassland, and metal plates. Local samples are extracted from the source domain (paviaU) and target domain (paviaC) data using a 13×13 pixel sliding window. 80% of the source domain labeled data is randomly sampled as the training set, and the remaining 20% ​​is used as the validation set; the target domain data is used independently as the test set and is not involved in either training or validation.

[0056] In step 102, when acquiring the spectral variation image corresponding to each hyperspectral image, two-dimensional and three-dimensional spatial features of the hyperspectral image are acquired to obtain a two-dimensional spectral image and a three-dimensional spectral image. The two-dimensional spectral image is processed by a residual structure consisting of multiple linear layers plus ReLU activation layers, and the processed two-dimensional spectral image is fused with the unprocessed two-dimensional spectral image. Then, it is processed again by a linear layer plus ReLU activation layer to obtain the target two-dimensional spectral image. The three-dimensional spectral image is processed by a convolutional layer plus ReLU activation layer, and the processed three-dimensional spectral image is fused with the target two-dimensional spectral image to obtain a fused image. Based on the mean and variance of the processed three-dimensional spectral image in the channel dimension corresponding to each hyperspectral image, a first spectral variation image corresponding to each processed three-dimensional spectral image is generated. Based on the mean and variance of the fused image in the channel dimension corresponding to each hyperspectral image, a second spectral variation image corresponding to each fused image is generated. The corresponding first spectral variation image and second spectral variation image are fused to obtain the spectral variation image corresponding to each hyperspectral image.

[0057] In this implementation, a spectral variation generator is designed, comprising three modules: a channel weight generation module, a spectral style variation module, and a spectral fusion module. These three modules aim to generate as many diverse spectral images as possible based on channel stability weights, while ensuring image authenticity and validity, thus forming a spectral image dataset. Its core content is as follows:

[0058] (1) Channel weight generation module:

[0059] For hyperspectral images, the information in each channel has different characteristics; some are susceptible to external interference, while others are relatively stable. Therefore, this embodiment calculates the channel dimension variance among similar ground features to determine the channel's contribution and stability in the classification process.

[0060] The specific steps are as follows: The source domain image x (256, 102, 13, 13) is input into the label channel weight generation module. The source domain image is processed sequentially according to the labels from category 1 to category 7. The variance (64, 1, 1) of the 64 channels in the same category is calculated, then normalized to generate channel weights for each category (7, 64, 1, 1). Higher weights indicate that the channel is more susceptible to external influences in reality, thus requiring more perturbation; lower weights indicate a more stable channel, potentially containing more features beneficial for classification, thus preserving as much source domain information as possible.

[0061] (2) Spectral style variation module:

[0062] To achieve spectral style variation, this embodiment extracts 3D and 2D information from the spectrum and uses randomization to perturb the style of the acquired 3D and 2D information, generating a new spectral image to achieve spectral style transformation. This is because, in some cases, hyperspectral image labels are sparse, and the uncertainty of unlabeled region labels may adversely affect subsequent model training. To avoid the potential impact of the above situation, the data cube is combined with the two-dimensional spectrum.

[0063] Input data processing: Input the preprocessed source domain image x_spe (dimension: 256×64×13×13) into the spectral variation module.

[0064] Feature decomposition and preprocessing: The input features are decomposed into two branches using Conv2d convolutional layers. The x_spe_2d (256×64×1×1) branch represents the compressed spectral features, while the x_spe_3d (256×64×13×13) branch represents the features that preserve spatial information. Both are used for subsequent processing.

[0065] Two-dimensional spectral branching: This branch employs a residual structure design, consisting of multiple linear layers and ReLU activation layers. This structure contains four linear layers with 64, 128, 128, and 64 neurons respectively. Each layer is equipped with a ReLU activation function, meaning that the channel dimension is transformed from 64 to 128 to 128 to 64 through three Linear+ReLU processing steps, finally obtaining x_spe_2d'. Then, x_spe_2d' is added to x_spe_2d, followed by one Linear+ReLU processing step (reducing from 64 to 32 channels) to obtain x_spe_2d'' (256, 32, 1, 1). This residual connection design effectively prevents the vanishing gradient problem in deep networks.

[0066] 3D Data Cube Two-Branch Processing: The 3D image is processed in two branches. The first branch, x_spe_3d, undergoes a Conv2d+ReLU process to become (256, 32, 1, 1). This is then added to x_spe_2d'' to obtain (256, 64, 1, 1), and the fused features are input to the Randomization spectral transformer for randomization, generating diverse feature representations. Finally, a single upsampling process restores the spatial dimension to (256, 64, 13, 13). The second branch first performs a Conv2d+ReLU process to obtain (256, 64, 13, 13), which is also input to the Randomization spectral transformer for randomization. Finally, the 2D and 3D results are concatenated to become (256, 128, 13, 13), and then processed again by Conv2d+ReLU to output the spectral variation result x_spe' (256, 64, 1, 1).

[0067] Based on this, the adaptability of the model to cross-domain work is improved through a dual-branch design and randomization mechanism. The residual connection ensures that important feature information is not lost in the depth transformation, while dimensionality compression and parallel branch processing reduce computational complexity while ensuring performance.

[0068] The randomization operation of the spectral converter in the above steps is as follows: First, the mean of each sample is calculated in the channel dimension. and variance The formula is as follows:

[0069] ;

[0070] ;

[0071] In the formula, H and W are the height and width of the image, respectively, X is the input image, and ε is a minimal constant.

[0072] The mean obtained The format is (256, 64), variance The format is (256, 64). Then, the indices of the input image are randomly shuffled, and a new index is generated based on the shuffled image. and variance :

[0073] ;

[0074] ;

[0075] In the formula, This is for images after indexing has been shuffled.

[0076] based on , , and Generate spectral style variation parameters and The formula is as follows:

[0077] ;

[0078] ;

[0079] In the formula, The learnable mixing weights control the mixing ratio of the original statistics and the random statistics. β is the translation parameter of spectral style variation, which is obtained by weighted mixing of the original mean and the random mean. γ is the scaling parameter of spectral style variation, which is obtained by weighted mixing of the original standard deviation and the random standard deviation.

[0080] Finally, a spectral variation image is generated based on the spectral style variation parameters. The formula is as follows:

[0081] ;

[0082] This formula applies new β and γ parameters after standardization (subtracting the mean and dividing by the standard deviation).

[0083] (3) Spectral fusion module:

[0084] To generate spectral perturbation samples more effectively, this experiment uses channel dimension stability and importance weights. The larger the weight, the more susceptible the channel (with large variance) is to external influences. By increasing the transform domain weight of this channel, the transformation in cross-domain scenarios in reality can be effectively simulated. Meanwhile, channels with small weights can be understood as channels with small variance between images, so more source domain images are used to preserve the important features of the images.

[0085] The specific steps are as follows: The source domain image, transform domain image, channel weights, and label input spectrum are mixed with the generated module. The source domain samples and generated domain samples are weighted and fused according to the channel dimension weights, as shown in the following formula: ;

[0086] In the formula, i represents the i-th type of image. Let be the channel weights of the i-th class of images. For the i-th type of image after spectral variation, For the i-th type of image that has not undergone spectral variation, the final returned image is the spectrally enhanced fused image. (256, 64, 1, 1).

[0087] In step 103, when generating the spatial variation image corresponding to each hyperspectral image, firstly, the center feature is obtained by acquiring the mean value of the center pixel of the hyperspectral image, and the edge feature is obtained by acquiring the mean value of the surrounding area of ​​the hyperspectral image. The pixel in the feature image corresponding to the center feature and the edge feature is subtracted from the pixel in the hyperspectral image to obtain a first matrix reflecting the similarity between the pixel in the hyperspectral image and the center feature, and a second matrix reflecting the similarity between the pixel in the hyperspectral image and the background feature. After subtracting the second matrix from the first matrix, the positive results in the new matrix are used as the background mask, and the negative results are used as the center mask. The hyperspectral image is then multiplied by the background mask and the center mask to obtain the background image and the center image. Then, the mean values ​​of all pixels in the center image and the background image are obtained respectively, and the absolute value of the difference between the two is taken; the absolute value is used to characterize the separation quality of the center image and the background image of each hyperspectral image; the center images and background images of several hyperspectral images with separation quality greater than a preset threshold are randomly shuffled, and the corresponding spatial variation image is generated by combining the center images and background images of each shuffled hyperspectral image.

[0088] In this implementation, to further enhance the model's generalization ability and robustness in cross-domain scenarios, this embodiment designs a SpaRandomization generator to simulate the changing patterns of background and target regions in real-world scenes. This module, based on the source domain image, introduces dynamic boundary estimation and weight control mechanisms to achieve conditional random recombination of the background and central object. The specific process is as follows:

[0089] (1) Generation of spatial center mask and background mask:

[0090] Input source image (256, 1, 13, 13). First, calculate and record the mean value of the center (3×3) pixels of each image to obtain the center feature. Then, calculate the mean value of the surrounding 2 rows and 2 columns to obtain the edge feature. Extend the center and edge features to the entire plane (256, 1, 13, 13). By subtracting the center similarity from each pixel in the source domain, obtain the center similarity and background similarity. Then, subtract the center similarity from the background similarity, setting positive results as the background mask and negative results as the center mask. Generate the mask accordingly.

[0091] Center mask: ;

[0092] Background mask: ;

[0093] In the formula, n is the image number, l is the number of channels, h is the height of the image, and w is the width of the image.

[0094] (2) Generation of exchange weights:

[0095] The input consists of a background mask, a center mask, and a source domain image (256, 1, 13, 13). The source domain image and the mask are multiplied to obtain the background image and the center image. The mean of the center image and the background image is calculated, and then the absolute value of the difference between them is used as a measure of image separation quality. A larger difference indicates more accurate boundaries and clearer separation, as shown below:

[0096] ;

[0097] In the formula, Weight_diff is the boundary fitting accuracy index value for each image, Inner_weight is the mean value within the boundary (center), and Outer_weight is the mean value outside the boundary (background). If A larger value indicates accurate boundary fitting. Finally, the difference is transformed into mixed weights mix_weights (256, 1, 1, 1) and center and background images within the range of [0,1] through normalization and other strategies.

[0098] (3) Exchange spatial neighborhood images:

[0099] This module takes the center image, background image, and mixing weights (mix_weights) as input, and selects the indices of the top 80% of images with the largest mix_weights values. Then, the background and center indices are randomly shuffled, and the background and center images are combined according to the new indices to obtain the spatial image after neighborhood transformation (256, 1, 13, 13).

[0100] In step 104, the spatially enhanced (i.e., mutated) representation and the spectrally enhanced (i.e., mutated) representation are first compared. and By combining, we obtain the spatial-spectral enhancement representation. and used as input by the decoder To form new samples .

[0101] Then, training is performed based on a generative adversarial learning (GAP) framework. The network architecture adopts a GAP framework, where the generator constructs a generated domain image by reconstructing the feature distribution of the target domain, and the discriminator determines whether the image comes from the generated domain or the source domain using a classification loss. The training parameters are set as follows: batch size 256, initial learning rate 1e-3, and maximum training epochs 20. In adversarial learning, the generator drives feature distribution alignment by minimizing the target domain classification loss and the contrastive adversarial loss, while the discriminator achieves cross-domain feature discrimination through the source domain classification loss, the target domain classification loss, and the contrastive loss, ultimately completing the hyperspectral image classification task.

[0102] In the generative adversarial learning framework of this embodiment, the generator of the generative adversarial network is used to generate the enhanced hyperspectral image corresponding to each hyperspectral image, i.e., the process described in steps 102 to 103 above. The discriminator includes a feature extractor, a residual classifier, and a main classifier; the feature extractor is used to extract the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image, and obtain the difference vector between the original features and the enhanced features as the residual features; the residual classifier is used to determine whether the residual features come from the hyperspectral image or the enhanced hyperspectral image; the main classifier is used to classify land cover in the hyperspectral image based on the enhanced features.

[0103] The feature extractor consists of two layers. The first layer uses convolutional kernels to perform channel dimensionality reduction on the hyperspectral image and the enhanced hyperspectral image respectively, and performs max pooling operation after nonlinear activation. The second layer uses convolutional kernels to perform channel dimensionality increase on the hyperspectral image and the enhanced hyperspectral image processed by the first layer respectively, and performs max pooling operation after nonlinear activation to obtain the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image.

[0104] In the specific implementation, the discriminator takes samples from the source domain and the generator domain (i.e., the generator) as input. By combining the classification loss and the residual loss, it distinguishes between samples from the source domain and the generator domain. The core process is as follows:

[0105] (1) Network architecture design:

[0106] Discriminator D contains a feature extractor A classifier and a residual classifier The discriminator D employs a dual-path feature extraction structure, constructing a feature pyramid through two consecutive convolutional layers. The feature extractor... It consists of two layers: the first layer uses 3×3 convolutional kernels to reduce the dimensionality of the input inchannel spectral channels to 64 channels, followed by ReLU nonlinear activation and then 2×2 max pooling; the second layer also uses 3×3 convolutions to increase the number of channels to 128 dimensions, maintaining the same activation and pooling process. This hierarchical design can progressively compress the spatial dimension of patch_size×patch_size image patches, ultimately outputting a compact 1×1×128 feature representation. The embedding network following the feature extractor consists of two fully connected layers, achieving feature dimensionality enhancement through a 512-dimensional hidden space, with ReLU activation function used to enhance nonlinear expressive power.

[0107] (2) Residual feature processing:

[0108] To address the inter-class similarity and intra-class variation issues in hyperspectral images, this project innovatively designs a residual classifier module to handle inter-domain differences. This module obtains residual components by calculating the difference between the feature representations of source and target samples. These residual components can capture inter-domain specific information. The residual feature processing mechanism is the core innovation of this architecture. During forward propagation, the system simultaneously processes source sample x1 and target sample x2, obtaining their high-dimensional embeddings out4 and out8 through a feature extraction network with shared weights. The residual classifier captures cross-domain difference information by calculating the difference vector nume = out8 - out4 between the two domain features. This difference is mapped to a class probability distribution through a linear layer netC. ​​In training mode, the network outputs the classification result x1_clss of the source sample, the classification result x2_clss of the target sample, and the residual probability residual_p. The residual probability is normalized using a softmax function to ensure that it conforms to the probability distribution characteristics.

[0109] ;

[0110] Among them, conditional entropy: measure The uncertainty ensures that the residual components are evenly distributed across all categories, guaranteeing that they do not contain domain-sensitive features relevant to the classification task.

[0111] The discriminator employs a dual-mode operation mechanism. Specifically, the network uses a dynamic size calculation mechanism to ensure architectural flexibility, automatically deriving the output size of the convolutional layers using the `_get_final_flattened_size` method. This method creates a dummy input tensor to simulate the forward propagation process, sequentially calculating the feature map size after `conv1+relu1+mp` and `conv2+relu2+mp`, and finally flattening it to obtain the input dimension of the fully connected layers. During training, the system simultaneously optimizes the main classifier `cls_head_src` and the residual classifier `netC`: the main classifier focuses on accurate classification of source domain samples, while the residual classifier extracts domain-invariant features by controlling the mutual information between residual features and class labels. In test mode, the network only returns the source domain classification result `clss`, maintaining the functional interface of the standard classifier. This dual-mode design satisfies both the domain adaptation requirements during training and ensures computational efficiency during deployment.

[0112] In addition, the model evaluation and testing are as follows:

[0113] (1) Evaluation: During evaluation, the optimal weights saved during the training phase are used to infer the target domain test set. A prediction map is generated using the obtained class probabilities. The classification accuracy is obtained by comparing the prediction map with the ground truth labels using the overall classification accuracy (OA), and the misclassification details of each class are obtained through the confusion matrix.

[0114] (2) Testing: The data in this embodiment comes from the Pavia University (source domain) and Pavia Centre (target domain) hyperspectral remote sensing datasets. The model was trained using PaviaU, with 80% of the data used for training and 20% for testing. PaviaC was then used as the test set to evaluate the model's image classification performance in domain generalization. The final accuracy OA reached 85.7%.

[0115] In summary, this invention proposes a hyperspectral image domain generalization classification method based on joint spatial-spectral extension, mainly in two aspects: First, by constructing a spectral variation generator and a spatial variation generator, the source domain data is jointly extended from both spectral and spatial dimensions. This not only simulates the spectral feature variations of hyperspectral images under different scenes, but also realistically reflects the diversity of different scenes through geometric texture deformation and object distribution adjustment in the spatial domain, thus effectively compensating for the shortcomings of existing methods in generating diverse data. Second, a residual classifier is introduced. By modeling the feature residuals between the source and target domains, the difference between the two is further reduced, ensuring high discrimination accuracy and robustness in cross-domain classification tasks.

[0116] In terms of technical implementation, this method uses two hyperspectral remote sensing datasets from Pavia University and Pavia Centre as experimental data. It employs 102 spectral bands, covering various ground cover scenarios such as asphalt pavement, grassland, and metal plates, and selects seven classes for training in the experiment. The entire model is trained based on a generative adversarial learning framework. The generator produces new images through spectral and spatial variations; the discriminator distinguishes between source and generated domain images through collaborative classification by the main classifier and the residual classifier. Through adversarial training, the model's generalization and robustness are improved under different sensor conditions, environmental conditions, and time phases.

[0117] By jointly simulating spectral and spatial variations and combining a classifier to achieve effective alignment between the source and target domains, this approach not only addresses the shortcomings of existing technologies in terms of data generation diversity, utilization of domain characteristics, and self-supervised optimization during the testing phase, but also provides more accurate, stable, and efficient technical support for hyperspectral image classification in practical applications such as precision agriculture and urban planning, demonstrating significant scientific value and broad application prospects.

[0118] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the protection scope of this invention.

[0119] Another embodiment of the present invention relates to a hyperspectral image domain generalization classification system. The implementation details of this embodiment's hyperspectral image domain generalization classification system are described below. The following details are provided for ease of understanding and are not essential for implementing this solution. This embodiment's hyperspectral image domain generalization classification system includes:

[0120] The image acquisition module is used to acquire hyperspectral images of several different land cover types;

[0121] The first image variation module is used to obtain the mean and variance of each hyperspectral image in the channel dimension, and randomly shuffle the mean and variance of each hyperspectral image. Based on the mean and variance of each hyperspectral image before and after random shuffling, the spectral variation parameters of the hyperspectral image are determined, so as to generate the corresponding spectral variation image through the spectral variation parameters.

[0122] The second image variation module is used to separate the center and background of each hyperspectral image to obtain their respective center and background images, and randomly shuffle the center and background images of each hyperspectral image. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated.

[0123] The image classification module is used to fuse the corresponding spectral variation images and spatial variation images to obtain several enhanced hyperspectral images. The hyperspectral image classification model is then trained using these enhanced hyperspectral images to classify land features from the hyperspectral images to be classified.

[0124] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0125] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0126] Another embodiment of the present invention relates to a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the hyperspectral image domain generalization classification methods described in the above embodiments.

[0127] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0128] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0129] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0130] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A hyperspectral image domain generalization classification method, characterized in that, The method includes: Acquire hyperspectral images of several different land cover types; Obtain the mean value of each hyperspectral image in the channel dimension. and variance After randomly shuffling the indices of each hyperspectral image, the mean value of the new hyperspectral image at each index is obtained. and variance and based on the mean and variance and mean and variance The spectral variation parameters of the hyperspectral image are determined so as to generate the corresponding spectral variation image based on the spectral variation parameters. The center and background of each hyperspectral image are separated to obtain their respective center and background images. The center and background images of each hyperspectral image are then randomly shuffled. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated. The corresponding spectral variation images and spatial variation images are fused to obtain several enhanced hyperspectral images. A hyperspectral image classification model is then trained using these enhanced hyperspectral images to classify land cover in the hyperspectral images to be classified. The spectral variation image is generated using the following formula: ； ； ； In the formula, For spectral variation images, These are the pixel values ​​of the hyperspectral image. and Here, β is the spectral variation parameter, and γ is the translation parameter and γ is the scaling parameter. As weight; The process of separating the center and background of each hyperspectral image to obtain their respective center and background images includes: For each hyperspectral image, the mean value of the center pixel is obtained to obtain the center feature, and the mean value of the surrounding pixels is obtained to obtain the edge feature. The difference between each pixel in the feature image corresponding to the center feature and the edge feature is calculated with each pixel in the hyperspectral image to obtain a first matrix reflecting the similarity between each pixel in the hyperspectral image and the center feature, and a second matrix reflecting the similarity between each pixel in the hyperspectral image and the background feature. After subtracting the second matrix from the first matrix, the positive results in the new matrix are used as the background mask, and the negative results are used as the center mask. The hyperspectral image is then multiplied by the background mask and the center mask respectively to obtain the background image and the center image. The process involves randomly shuffling the center and background images of each hyperspectral image, and then combining the shuffled center and background images to generate a corresponding spatial variation image, including: The mean values ​​of all pixels in the center image and the background image are obtained separately, and the absolute value of the difference between the two is taken. The absolute value is used to characterize the separation quality of the center image and the background image of each hyperspectral image. The center images and background images of several hyperspectral images with separation quality greater than a preset threshold are randomly shuffled, and the corresponding spatial variation image is generated by combining the center images and background images of each shuffled hyperspectral image.

2. The hyperspectral image domain generalization classification method according to claim 1, characterized in that, The step of generating the corresponding spectral variation image through spectral variation parameters includes: For each hyperspectral image, the two-dimensional and three-dimensional spatial features of the hyperspectral image are obtained to obtain the two-dimensional spectral image and the three-dimensional spectral image; The two-dimensional spectral image is processed by a residual structure consisting of multiple linear layers plus ReLU activation layers. The processed two-dimensional spectral image is then fused with the unprocessed two-dimensional spectral image and processed again by a linear layer plus ReLU activation layer to obtain the target two-dimensional spectral image. The three-dimensional spectral image is processed by a structure with a convolutional layer and a ReLU activation layer, and then the processed three-dimensional spectral image is fused with the target two-dimensional spectral image to obtain a fused image. Based on the mean and variance of the processed 3D spectral image in the channel dimension corresponding to each hyperspectral image, a first spectral variation image corresponding to each processed 3D spectral image is generated. Based on the mean and variance of the fusion image corresponding to each hyperspectral image in the channel dimension, a second spectral variation image is generated for each fusion image; The corresponding first and second spectral variation images are fused to obtain the spectral variation image corresponding to each hyperspectral image.

3. The hyperspectral image domain generalization classification method according to claim 1, characterized in that, The process of fusing the corresponding spectral variation images and spatial variation images to obtain several enhanced hyperspectral images includes: Based on the variance of hyperspectral images of the same type of land cover in the channel dimension among several hyperspectral images, the weight of each channel of the hyperspectral image of each land cover type is determined; whereby the weight is used to characterize the stability and contribution of the hyperspectral image channel to the hyperspectral image classification. Based on the weight of each channel of the hyperspectral image, each hyperspectral image and its corresponding spectral variation image are fused to obtain the corresponding spectral enhancement fused image; The spectral enhancement fusion image and spatial variation image corresponding to each hyperspectral image are fused to obtain several enhanced hyperspectral images.

4. The hyperspectral image domain generalization classification method according to claim 1, characterized in that, The hyperspectral image classification model was trained using a generative adversarial network. The generator of the generative adversarial network is used to generate an enhanced hyperspectral image corresponding to each hyperspectral image; The discriminator in a generative adversarial network includes a feature extractor, a residual classifier, and a master classifier; The feature extractor is used to extract the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image, and obtain the difference vector between the original features and the enhanced features as the residual features; The residual classifier is used to determine whether the residual features come from hyperspectral images or enhanced hyperspectral images; The master classifier is used to classify ground features in hyperspectral images based on enhanced features.

5. The hyperspectral image domain generalization classification method according to claim 4, characterized in that, The feature extractor consists of two layers. The first layer uses convolution kernels to perform channel dimensionality reduction on the hyperspectral image and the enhanced hyperspectral image respectively, and performs max pooling operation after nonlinear activation. The second layer uses convolution kernels to perform channel dimensionality increase on the hyperspectral image and the enhanced hyperspectral image processed by the first layer respectively, and performs max pooling operation after nonlinear activation, so as to obtain the original features of the hyperspectral image and the enhanced features of the enhanced hyperspectral image.

6. A hyperspectral image domain generalization classification system, characterized in that, The system includes: The image acquisition module is used to acquire hyperspectral images of several different land cover types; The first image variation module is used to obtain the mean value of each hyperspectral image in the channel dimension. and variance After randomly shuffling the indices of each hyperspectral image, the mean value of the new hyperspectral image at each index is obtained. and variance and based on the mean and variance and mean and variance The spectral variation parameters of the hyperspectral image are determined so as to generate the corresponding spectral variation image based on the spectral variation parameters. The second image variation module is used to separate the center and background of each hyperspectral image to obtain their respective center and background images, and randomly shuffle the center and background images of each hyperspectral image. By combining the shuffled center and background images of each hyperspectral image, a corresponding spatial variation image is generated. The image classification module is used to fuse the corresponding spectral variation image and spatial variation image to obtain several enhanced hyperspectral images, and to train a hyperspectral image classification model through several enhanced hyperspectral images to classify land features in the hyperspectral images to be classified. The spectral variation image is generated using the following formula: ； ； ； In the formula, For spectral variation images, These are the pixel values ​​of the hyperspectral image. and Here, β is the spectral variation parameter, and γ is the translation parameter and γ is the scaling parameter. As weight; The second image mutation module is further configured to, for each hyperspectral image, obtain the mean value of the center pixel of the hyperspectral image to obtain the center feature, and obtain the mean value of the surrounding area of ​​the hyperspectral image to obtain the edge feature; respectively, subtract each pixel in the feature image corresponding to the center feature and the edge feature from each pixel in the hyperspectral image to obtain a first matrix reflecting the similarity between each pixel in the hyperspectral image and the center feature, and a second matrix reflecting the similarity between each pixel in the hyperspectral image and the background feature; after subtracting the second matrix from the first matrix, the positive result in the new matrix is ​​used as the background mask, and the negative result is used as the center mask, and the hyperspectral image is multiplied by the background mask and the center mask respectively to obtain the background image and the center image; The second image variation module is also used to obtain the mean value of all pixels in the center image and the background image respectively, and take the absolute value after subtracting the two; wherein, the absolute value is used to characterize the separation quality of the center image and the background image of each hyperspectral image; randomly shuffle the center images and background images of several hyperspectral images in which the separation quality is greater than a preset threshold, and generate the corresponding spatial variation image by combining the center images and background images of each shuffled hyperspectral image.

7. A computer device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the hyperspectral image domain generalization classification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the hyperspectral image domain generalization classification method as described in any one of claims 1 to 5.

Images