Hyperspectral image classification method and system based on spectral-spatial diffusion generation

Abstract

The application provides a hyperspectral image classification method and system based on spectral-space diffusion generation, introduces a diffusion model to generate a hyperspectral sample, and the progressive denoising generation strategy of the diffusion model can generate more diversified hyperspectral samples in line with actual spectral physical laws while preserving spectral details and spatial structures; a class label embedding mechanism is introduced to fine-tune the pre-trained diffusion model and optimize the generation capability of the diffusion model, thereby fundamentally relieving the problem of small-sample label scarcity. Multiple screening standards are introduced, including spectral physical constraints, feature space alignment and semantic consistency verification, to ensure the authenticity and semantic consistency of the generated sample in the spectral and spatial dimensions.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral processing technology, and particularly relates to a hyperspectral image classification method and system based on spectral-spatial diffusion generation. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Hyperspectral image classification technology, by fusing spatial and spectral information, enables accurate identification of ground features and has become a core means of intelligent remote sensing interpretation. However, in practical applications, the scarcity of manually labeled samples leads to prominent small-sample classification problems. Researchers have attempted to introduce generative techniques to expand the data, but existing solutions have significant limitations.

[0004] Generative Adversarial Networks (GANs), as a mainstream generative method, are widely used to alleviate the problem of insufficient sample size. For example, the proposed 3D-HyperGAMO model synthesizes minority class samples through a 3D hyperspectral patch generator to improve inter-class balance. However, GANs generally suffer from pattern collapse during training, where the generator tends to repeatedly output samples with the same pattern, resulting in a severe lack of generative diversity. This problem is further exacerbated in small-sample scenarios, as the sparse data distribution makes the adversarial game between the generator and the discriminator more prone to imbalance. In addition, hyperspectral samples generated by GANs often exhibit spectral-spatial structural distortion, such as discontinuities between bands or blurred spatial textures, because their adversarial training mechanism struggles to accurately model the correlations of high-dimensional features.

[0005] To overcome the generation bottleneck of generative adversarial networks (GANs), diffusion models have attracted attention due to their progressive denoising generation strategy. The SpectralDiff method was the first to introduce diffusion models into the hyperspectral classification domain, designing a spectral-spatial diffusion module to assist feature extraction. However, this method only uses it as a feature enhancement tool and cannot substantially expand the training dataset. It is worth noting that even in mature application domains such as RGB images, current diffusion generation schemes still lack a sample selection mechanism. The generated samples are evaluated for visual quality only through general metrics, leading to insufficient semantic alignment. If the above techniques are directly transferred to the hyperspectral image generation domain, their complex spectral-spatial coupling characteristics will significantly exacerbate the semantic inaccuracy problem. Moreover, existing small-sample hyperspectral image classification models still rely on complex network structures, and their parameter count and computational cost pose obstacles to embedded deployment in small-sample scenarios. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a hyperspectral image classification method and system based on spectral-spatial diffusion generation. It generates hyperspectral samples through a diffusion model, introduces a category label embedding mechanism, fine-tunes the pre-trained diffusion model, optimizes the generation capability of the diffusion model, and fundamentally alleviates the problem of scarce small sample labels.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a hyperspectral image classification method based on spectral-spatial diffusion generation, comprising:

[0009] The spectral-spatial diffusion model was pre-trained using a hyperspectral image training set to obtain a pre-trained spectral-spatial diffusion model;

[0010] A category label embedding mechanism is introduced, and category information is incorporated into the training objective of the spectral-spatial diffusion model through feature-wise linear modulation. The pre-trained spectral-spatial diffusion model is fine-tuned using labeled small sample data to obtain the first spectral-spatial diffusion model. Hyperspectral generated samples are obtained based on the first spectral-spatial diffusion model.

[0011] The generated hyperspectral samples are screened to obtain the filtered hyperspectral samples; the screening criteria include spectral physical constraints, feature space alignment, and semantic consistency verification.

[0012] The classification model is trained using the selected hyperspectral samples. The hyperspectral image to be classified is then input into the trained classification model to obtain the classification result.

[0013] In a second aspect, the present invention provides a hyperspectral image classification system based on spectral-spatial diffusion generation, comprising:

[0014] The pre-training module is configured to: pre-train the spectral-spatial diffusion model using a hyperspectral image training set to obtain a pre-trained spectral-spatial diffusion model;

[0015] The training and generation module is configured to: introduce a category label embedding mechanism, integrate category information into the training objective of the spectral-spatial diffusion model through feature-wise linear modulation, fine-tune the pre-trained spectral-spatial diffusion model using labeled small sample data to obtain the first spectral-spatial diffusion model, and obtain hyperspectral generated samples based on the first spectral-spatial diffusion model;

[0016] The filtering module is configured to filter the hyperspectral generated samples to obtain filtered hyperspectral generated samples; the filtering conditions include spectral physical constraints, feature space alignment and semantic consistency verification.

[0017] The classification module is configured to: train the classification model using the selected hyperspectral generated samples, input the hyperspectral image to be classified into the trained classification model, and obtain the classification result.

[0018] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0019] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0020] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0021] The above one or more technical solutions have the following beneficial effects:

[0022] In this invention, hyperspectral samples are generated through a diffusion model. The progressive denoising generation strategy of the diffusion model can generate more diverse hyperspectral samples that conform to the actual physical laws of spectroscopy while preserving spectral details and spatial structure. A category label embedding mechanism is introduced to fine-tune the pre-trained diffusion model, optimize the generation capability of the diffusion model, and fundamentally alleviate the problem of scarce small sample labels.

[0023] This invention introduces multiple screening criteria, including spectral physical constraints, feature space alignment, and semantic consistency verification, to ensure the authenticity and semantic consistency of the generated samples in both spectral and spatial dimensions. This screening mechanism effectively avoids the problems of spectral continuity breaks, spatial texture blurring, and semantic misalignment in existing generation methods, thereby improving the quality of the generated samples and ensuring that they can effectively support subsequent classification tasks of hyperspectral images.

[0024] This invention designs a lightweight classification module that significantly reduces computational redundancy and parameter count by the collaborative work of three main components: a multi-scale lightweight bottleneck module, a dual-attention fusion module, and a prototype-enhanced feature classification head. In particular, it achieves a good balance between computational overhead and classification accuracy, enabling efficient operation in low-resource environments and enhancing the practical application value of hyperspectral image classification.

[0025] This invention addresses the problems of unstable prototypes and unclear class boundaries in traditional few-sample classification methods by introducing dynamic prototype aggregation and optimized contrast loss design. The EMA mechanism smooths the update of class prototypes, while triplet loss and prototype center loss ensure that similar samples cluster towards class prototypes, enhancing the discriminative power between classes. Furthermore, the optimized combination of focus loss and label smoothing loss effectively avoids overfitting and strengthens the model's ability to identify difficult-to-classify samples, further improving the accuracy and robustness of few-sample hyperspectral image classification.

[0026] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a schematic diagram of the overall process in Embodiment 1 of the present invention;

[0029] Figure 2 This is a schematic diagram of the sample quality optimization process in Embodiment 1 of the present invention;

[0030] Figure 3 This is a schematic diagram of the classification model network structure in Embodiment 1 of the present invention;

[0031] Figure 4 This is a schematic diagram of the diffusion model network structure in Embodiment 1 of the present invention. Detailed Implementation

[0032] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0033] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0034] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0035] Example 1

[0036] This embodiment discloses a hyperspectral image classification method based on spectral-spatial diffusion, including:

[0037] The spectral-spatial diffusion model was pre-trained using a hyperspectral image training set to obtain a pre-trained spectral-spatial diffusion model;

[0038] A category label embedding mechanism is introduced, and category information is incorporated into the training objective of the spectral-spatial diffusion model through feature-wise linear modulation. The pre-trained spectral-spatial diffusion model is fine-tuned using labeled small sample data to obtain the first spectral-spatial diffusion model. Hyperspectral generated samples are obtained based on the first spectral-spatial diffusion model.

[0039] The generated hyperspectral samples are screened to obtain the filtered hyperspectral samples; the screening criteria include spectral physical constraints, feature space alignment, and semantic consistency verification.

[0040] The classification model is trained using the selected hyperspectral samples. The hyperspectral image to be classified is then input into the trained classification model to obtain the classification result.

[0041] This embodiment provides a hyperspectral image classification method based on spectral-spatial diffusion generation. The method comprises three parts: spectral-spatial joint diffusion generation, sample quality optimization, and a lightweight classification model. These three parts work together to achieve a complete "generation-screening-classification" process. Specific technical measures are as follows:

[0042] The spectral-spatial joint diffusion generation aims to solve the problem of insufficient hyperspectral sample generation capability in small sample scenarios. Based on a three-dimensional diffusion model, it achieves sample generation through a progressive architecture of "noise input layer → spectral-spatial joint denoising network → sample output layer".

[0043] The spectral-spatial joint denoising network comprises a spectral feature modeling submodule (using 1D convolution combined with a temporal attention mechanism to capture band continuity) and a spatial feature modeling submodule (using 2D convolution combined with a spatial self-attention mechanism to mine pixel neighborhood dependencies), and integrates spectral and spatial features through a cross-dimensional fusion layer. The generation process adopts a two-step strategy of "unsupervised pre-training (learning the basic distribution of unlabeled data) + small sample fine-tuning (introducing class condition constraints)", which ultimately generates hyperspectral samples that conform to the real data distribution and have both spectral details and spatial structure, thus alleviating the problem of scarce small sample annotations from the data source.

[0044] To address the issues of semantic inaccuracies and lack of quality screening in generated samples, the sample quality optimization section uses the output generated by spectral-spatial joint diffusion as input and improves sample quality through a multi-index screening mechanism. Specifically, it designs a triple screening criterion: spectral physical constraints (calculating the spectral angular distance and Euclidean distance between the generated sample and the real sample to ensure spectral continuity and physical authenticity), feature space alignment (calculating the feature distance between the generated sample and the real sample prototype through a feature extractor to ensure consistent feature distribution), and semantic consistency verification (using the classification confidence of a pre-trained classifier to ensure that the category semantics of the generated sample are aligned with the target task). After screening, the output is a high-quality generated sample that combines structural fidelity and semantic accuracy.

[0045] The lightweight classification model addresses the issues of high computational cost and poor adaptability of classification networks by employing a multi-scale lightweight bottleneck module (HLB), a dual attention fusion module (DAFM), and a prototype-enhanced feature classification head (PEFC). The multi-scale lightweight bottleneck module reduces parameters through a decomposition structure of "spectral 1D convolution + spatial 2D convolution" and extracts complex ground feature features using multi-scale convolution kernels. The dual attention fusion module includes a spectral attention mechanism and a spatial attention sub-module, enhancing key features through weighted fusion. The prototype-enhanced feature classification head calculates class prototypes using a small number of labeled samples, mapping the distance between features and prototypes to classification probabilities, thus improving robustness in small-sample scenarios. The overall architecture reduces computational cost while maintaining the integrity of high-dimensional spectral information, making it suitable for embedded deployments.

[0046] The following is combined Figures 1-2 The hyperspectral image classification method based on spectral-spatial diffusion proposed in this embodiment will be described in detail:

[0047] The spectral-spatial joint diffusion generation process completes the generation of hyperspectral samples in small-sample scenarios through a three-stage process: "unsupervised pre-training to learn the basic distribution of data → category conditional embedding to fuse label information → small-sample fine-tuning to achieve category-controllable generation".

[0048] The overall process is as follows: First, the 3D U-Net diffusion model is pre-trained in unsupervised mode using a large amount of unlabeled hyperspectral data to enable it to master the basic distribution of spectral-spatial joint features. Next, a class label embedding mechanism is introduced to integrate class information into the diffusion model through feature-wise linear modulation. Finally, the model is fine-tuned using a small number of labeled samples to enable it to generate hyperspectral samples of specific classes, thus meeting the data augmentation requirements for small sample classification.

[0049] Specifically, it includes:

[0050] Step 1: Unsupervised pre-training.

[0051] Given a hyperspectral image (HSI), denoted as ,in and These represent the height and width of the image, respectively. Indicates the number of spectral channels.

[0052] A forward diffusion process is used to gradually add Gaussian noise to the hyperspectral image. The specific noise addition process is as follows:

[0053]

[0054] in, The noise variance parameter increases over time. Indicates the first Hyperspectral image features after the first diffusion process It is obedience Distributed noise, It is a standard normal distribution. It is the identity matrix. With the addition of noise, the image deteriorates from the original data. It transforms into a noise distribution.

[0055] Through recursive expansion, the noise attenuation relationship can be expressed as:

[0056]

[0057] in, This is the cumulative noise attenuation factor, at time step t=T. The noise is added to approximate a standard Gaussian distribution, completing the forward diffusion process.

[0058] In the reverse denoising process, the model uses the trained neural network To predict noise and gradually restore the original image.

[0059] The update formula for reverse denoising is:

[0060]

[0061] in, This is for predicting the mean of the denoised image.

[0062] The denoising process gradually removes noise and restores the original image structure. For example... Figure 4As shown, a symmetric 3D U-Net architecture is employed in the diffusion model. The U-Net consists of an encoder and a decoder. The encoder utilizes 3D convolution to extract multi-scale spectral-spatial features from the hyperspectral image, while the decoder restores the data dimensionality through upsampling and convolution. The network preserves low-level details through skip connections to enhance the accuracy of image restoration. To strengthen the representation capability of spectral features, a spectral attention module is added after the residual block in each stage of the encoder and before the downsampling operation.

[0063] The spectral attention module adaptively adjusts the feature weights for different spectral bands. The specific calculation process is as follows:

[0064]

[0065] GAP stands for Global Average Pooling. These are the input features of the spectral attention module. For activation function, and These are learnable weights.

[0066] During training, L1 loss, L2 loss, and spectral angle (SAM) loss are used for joint optimization to ensure that the generated images are faithful in both spectral and spatial dimensions.

[0067] The loss function is defined as follows:

[0068]

[0069] in, It is true standard normal distribution noise. This represents the features of the generated hyperspectral image. SAM loss ensures that the angle between the generated spectral curve and the real sample spectral curve is minimized, thus maintaining the physical authenticity of the spectrum.

[0070] Step 2: Embed and merge tag information based on category conditions.

[0071] By effectively integrating category label information into the diffusion model and using a category-conditional embedding mechanism, the generated hyperspectral image can be controlled according to the given category label, thereby achieving category-controllable image generation.

[0072] First, the discrete category labels It is mapped to a continuous D-dimensional real vector through the embedding layer. This vector is related to the time step embedding vector. By combining elements, a category conditional embedding vector is formed. The specific formula is as follows:

[0073]

[0074] in, This is the sinusoidal encoding vector for each time step.

[0075] To effectively incorporate category information into the model, a feature-wise linear modulation (FiLM) mechanism is employed to adaptively adjust the features. In the residual blocks of the U-Net network, the FiLM mechanism calculates a scaling factor. and offset factor Linear modulation is applied to each feature channel.

[0076] The specific calculation process is as follows:

[0077]

[0078] in, The conditional embedding is obtained by using the features input to the FiLM mechanism for linear modulation of each feature channel. Then, a fully connected layer is needed to... Mapped to match the number of H channels and Through two fully connected layers ( , Using the ReLU activation function, the class conditional embedding vector is... Mapped to scaling factor:

[0079]

[0080] Adopted and Two parallel fully connected layers ( , ) structure, independent mapping class conditional embedding vector Offset factor;

[0081]

[0082] This is used after unsupervised pre-training, when introducing class labels for training the conditional diffusion model. Accurately integrating class conditional information into the feature extraction and denoising process of the diffusion model is the key to connecting unsupervised pre-training with controllable class generation.

[0083] During training, class labels are introduced. As a condition, it is incorporated into the training objective of the diffusion model.

[0084] The loss function of the categorical conditional diffusion model can be expressed as:

[0085]

[0086] in, In the known number of Step diffusion characteristics Time step and category labels In this case, the model predicts noise. This loss function, based on the unsupervised pre-trained L1 loss, forces the model to learn the influence of class labels on spectral-spatial features, enabling the model to control the spectral-spatial structure of the generated image according to the class labels.

[0087] Step 3: Fine-tuning with small samples to achieve controllable generation of categories.

[0088] In scenarios with limited labeled data, this embodiment employs a two-step training strategy. The first step involves unsupervised pre-training of the diffusion model using a large number of unlabeled hyperspectral images to learn the basic distribution characteristics of the data. The second step involves fine-tuning on a small number of labeled hyperspectral image samples, optimizing the diffusion model's generative capabilities by introducing category label information. This is because overfitting is prone to occur in small-sample scenarios, and a transfer learning strategy is used to avoid this problem. During the fine-tuning phase, most of the model's weights are retained to maintain its ability to learn the overall data distribution, and only high-level parameters related to category labels are updated.

[0089] Specifically, the first five layers of the model are frozen, and only the parameters of the embedding layer and the last three layers of U-Net are updated to ensure effective adaptation to category information. During fine-tuning, a small number of labeled hyperspectral image patches are used as input, and the model parameters are optimized through category conditional diffusion loss.

[0090] The loss function is:

[0091] in, For category-conditional loss, The SAM loss weights are hyperparameters. The SAM loss ensures that the angle between the generated image and the spectral curve of the real sample is minimized, preserving spectral physical authenticity. Generation is performed using the fine-tuned model, which samples initial noise from a standard Gaussian distribution. Combined with target category embedding vector Perform the reverse denoising process.

[0092] The noise prediction network in the generation process estimates the noise components and updates the image step by step. The reverse process is optimized by linear interpolation, which improves the generation speed while maintaining image quality.

[0093] The specific reverse update formula is as follows:

[0094]

[0095] Finally, samples of the corresponding categories are generated based on a small number of category labels, providing sufficient generated data for the next module to perform filtering.

[0096] While conditional diffusion models can generate class-controllable hyperspectral samples, pseudo-samples generated directly in small-sample scenarios may suffer from issues such as spectral physical inaccuracies or semantic ambiguity. Therefore, the sample quality optimization module proposes a sample quality improvement framework based on multi-index screening.

[0097] By employing spectral physical constraints, feature space alignment, and semantic consistency verification, the quality and usability of generated samples are systematically improved. Specifically, the following triple screening criteria are designed to optimize the quality of generated samples from different dimensions, ensuring their effectiveness.

[0098] The first is the spectral physical rationality constraint, which is to ensure that the generated hyperspectral samples have real physical properties in the spectral dimension, so that the generated images can better match the spectral characteristics of real ground objects.

[0099] Using Spectral Angle Mapper (SAM) as the core metric, the sample spectrum is calculated and generated. Spectra of real samples of the target category The included angle is given by the following formula:

[0100]

[0101] Wherein, the symbol "·" represents the vector dot product, This represents the Euclidean norm.

[0102] To ensure the spectral physical authenticity of the generated samples, only SAM values ​​below a certain threshold are retained. The threshold was determined through statistical analysis of intra-class spectral differences in real samples of the target category. The screening criterion was to retain only samples that met the SAM < 0. The generated samples ensure that the spectral characteristics of the generated samples are highly consistent with the physical characteristics of the target ground objects, thereby improving the spectral authenticity.

[0103] The second is the category prototype consistency constraint to achieve feature space alignment. This is to ensure that the generated samples are close to the category prototype of the target category in the feature space, thereby enhancing the category specificity and discriminative power of the generated samples. First, a pre-trained classifier is used to extract the category prototype.

[0104] In target category Above, by extracting all real samples The feature vectors in the penultimate layer of the classifier's feature extractor are used to calculate their mean, thus obtaining the class prototype vector. :

[0105]

[0106] in, For category The number of samples, This is the feature extraction function for the pre-trained classifier. For category The One real sample. Then, for the generated samples... Extract feature representations at the same level And calculate its relationship with the category prototype. Euclidean distance :

[0107]

[0108] The distance and the threshold Compare and filter those that meet the requirements. The generated samples are designed to closely approximate the target category prototype in the feature space, thereby further enhancing category specificity and discriminative power.

[0109] The third is semantic consistency verification. To ensure that the generated samples have clear class separability and avoid semantic ambiguity, a pre-trained classifier is used to verify the generated samples. Perform category prediction and calculate prediction confidence. Only retain predictions with a confidence level higher than a set threshold. The purpose of this step is to ensure that the generated samples not only meet expectations in the spectral and feature space, but also have high identifiability at the classification semantic level, preventing the generated samples from having ambiguous categories or misleading the classifier.

[0110] By employing multiple screening methods, including constraints on spectral physical rationality, category prototype consistency, and semantic consistency verification, the quality of generated hyperspectral samples can be effectively improved. This ensures that the generated samples meet the expected requirements in terms of spectral characteristics, feature space, and semantics, greatly enhancing the practical usability and classification performance of the generated samples in small-sample scenarios.

[0111] This embodiment, after expanding the training set with high-quality generated samples, designs a novel lightweight hyperspectral image classification module. It aims to achieve efficient and accurate spectral-spatial feature representation through a collaborative design of multi-scale feature fusion, dual attention mechanisms, and prototype reinforcement learning. This lightweight classification module improves the classification accuracy of hyperspectral images while maintaining a lightweight structure, making it particularly suitable for small-sample scenarios.

[0112] The lightweight classification module comprises three core components: a multi-scale lightweight bottleneck module (HLB), a dual attention fusion module (DAFM), and a prototype augmented feature classification head (PEFC).

[0113] The multi-scale lightweight bottleneck module aims to extract efficient spectral-spatial features through multi-scale convolution and a lightweight bottleneck structure to improve the accuracy of hyperspectral image classification, especially by extracting as much local and global feature information as possible while maintaining a lightweight structure. Through dilated convolution, this module can effectively expand the receptive field and capture image information at different scales. Given a hyperspectral image cube as input... ,in Indicates the number of bands in a hyperspectral image. It is the spatial size of each sample.

[0114] To extract multi-scale features, the multi-scale lightweight bottleneck module uses three parallel dilated convolution branches to extract features at different scales. Specifically, the original scale branch employs a depthwise separable convolution operation. The receptive field is 3×3, used to capture local detail features, generating features. The mesoscale branch uses a 3×3 dilated convolution with an inflation rate of d=2. The receptive field is 5×5, used to extract regional structural features, generating features. Large-scale branches employ 3×3 dilated convolutions with an inflation rate of d=3. The receptive field is 7×7, used to extract structural features from a larger region, generating features. Finally, multi-scale feature fusion is performed to combine the features extracted from the three branches. , and By stitching along the channel dimension, multi-scale fused features are generated. Subsequently, the dimensionality is reduced to B / 4 channels through 1×1 convolution. The reduced features are then subjected to global average pooling to generate multi-scale feature vectors. .

[0115] To enhance the discriminative power of features, this embodiment utilizes spectral weights and spatial weights for feature weighting in the multi-scale lightweight bottleneck module. Spectral weights are applied to the feature vector... We obtain the following results through one-dimensional convolution learning:

[0116] in, This represents a 1D convolution operation with B / 8 input channels and B output channels, used to restore the feature channel count to B.

[0117] Spatial weight By analyzing the original input Perform two-dimensional convolution extraction; A 3×3 convolution operation with B input channels and 1 output channel transforms the multi-channel input into a single-channel spatial weight map. Then, the features are spectrally recalibrated using a Hadamard product, and spatially focused using a dot product operation and spatial weights, ultimately yielding the enhanced features.

[0118]

[0119] To control the number of parameters and computational cost, the number of channels is compressed to B / 2 using 1×1 convolutions, ensuring the efficiency and accuracy of the model in small sample scenarios.

[0120] To further enhance the discriminative power of the output features of the multi-scale lightweight bottleneck module, a dual-attention fusion module dynamically calibrates spectral-spatial features through a dual-branch attention mechanism. Unlike the cross-dimensional weight fusion in the multi-scale lightweight bottleneck module, the dual-attention fusion module enhances features with finer granularity through independent spectral and spatial attention branches, aiming to improve the accuracy of hyperspectral image classification. This module focuses on key information in the spectral and spatial dimensions respectively through spectral and spatial attention mechanisms, thereby enhancing the discriminative power in classification tasks. The spectral attention effect on the output features of the multi-scale lightweight bottleneck module... Applying one-dimensional convolution To extract spectral features, and retain the original information through residual connections, the feature is obtained. :

[0121] Representative features are obtained through two-dimensional average pooling and two-dimensional max pooling operations, respectively. and :

[0122]

[0123]

[0124] Using adaptive weights , , The pooling features are weighted and then activated by a Sigmoid function to generate spectral weights.

[0125]

[0126] Where σ is the Sigmoid activation function.

[0127] The spatial attention branch focuses on discriminative spatial regions in hyperspectral images, targeting the output features of the multi-scale lightweight bottleneck module. Spatial structure features are extracted using 3×3 convolution, and feature integrity is maintained through residual connections to obtain spatial features. :

[0128]

[0129] Then, a one-dimensional pooling operation is performed along the spectral dimension to obtain the spatial feature matrix. :

[0130]

[0131] The dimensionality is reduced to a single channel through 1×1 convolution, and a spatial weight map is generated through an activation function. And perform a Hadamard product with the input features:

[0132]

[0133]

[0134] Finally, adaptive weights are used. By fusing spectral and spatially weighted features, the final output is formed:

[0135]

[0136] The dual-attention fusion module enhances the spectral and spatial features of hyperspectral images through two independent attention mechanisms: spectral and spatial. The spectral attention branch focuses on strengthening the discriminative power of key bands, while the spatial attention branch focuses on important spatial regions. Through the combined effect of these two mechanisms, this module effectively improves the expressive power of spectral-spatial features, ultimately outputting fused features. This provides more discriminative input for subsequent prototype enhancement feature classification heads, further improving the performance of hyperspectral image classification models.

[0137] The Prototype-Enhanced Feature Classifier Head aims to strengthen the discriminative power of hyperspectral image classification, especially in low-shot scenarios, through the design of prototype learning and contrastive loss. By tightly integrating with the aforementioned dual-attention fusion module, the Prototype-Enhanced Feature Classifier Head further improves classification performance, ensuring that the discriminative power of features accurately characterizes different categories, thereby achieving higher classification accuracy in low-shot learning. The Prototype-Enhanced Feature Classifier Head receives output features from the dual-attention fusion module. And based on these features, prototype learning and optimization are carried out.

[0138] First, a dynamic prototype aggregation strategy is used to update the prototype for each category. In small sample scenarios, fluctuations in category prototypes may affect the model's learning performance. Therefore, the prototype-enhanced feature classifier introduces an exponential moving average (EMA) mechanism to smooth the prototype update process.

[0139] Specifically, the prototype of each category Represented by the features of this category of samples Together with the EMA value of the historical prototype, the update formula is as follows:

[0140] in, For category Sample size It is a sample The feature representation after global average pooling by the dual attention fusion module is... Obtained through global average pooling. The EMA weighting coefficients determine the smoothness of the current prototype by which the historical prototype is applied. Initially, Set as a prototype matrix with random initialization ,in For the number of categories, For feature dimensions.

[0141] Next, calculate the feature vector. With each category prototype Cosine similarity between :

[0142]

[0143] This similarity metric helps the model determine how well the input sample matches each category.

[0144] To enhance gradient propagation efficiency and prevent overconfidence in simple samples, a temperature coefficient is introduced. This softens the similarity score. The probability distribution is obtained by performing an exponential operation on the cosine similarity and then normalizing it. The formula is as follows:

[0145]

[0146] This transforms similarity into a probability distribution, thus providing guidance for subsequent prototype learning and classification decisions.

[0147] The final classification probability is calculated by combining prototype similarity and a linear classifier. Based on similarity... and feature mapping The final classification probability It can be obtained through the following formula:

[0148]

[0149] in, and For the parameters of a linear classifier, To determine the contribution of spectral similarity and the output of the linear classifier, we need to integrate the weights.

[0150] This study combines triplet loss and prototype center loss to optimize feature-prototype matching. Triplet loss ensures that the distance between features of the same class and their prototypes is less than the distance between the furthest negative class feature, while prototype center loss constrains features to cluster towards their respective class prototypes. The final contrastive loss function is:

[0151] in, For Euclidean distance, For the sample The true category prototype, The farthest negative class prototype, that is, with The non-realistic category prototype with the largest distance. λ is the marginal value, and λ is the regularity coefficient.

[0152] To further improve classification accuracy, the focus loss and label smoothing loss are combined with the contrastive loss for optimization. The focus loss avoids the model's overconfidence in easily classifiable samples by focusing on difficult-to-classify samples, and its formula is as follows:

[0153]

[0154] in, It is the predicted probability of the true category. ≥ is the focus parameter, and K is the number of categories. This is the one-hot representation of the true label. The weighting factor for focus loss. By adjusting This adjusts the loss weights for easy and difficult samples. Label smoothing mitigates the impact of noisy labels by softening the label distribution, as shown in the following formula:

[0155] ,

[0156] in, The smoothing coefficient is set to 0.1. These are the smoothed labels; hard labels are 0 or 1, and the softened labels are the true category labels. The non-real category is This loss function constrains the entropy of the model's output distribution, preventing the model from overconfident in a single class. The final total loss function is:

[0157]

[0158] in, and The loss balance coefficient is used. The prototype-enhanced feature classifier further enhances the classification capability of hyperspectral images through dynamic prototype aggregation and contrastive loss design. By smoothing class prototype updates through the EMA mechanism, strengthening class clustering through triplet loss and prototype center loss, and jointly optimizing focal loss and label smoothing loss, the prototype-enhanced feature classifier not only improves the model's classification accuracy but also effectively solves the overfitting problem in small sample scenarios. Ultimately, the prototype-enhanced feature classifier provides the model with a powerful classification capability, particularly suitable for complex hyperspectral image classification tasks.

[0159] This invention addresses the problems of insufficient diversity and spectral-spatial feature distortion in traditional generation models under small sample scenarios by introducing a spectral-spatial joint generation module based on a diffusion model. The progressive denoising generation strategy of the diffusion model can generate more diverse hyperspectral samples that conform to the actual spectral physics while preserving spectral details and spatial structure, fundamentally alleviating the problem of scarce small sample annotations.

[0160] This invention introduces multiple screening criteria by optimizing the sample quality module, including spectral physical constraints, feature space alignment, and semantic consistency verification, to ensure the authenticity and semantic consistency of the generated samples in both spectral and spatial dimensions. This screening mechanism effectively avoids the problems of spectral continuity breaks, spatial texture blurring, and semantic misalignment in existing generation methods, thereby improving the quality of the generated samples and ensuring that they can effectively support subsequent classification tasks of hyperspectral images.

[0161] To meet the embedded deployment requirements in small-sample scenarios, this invention designs a lightweight classification module. Through the collaborative work of three main components—a multi-scale lightweight bottleneck module (HLB), a dual attention fusion module (DAFM), and a prototype enhanced feature classification head (PEFC)—the computational redundancy and parameter count of the model are significantly reduced. In particular, a good balance is achieved between computational overhead and classification accuracy, enabling efficient operation in low-resource environments and enhancing the practical application value of hyperspectral image classification.

[0162] This invention addresses the problems of unstable prototypes and unclear class boundaries in traditional few-sample classification methods by introducing dynamic prototype aggregation and optimized contrast loss design. The EMA mechanism smooths the update of class prototypes, while triplet loss and prototype center loss ensure that similar samples cluster towards class prototypes, enhancing the discriminative power between classes. Furthermore, the optimized combination of focus loss and label smoothing loss effectively avoids overfitting and strengthens the model's ability to identify difficult-to-classify samples, further improving the accuracy and robustness of few-sample hyperspectral image classification.

[0163] In summary, this invention successfully overcomes multiple challenges in classifying small-sample hyperspectral images by innovatively combining a diffusion generation model, an optimized sample selection mechanism, and a lightweight classifier design. Through these innovative designs, this invention improves the quality of generated samples and classification accuracy, making the hyperspectral image classification method more practical and adaptable to deployment, particularly suitable for resource-constrained embedded systems and real-world application scenarios.

[0164] Example 2

[0165] The purpose of this embodiment is to provide a hyperspectral image classification system based on spectral-spatial diffusion generation, including:

[0166] The pre-training module is configured to: pre-train the spectral-spatial diffusion model using a hyperspectral image training set to obtain a pre-trained spectral-spatial diffusion model;

[0167] The training and generation module is configured to: introduce a category label embedding mechanism, integrate category information into the training objective of the spectral-spatial diffusion model through feature-wise linear modulation, fine-tune the pre-trained spectral-spatial diffusion model using labeled small sample data to obtain the first spectral-spatial diffusion model, and obtain hyperspectral generated samples based on the first spectral-spatial diffusion model;

[0168] The filtering module is configured to filter the hyperspectral generated samples to obtain filtered hyperspectral generated samples; the filtering conditions include spectral physical constraints, feature space alignment and semantic consistency verification.

[0169] The classification module is configured to: train the classification model using the selected hyperspectral generated samples, input the hyperspectral image to be classified into the trained classification model, and obtain the classification result.

[0170] In further embodiments, the following is also provided:

[0171] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0172] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0173] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0174] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0175] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0176] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0177] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0178] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0179] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0180] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method based on spectral-spatial diffusion generation, characterized in that, The application comprises the following steps: Pre-training the spectral-spatial diffusion model by using a hyperspectral image training set, to obtain a pre-trained spectral-spatial diffusion model; Mapping discrete class labels into real number vectors, and combining them with time step embedding vectors to form class-conditioned embedding vectors; Based on the class-conditioned embedding vectors, linear modulation is performed on each feature channel by using a per-feature linear modulation mechanism, and the introduced class labels are used as conditions to integrate class information into the training target of the spectral-spatial diffusion model; Fine-tuning the pre-trained spectral-spatial diffusion model by using small sample data with labels to obtain a first spectral-spatial diffusion model, and obtaining hyperspectral generation samples based on the first spectral-spatial diffusion model; Screening the hyperspectral generation samples to obtain screened hyperspectral generation samples; wherein the screening conditions include spectral physical constraints, class prototype consistency constraints and semantic consistency verification. The spectral physical constraint is that the included angle between the hyperspectral generation sample and the target class real sample spectrum is calculated as a spectral angle distance, and the hyperspectral generation sample with a calculated spectral angle distance less than a set angle threshold is retained. The class prototype consistency constraint is that a class prototype vector is obtained by extracting feature vectors of real samples and calculating the mean value by using a pre-trained classifier; feature vectors of the same level of the hyperspectral generation sample are extracted by using the pre-trained classifier, and the Euclidean distance with the class prototype vector is calculated to screen the hyperspectral generation sample with a calculated Euclidean distance less than a set distance threshold. The semantic consistency verification is that the class prediction of the hyperspectral generation sample is performed by using a pre-trained classifier, and the prediction confidence is calculated to retain the hyperspectral generation sample with a prediction confidence higher than a set threshold. The classification model is trained by using the screened hyperspectral generation sample, and the classification result is obtained by inputting the hyperspectral image to be classified into the trained classification model. The classification model comprises three core components: a multi-scale lightweight bottleneck module, a double attention fusion module and a prototype enhanced feature classification head. The processing process of the classification model for the input hyperspectral image to be classified is as follows: The multi-scale lightweight bottleneck module is used to capture the spectral features and spatial features of different scales of the hyperspectral image to be classified, and the spectral features and spatial features are weighted and fused to obtain spectral-spatial features. Based on the double attention fusion module, the spectral attention branch and the spatial attention branch are used to improve the expression ability of the spectral-spatial features. Based on the prototype enhanced feature classification head, the prototype of each class is updated by a prototype aggregation strategy, the cosine similarity between the feature vector output by the double attention fusion module and each class prototype is calculated, the classification probability is obtained by performing exponential operation and normalization processing according to the calculated cosine similarity.

2. The hyperspectral image classification method based on spectral-spatial diffusion generation of claim 1, wherein, In the fine-tuning stage of the pre-trained diffusion model, only the high-level parameters related to the class label are updated.

3. The method of hyperspectral image classification based on spectral-spatial diffusion generation of claim 1, wherein, The loss function of the classification model is determined by a focal loss, a label smoothing loss and a contrastive loss; wherein the contrastive loss includes a triplet loss which ensures that the distance between the features of the same class and the prototype is less than the distance of the farthest negative class features, and a prototype center loss which constrains the prototype of the feature belonging to the class to be gathered.

4. A hyperspectral image classification system based on spectral-spatial diffusion generation, characterized by, The application comprises the following steps: The pre-training module is configured to pre-train the spectral-spatial diffusion model by using a hyperspectral image training set, to obtain a pre-trained spectral-spatial diffusion model; The training and generation module is configured to map discrete class labels into real number vectors, and combine the time step embedding vectors to form class-conditioned embedding vectors; Based on the class-conditioned embedding vectors, linear modulation is performed on each feature channel by using a per-feature linear modulation mechanism, and the class labels introduced as conditions are used to integrate class information into the training target of the spectral-spatial diffusion model; The pre-trained spectral-spatial diffusion model is fine-tuned by using labeled small sample data, to obtain a first spectral-spatial diffusion model, and hyperspectral generation samples are obtained based on the first spectral-spatial diffusion model; The screening module is configured to screen the hyperspectral generation samples, to obtain screened hyperspectral generation samples; wherein the screening conditions include spectral physical constraints, class prototype consistency constraints, and semantic consistency verification. The spectral physical constraints are that an included angle between the hyperspectral generation samples and target class real sample spectra is calculated as a spectral angle distance, and the hyperspectral generation samples with a calculated spectral angle distance less than a set angle threshold are retained. The class prototype consistency constraints are that a feature vector of a real sample is extracted by using a pre-trained classifier and a mean value is calculated, to obtain a class prototype vector; a feature vector of the same level of the hyperspectral generation samples is extracted by using the pre-trained classifier, and an Euclidean distance with the class prototype vector is calculated, to screen the hyperspectral generation samples with a calculated Euclidean distance less than a set distance threshold. The semantic consistency verification is that a class prediction is performed on the hyperspectral generation samples by using the pre-trained classifier and a prediction confidence is calculated, to retain the hyperspectral generation samples with a prediction confidence higher than a set threshold. The classification module is configured to train a classification model by using the screened hyperspectral generation samples, input the hyperspectral image to be classified into the trained classification model, and obtain a classification result; the classification model includes three core components: a multi-scale lightweight bottleneck module, a dual attention fusion module, and a prototype enhanced feature classification head. The processing process of the classification model on the input hyperspectral image to be classified is as follows: The multi-scale lightweight bottleneck module is used to capture spectral features and spatial features of different scales of the hyperspectral image to be classified, and the spectral features and the spatial features are weighted and fused to obtain spectral-spatial features; The dual attention fusion module is used to improve the expression ability of the spectral-spatial features by using a spectral attention branch and a spatial attention branch; The prototype enhanced feature classification head is used to update the prototype of each class by a prototype aggregation strategy, calculate the cosine similarity between the feature vector output by the dual attention fusion module and each class prototype, perform exponential operation and normalization processing according to the calculated cosine similarity, and obtain a classification probability.

5. An electronic device, comprising: The computer instructions stored on the memory and run on the processor complete the method of any one of claims 1-3 when the computer instructions are run by the processor.

6. A computer-readable storage medium, characterized in that, A computer program product for storing computer instructions which, when executed by a processor, perform the method of any one of claims 1-3.

7. A computer program product, characterised in that, A computer program which, when executed by a processor, performs the method of any one of claims 1-3.

Images