Self-supervised hyperspectral image classification method suitable for low-label sample scene

Abstract

The invention discloses a self-supervised hyperspectral image classification method suitable for a low-annotation sample scene, and relates to the technical field of hyperspectral remote sensing image processing, comprising a self-supervised category sensing network oriented to the low-annotation scene; in the pre-training stage, a grouping spectrum enhancement module, a spectrum self-attention module and mask reconstruction are adopted, and the model is guided to focus on category-sensitive space-spectrum features under the label-free condition by minimizing the difference between a reconstructed image and an original shielded area; in the fine tuning stage, pre-trained network parameters are used as initialization parameters, and feature expression is further refined through classification loss. Therefore, by adopting the self-supervised hyperspectral image classification method suitable for the low-label sample scene, the lossless transmission of difficult sample features is realized, the distinguishing feature expression of mixed pixels is enhanced, and the classification balance of few sample categories is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral remote sensing image processing, and in particular to a self-supervised hyperspectral image classification method suitable for a low-labeled sample scene. BACKGROUND

[0002] Hyperspectral image (HSI) has dozens to hundreds of continuous and narrow spectral bands, which can simultaneously capture the rich spectral and spatial information of the ground object, and is widely used in fine agricultural monitoring, mineral resource exploration, urban land use analysis, ecological environment assessment and other fields. Its high spectral resolution provides an unprecedented fine-grained description capability for feature material identification and classification, and has become a key data source in remote sensing intelligent interpretation tasks. However, the current HSI classification technology still faces many core challenges to be solved in practical application.

[0003] Firstly, the problem of labeled sample scarcity is prominent. Due to the high complexity of hyperspectral images in spectral dimension and spatial range, it is usually necessary for domain experts to manually label according to field investigation or auxiliary data, which has high labeling cost and low efficiency, and seriously restricts the acquisition of high-quality training samples. The existing mainstream supervised classification methods are severely dependent on a large number of high-quality labeled samples, and are prone to overfitting under the condition of limited training data, which affects the generalization ability of the model in actual scenes. For example, although traditional SVM, Random Forest, and supervised network based on 3D-CNN perform well in full supervision, their accuracy significantly decreases under the condition of limited samples.

[0004] Secondly, the class distinction ability is insufficient. Different ground object classes in hyperspectral images have high similarity in some bands, and the spectral difference between classes is small. The redundancy and noise of the superimposed spectral data itself make it difficult for the model to effectively distinguish similar classes. For example, the classic ResNet and DenseNet do not explicitly consider spectral channel selection in the HSI scene, which may lead to insufficient response to different class bands and blurred boundaries.

[0005] Thirdly, the spectral band dependence modeling ability is weak. Existing methods mostly compress high-dimensional spectral information based on local convolution, dimensionality reduction mapping or principal component analysis, which is difficult to explicitly capture the long-distance dependence relationship between spectral bands. For example, although some Transformer structures (such as ViT-HSI) introduce global modeling capability, they do not sufficiently describe the semantic dependence of spectral bands, especially without the guidance of position encoding or spectral perception mechanism to accurately capture the combined features between key channels.

[0006] In addition, the self-supervised learning method emerging in recent years shows strong feature modeling potential without manual annotation, such as the methods of MAE, SimCLR-HSI, etc. have been introduced into the hyperspectral classification task. However, most self-supervised strategies focus on the reconstruction or contrast of general image structures, lack of guiding mechanism for class-sensitive spectral bands in hyperspectral data, and are difficult to achieve fine-grained class difference modeling and enhancement.

[0007] Therefore, at the present stage, the hyperspectral image classification task urgently needs a new network structure that can fully mine key discriminant spatial features and class-sensitive spectral band information under the condition of low-labeled samples. SUMMARY

[0008] The purpose of the present application is to provide a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios to solve the problems in the background art.

[0009] To achieve the above-mentioned purpose, the present application provides a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios, comprising the following steps: S1, collecting an original hyperspectral image block , and constructing a self-supervised class-aware network for low-labeled scenarios, including a grouping spectral enhancement module, a spectral self-attention module, a mask reconstruction and fine-tuning stage; S2, in the pre-training stage, guiding the model to focus on class-sensitive spatial-spectral features under the condition of no label; S21, based on the original hyperspectral image block , through the grouping spectral enhancement module, spatial-spectral features with key spatial structure and class responsiveness are generated; S22, based on the spatial-spectral features, through the spectral self-attention module, the non-local dependence between spectral bands is captured, the discriminant ability across spectral bands is enhanced, and the spatial-spectral joint features that fuse the global dependence of spectral bands and spatial structure are output; S23, by mask reconstruction, the spatial-spectral joint features are divided into N small blocks of the same size, part of the small blocks are randomly masked, and learnable classification labels and position information are added, to generate mask labels, unmasked labels and classification labels; S24, the classification labels and unmasked labels are jointly input into the encoder for feature encoding to generate high-level semantic latent features; S25, the high-level semantic latent features output by the encoder and the mask labels are jointly input into the decoder to reconstruct the masked area, to generate a reconstructed image; S26, by calculating the difference between the reconstructed image and the original masked area, the model is guided to optimize the feature extraction ability under the condition of no label supervision, and then the network parameters are updated; S3, in the fine-tuning stage, taking the pre-trained network parameters as the initialization parameters, further refining the feature expression through the classification loss.

[0010] Preferably, in S1, the spatial size of the original hyperspectral image block is , and the number of spectral channels is , that is .

[0011] Preferably, S21 comprises: S211, adjusting the spectral channels of the original hyperspectral data by convolution, the number of adjusted spectral channels being , and being set as an integer multiple of the preset grouping number; S212, sequentially using different sizes of grouping convolution to reserve the local spatial structure and the interaction characteristics of the spectral channels of different scales, and then obtaining the local spatial features through batch normalization processing and nonlinear activation operation; S213, extracting the global features through global average pooling, learning the channel weight through two-layer fully connected network to obtain the important channel feature weight, and then obtaining the enhanced spatial-spectral features through residual connection of the local spatial features.

[0012] Preferably, in S213, the important channel feature weight is as follows: ; In the formula, is the important channel feature weight, , is the weight matrix, is the global feature, is the corresponding activation function.

[0013] Preferably, S22 comprises: S221, flattening the spatial-spectral features to obtain , calculating the similarity between any two channels in to obtain : ; S222, multiplying with the value matrix to obtain the attention output, and performing residual connection: ; S223, restoring to the original shape to obtain the spatial-spectral joint features .

[0014] ​Preferably, S24 comprises splicing the classification label and the unmasked label, and then generating high-level semantic latent features through a Transformer encoder, as follows: ; ; In the formula, represents the spliced input features, represents the classification label, represents the unmasked label, represents the mapping function of the encoder, represents the high-level semantic latent features.

[0015] Preferably, in S25, the process of reconstructing the image comprises: ; In the formula, represents the reconstructed image, represents filling the mask label into the high-level semantic latent features as the input of the decoder, represents the mapping function of the decoder.

[0016] Preferably, S3 comprises: S31, initializing pre-training parameters; S32, extracting important features of a specific class in a key waveband and its spatial neighborhood, i.e., spatial-spectral features, through a grouping spectral enhancement module ; S33, dividing into non-overlapping small blocks , wherein represents the size of the image block divided in space, and the flattened linear mapping generates small block labels , introduces classification labels , and all position encodings are used as the input of the encoder to perform high-level feature modeling and latent discriminative semantic extraction, as follows: ; In the formula, represents the mapping function of the encoder, represents the high-level semantic latent features; S33, the encoder outputs , the high-dimensional features are mapped into the final class probability distribution through a classification head, and a cross-entropy loss function is used to measure the difference between the model prediction and the real label: ; In the formula, is the label of the real class, is the class probability distribution predicted by the model, is the number of classes.

[0017] Therefore, the present application adopts the above-mentioned self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios, which has the following technical effects: (1) Improve the class discrimination ability: the present application combines the grouping feature enhancement mechanism with the spectral self-attention module to effectively extract the discriminative spatial-spectral features strongly related to the class, and improve the discrimination degree between similar classes.

[0018] (2) Strengthen the global modeling between spectral bands: the present application introduces the spectral self-attention structure to explicitly model the long-distance dependency relationship between spectral bands, breaking the limitation of traditional convolutional networks on local feature modeling.

[0019] (3) Reduce the dependence on labeled data: the present application uses the mask reconstruction task to realize self-supervised feature learning, which can pre-train the model on unlabeled samples and maintain high classification performance in low-labeled scenarios.

[0020] (4) Accelerate model convergence: the present application obtains discriminative features through the pre-training stage, providing high-quality initialization for fine-tuning, thereby significantly accelerating the convergence speed of the downstream classification task.

[0021] The technical solutions of the present application will be described in further detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a structural diagram of a self-supervised class perception network for low-labeled scenarios in an embodiment of a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios; Figure 2 is a structural diagram of a grouping spectral enhancement module in an embodiment of a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios. DETAILED DESCRIPTION

[0023] The present application can be explained in more detail through the following examples, and the purpose of disclosing the present application is to protect all variations and improvements within the scope of the present application, and the present application is not limited to the following examples.

[0024] The present application provides a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios, which relates to a self-supervised class perception network for low-labeled scenarios, as shown in Figure 1 The input of the network is the original hyperspectral image block ( Figure 1 The red box part in , is the spatial size, The number of spectral channels represents the target; the objective is to achieve category-aware spatial-spectral joint feature extraction through intra-group feature enhancement.

[0025] Specifically, the self-supervised category perception network for low-label scenarios consists of four parts: Grouped Spectral Enhancement Module (GSW), Spectral Self-Attention Module (SSA), Mask Reconstruction, and Fine-tuning Stage.

[0026] I. Grouped Spectral Enhancement Module (e.g.) Figure 2 As shown): Adaptively extracts important features of specific categories in key bands and their spatial neighborhoods, effectively suppressing redundant information interference, as detailed below: 1. Spectral channel adjustment: via Convolution transforms the raw hyperspectral data Adjust the channel: ; Output dimensions: ,in This indicates the adjusted number of spectral channels, making It can be divided by a preset number of groups, thus adapting to subsequent grouping operations.

[0027] 2. Local spatial feature extraction: This is achieved by sequentially employing... Grouped convolution and Grouped convolution preserves the local spatial structure at different scales and the interactive properties of spectral channels: ; Output dimensions: ,in for The dimensions after grouped convolution processing.

[0028] 3. Stable network: The output is obtained through batch normalization (BN) and nonlinear activation (ReLU) operations. : .

[0029] 4. Extract global features: Extract global features using global average pooling (GAP): ; Output dimensions: ,in Indicates the batch size.

[0030] 5. Important Feature Learning: Channel weights are learned through a two-layer fully connected network to obtain the important channel feature weights, as follows: ; in, denotes the sigmoid function, which is used to rescale the spectral weights between 0 and 1. denotes the convolution operation, and denotes the weights of the two fully connected layers, is set to 16.

[0031] Output dimension: .

[0032] 6. Reinforced feature: the weight is element-wise multiplied with to obtain the class-responsive spatial-spectral feature , in which the key information is enhanced and the redundant information is suppressed.

[0033] II. Spectral self-attention module: modeling the long-range dependency between spectral bands.

[0034] Input: .

[0035] Objective: input the class-responsive spatial-spectral feature into the spectral self-attention module to fuse the global dependency between spectral bands and the local spatial information, and generate a joint feature representation with class sensitivity and spatial discriminability, which specifically includes: 1. Similarity calculation: input into the spectral self-attention module to capture the global dependency in the spectral dimension, which is first flattened into , and then the similarity between any two channels in is calculated: .

[0036] 2. Multiply with to obtain the attention output, and perform residual connection: ; Output dimension: .

[0037] 3. Feature restoration: restore to the original shape to fuse the global dependency between spectral bands and the local spatial distribution information, and generate a spatial-spectral joint feature with both class sensitivity and spatial discriminability.

[0038] III. Mask reconstruction: guide the model to focus on the class-sensitive spatial-spectral joint feature under the condition of no label.

[0039] Input: .

[0040] Objective: Use the mask reconstruction task as a self-supervised training method, guide the model to learn class-sensitive spatial-spectral discriminative features on unlabeled data by masking part of the input and restoring its content, which includes: 1. Divide the spatial-spectral joint feature into N small blocks , and perform a random masking operation, input semantic labels and position embedding modules, map shallow features to deep semantic labels, add a learnable classification label, and introduce position information to generate mask labels , unmasked labels and classification labels , 2. Feature encoding: unmasked labels and classification labels are input into the Transformer encoder for spectral-spatial feature extraction to obtain latent representation: ; ; where, is the mapping function of the encoder; , is the number of unmasked blocks, is the output dimension of the encoder.

[0041] 3. Decoder reconstruction: encoder output and mask label are input into the decoder to map to the reconstruction result: ; where, means filling the mask label into the encoder output data as input to the decoder; is the mapping function of the decoder.

[0042] Output dimension: .

[0043] 4. Calculate reconstruction loss: remove from , and calculate the difference between the reconstructed data and the original data at the same pixel position: ; where, and are the reconstructed pixel value and the original pixel value at the same position.

[0044] Four, fine-tuning stage: based on the parameters in the pre-training stage, load a small amount of labeled samples to further supervise the optimization of the model.

[0045] Input: Raw hyperspectral image patch Pre-trained GSW, Patch embedding matrix, and Transformer encoder parameters.

[0046] Objective: Guided by category labels and based on pre-trained features, further optimize the discriminative power of the spatial-spectral joint features to complete the final hyperspectral image classification task using a classification head. Specifically, this includes: 1. Initialize the pre-training parameters.

[0047] 2. Extracting Spatial-Spectral Features: Using GSW, important features of specific categories in key bands and their spatial neighborhoods are extracted. .

[0048] 3. Feature encoding: [This refers to the process of encoding features.] Divided into Non-overlapping small blocks ,in The image patch size, representing the spatial division, is flattened and then linearly mapped to generate smaller patch labels. Introduce learnable classification labels All positions are encoded and then used as input to the encoder to achieve high-level feature modeling and extraction of latent discriminative semantics. .

[0049] 4. Classification loss: Encoder output The high-dimensional features are mapped to the final class probability distribution using a classification head. The cross-entropy loss function is used to measure the difference between the model's predictions and the true labels. ; in, These are labels for the real category. It is the class probability distribution predicted by the model. It represents the number of categories.

[0050] Example 1 The above method is used to classify the Indian Pines dataset, specifically including: Data preprocessing: Extract 27×27 pixel blocks.

[0051] Pre-training phase: Epoch=300, 80% mask ratio.

[0052] Encoder: 3-head self-attention, feature dimension 128.

[0053] Fine-tuning phase: Epoch=200.

[0054] Training parameters: Adam optimizer (lr=0.03), Batch Size=512.

[0055] Results: In class 4 (Corn), which is spectrally highly similar to classes such as Corn-mintill and Corn-notill, traditional methods struggle due to blurred class boundaries. Our invention achieves a high classification accuracy of 99.08% in this class. For class 8, only 20 samples were randomly selected from 478 samples as training samples, achieving 100% classification accuracy. Furthermore, classes 7, 9, 13, and 16 also achieved completely correct classification on the test set with very few training samples. These results demonstrate that despite significant spectral aliasing and blurred class boundaries in this dataset, our method effectively mitigates inter-class confusion even with low-label sample scenarios, exhibiting strong discriminative ability and generalization performance.

[0056] Example 2 Applying the Pavia University dataset: Results: The model achieved an average classification accuracy of 96.89%, with more than half of the land cover categories achieving an accuracy higher than 99%. Even under unlabeled pre-training conditions, the model was able to construct a highly discriminative spatial-spectral joint representation and achieve good classification accuracy for complex land covers.

[0057] Therefore, the present invention adopts the above-mentioned self-supervised hyperspectral image classification method suitable for low-label sample scenarios, which realizes lossless transfer of features of difficult samples, enhances the discriminative feature expression of mixed pixels, and improves the classification balance of few sample categories.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A self-supervised hyperspectral image classification method suitable for low-label sample scenarios, characterized in that, Includes the following steps: S1. Acquire raw hyperspectral image blocks Furthermore, a self-supervised category perception network for low-label scenarios is constructed, including a grouped spectral enhancement module, a spectral self-attention module, a mask reconstruction, and a fine-tuning stage. S2. During the pre-training phase, guide the model to focus on category-sensitive spatial-spectral features under unlabeled conditions; S21, Based on the original hyperspectral image patch By using the grouped spectral enhancement module, spatial-spectral features that combine key spatial structure and class responsiveness are generated; S22. Based on spatial-spectral features, the non-local dependencies between spectral bands are captured through a spectral self-attention module, enhancing the cross-spectral discrimination capability and outputting spatial-spectral joint features that fuse global dependencies of spectral bands and spatial structure. S23. Through mask reconstruction, the spatial-spectral joint features are divided into N small blocks of the same size, some small blocks are randomly masked, and learnable classification labels and location information are added to generate masked labels, unmasked labels and classification labels. S24. Input the classification label and the unmasked label together into the encoder for feature encoding to generate high-level semantic latent features; S25. Input the high-level semantic latent features and mask tags output by the encoder into the decoder to reconstruct the occluded region and generate a reconstructed image. S26. By calculating the difference between the reconstructed image and the original occluded region, the model is guided to optimize its feature extraction capability under unlabeled supervision, thereby updating the network parameters. S3. In the fine-tuning stage, the pre-trained network parameters are used as initialization parameters, and the feature expression is further refined through classification loss.

2. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, In S1, the original hyperspectral image patch The space size is The number of spectral channels is ,Right now .

3. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, S21 includes: S211, Through Convolution pairs of raw hyperspectral data Perform spectral channel adjustment; the adjusted number of spectral channels is: ,and Set to an integer multiple of the preset group number; S212. Different sizes of grouped convolutions are used sequentially to preserve the local spatial structure and spectral channel interaction characteristics at different scales. Then, through batch normalization and nonlinear activation operations, local spatial features are obtained. S213. Global features are extracted through global average pooling, and channel weights are learned through a two-layer fully connected network to obtain the weights of important channel features. Then, local spatial features are obtained through residual connections to obtain enhanced spatial-spectral features.

4. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 3, characterized in that, In S213, the weights of important channel features are as follows: ； In the formula, As important channel feature weights, , This is the weight matrix. As a global feature, This is the corresponding activation function.

5. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, S22 includes: S221. Flatten the spatial-spectral features to obtain ,right Similarity calculation is performed on any two channels to obtain : ； S222, will Multiplying the value matrix yields the attention output, which is then used to perform residual connections. ； S223, will Restored to its original shape, spatial-spectral joint features were obtained. .

6. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, S24 involves concatenating the classification label and the unmasked label, and then generating high-level semantic latent features through a Transformer encoder, as follows: ； ； In the formula, This represents the concatenated input features. Indicates classification label, Indicates an unmasked flag. Represents the mapping function of the encoder. It represents the latent features of high-level semantics.

7. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, In S25, the process of reconstructing the image includes: ； in, Indicates the reconstructed image. Indicates to mark the mask It is filled into the high-level semantic latent features and used as input to the decoder. This represents the mapping function of the decoder.

8. The self-supervised hyperspectral image classification method suitable for low-label sample scenarios according to claim 1, characterized in that, S3 include: S31. Initialize pre-training parameters; S32. Extract important features of specific categories in key bands and their spatial neighborhoods through the grouped spectral enhancement module, i.e., spatial-spectral features. ; S33, will Divided into Non-overlapping small blocks ,in The image patch size, representing the spatial division, is flattened and then linearly mapped to generate smaller patch labels. Introducing classification tags All positions are encoded and then used as input to the encoder for high-level feature modeling and latent discriminative semantic extraction, as follows: ； In the formula, Represents the mapping function of the encoder. Represents high-level semantic latent features; S33, Encoder Output The high-dimensional features are mapped to the final class probability distribution through the classification head, and the cross-entropy loss function is used to measure the difference between the model prediction and the true label. ； in, These are labels for the real category. It is the class probability distribution predicted by the model. It represents the number of categories.

Images