A self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios

Abstract

This invention discloses a self-supervised hyperspectral image classification method suitable for low-label sample scenarios, relating to the field of hyperspectral remote sensing image processing technology. It includes a self-supervised category-aware network for low-label scenarios. In the pre-training stage, a grouped spectral enhancement module, a spectral self-attention module, and mask reconstruction are employed. By minimizing the difference between the reconstructed image and the original masked region, the model is guided to focus on category-sensitive spatial-spectral features under unlabeled conditions. In the fine-tuning stage, the pre-trained network parameters are used as initialization parameters, and the feature expression is further refined through classification loss. Therefore, this self-supervised hyperspectral image classification method suitable for low-label sample scenarios achieves lossless transfer of features from difficult samples, enhances the discriminative feature expression of mixed pixels, and improves the classification balance of few sample classes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral remote sensing image processing, and in particular to a self-supervised hyperspectral image classification method suitable for a low-labeled sample scene. BACKGROUND

[0002] Hyperspectral image (HSI) has dozens to hundreds of continuous and narrow spectral bands, which can simultaneously capture rich spectral and spatial information of ground objects, and is widely used in fine agricultural monitoring, mineral resource exploration, urban land use analysis, ecological environment assessment and other fields. Its high spectral resolution provides an unprecedented fine-grained description capability for feature material identification and classification, and has become a key data source in remote sensing intelligent interpretation tasks. However, the current HSI classification technology still faces many core challenges to be solved in practical application.

[0003] Firstly, the problem of labeled sample scarcity is prominent. Due to the high complexity of hyperspectral images in spectral dimension and spatial range, it is usually necessary for domain experts to manually label according to field investigation or auxiliary data, which has high labeling cost and low efficiency, and seriously restricts the acquisition of high-quality training samples. The existing mainstream supervised classification methods are severely dependent on a large number of high-quality labeled samples, and are prone to overfitting under the condition of limited training data, which affects the generalization ability of the model in actual scenes. For example, although traditional SVM, Random Forest, and supervised network based on 3D-CNN perform well in full supervision, their accuracy significantly decreases under the condition of limited samples.

[0004] Secondly, the class distinction ability is insufficient. Different ground object classes in hyperspectral images have high similarity in some bands, and the spectral difference between classes is small. The redundancy and noise of the superimposed spectral data itself make it difficult for the model to effectively distinguish similar classes. For example, the classic ResNet and DenseNet do not explicitly consider spectral channel selection in the HSI scene, which may lead to insufficient response to different class bands and blurred boundaries.

[0005] Thirdly, the spectral band dependence modeling ability is weak. Existing methods mostly compress high-dimensional spectral information based on local convolution, dimensionality reduction mapping or principal component analysis, which is difficult to explicitly capture long-distance dependence between spectral bands. For example, although some Transformer structures (such as ViT-HSI) introduce global modeling capability, they do not adequately describe the semantic dependence of spectral bands, especially without the guidance of position encoding or spectral perception mechanism to accurately capture the combined features between key channels.

[0006] In addition, the self-supervised learning method emerging in recent years shows strong feature modeling potential without manual annotation, such as MAE, SimCLR-HSI and other methods have been introduced into the hyperspectral classification task. However, most self-supervised strategies focus on the reconstruction or contrast of general image structure, lack of guiding mechanism for class-sensitive spectral bands in hyperspectral data, and it is difficult to realize fine class difference modeling and enhancement.

[0007] Therefore, at present, the hyperspectral image classification task urgently needs a new network structure that can fully mine key discriminant spatial features and class-sensitive spectral band information under the condition of low-labeled samples. SUMMARY

[0008] The purpose of the present application is to provide a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios to solve the problems in the background art.

[0009] To achieve the above purpose, the present application provides a self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios, comprising the following steps:

[0010] S1, collecting an original hyperspectral image block , and constructing a self-supervised class-aware network for low-labeled scenarios, including a grouping spectral enhancement module, a spectral self-attention module, a mask reconstruction and a fine-tuning stage;

[0011] S2, in the pre-training stage, guiding the model to focus on class-sensitive spatial-spectral features under the condition of no label;

[0012] S21, based on the original hyperspectral image block , through the grouping spectral enhancement module, spatial-spectral features with key spatial structure and class responsiveness are generated;

[0013] S22, based on the spatial-spectral features, through the spectral self-attention module, the non-local dependence between spectral bands is captured, the discriminant ability across spectral bands is enhanced, and the spatial-spectral joint features integrating the global dependence of spectral bands and spatial structure are output;

[0014] S23, by mask reconstruction, the spatial-spectral joint features are divided into N small blocks of the same size, part of the small blocks are randomly masked, and the mask label, the unmasked label and the classification label are generated by adding the learnable classification label and the position information;

[0015] S24, the classification label and the unmasked label are jointly input into the encoder for feature encoding to generate high-level semantic latent features;

[0016] S25, the high-level semantic latent features output by the encoder and the mask label are jointly input into the decoder to reconstruct the masked area, and the reconstructed image is generated;

[0017] S26, guiding the model to optimize the feature extraction ability under the condition of no label supervision by calculating the difference between the reconstructed image and the original shaded area, and then updating the network parameters;

[0018] S3, in the fine-tuning stage, taking the pre-trained network parameters as the initialization parameters, and further refining the feature expression through the classification loss.

[0019] Preferably, in S1, the original hyperspectral image block has a spatial size of and a spectral channel number of , that is, .

[0020] Preferably, S21 comprises:

[0021] S211, adjusting the spectral channel of the original hyperspectral data by convolution, the adjusted spectral channel number being , and being set as an integer multiple of the preset grouping number;

[0022] S212, sequentially adopting group convolutions of different sizes to reserve the local spatial structure and the interaction characteristics of the spectral channels of different scales, and then obtaining the local spatial features through batch normalization processing and nonlinear activation operation;

[0023] S213, extracting the global features through global average pooling, learning the channel weight through two-layer fully connected network, obtaining the important channel feature weight, and then obtaining the enhanced spatial-spectral features through residual connection of the local spatial features.

[0024] Preferably, in S213, the important channel feature weight is as follows:

[0025] ;

[0026] In the formula, is the important channel feature weight, , is the weight matrix, is the global feature, is the corresponding activation function.

[0027] Preferably, S22 comprises:

[0028] S221, flattening the spatial-spectral features to obtain , and calculating the similarity between any two channels in to obtain :

[0029] ​ ;

[0030] S222, will Multiplying the value matrix yields the attention output, which is then used to perform residual connections.

[0031] ;

[0032] S223, will Restored to its original shape, spatial-spectral joint features were obtained. .

[0033] Preferably, S24 includes concatenating the classification label and the unmasked label, and then generating high-level semantic latent features through a Transformer encoder, as follows:

[0034] ;

[0035] ;

[0036] In the formula, This represents the concatenated input features. Indicates classification label, Indicates an unmasked flag. Represents the mapping function of the encoder. It represents the latent features of high-level semantics.

[0037] Preferably, in S25, the image reconstruction process includes:

[0038] ;

[0039] in, Indicates the reconstructed image. Indicates to mark the mask It is filled into the high-level semantic latent features and used as input to the decoder. This represents the mapping function of the decoder.

[0040] Preferably, S3 includes:

[0041] S31. Initialize pre-training parameters;

[0042] S32. Extract important features of specific categories in key bands and their spatial neighborhoods through the grouped spectral enhancement module, i.e., spatial-spectral features. ;

[0043] S33, will Divided into Non-overlapping small blocks ,in The image patch size, representing the spatial division, is flattened and then linearly mapped to generate smaller patch labels. , introduce classification labels , all position encodings are taken as inputs of the encoder to perform high-level feature modeling and potential discriminative semantic extraction as follows:

[0044] ;

[0045] wherein, represents a mapping function of the encoder, represents a high-level semantic potential feature;

[0046] S33, the encoder outputs , the high-dimensional features are mapped into final category probability distribution through a classification head, and a cross-entropy loss function is used to measure the difference between model prediction and real label:

[0047] ;

[0048] wherein, is a label of a real category, is a category probability distribution predicted by the model, is the number of categories.

[0049] Therefore, the self-supervised hyperspectral image classification method suitable for the low-labeled sample scene has the following technical effects:

[0050] (1) Improve the category discrimination ability: the present application combines the grouping feature enhancement mechanism with the spectral self-attention module to effectively extract the discriminative spatial-spectral features strongly related to the category and improve the discrimination degree between similar categories.

[0051] (2) Strengthen the global modeling between spectral bands: the present application introduces a spectral self-attention structure to explicitly model the long-distance dependence relationship between spectral bands, breaking the limitation of traditional convolutional networks on local feature modeling.

[0052] (3) Reduce the dependence on labeled data: the present application uses a mask reconstruction task to realize self-supervised feature learning, which can pre-train the model on unlabeled samples and maintain high classification performance in low-labeled scenarios.

[0053] (4) Accelerate model convergence: the present application obtains discriminative features through the pre-training stage to provide high-quality initialization for fine-tuning, thereby significantly accelerating the convergence speed of the downstream classification task.

[0054] The technical solutions of the present application will be further described in detail below through the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1It is a kind of self-supervised hyperspectral image classification method suitable for low-labeled sample scene.

[0056] Figure 2 It is a kind of self-supervised hyperspectral image classification method suitable for low-labeled sample scene. DETAILED DESCRIPTION

[0057] The purpose of disclosing the present application is to protect all changes and improvements within the scope of the present application, and the present application is not limited to the following examples.

[0058] The present application provides a kind of self-supervised hyperspectral image classification method suitable for low-labeled sample scene, it is low-labeled scene-oriented self-supervised class perception network, as shown in Figure 1 The input of the network is the original hyperspectral image block ( Figure 1 Red box part in) , Space size, Spectrum channel number; The target is to realize the spatial-spectral joint feature extraction of class perception by group feature enhancement.

[0059] Specifically, the low-labeled scene-oriented self-supervised class perception network includes four parts of grouping spectral enhancement module (GSW), spectral self-attention module (SSA), mask reconstruction and fine-tuning stage.

[0060] I. Grouping spectral enhancement module (as shown in Figure 2 ): adaptively extract important features of specific classes in key wavebands and their spatial neighborhoods, effectively suppress redundant information interference, as follows:

[0061] 1. Spectral channel adjustment: adjust the original hyperspectral data by convolution:

[0062] ;

[0063] Output dimension: , wherein Indicates the adjusted spectral channel number, so that Can be divided by the preset grouping number, so as to adapt to the subsequent grouping operation.

[0064] 2. Local spatial feature extraction: in turn adopt Grouping convolution and Grouping convolution, the local spatial structure of different scales and the interaction characteristics of spectral channels are reserved:

[0065] ;

[0066] Output dimension: where is the dimension after grouped convolution processing.

[0067] 3. Stable network: The output is obtained by batch normalization processing (BN) and nonlinear activation (ReLU) operation :

[0068] .

[0069] 4. Global feature extraction: Global average pooling (GAP) is used to extract global features

[0070] ;

[0071] Output dimension: where denotes the batch size.

[0072] 5. Important feature learning: Channel weight learning is performed through a two-layer fully connected network to obtain important channel feature weights, which are

[0073] ;

[0074] where, denotes the Sigmoid function, which is used to rescale the spectrum weight between 0 and 1. denotes the convolution operation, and denotes the weights of the two fully connected layers, which is set to 16.

[0075] Output dimension: .

[0076] 6. Reinforced features: The weights are multiplied element-wise with to obtain spatial-spectral features that are responsive to categories, and key information is enhanced while redundant information is suppressed.

[0077] II. Spectral self-attention module: Modeling long-range dependencies between spectral bands.

[0078] Input: .

[0079] Objective: Input the class-responsive spatial-spectral features into the spectral segment self-attention module, fuse the inter-spectral global dependency and spatial local information, and generate a joint feature representation with class sensitivity and spatial discriminability, which includes:

[0080] 1. Similarity calculation: Calculate the similarity between and .

[0081] .

[0082] 2. Multiply and to get the attention output, and perform residual connection:

[0083] ;

[0084] Output dimension: .

[0085] 3. Feature restoration: Restore to its original shape, fuse the global dependency between spectral segments and local spatial distribution information, and generate spatial-spectral joint features with class sensitivity and spatial discriminability .

[0086] Three, mask reconstruction: Guide the model to focus on class-sensitive spatial-spectral joint features under the condition of no label.

[0087] Input: .

[0088] Objective: Use the mask reconstruction task as a self-supervised training method, mask part of the input and restore its content, guide the model to learn class-sensitive spatial-spectral discriminative features on unlabeled data, which includes:

[0089] 1. Divide the spatial-spectral joint feature into N small blocks , and perform random masking operation to input semantic labels and position embedding module, map shallow features to deep semantic labels, add a learnable classification label, and introduce position information, to generate mask labels , unmasked labels and classification labels ,

[0090] 2. Feature encoding: Unmasked labels and classification labels ​together into the Transformer encoder for spectral-spatial feature extraction to obtain latent representation:

[0091] ;

[0092] ;

[0093] wherein, represents the mapping function of the encoder; , is the number of unmasked small blocks, is the output dimension of the encoder.

[0094] 3. Decoder reconstruction: the encoder output is input into the decoder together with the mask label to be mapped into the reconstruction result:

[0095] ;

[0096] wherein, represents filling the mask label into the encoder output data to be input into the decoder together; represents the mapping function of the decoder.

[0097] Output dimension: .

[0098] 4. Calculate the reconstruction loss: remove from , and calculate the difference between the reconstruction data and the original data at the same pixel position:

[0099] ;

[0100] wherein, and are the reconstruction pixel value and the original pixel value at the same position.

[0101] Four, fine-tuning stage: on the basis of the parameters in the pre-training stage, a small amount of labeled samples are loaded to further optimize the model in a supervised manner.

[0102] Input: original hyperspectral image block , pre-trained GSW, patch embedding matrix and Transformer encoder parameters.

[0103] Objective: under the guidance of the class label, based on the pre-trained features, further optimize the discriminability of the spatial-spectral joint features, and complete the final hyperspectral image classification task through the classification head, which specifically includes:

[0104] 1. Initialize the pre-training parameters.

[0105] 2. Extract spatial-spectral features: extract important features of specific classes in key wavebands and their spatial neighborhoods by GSW .

[0106] 3. Feature encoding: divide the into non-overlapping small patches , where represents the size of the spatially divided image patches, and the flattened linear mapping generates patch labels , introducing learnable class labels , all position encodings are encoded as inputs to the encoder, realizing high-level feature modeling and extraction of latent discriminative semantics:

[0107] .

[0108] 4. Classification loss: the encoder output is mapped to the final class probability distribution by the classification head. The cross-entropy loss function is used to measure the difference between the model prediction and the true label:

[0109] ;

[0110] where is the label of the true class, is the class probability distribution predicted by the model, is the number of classes.

[0111] Example 1

[0112] The above method is used for Indian Pines data set classification, which specifically includes:

[0113] Data preprocessing: intercept 27x27 pixel blocks.

[0114] Pre-training stage: Epoch=300, 80% mask ratio.

[0115] Encoder: 3-head self-attention, feature dimension 128.

[0116] Fine-tuning stage: Epoch=200.

[0117] Training parameters: Adam optimizer (lr=0.03), Batch Size=512.

[0118] Results: In the class 4 (Corn) and Corn-mintill, Corn-notill, etc. The spectral similarity category, due to the fuzzy boundary of the category, the traditional method is difficult to classify. The invention realizes the high classification accuracy of 99.08% in this class. For class 8, only 20 samples are randomly selected from 478 samples as training samples, and the final classification accuracy reaches 100%. At the same time, classes 7, 9, 13 and 16 also achieve complete correct classification on the test set under the condition of providing only a few training samples. The above results show that although this data set has significant spectral overlap and inter-class boundary ambiguity, the method of the invention can still effectively alleviate the inter-class confusion in the low-labeled sample scene, and has strong discriminant ability and generalization performance.

[0119] Embodiment two

[0120] Apply Pavia University data set:

[0121] Results: The average classification accuracy is 96.89%, more than half of the classification accuracy of the ground object category is higher than 99%. Even under the condition of no label pre-training, the model can still construct a spatial-spectral joint representation with significant discriminability, and can achieve good classification accuracy for complex ground objects.

[0122] Therefore, the invention adopts the above-mentioned self-supervised hyperspectral image classification method suitable for low-labeled sample scene, realizes the lossless transmission of difficult sample features, enhances the discriminative feature expression of mixed pixels, and improves the classification balance of few sample classes.

[0123] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application but not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalent replace the technical solutions of the present application, and these modifications or equivalent replacements also cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A self-supervised hyperspectral image classification method suitable for low-labeled sample scenarios, characterized in that, The method comprises the following steps: S1, collect original hyperspectral image blocks And construct a self-supervised class-aware network for low-labeled scenarios, including a grouping spectral enhancement module, a spectral self-attention module, a mask reconstruction and a fine-tuning stage. S2, in the pre-training stage, guiding the model to focus on the class-sensitive spatial-spectral features under the condition of no label; S21, based on the original hyperspectral image block , through the grouping spectral enhancement module, generate spatial-spectral features with key spatial structure and category responsiveness; S22, based on the spatial-spectral features, the spectral self-attention module is used to capture the non-local dependence between spectral bands, enhance the discriminative ability across spectral bands, and output spatial-spectral joint features that fuse the global dependence of spectral bands and spatial structure; S23, the spatial-spectral joint features are divided into N small blocks of the same size by mask reconstruction, a part of the small blocks are randomly masked, and learnable classification labels and position information are added to generate mask labels, unmasked labels and classification labels; S24, the classification labels and unmasked labels are jointly input into an encoder for feature encoding to generate high-level semantic latent features; S25, the high-level semantic latent features output by the encoder and the mask labels are jointly input into a decoder to reconstruct the masked area and generate a reconstructed image; S26, by calculating the difference between the reconstructed image and the original masked area, the model is guided to optimize the feature extraction ability under the condition of no label supervision, and then the network parameters are updated; S3, in the fine-tuning stage, the pre-trained network parameters are used as the initial parameters, and the feature expression is further refined through the classification loss; S21 comprises: S211、by convolution on the original hyperspectral data spectrum channel adjustment, the number of adjusted spectrum channels is , and is set to an integer multiple of the preset grouping number; S212, different sizes of grouped convolutions are used in sequence to retain the interactive characteristics of local spatial structures and spectral channels of different scales, and then batch normalization processing and nonlinear activation operations are performed to obtain local spatial features; S213, global features are extracted through global average pooling, channel weight learning is performed through a two-layer fully connected network to obtain important channel feature weights, and then the local spatial features are connected through a residual connection to obtain enhanced spatial-spectral features.

2. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, characterized in that, In S1, the original hyperspectral image block has a spatial size of and a spectral channel number of , i.e. .

3. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, characterized in that, In S213, the important channel feature weights are as follows: ； wherein, is an important channel feature weight, , is a weight matrix, is a global feature, is a corresponding activation function.

4. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, characterized in that, S22 comprises: S221, performing flattening processing on the space-spectrum features to obtain , wherein the similarity calculation is performed on any two channels in the space-spectrum features , wherein the similarity calculation is performed on any two channels in the space-spectrum features : ； S222、 the The attention output is obtained by multiplying the value matrix, and a residual connection is performed: ； S223、will recovery to the original shape, get space-spectrum joint features .

5. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, characterized in that, S24 comprises splicing the classification labels and unmasked labels, and then generating high-level semantic latent features through a Transformer encoder, as follows: ； ； In the formula, represents the input feature after splicing, represents the classification label, represents the unmasked label, represents the mapping function of the encoder, represents the high-level semantic latent feature.

6. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, characterized in that, In S25, the process of the reconstructed image comprises: ； wherein, denotes a reconstructed image, denotes marking the mask filling into the high-level semantic latent feature as an input of the decoder, denotes a mapping function of the decoder.

7. The self-supervised hyperspectral image classification method for low-labeled sample scenes according to claim 1, wherein S3 Comprise: S31, initialize the pre-training parameters; S32, extract important features of specific categories in the key waveband and its spatial neighborhood, i.e., spatial-spectral features, through the grouped spectral enhancement module ; S33, will Divided into Non-overlapping small blocks ,in The image patch size, representing the spatial division, is flattened and then linearly mapped to generate smaller patch labels. Introducing classification tags All positions are encoded and then used as input to the encoder for high-level feature modeling and latent discriminative semantic extraction, as follows: ； In the formula, represents a mapping function of the encoder, represents a high-level semantic latent feature; S33, encoder output The high-dimensional features are mapped to the final class probability distribution by the classification head, and the cross-entropy loss function is used to measure the difference between the model prediction and the true label: ； wherein, is a label of a true class, is a model-predicted class probability distribution, is a number of classes.

Images