A hyperspectral image classification method based on spectral feature reconstruction

Abstract

The present invention discloses a hyperspectral image classification method based on spectral feature reconstruction, which belongs to the technical field of hyperspectral image processing. The method comprises the following steps: S1, collecting original hyperspectral image blocks; S2, constructing a spectral space feature reconstruction reversible fusion network, including FIR, SSIF, SP and TE; S3, inputting FIR to generate reconstructed features; S4, inputting the reconstructed features to SSIF to generate enhanced features; S5, inputting the enhanced features to SP to generate a semantic tag sequence; S6, inputting the semantic tag sequence to TE to generate; The hyperspectral image classification method based on spectral feature reconstruction provided by the present invention realizes the lossless transmission of difficult sample features, enhances the discriminative feature expression of mixed pixels and improves the classification balance of few sample categories.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and in particular to a hyperspectral image classification method based on spectral feature reconstruction. Background Art

[0002] Hyperspectral imagery (HSI), with its rich spectral information and spatial detail, plays a key role in geological exploration, agricultural monitoring, environmental remote sensing, and other fields. Accurate image classification is crucial for realizing its application value. However, current HSI classification technology still faces many core challenges that need to be addressed in practical applications.

[0003] First, the imbalanced distribution of classes significantly hinders classification performance. In HSI data, the number of samples for some key features (such as rare minerals) is small, significantly outstripping the number of samples for the majority class. This imbalanced distribution can cause the classifier to overfit the characteristics of the majority class samples during training, significantly reducing its ability to identify minority class samples. This makes it difficult to accurately detect and classify key features, seriously impacting the effectiveness of HSI applications in resource exploration, ecological protection, and other fields.

[0004] Secondly, mixed pixel interference is another major obstacle to HSI classification. At the edge of the HSI, the spectral characteristics of different objects intermingle, blurring the boundaries and forming mixed pixels. Traditional classification methods have difficulty effectively distinguishing the characteristics of different objects in mixed pixels.

[0005] Furthermore, feature loss during deep network training is a prominent issue. While deep neural networks possess powerful feature extraction capabilities when processing complex HSI data, during training, the effective features of difficult samples with complex and indistinguishable spectral characteristics are easily corrupted by noise or lost during multi-layer network transmission, making them unable to provide a reliable basis for classification decisions. This significantly limits the performance of the classification model.

[0006] Existing HSI classification technologies have obvious shortcomings in addressing the above challenges. Traditional methods such as SVM and PCA can only utilize shallow spectral features, completely ignoring the value of image spatial information, and are unable to meet the needs of high-precision HSI classification. Although 3D-CNN attempts to capture spatial and spectral information, its computational complexity is extremely high. When processing HSI containing long sequences of spectral data, it is difficult to achieve efficient spectral dependency modeling. Deep learning methods that have emerged in recent years also have limitations. For example, MorphFormer (Reference: SK Roy et al., IEEE TGRS, 2023) enhances spatial feature extraction through morphological operations, but its ability to retain key features is insufficient when processing small sample data. SSFTT (Reference: L. Sun et al., IEEE TGRS, 2022) combines CNN and Transformer architectures for feature fusion. However, its irreversible fusion method leads to a large loss of key information in mixed pixels, seriously affecting the classification effect.

[0007] In summary, existing HSI classification technology is difficult to effectively solve problems such as uneven category distribution, mixed pixel interference, and feature transmission loss. A new classification method and system is urgently needed to improve the accuracy and reliability of HSI classification. Summary of the Invention

[0008] The purpose of the present invention is to provide a hyperspectral image classification method based on spectral feature reconstruction to solve the problems existing in the above background technology.

[0009] To achieve the above object, the present invention provides a hyperspectral image classification method based on spectral feature reconstruction, comprising the following steps:

[0010] S1. Collect original hyperspectral image blocks ;

[0011] S2. Construct a spectral-spatial feature reconstruction and reversible fusion network, including a feature information reconstruction module, a spectral-spatial reversible fusion module, a semantic tagging and position embedding module, and a Transformer encoder module.

[0012] S3. Input feature information reconstruction module to generate reconstructed features ;

[0013] S4, reconstruct features Input the spectral-spatial reversible fusion module, which realizes the dynamic interaction of spectral and spatial features through the reversible structure to generate enhanced features ;

[0014] S5. Input semantic tag and position embedding module, map shallow features into deep semantic tags, and introduce position information to generate semantic tag sequence ;

[0015] S6. Input Transformer encoder module, capture global dependencies through self-attention mechanism, and generate .

[0016] Preferably, the original hyperspectral image block in step S1 for:

[0017] ;

[0018] in, is the window size, is the number of spectral channels.

[0019] Preferably, step S3 specifically includes:

[0020] S31, Spectral channel interaction: Use 1×1 point convolution to expand the spectral channel to 256 dimensions to enhance the information interaction between different bands, as follows:

[0021] ;

[0022] S32, local spatial feature extraction: Use 3×3 convolution to extract local spatial features:

[0023] ;

[0024] S33, feature splicing: splicing the original input and convolution results along the channel dimension to generate 768-dimensional reconstructed features :

[0025] .

[0026] Preferably, the output dimensions in step S31, step S32 and step S33 are respectively: 、 、 .

[0027] Preferably, step S4 specifically includes:

[0028] S41, three-branch split: It is divided into three sub-parts along the channel dimension:

[0029] ;

[0030] S42, set bottleneck feature extraction unit, 、 、 Perform feature extraction and output is , specifically:

[0031] S421, perform 3×3 depth-wise separable convolution on each sub-part, as follows:

[0032] ;

[0033] Among them, the number of convolution kernels of DWConv is 64, the stride is 1, and the output dimension is consistent with the input;

[0034] S422, through the attention weight matrix Enhance important features:

[0035] ;

[0036] in, is the number of channels;

[0037] Add the enhanced features to the original input:

[0038] ;

[0039] S43, concatenate the outputs of the three sub-parts along the channel dimension, and reduce the dimension to the original spectral channel number B through 1×1 point convolution, and compare the dimension reduction result with the original input. Add, preserving the original information:

[0040] ;

[0041] .

[0042] Preferably, step S5 specifically includes:

[0043] S51, Feature Flattening:

[0044] ;

[0045] S52. Define two weight matrices that obey Gaussian distribution ,in, is the number of markers;

[0046] Generate semantic markup :

[0047] ;

[0048] ;

[0049] ;

[0050] S53. Introducing learnable classification labels and concatenated with semantic markup:

[0051] ;

[0052] Add trainable position embeddings :

[0053] ;

[0054] And apply Dropout to prevent overfitting:

[0055] .

[0056] Preferably, step S6 specifically includes:

[0057] S61. Multi-head self-attention: Initializing the query ,key ,value Matrix, where the number of heads , the dimensions of each head =16:

[0058] ;

[0059] Single-head attention calculation:

[0060] ;

[0061] Multiple result splicing:

[0062] ;

[0063] in, is the output projection matrix;

[0064] S62, cross-layer adaptive fusion: output the current layer With the Layer output Fusion:

[0065] ;

[0066] in, is a learnable parameter with an initial value of 0.5;

[0067] S63, Multilayer Perceptron contains two fully connected layers and GELU activation function:

[0068] ;

[0069] in, 、 Represent the weight matrix respectively; 、 Indicates bias.

[0070] Therefore, the present invention adopts the above-mentioned hyperspectral image classification method based on spectral feature reconstruction, which has the following beneficial effects:

[0071] (1) Achieve lossless interaction of spectral and spatial features through the SSIF module, reducing the loss of mixed pixel information;

[0072] (2) Feature weighting mechanism based on Hadamard product to improve the distinguishability of difficult samples;

[0073] (3) The CAF strategy strengthens inter-layer feature transfer and alleviates the gradient vanishing problem;

[0074] (4) Computational efficiency: Depthwise separable convolution (DWConv) reduces the number of parameters.

[0075] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a schematic diagram of the structure of a reversible fusion network for spectral spatial feature reconstruction according to an embodiment of the present invention;

[0077] Figure 2 Schematic diagram of the three-branch structure and bottleneck feature extraction unit details of the spectral-spatial reversible fusion module according to an embodiment of the present invention;

[0078] Figure 3 Schematic diagram of the fusion of multi-head self-attention and CAF in the Transformer encoder module of an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0080] A hyperspectral image classification method based on spectral feature reconstruction, comprising:

[0081] Collect raw hyperspectral image patches ,in, is the window size, is the number of spectral channels.

[0082] Construct spectral spatial feature reconstruction reversible fusion network (SSRIFT), such as Figure 1 As shown, including:

[0083] 1. Feature Information Reconstruction Module (FIR): Enhances the expressiveness of spectral-spatial features and solves the problem of sparse features in small samples.

[0084] Input: original hyperspectral image patch ;

[0085] Objective: Reconstruct highly discriminative features through spectral channel interaction and spatial feature extraction.

[0086] Specifically include:

[0087] 1. Spectral channel interaction: Use 1×1 point convolution (PWConv1) to expand the spectral channel to 256 dimensions to enhance information interaction between different bands:

[0088] ;

[0089] Among them, the number of convolution kernels of PWConv1 is 256 and the stride is 1.

[0090] Output dimensions: .

[0091] 2. Local spatial feature extraction: Use 3×3 convolution (Conv) to extract local spatial features, with 256 convolution kernels and a step size of 1:

[0092] ;

[0093] Output dimensions: .

[0094] Activation function: ReLU (to suppress gradient explosion), followed by batch normalization (BatchNorm).

[0095] 3. Feature splicing: Splice the original input and convolution result along the channel dimension to generate 768-dimensional reconstructed features :

[0096] .

[0097] Output dimensions: By fusion of multi-scale features, the feature density of few-sample categories is enhanced.

[0098] 2. Spectral Spatial Reversible Fusion Module (SSIF): It realizes lossless feature transfer through reversible network structure and retains mixed pixel information, such as Figure 2 shown.

[0099] enter: ;

[0100] Objective: To achieve dynamic interaction between spectral and spatial features through reversible structures and avoid feature loss.

[0101] Specifically include:

[0102] 1. Three-branch segmentation: Divided into three sub-parts along the channel dimension

[0103] ;

[0104] 2. Bottleneck Feature Extraction Unit (BFE):

[0105] (1) Depthwise Separable Convolution (DWConv): Perform 3×3 depthwise separable convolution on each sub-part to reduce the number of parameters while extracting local features:

[0106] ;

[0107] Among them, the number of convolution kernels of DWConv is 64, the stride is 1, and the output dimension is consistent with the input;

[0108] (2) Feature enhancement and suppression:

[0109] Hadamard product ( ): Through the attention weight matrix Enhance important features:

[0110] ;

[0111] in, is the number of channels;

[0112] Pixel-by-pixel addition (+): Add the enhanced features to the original input to suppress noise:

[0113] ;

[0114] (3) Feature fusion and dimensionality reduction: The outputs of the three sub-parts are concatenated along the channel dimension and reduced to the number of original spectral channels B through 1×1 point convolution (PWConv2):

[0115] ;

[0116] ;

[0117] Residual connection: Combine the dimensionality reduction result with the original input Add, preserving the original information:

[0118] .

[0119] By achieving lossless feature transfer through a reversible structure, the classification accuracy of mixed pixel samples is improved by 12.6%.

[0120] 3. Semantic Labeling and Position Embedding Module (SP): Maps shallow features into deep semantic labels and fuses them with position information.

[0121] Input: Output of SSIF module ;

[0122] Goal: Map shallow features into deep semantic tags and introduce position information.

[0123] Specifically include:

[0124] 1. Feature flattening:

[0125] ;

[0126] 2. Gaussian weight mapping:

[0127] Define two weight matrices that obey Gaussian distribution ,in, is the number of markers;

[0128] Generate semantic markup :

[0129] ;

[0130] ;

[0131] ;

[0132] 3. CLS Token and location embedding:

[0133] Introducing learnable classification labels and concatenated with semantic markup:

[0134] ;

[0135] Add trainable position embeddings :

[0136] ;

[0137] And apply Dropout (ratio 0.1) to prevent overfitting:

[0138] .

[0139] 4. Transformer Encoder Module (TE): Captures global sequence dependencies and optimizes feature flow in combination with cross-layer adaptive fusion (CAF), such as Figure 3 shown.

[0140] enter: ;

[0141] Objective: Capture global dependencies through self-attention mechanism and optimize classification features.

[0142] Specifically include:

[0143] 1. Multi-head self-attention (MSA): Initialize query ,key ,value Matrix, where the number of heads , the dimensions of each head =16:

[0144] ;

[0145] Single-head attention calculation:

[0146] ;

[0147] Multiple result splicing:

[0148] ;

[0149] in, is the output projection matrix;

[0150] 2. Cross-layer adaptive fusion (CAF): Output the current layer With the Layer output Fusion:

[0151] ;

[0152] in, is a learnable parameter with an initial value of 0.5;

[0153] S63, Multilayer Perceptron (MLP) contains two fully connected layers (dimensions 64→256→64) and GELU activation function:

[0154] ;

[0155] in, 、 Represent the weight matrix respectively; 、 Indicates bias.

[0156] Example 1

[0157] The above method is used to classify the Indian Pines dataset, specifically including:

[0158] Data preprocessing: intercept 7×7 pixel blocks.

[0159] Network construction: FIR module: 1×1 convolution (256 kernels) → 3×3 convolution (256 kernels) → splicing.

[0160] SSIF module: segmentation ratio =256, =512, number of DWConv cores 64.

[0161] TE module: 4-head self-attention, hidden layer dimension 64.

[0162] Training parameters: Adam optimizer (lr=5e-4), Batch Size=64, Epoch=400.

[0163] For example, there are only two training samples for categories 7 and 9, but the classification accuracy reached 100% on the test samples. Good classification accuracy can also be achieved in categories with uneven sample distribution.

[0164] Example 2

[0165] Applying a small sample scenario in Botswana

[0166] Improvements: The number of tokens in the SP module is n=64n=64, and the CAF fusion weight is dynamically adjusted.

[0167] Results: OA achieved 99.16%, a 0.26% improvement over MorphFormer. On this dataset, the sample distribution is relatively even, but the number of training samples is small, with only 4 to 15 samples per class. Even with this small sample size, high classification accuracy is achieved.

[0168] Therefore, the present invention adopts the above-mentioned hyperspectral image classification method based on spectral feature reconstruction to achieve lossless transmission of difficult sample features, enhance the discriminative feature expression of mixed pixels, and improve the classification balance of small sample categories.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A hyperspectral image classification method based on spectral feature reconstruction, characterized in that: The following steps are involved: S1. Collect original hyperspectral image blocks ; S2. Construct a spectral-spatial feature reconstruction and reversible fusion network, including a feature information reconstruction module, a spectral-spatial reversible fusion module, a semantic tagging and position embedding module, and a Transformer encoder module. S3. Input feature information reconstruction module to generate reconstructed features ; S4, reconstruct features Input the spectral-spatial reversible fusion module, which realizes the dynamic interaction of spectral and spatial features through the reversible structure to generate enhanced features ; S5. Input semantic tag and position embedding module, map shallow features into deep semantic tags, and introduce position information to generate semantic tag sequence ; S6. Input Transformer encoder module, capture global dependencies through self-attention mechanism, and generate ; Step S4 specifically includes: S41, three-branch split: It is divided into three sub-parts along the channel dimension: ； S42, set bottleneck feature extraction unit, 、 、 Perform feature extraction, specifically: S421, perform 3×3 depth-wise separable convolution on each sub-part, as follows: ； Among them, the number of convolution kernels of DWConv is 64, the stride is 1, and the output dimension is consistent with the input; S422, through the attention weight matrix Enhance important features: ； in, is the number of channels; Add the enhanced features to the original input: ； S43, concatenate the outputs of the three sub-parts along the channel dimension and reduce the dimension to the original spectral channel number B through 1×1 point convolution, which is: ； ； Compare the dimensionality reduction result with the original input Add, preserving the original information: 。 2. The hyperspectral image classification method based on spectral feature reconstruction according to claim 1 is characterized in that: The original hyperspectral image block in step S1 for: ； in, is the window size, is the number of spectral channels.

3. The hyperspectral image classification method based on spectral feature reconstruction according to claim 1 is characterized in that: Step S3 specifically includes: S31, Spectral channel interaction: Use 1×1 point convolution to expand the spectral channel to 256 dimensions to enhance the information interaction between different bands, as follows: ； S32, local spatial feature extraction: Use 3×3 convolution to extract local spatial features: ； S33, feature splicing: splicing the original input and convolution results along the channel dimension to generate 768-dimensional reconstructed features : 。 4. The hyperspectral image classification method based on spectral feature reconstruction according to claim 3 is characterized in that: The output dimensions in step S31, step S32 and step S33 are: 、 、 .

5. The hyperspectral image classification method based on spectral feature reconstruction according to claim 1 is characterized in that: Step S5 specifically includes: S51, Feature Flattening: ； S52. Define two weight matrices that obey Gaussian distribution ,in, is the number of markers; Generate semantic markup : ； ； ； S53. Introducing learnable classification labels and concatenated with semantic markup: ； Add trainable position embeddings : ； And apply Dropout to prevent overfitting: 。 6. The hyperspectral image classification method based on spectral feature reconstruction according to claim 1, characterized in that: Step S6 specifically includes: S61. Multi-head self-attention: Initializing the query ,key ,value Matrix, where the number of heads , the dimensions of each head =16: ； Single-head attention calculation: ； Multiple result splicing: ； in, is the output projection matrix; S62, cross-layer adaptive fusion: output the current layer With the Layer Output Fusion: ； in, is a learnable parameter with an initial value of 0.5; S63, Multilayer Perceptron contains two fully connected layers and GELU activation function: ； in, 、 Represent the weight matrix respectively; 、 Indicates bias.

Images