Space-spectrum cross-scale interaction network and method for hyperspectral image classification
By using a spatial-spectral cross-scale interactive network for hyperspectral image classification, the problem of background pixel interference in hyperspectral image classification is solved, improving the model's ability to identify complex boundaries and the accuracy of foreground classification, making it suitable for practical applications.
Patent Information
- Application Number
- CN202610159636.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies in hyperspectral image classification suffer from inflated performance metrics due to the "no background" classification setting, making it unable to effectively handle issues involving background pixels. Furthermore, the models lack the ability to recognize complex boundaries, leading to a decrease in foreground classification accuracy.
A spatial-spectral cross-scale interactive network for hyperspectral image classification is adopted. Through principal component analysis dimensionality reduction, dual-path feature extraction, multi-scale feature pyramid, and cross-scale interactive module, combined with a multi-round perturbation mechanism guided by spectral-spatial consistency, background pixel interference is suppressed and classification accuracy is improved.
It effectively handles complex boundaries, enhances the model's ability to identify spectral-spatial mixed boundaries, significantly reduces the erosion of foreground classification by background classes, improves foreground classification accuracy, is suitable for real-world scenarios, and has strong generalization and robustness.
Smart Images
Figure CN122067012A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hyperspectral image processing technology, and in particular to a spatial-spectral cross-scale interactive network for hyperspectral image classification, and more specifically to the interaction method of the aforementioned spatial-spectral cross-scale interactive network for hyperspectral image classification. Background Technology
[0002] Hyperspectral images (HSI), acquired by airborne or spaceborne sensors, are three-dimensional data cubes featuring two spatial dimensions and one continuous spectral dimension. Each pixel in an HSI corresponds to a reflectance spectrum spanning hundreds of narrow and continuous wavelength bands, enabling the differentiation of subtle spectral differences between different land cover categories. This has significant value in many fields, including precision agriculture, environmental monitoring, urban planning, and mineral exploration.
[0003] However, state-of-the-art methods in deep learning hyperspectral image classification are evaluated under a "no-background" classification setting, where intrusive background pixels are actively removed during training and testing, leading to artificially inflated performance metrics. We design a novel network for hyperspectral image classification that includes BKG categories. Summary of the Invention
[0004] The technical problem to be solved by this invention is to address the shortcomings of existing technologies by providing a spatial-spectral cross-scale interactive network for hyperspectral image classification. This network can solve the problem that existing technologies lead to inflated performance indicators when evaluated under "no background" classification settings, and cannot effectively handle the classification of real hyperspectral images containing background pixels. It can improve the model's ability to recognize complex boundaries, suppress background pixel interference, and improve classification accuracy.
[0005] The technical problem to be solved by this invention is achieved through the following technical solution. This invention is a spatial-spectral cross-scale interactive network for hyperspectral image classification, comprising the following modules: (1) Data preprocessing module: Principal component analysis is applied to the input hyperspectral image for dimensionality reduction. The image patch size is set to 15×15, and the feature dimension after principal component analysis is set to 30. The image patch centered on the target pixel is then input into the subsequent module. (2) Dual-path low-level feature extraction module: (2.1) Spectral hierarchical encoding module: A multi-level spectral representation is constructed using a hierarchical one-dimensional convolutional structure. The spectral branch contains three consecutive one-dimensional convolutional layers in the channel direction, with the number of channels being 32, 64 and 128, respectively. Each layer follows the order Conv1D→BatchNorm(BN)→ReLU, where Conv1D is a one-dimensional convolution operation, BatchNorm(BN) is a batch normalization operation, and ReLU is an activation function. The features output from the three layers are concatenated along the channel dimension and projected onto a unified embedding dimension D=256 through a (1×1) pointwise convolution to obtain the basic spectral features. (2.2) Spatial multi-kernel context module: A progressive multi-kernel cascade design is adopted. The spatial branch applies three convolutional groups in sequence, with convolutional kernel sizes of 3×3, 5×5 and 7×7 respectively. Each group follows the order of Conv2D→BN→ReLU, where Conv2D is a one-dimensional convolution operation. Intermediate responses are aggregated by element-wise summation and a 1×1 convolution is used to integrate channels and stabilize semantics, while preserving the original channel dimension C to maintain the low-level spatial response aligned with the input spectral channels; (3) Dual-path multi-scale feature pyramid module: (3.1) Spectral pyramid path: The spatial response of the spectrum is modeled at three different spatial scales. At the fine scale, the original spatial resolution (H, W) is maintained and 1×1 convolution is used for channel adjustment. At the medium scale, 1×1 convolution, 3×3 depthwise separable convolution and 2×2 max pooling are applied in sequence. At the coarse scale, 1×1 convolution, 5×5 depthwise separable convolution and global average pooling are applied in sequence. A dual attention mechanism is applied, which enhances the feature discrimination power based on window-based multi-head self attention and channel attention. Each scale outputs a gated feature as the input of the cross-scale interaction module. (3.2) Spatial Pyramid Path: Used for hierarchical spatial feature extraction. At the fine scale, two consecutive 3×3 standard convolutions with a stride of 1 are applied; at the medium scale, spatial abstraction is performed through a convolution with a stride of 2, followed by a refined convolution; at the coarse scale, a convolution with a stride of 2 is performed first, followed by global average pooling; all three scales use channel attention to emphasize discriminative spatial feature channels, and the output forms a spatial feature pyramid. (4) Cross-scale spatial-spectral interaction module: (4.1) Diagonal interaction block: Perform pixel-level deep fusion between spectral and spatial features at the same scale. If the spatial sizes of the two feature maps are different, perform bilinear interpolation on the spatial features to align them with the size of the spectral features. Then, stitch the aligned spectral and spatial features along the channel dimension and adaptively recalibrate the features using the channel attention mechanism. After the weighted features are subjected to 1×1 convolution, batch normalization and ReLU activation, the local context is further fused and the expressive power is enhanced through window multi-head self-attention and position feedforward network. (4.2) Cross-scale block: The information complementarity between features of different scales is achieved through an asymmetric attention mechanism. The key-value features are interpolated and aligned to the query scale. The query, key, and value vectors are generated through independent 1×1 convolutions. Cross-scale attention is calculated and output. (4.3) Modal adaptive fusion: The spectral and spatial pyramids output multi-scale feature sets {s1,s2,s3} and {p1,p2,p3} respectively. First, they are fused at the same scale through diagonal interactive blocks, and then cross-scale compensation is performed through cross-scale blocks. All features are upsampled to the finest scale and averaged to obtain unified cross-scale interactive features, which are reshaped into spatial token sequences and input into the adaptive fusion module in parallel with the spectral token sequences. The dual-modal features are dynamically weighted and fused through learnable scalar coefficients. (5) Spectral-spatial consistency-guided multi-round perturbation module: The feature representation of suspicious background samples is finely adjusted. The baseline fusion features of the network are retained during initialization without disturbance. R rounds of perturbation are executed iteratively. In each round, the consistency weight is calculated, the unified attention fusion features are generated, the original decision style is retained, and the uncertainty-aware weighted pooling is performed to obtain the features after aggregation in this round. The features are then mixed with the original baseline features to obtain the final feature output of this round. (6) Multi-round prospect-priority decision-making module: Based on the baseline prediction, the foreground class with the highest probability is selected as the initial candidate. For each round of perturbation, the corrected features are fed into the classifier to obtain the predicted probability, and the probability of the highest foreground class and its corresponding class index are recorded. When both class consistency and probability threshold conditions are met, the sample is reclassified as foreground. The technical problem to be solved by this invention can also be further achieved through the following technical solution: For the spatial-spectral cross-scale interactive network for hyperspectral image classification described above, the network further includes a loss function module. The loss function consists of weighted cross-entropy loss and spectral-spatial consistency regularization loss, and the total loss is defined as:
[0006] in is a hyperparameter used to balance the strength of the classification loss and the consistency regularization term; the weighted cross-entropy loss affects the final fused classification result of the network. For the i-th sample in a batch, the logical value of its network output is . The real label is The weighted cross-entropy loss is calculated as follows:
[0007] Where N is the batch size, and s(⋅) represents the softmax function. This is a category The pre-defined weights are assigned; the spectral-spatial consistency regularization loss, for sample i, is given by the predicted probability vector output by its spectral branch. The predicted probability vector output by the spatial branch is The Kullback-Leibler divergence is used to measure the difference between the two probability distributions, and a confidence-aware mask is introduced. The final consistency regularization loss is the average of the KL divergences of all activated samples: .
[0008] The technical problem to be solved by this invention can also be further achieved through the following technical solution: For the spatial-spectral cross-scale interactive network for hyperspectral image classification described above, a spatial-spectral cross-scale interactive method for hyperspectral image classification is proposed. First, principal component analysis is applied to the input hyperspectral image for dimensionality reduction. Then, the image block centered on the target pixel is input into a dual-path low-level feature extraction module—a spectral hierarchical encoding module and a spatial multi-kernel context module. Second, based on the extracted spectral and spatial features, a spectral pyramid path and a spatial pyramid path are constructed respectively. Then, a cross-scale spatial-spectral interactive module composed of diagonal interactive blocks and cross-scale blocks is used to perform deep multi-scale and multi-feature fusion. Finally, features are refined through a multi-round perturbation mechanism guided by spectral-spatial consistency, and a multi-round foreground priority decision rule is applied to generate the final classification result.
[0009] The technical problem to be solved by this invention can also be further achieved through the following technical solution: the spatial-spectral cross-scale interaction method for hyperspectral image classification described above includes the following steps: (1) Data preprocessing: Principal component analysis is applied to the input hyperspectral image for dimensionality reduction. The image patch size and feature dimension after principal component analysis are set. Image patches are obtained with the target pixel as the center. (2) Low-level feature extraction: Spectral features are extracted using a spectral hierarchical coding module, and a multi-level spectral representation is constructed using a hierarchical one-dimensional convolutional structure. The basic spectral features are obtained by processing the three-layer output features. Spatial multi-core context module is used; spatial features are extracted, and a progressive multi-core cascade design is adopted to aggregate intermediate responses and integrate channels; (3) Constructing a multi-scale feature pyramid: Construct a spectral pyramid path to model the spatial response of spectral correlation at different spatial scales, and apply a dual attention mechanism to enhance feature discrimination. A spatial pyramid path is constructed to extract hierarchical spatial features, and channel attention is used to emphasize discriminative spatial feature channels. (4) Cross-scale spatial-spectral interaction: Pixel-by-pixel deep fusion of spectral and spatial features at the same scale is achieved using diagonal interactive blocks; Use cross-scale blocks to achieve information complementarity between features of different scales; Modality adaptive fusion is performed by processing the multi-scale feature sets of the spectral and spatial pyramid outputs to obtain the fused features; (5) Multi-round perturbation guided by spectral-spatial consistency: The feature representation of suspicious background samples is finely adjusted, the baseline fusion features are initialized and preserved, and multi-round perturbation is executed iteratively. In each round, consistency weight calculation, unified attention fusion feature generation, original decision style preservation, weighted pooling of uncertainty perception and progressive feature update are performed. (6) Multi-round foreground priority decision: Based on the baseline prediction, the initial candidate foreground category is selected, and the corrected features after each round of perturbation are predicted. When the category consistency and probability threshold conditions are met, the sample is reclassified as a foreground.
[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Effective handling of complex boundaries: This invention enhances the model's ability to identify spectral-spatial mixed boundaries through a cross-scale interaction mechanism of spectral and spatial dual paths; 2. Suppressing Background Pixel Interference: In deep learning, due to the large number of background category samples and their complex features, models often tend to classify uncertain pixels as background by default, leading to a decrease in foreground classification accuracy. This invention introduces a background category correction module, which significantly reduces the erosion of background class on foreground classification decisions through multi-round perturbation guided by spectral-spatial consistency and a foreground-priority decision-making mechanism, thereby improving foreground classification accuracy. 3. Balancing local and global features: This invention combines a multi-scale pyramid structure with a cross-scale attention mechanism to achieve efficient modeling of both local details and global context; 4. Applicable to real-world scenarios: This invention maintains superior performance even with HSIC-B settings that preserve background pixels, making it closer to actual application needs; 5. Strong model generalization ability: This invention achieves better classification accuracy than existing methods on multiple public hyperspectral datasets, demonstrating strong generalization and robustness. Attached Figure Description
[0011] Figure 1 Experimental comparison between the present invention and existing methods Figure 1 ; Figure 2 Experimental comparison between the present invention and existing methods Figure 2 ; Figure 3 Experimental comparison between the present invention and existing methods Figure 3 . Detailed Implementation
[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] A spatial-spectral cross-scale interaction method for hyperspectral image classification is proposed. First, principal component analysis (PCA) is applied to the input hyperspectral image for dimensionality reduction. Then, image patches centered on the target pixel are input into a dual-path low-level feature extraction module—a spectral hierarchical coding module (SHEM) and a spatial multi-kernel context module (SMCM). Second, based on the extracted spectral and spatial features, a spectral pyramid path (SPP) and a spatial pyramid path (PSPP) are constructed, respectively. Subsequently, a cross-scale spatial-spectral interaction module consisting of a diagonal interaction block (DIB) and a cross-scale block (CSB) is used to achieve deep multi-scale and multi-feature fusion. Finally, a multi-round perturbation mechanism guided by spectral-spatial consistency is used to refine the features, and a multi-round foreground priority decision rule is applied to generate the final classification result.
[0014] To clearly describe the dimensional changes of the feature map throughout the process, the image patch size was set to 15×15, and the feature dimension after PCA was set to 30.
[0015] This method uses a spatial-spectral cross-scale interactive network for hyperspectral image classification, specifically including: (1) Spectral Hierarchical Encoding Module (SHEM): A hyperspectral pixel consists of hundreds of consecutive spectral bands, exhibiting a clear sequence structure. To effectively encode first- and second-order spectral variation trends, a hierarchical one-dimensional convolutional structure is used to construct a multi-level spectral representation. The spectral branch contains three consecutive one-dimensional convolutional layers in the channel direction, with 32, 64, and 128 channels, respectively. Each layer follows the following order: Conv1D → BatchNorm (BN) → ReLU. As the network deepens, the convolutional kernel senses higher-order spectral variations along the sequence direction, forming a multi-level spectral response (e.g., material absorption peaks, reflectance variation trends).
[0016] The features output from the three layers are concatenated along the channel dimension and projected onto a unified embedding dimension D=256 through a (1×1) pointwise convolution, thereby aggregating features from different semantic levels and aligning their feature domains. The forward process can be mathematically expressed as follows:
[0017] in The input spectral sequence has batch size B, number of channels C, and sequence length L (for spatial expansion, L = H × W), and the final output is... These will serve as the basis for subsequent spectral features.
[0018] (2) Spatial Multicore Context Module (SMCM): Spatial patterns encode key cues such as texture, edges, and local neighborhood layouts, which are particularly informative for resolving blurred category boundaries in hyperspectral scenes. To represent spatial context across multiple effective receptive fields while keeping modules lightweight, a progressive multi-core cascaded design is employed.
[0019] Specifically, the spatial branches apply three convolutional groups sequentially, with kernel sizes of 3×3, 5×5, and 7×7. Unlike parallel multi-branch designs, this cascaded approach progressively expands the receptive field and allows subsequent convolutional kernels to operate on the refined responses, resulting in a spatial context hierarchy from fine-grained to coarse-grained. Each group follows the order Conv2D→BN→ReLU. Intermediate responses are then aggregated through element-wise summation, and a 1×1 convolution is used to integrate channels and stabilize semantics.
[0020] The forward process is defined as follows:
[0021] in This represents the spatial feature map of the input. In our implementation, The original channel dimension C is preserved to maintain a low-level spatial response aligned with the input spectral channels. Subsequent spectral and spatial pyramid paths project features to a uniform embedding dimension D via 1×1 convolutions, enabling consistent multi-scale interactions and feature fusion in later stages.
[0022] (3) Dual-path multi-scale feature pyramid Based on enhanced low-level spatial features, this work constructs two independent multi-scale feature pyramids: the spectral pyramid path (SPP) and the spatial pyramid path (PSPP). These two paths model multi-scale semantics from the perspectives of spectral correlation and spatial structure, respectively, and employ an attention mechanism to achieve adaptive modulation of scale.
[0023] Spectral Pyramid Path (SPP): The Spectral Pyramid Path models the spatial response of spectral correlation at three different spatial scales, thereby enhancing the consistency of spectral dominant semantics across scales.
[0024] Fine-scale: Preserve the original spatial resolution (H, W). Channel adjustment is performed using 1×1 convolution to extract local fine structures and spatially sensitive spectral textures.
[0025] At a medium scale: 1×1 convolution, 3×3 depthwise separable convolution, and 2×2 max pooling are applied sequentially to aggregate the neighborhood context. Depthwise separable convolution groups channels, enhancing feature discriminative power without increasing the number of parameters.
[0026] Coarse scale: Apply 1×1 convolution, 5×5 depthwise separable convolution and global average pooling in sequence to model global spectral dependence.
[0027] To dynamically enhance feature discriminative power, a dual attention mechanism was applied, specifically window-based multi-head self-attention (W-MHSA). and after upsampling The above operations are performed to model long-range dependencies within a local window. Channel attention (CA) is applied across all scales to recalibrate channel importance. For feature map F, the channel attention weights are... The calculation is as follows:
[0028] in For learnable weights, It is the ReLU function. It's the Sigmoid function, and GAP represents global average pooling. The weighted feature is... ,in This represents channel-wise multiplication. Ultimately, each scale outputs a gated feature, which serves as the input to the cross-scale interaction module.
[0029] Spatial Pyramid Path (PSPP): As a complement to the spectral path, the spatial pyramid path focuses on hierarchical spatial feature extraction, emphasizing geometric shapes, edges, and region patterns at multiple resolutions.
[0030] Fine-scale (P1): Apply two consecutive 3×3 standard convolutions with a stride of 1 to enhance local spatial pattern modeling.
[0031] Mesoscale (P2): Spatial abstraction is performed through a convolution with a stride of 2, followed by a refined convolution.
[0032] Coarse scale (P3): In order to obtain the global spatial context, a convolution with a stride of 2 is performed first, followed by global average pooling.
[0033] All three scales employ the same channel attention (CA) formula as in the spectral path to emphasize discriminative spatial feature channels. The output forms a spatial feature pyramid. , , Then input the cross-scale interaction module.
[0034] (4) Cross-scale spatial-spectral interaction module The core architecture of the proposed network is detailed, including Diagonal Interaction Blocks (DIB), Cross-Scale Blocks (CSB), and Modality Adaptive Fusion. These modules constitute the cross-modal and cross-scale feature modeling mechanism within the backbone network, enabling the model to fully utilize the complex spatial-spectral correlations and global contextual information in hyperspectral images.
[0035] Diagonal Interaction Block (DIB): This module is used to achieve pixel-by-pixel-level deep fusion between spectral and spatial features at the same scale.
[0036] Input: Let the output of the spectral pyramid at scale l be... The output of the spatial pyramid is All of their dimensions are .
[0037] Spatial Alignment: If the two feature maps have different spatial dimensions, first perform bilinear interpolation on the spatial features to align them with the spectral feature dimensions.
[0038] in This indicates a bilinear interpolation operation. Feature stitching and attention recalibration: Stitching the aligned spectral and spatial features along the channel dimension: Then, the features are adaptively recalibrated using a channel attention mechanism: Where GAP(⋅) represents global average pooling. It is the ReLU activation function. This is the Sigmoid function.
[0039] Feature Enhancement and Output: The weighted features are subjected to 1×1 convolution, batch normalization, and ReLU activation, and then further integrated with local context and enhanced expressive power through windowed multi-head self-attention (W-MHSA) and positional feedforward network (FFN). Output This is the diagonal interaction result at this scale.
[0040] Cross-Scale Block (CSB) This module achieves information complementarity between features at different scales through an asymmetric attention mechanism, enhancing... The model's ability to collaboratively model both global context and local details.
[0041] Scale alignment: Assume the query features are derived from the scale. (size Key-value features come from scale (size ).
[0042] First, align the key-value feature interpolation to the query scale: Attention computation: Generate query, key, and value vectors through independent 1×1 convolutions: Calculate cross-scale attention and output:
[0043]
[0044] Modal adaptive fusion Multi-scale feature interaction: The spectral and spatial pyramids output multi-scale feature sets {s1,s2,s3} and {p1,p2,p3}, respectively. First, same-scale fusion is performed using DIB: Then, cross-scale compensation is performed using CSB: Feature aggregation and serialization: After upsampling all features to the finest scale, the average is taken to obtain a unified cross-scale interactive feature. in This is an upsampling operation. Remodeling into a spatial token sequence and with spectral token sequence Parallel input is fed to the adaptive fusion module.
[0045] Adaptive fusion: Dynamically weighted fusion of bimodal features using learnable scalar coefficients. This mechanism enables the model to adaptively adjust the spectral and spatial contribution weights based on the content, with all parameters optimized end-to-end.
[0046] (5) Loss function To jointly optimize the entire network and effectively promote collaborative learning between the spectral and spatial paths, we designed a composite loss function. This loss consists of a weighted cross-entropy loss for supervision of the classification task and a spectral-spatial consistency regularization loss designed to bridge the gap between the semantic representations of spectral and spatial features.
[0047] Total loss is defined as: in It is a hyperparameter used to balance the classification loss with the strength of the consistency regularization term.
[0048] Weighted cross-entropy loss This loss is applied to the network's final fused classification result to optimize classification accuracy and address class imbalance. For the i-th sample in a batch, the logical value of its network output is... The real label is The weighted cross-entropy loss is calculated as follows: here: N is the batch size. s(⋅) represents the softmax function, used to convert logical values into a class probability distribution. This is a category The pre-defined weights assigned are usually proportional to the inverse of the number of samples in each class, in order to mitigate the class imbalance problem.
[0049] Spectral-spatial consistency regularization loss This regularization term aims to encourage the spectral and spatial branches of the network to make as consistent classification predictions as possible for the same location, especially when the model's own prediction confidence is high, thereby imposing semantic consistency constraints during the feature learning process of the two modalities.
[0050] For sample i, let its spectral branch output prediction probability vector be... The predicted probability vector output by the spatial branch is First, the Kullback-Leibler divergence is used to measure the difference between the two probability distributions:
[0051] To avoid introducing noise by forcibly imposing consistency constraints on predictions that are inherently uncertain (low confidence), we introduce a confidence-aware mask. This mask is only applied in spectral and spatial branch predictions where the maximum prediction probability of either branch exceeds a preset threshold. Activation at time (with a value of 1) indicates that the model has sufficient confidence in the sample: Ultimately, the consistency regularization loss is the average of the KL divergences of all activated samples: This hybrid loss function, on the one hand, through... On the one hand, it ensures accurate classification and discrimination capabilities. At the feature level, guide the network to learn robust and consistent semantic representations across modalities.
[0052] (6) Spectral-spatial consistency-guided multi-round perturbation This section introduces a multi-round perturbation mechanism for background interference correction. This mechanism refines the feature representation of suspicious background samples through iterative, consistency-based feature reweighting to suppress the interference of background pixels on classification decisions.
[0053] Initialization: For each sample marked as suspicious background The SSCSIN backbone network outputs a spectral feature token sequence after multi-scale interaction. and spatial feature token sequence Where N is the number of spatial locations and D is the feature dimension, while preserving the undisturbed baseline fusion features of the network. .
[0054] The perturbation process: This mechanism iterates through R rounds (typically 2 to 3 rounds). In each round r, four core hyperparameters are dynamically adjusted: These control the neutralization of fusion intensity, the preservation of the original fusion result, the smoothness (temperature) of uncertainty weights, and the mixing ratio of perturbation features, respectively.
[0055] Specific steps for round r: Calculate the consistency weight: For each position n, calculate the normalized cosine similarity between its spectrum and the spatial token. And map it to consistency weights and uncertainty : Generate unified attention fusion features: To prevent over-reliance on a single modality, the consistency weights are neutralized. Then, the spectral and spatial tokens are fused using the neutralized weights: Preserve the original decision-making style: use the fusion tokens from the previous round (or the initial round). Linear interpolation is performed between the newly generated unified attention fusion result and the network to preserve the network's original "decision-making style": Uncertainty-aware weighted pooling: based on uncertainty at each location Calculate its weight. The lower the uncertainty (the higher the consistency), the greater the contribution of that position: Subsequently, the corrected tokens of the entire sample are weighted and summed to obtain the features after this round of aggregation: Progressive feature update: To stabilize training and avoid overcorrection, the features aggregated in this round are updated. Compared with the original baseline characteristics By mixing, we obtain the final feature output for this round: Mechanism advantages: This multi-round, gradual perturbation allows the model to gradually enhance the foreground signal and suppress background interference in a controlled and stable manner, while ensuring the stability of the optimization process through continuous connection with baseline features.
[0056] (7) Multi-round prospect-first decision rule After multiple rounds of perturbation, a robust rule is needed to determine whether a sample initially classified as background should be reclassified as foreground; a multi-round foreground priority decision rule is proposed.
[0057] Rules and procedures: 1. Initialization: Based on baseline prediction Select the foreground category with the highest probability (index) As an initial candidate: Category index 0 represents the background.
[0058] 2. Round-by-round prediction: For each round of perturbation r=1,...,R, the corrected features are... The data is fed into a classifier to obtain the predicted probability for that round. And record its highest foreground class probability. and the corresponding category index ; Decision condition: A sample will be reclassified as a foreground (class k*) if and only if the following two strict conditions are met: Category consistency: The predicted foreground category must be consistent across all rounds (including the baseline round). Probability threshold: The predicted probability of this foreground category must exceed a minimum threshold in all rounds. : 4. Final output: Significance of the rule: By requiring high consistency and high confidence across rounds, this rule ensures that the reclassification decision is robust and reliable, greatly reducing misjudgments and thus improving the clarity of classification boundaries and overall accuracy in complex scenarios.
[0059] In terms of training settings, all unlabeled pixels (non-foreground categories) were treated as background (BKG) during both training and testing. For the three public datasets, training samples were randomly sampled stratified by category, preserving the original data distribution: PaviaUniversity: 1% of labeled samples per category; Salinas: 0.5% of labeled samples per category; IndianPines: 5% of labeled samples per category. The remaining samples were used for testing to ensure the objectivity and reproducibility of the evaluation.
[0060] In our experimental evaluation, for the hyperspectral image classification (HSIC-B) scenario that includes background pixels, we used background-aware (BKG-aware) evaluation metrics to comprehensively assess the model performance. These metrics include: Producer's Overall Accuracy (OA): Overall classification accuracy; Producer's Average Accuracy (AA): Average classification accuracy for each category; OverallProducer'sPrecisionRate( Overall producer accuracy (including background category); Producer'sAveragePrecisionRate( ): Average producer accuracy for each category (including background category).
[0061] These metrics are all calculated based on producer precision (PR) for each category, which can effectively reflect the model's classification performance under background interference.
[0062] To comprehensively benchmark the performance of the proposed method, it was compared with a series of representative deep learning models established in recent years. The benchmark comparison methods include: Dual-Branch Dual Attention Network (DBDA), Group-Aware Hierarchical Transformer (GAHT), Spectral-Spatial Tokenization Transformer (SSFTT), Spectral Query Space Transformer (SQSFormer), Spectral-Spatial Transformer Network (SSTN), MorphFormer, a lightweight architecture employing Grouped Separable Convolutions and Visual Transformer (ViT) (GSC-VIT), and Spatial-Spectral Augmentation and Fusion Network (SSEFN). Specific results can be found in [link to results]. Figure 1-3 .
[0063] To ensure the fairness and reproducibility of the experimental framework, the hyperparameters and training configurations of all comparison models strictly followed the implementation methods reported in their original literature to guarantee the reproducibility of the experiments.
Claims
1. A spatial-spectral cross-scale interactive network for hyperspectral image classification, characterized in that: Includes the following modules: (1) Data preprocessing module: Principal component analysis is applied to the input hyperspectral image for dimensionality reduction. The image patch size is set to 15×15, and the feature dimension after principal component analysis is set to 30. The image patch centered on the target pixel is then input into the subsequent module. (2) Dual-path low-level feature extraction module: (2.1) Spectral hierarchical encoding module: A multi-level spectral representation is constructed using a hierarchical one-dimensional convolutional structure. The spectral branch contains three consecutive one-dimensional convolutional layers in the channel direction, with the number of channels being 32, 64 and 128, respectively. Each layer follows the order Conv1D→BatchNorm(BN)→ReLU, where Conv1D is a one-dimensional convolution operation, BatchNorm(BN) is a batch normalization operation, and ReLU is an activation function. The features output from the three layers are concatenated along the channel dimension and projected onto a unified embedding dimension D=256 through a (1×1) pointwise convolution to obtain the basic spectral features. (2.2) Spatial multi-kernel context module: A progressive multi-kernel cascaded design is adopted. The spatial branch applies three convolutional groups in sequence, with convolutional kernel sizes of 3×3, 5×5 and 7×7 respectively. Each group follows the order of Conv2D→BN→ReLU, where Conv2D is a two-dimensional convolutional operation. Intermediate responses are aggregated by element-wise summation and a 1×1 convolution is used to integrate channels and stabilize semantics, while preserving the original channel dimension C to maintain the low-level spatial response aligned with the input spectral channels; (3) Dual-path multi-scale feature pyramid module: (3.1) Spectral pyramid path: The spatial response of the spectrum is modeled at three different spatial scales. At the fine scale, the original spatial resolution (H, W) is maintained and 1×1 convolution is used for channel adjustment. At the medium scale, 1×1 convolution, 3×3 depthwise separable convolution and 2×2 max pooling are applied in sequence. At the coarse scale, 1×1 convolution, 5×5 depthwise separable convolution and global average pooling are applied in sequence. A dual attention mechanism is applied, which enhances the feature discrimination power based on window-based multi-head self attention and channel attention. Each scale outputs a gated feature as the input of the cross-scale interaction module. (3.2) Spatial Pyramid Path: Used for hierarchical spatial feature extraction. At the fine scale, two consecutive 3×3 standard convolutions with a stride of 1 are applied; at the medium scale, spatial abstraction is performed through a convolution with a stride of 2, followed by a refined convolution; at the coarse scale, a convolution with a stride of 2 is performed first, followed by global average pooling; all three scales use channel attention to emphasize discriminative spatial feature channels, and the output forms a spatial feature pyramid. (4) Cross-scale spatial-spectral interaction module: (4.1) Diagonal interaction block: Perform pixel-level deep fusion between spectral and spatial features at the same scale. If the spatial sizes of the two feature maps are different, perform bilinear interpolation on the spatial features to align them with the size of the spectral features. Then, stitch the aligned spectral and spatial features along the channel dimension and adaptively recalibrate the features using the channel attention mechanism. After the weighted features are subjected to 1×1 convolution, batch normalization and ReLU activation, the local context is further fused and the expressive power is enhanced through window multi-head self-attention and position feedforward network. (4.2) Cross-scale block: The information complementarity between features of different scales is achieved through an asymmetric attention mechanism. The key-value features are interpolated and aligned to the query scale. The query, key, and value vectors are generated through independent 1×1 convolutions. Cross-scale attention is calculated and output. (4.3) Modal adaptive fusion: The spectral and spatial pyramids output multi-scale feature sets {s1,s2,s3} and {p1,p2,p3} respectively. First, they are fused at the same scale through diagonal interactive blocks, and then cross-scale compensation is performed through cross-scale blocks. All features are upsampled to the finest scale and averaged to obtain unified cross-scale interactive features, which are reshaped into spatial token sequences and input into the adaptive fusion module in parallel with the spectral token sequences. The dual-modal features are dynamically weighted and fused through learnable scalar coefficients. (5) Spectral-spatial consistency-guided multi-round perturbation module: The feature representation of suspicious background samples is finely adjusted. The baseline fusion features of the network are retained during initialization without disturbance. R rounds of perturbation are executed iteratively. In each round, the consistency weight is calculated, the unified attention fusion features are generated, the original decision style is retained, and the uncertainty-aware weighted pooling is performed to obtain the features after aggregation in this round. The features are then mixed with the original baseline features to obtain the final feature output of this round. (6) Multi-round prospect-priority decision-making module: Based on the baseline prediction, the foreground category with the highest probability is selected as the initial candidate. For each round of perturbation, the corrected features are fed into the classifier to obtain the predicted probability and record the probability of the highest foreground category and the corresponding category index. When the two conditions of category consistency and probability threshold are met, the sample is reclassified as foreground.
2. The spatial-spectral cross-scale interactive network for hyperspectral image classification according to claim 1, characterized in that: The network also includes a loss function module, which consists of a weighted cross-entropy loss and a spectral-spatial consistency regularization loss. The total loss is defined as: ； in It is a hyperparameter used to balance the classification loss with the strength of the consistency regularization term; The weighted cross-entropy loss is applied to the final fusion classification result of the network. For the i-th sample in a batch, the logical value of the network output is... The real label is The weighted cross-entropy loss is calculated as follows: ； Where N is the batch size, and s(⋅) represents the softmax function. This is a category The pre-defined weights are assigned; the spectral-spatial consistency regularization loss, for sample i, is given by the predicted probability vector output by its spectral branch. The predicted probability vector output by the spatial branch is The predicted probability vector output by the spatial branch is The Kullback-Leibler divergence is used to measure the difference between the two probability distributions, and a confidence-aware mask is introduced. The final consistency regularization loss is the average of the KL divergences of all activated samples: ； A spatial-spectral cross-scale interaction method for hyperspectral image classification is characterized by the following: the method uses the spatial-spectral cross-scale interaction network for hyperspectral image classification as described in any one of claims 1-2. First, principal component analysis is applied to the input hyperspectral image for dimensionality reduction. Then, image blocks centered on the target pixel are input into a dual-path low-level feature extraction module—a spectral hierarchical encoding module and a spatial multi-kernel context module. Second, based on the extracted spectral and spatial features, spectral pyramid paths and spatial pyramid paths are constructed respectively. Then, a cross-scale spatial-spectral interaction module composed of diagonal interaction blocks and cross-scale blocks is used to perform deep multi-scale and multi-feature fusion. Finally, features are refined through a multi-round perturbation mechanism guided by spectral-spatial consistency, and a multi-round foreground priority decision rule is applied to generate the final classification result.
3. The spatial-spectral cross-scale interactive method for hyperspectral image classification according to claim 3, characterized in that: The method includes the following steps: (1) Data preprocessing: Principal component analysis is applied to the input hyperspectral image for dimensionality reduction. The image patch size and the feature dimension after principal component analysis are set. Image patches are obtained with the target pixel as the center. (2) Low-level feature extraction: Spectral features are extracted using a spectral hierarchical coding module, and a multi-level spectral representation is constructed using a hierarchical one-dimensional convolutional structure. The basic spectral features are obtained by processing the three-layer output features. Spatial multi-core context module is used; spatial features are extracted, and a progressive multi-core cascade design is adopted to aggregate intermediate responses and integrate channels; (3) Constructing a multi-scale feature pyramid: Construct a spectral pyramid path to model the spatial response of spectral correlation at different spatial scales, and apply a dual attention mechanism to enhance feature discrimination. A spatial pyramid path is constructed to extract hierarchical spatial features, and channel attention is used to emphasize discriminative spatial feature channels. (4) Cross-scale spatial-spectral interaction: Pixel-by-pixel deep fusion of spectral and spatial features at the same scale is achieved using diagonal interactive blocks; Use cross-scale blocks to achieve information complementarity between features of different scales; Modality adaptive fusion is performed by processing the multi-scale feature sets of the spectral and spatial pyramid outputs to obtain the fused features; (5) Multi-round perturbation guided by spectral-spatial consistency: The feature representation of suspicious background samples is finely adjusted, the baseline fusion features are initialized and preserved, and multi-round perturbation is executed iteratively. In each round, consistency weight calculation, unified attention fusion feature generation, original decision style preservation, weighted pooling of uncertainty perception and progressive feature update are performed. (6) Multi-round foreground priority decision: Based on the baseline prediction, the initial candidate foreground category is selected, and the corrected features after each round of perturbation are predicted. When the category consistency and probability threshold conditions are met, the sample is reclassified as a foreground.