Single hyperspectral image super-resolution method based on mixed spatial-spectral aggregation network

Abstract

The invention discloses a single hyperspectral image super-resolution method based on a mixed spatial-spectral aggregation network, which belongs to the technical field of computer vision and comprises the following steps: dividing an input image into G overlapped spectrum groups; an MDAF is designed, and multi-scale spatial spectrum features of each spectrum group are extracted and spliced to form a local multi-scale feature map; performing initial feature extraction on the local multi-scale feature map by using a convolutional layer, and mapping the local multi-scale feature map to a hyperspectral dimension to obtain a spatial-spectral feature; s2AM is designed, input features are sequentially sent to N serially connected S2AMs, global depth spatial spectrum features are extracted, and local residual learning is adopted to obtain a depth spatial spectrum feature fusion result; performing up-sampling on the deep features to obtain a target resolution; and introducing a two-stage feature alignment strategy to obtain a super-resolution hyperspectral image. According to the single hyperspectral image super-resolution method based on the mixed spatial-spectral aggregation network, efficient and coordinated spatial-spectral characteristic representation is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a single hyperspectral image super-resolution method based on a hybrid spatial spectrum aggregation network. Background Technology

[0002] Hyperspectral imaging (HSI) is an imaging technique that acquires the reflectance characteristics of ground objects across tens to hundreds of consecutive narrow bands, enabling precise capture of subtle spectral differences. Compared to traditional RGB imaging with its limited spectral range, HSI provides richer spectral information, thus achieving high-resolution representation of material properties. Leveraging this advantage, hyperspectral imaging technology has been widely applied in target detection, geological exploration, and medical diagnostics. However, due to limitations in sensor hardware, hyperspectral imaging systems typically involve an inherent trade-off between spatial and spectral resolution. Given the high cost and difficulty of hardware upgrades, improving the spatial resolution of HSI using algorithmic methods has become a research direction of significant value.

[0003] Super-resolution (SR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) images. It is widely used in computer vision, remote sensing, and medical imaging to alleviate the problem of insufficient spatial resolution in hyperspectral images. Based on whether it relies on auxiliary image information, HSI super-resolution methods are generally divided into fusion-based methods and single-image hyperspectral super-resolution (SHSR). The former typically combines panchromatic, RGB, or multispectral images with higher spatial resolution for reconstruction to enhance spatial detail. Although this type of method generally outperforms SHSR in reconstruction performance, it relies on the assumption of accurate registration between the auxiliary image and the low-resolution HSI, a high-quality registration that is often difficult to achieve in practical applications.

[0004] In contrast, SHSR does not rely on external images, but only utilizes information from the low-resolution HSI itself for reconstruction, offering greater flexibility in data availability and practical applications, and thus attracting increasing attention. Existing SHSR research mainly includes two categories: traditional models and deep learning-based models. The former typically involves manually designing prior constraints on the reconstruction process, such as introducing spectral priors using sparse representations or Bayesian modeling to improve reconstruction stability. However, these methods rely on hand-designed features and have limited ability to characterize data distribution and high-dimensional spatial-spectral relationships, making it difficult to achieve ideal reconstruction results.

[0005] In recent years, the development of deep learning technology has made it possible to learn complex mappings between low-resolution and high-resolution images end-to-end. Convolutional Neural Networks (CNNs), with their efficient local feature extraction capabilities, have been widely explored in image super-resolution applications. CNN-based methods perform well in local feature extraction, but generally lack effective modeling of global spatial-spectral relationships. To overcome the locality limitation of CNNs, Transformer-based architectures have shown great potential in modeling global dependencies in recent years. A large amount of work has verified the effectiveness of Transformers in various low-level vision tasks. However, although existing methods have made progress in performance, the introduction of 3D convolution and self-attention mechanisms significantly increases computational complexity, limiting their efficiency in high-resolution HSI reconstruction. Summary of the Invention

[0006] The purpose of this invention is to provide a single hyperspectral image super-resolution method based on a hybrid spatial spectrum aggregation network to solve the problems existing in the background technology.

[0007] To achieve the above objectives, this invention provides a single hyperspectral image super-resolution method based on a hybrid spatial-spectral aggregation network, for low-resolution hyperspectral images. ,in , and These represent the image's height, width, and number of spectral bands, respectively; the corresponding high-resolution image is... ;include: Divide the input image into G overlapping spectral groups: ; Each spectral group It contains c spectral bands, and there are 0 overlapping bands; Design a multi-scale dual attention fusion module MDAF to extract multi-scale spatial spectral features for each spectral group and stitch them together in the spectral dimension to form a local multi-scale feature map; Convolutional layers are applied to extract initial features from local multi-scale feature maps, effectively mapping them to higher hyperspectral dimensions to obtain spatial-spectral features. ; Design a Transformer module for spatial spectrum aggregation S 2 AM, spatial spectral features The system extracts global contextual information across bands, sequentially feeding the input features into N cascaded S-bands. 2 AM, extracting global depth spatial-spectral features The deep spatial-spectral feature fusion result is obtained by using local residual learning. ; In the reconstruction stage, the PixelShuffle algorithm is used to upsample deep features to obtain the target resolution; a two-stage feature alignment strategy is introduced to finally obtain a super-resolution hyperspectral image. .

[0008] Preferably, the process of extracting multi-scale spatial spectral features for each spectral group and stitching them together along the spectral dimension to form a local multi-scale feature map is as follows: No. The multi-scale spatial spectral features can be expressed as: ; in, This represents the feature extraction function of the module; After extraction, the features from each group are stitched together along the spectral dimension to form a local multi-scale feature map: ; in, This indicates a spectral dimension splicing operation.

[0009] Preferably, convolutional layers are applied to extract initial features from the local multi-scale feature map and obtain spatial spectral features. The process is as follows: Before inputting the concatenated features into the global network, convolutional layers are applied to extract initial features and effectively map them to a higher hyperspectral dimension. This process is represented as follows: ; in This is a 1×1 convolution operation; This represents the spatial spectral features obtained by mapping the spectral dimensions.

[0010] The preferred content of the multi-scale dual attention fusion module MDAF is as follows: First, the input features are fed into three parallel convolutional branches, using 3×3, 5×5, and 7×7 kernels respectively; then, features of different scales are fused in pairs to enhance the diversity of feature representation; let... For the first The input to MDAF, and the features extracted at different scales are: The process is represented as follows: ; ; ; in, , , These represent convolution operations with kernel sizes of 3×3, 5×5, and 7×7, respectively. A dual attention mechanism is introduced to optimize feature responses from both channel and spatial dimensions. Channel attention (CA) models the correlation between channels through global average pooling dimensionality reduction, ReLU activation, and the sigmoid function, adaptively adjusting the importance weights of each spectral channel. ; Spatial attention (SA) combines global average pooling and max pooling operations, and the concatenated result is then processed by convolution and sigmoid activation to obtain a spatial weight map, which is used to highlight key regions and capture spatial spectral dependencies. ; Finally, channel and spatial attention features are integrated through 1×1 convolution, and the enhanced output is obtained by using residual connections: ; in, This represents a 1×1 convolution operation. Indicates the first The output features of each MDAF.

[0011] Preferably, the deep spatial-spectral feature fusion result is obtained. The process is as follows: Design a Transformer module for spatial spectrum aggregation S 2 AM, from spatial spectral features Extract global context information across bands; sequentially feed the input features into N cascaded S... 2 AM, extracting global depth spatial-spectral features The process is represented as follows: ; in, Indicates the Nth S 2 AM; subsequently, local residual learning is used to obtain the deep spatial-spectral feature fusion result. The process is represented as follows: .

[0012] Preferably, the spatial spectrum aggregation Transformer module S 2 AM content is as follows: S 2 AM consists of three parts: spectral grouping self-attention S-GSA, windowed self-attention S-WSA, and a 3×3 convolutional layer containing two residual blocks. The first residual block consists of a LayerNorm layer and a self-attention layer, and the second residual block consists of a LayerNorm layer and a multilayer perceptron (MLP). The process is represented as follows: ; ; in, This represents the LayerNorm layer function. This represents a multilayer perceptron. This refers to global features within the module.

[0013] Preferably, the spectral grouping self-attention mechanism S-GSA combines the complementary properties of deformable convolution and grouped convolution to achieve efficient modeling and dynamic adaptive sampling of hyperspectral channels. Deformable convolution introduces a learnable offset to dynamically adjust the sampling position during convolution calculation, thereby adapting to the non-uniformity of spectral distribution. Grouped convolution, through channel grouping and independent convolution calculation, reduces redundant dependencies between high-dimensional spectral channels, achieving efficient spectral grouping representation. Let the input features be... After deformable convolution, grouped convolution, and two 1×1 pointwise convolutions, the query is mapped to the result. ,key Sum Taking a single attention head as an example, the S-GSA calculation is expressed as follows: ; ; ; ; in, This represents a deformable convolution operation. Represents grouped convolution. and For pointwise convolution functions, For the dimension of Q, specifically Finally, the outputs of each attention head are concatenated along the spectral dimension to obtain the output features of the spectral grouping self-attention module.

[0014] The preferred spatial window self-attention mechanism, S-WSA, effectively reduces computational complexity by limiting the attention range through local windows in the spatial dimension, and enhances local spatial modeling capabilities through relative position encoding. Specifically, the input features are divided into M×M windows in the spatial dimension, with each window treated as a "token." The input features are then reshaped into... (in Indicates the number of windows. (Number of pixels contained in each window). This is then transformed into a query vector via linear projection. Key vector Sum value vector ,Right now: ; ; ; in, This is the learnable projection matrix. The window size M is set to 8. For a single attention head, the calculation process is as follows: ; in, For the dimension of Q, specifically B is a learnable relative position offset matrix used to model the spatial relative positional relationships between pixels within the window.

[0015] Preferably, a super-resolution hyperspectral image is obtained. as follows: During the reconstruction phase, the PixelShuffle algorithm is used to upsample deep features to obtain the target resolution. ; in, This represents the upsampling function, and the spatial size of the output features is... The spectral dimension is C.

[0016] To further improve reconstruction efficiency and high-frequency detail representation, a two-stage feature alignment strategy is introduced. First, the input low-resolution image is spatially upsampled using bicubic interpolation to match the target size. Then, a 1×1 convolutional layer is used to lightweightly adjust the aligned feature channels, achieving efficient residual fusion. Furthermore, the 1×1 convolution enables lightweight feature adaptation, significantly improving efficiency while effectively preserving spatial-spectral correlation characteristics. The final reconstruction process can be summarized as follows: ; ; in, and All are 1×1 convolutional layers. This indicates the output of the final super-resolution hyperspectral image.

[0017] Therefore, the present invention employs the above-mentioned single hyperspectral image super-resolution method based on a hybrid spatial spectrum aggregation network, which has the following beneficial effects: (1) Local multi-scale spatial-spectral features are extracted through the multi-scale dual attention fusion module (MDAF), and combined with the spatial-spectral aggregation Transformer module (S 2 AM) models global spectral and spatial dependencies, achieving efficient synergy between the local feature extraction advantages of CNN and the global modeling capabilities of Transformer, significantly improving the expressive power of spatial and spectral features and enhancing the structural and spectral fidelity of reconstructed images; (2) In MDAF, channel attention (CA) and spatial attention (SA) mechanisms are introduced to adaptively adjust feature weights from the spectral and spatial dimensions, respectively, highlighting important information and suppressing redundant features, thereby improving the selectivity and expressive power of features in the multi-scale fusion process, which is conducive to recovering detailed information; (3) By using the spectral group self-attention (S-GSA) and spatial window self-attention (S-WSA) mechanisms, the attention calculation is restricted to the spectral group and the local spatial window, which significantly reduces the computational burden of the traditional Transformer on hyperspectral data, while maintaining strong global modeling capabilities, making it suitable for processing high-dimensional hyperspectral data. (4) In the reconstruction stage, the PixelShuffle upsampling and two-stage feature alignment strategy is adopted. First, spatial alignment is performed by bicubic interpolation, and then the feature channels are adjusted by lightweight convolution, which realizes efficient feature fusion and residual learning, and enhances the recovery ability of high-frequency details while improving reconstruction efficiency.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall architecture of HSANet according to an embodiment of the present invention; Figure 2 This is an embodiment S of the present invention. 2 A schematic diagram of the AM structure; Figure 3 This is a schematic diagram of the structure of the multi-scale dual-attention fusion module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of spatial attention and channel attention in an embodiment of the present invention; wherein, (a) is a schematic diagram of spatial attention; and (b) is a schematic diagram of channel attention. Figure 5 This is a schematic diagram of the structure of the spectral grouping self-attention in an embodiment of the present invention; Figure 6 This is a schematic diagram of the self-attention structure of the spatial window in an embodiment of the present invention; Figure 7 This is a visualization result on the Chikusei dataset when the scaling factor is ×4 according to an embodiment of the present invention; Figure 8 This is a visualization result on the CAVE dataset when the scaling factor is ×4 in an embodiment of the present invention; Figure 9 This is a visualization result on the Pavia Center dataset when the scaling factor is ×4 in an embodiment of the present invention; Figure 10This is the visualization result on the XiongAn dataset when the scaling factor is ×4 in this embodiment of the invention; Figure 11 This is an error graph of the Chikusei dataset at a scaling factor of ×4, as shown in this embodiment of the invention. Figure 12 This is an error graph of the CAVE dataset in an embodiment of the present invention at a scaling factor of ×4; Figure 13 This is an error graph of the Pavia Center dataset at a scaling factor of ×4, as shown in this embodiment of the invention. Figure 14 This is an error graph of the XiongAn dataset at a scaling factor of ×4, as shown in this embodiment of the invention. Figure 15 The figures show the average spectral difference curves of different datasets under a ×4 scaling factor in embodiments of the present invention, wherein (a) is the average spectral difference curve of the Chikusei dataset under a ×4 scaling factor; and (b) is the average spectral difference curve of the CAVE dataset under a ×4 scaling factor. Figure 16 The figures show the average spectral difference curves of different datasets under a ×4 scaling factor in the embodiments of the present invention, wherein (a) is the average spectral difference curve of the Pavia Center dataset under a ×4 scaling factor; and (b) is the average spectral difference curve of the XiongAn dataset under a ×4 scaling factor. Detailed Implementation

[0020] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] Please see Figures 1-6 This paper presents a single hyperspectral image super-resolution method based on a hybrid spatial-spectral aggregation network. The overall network consists of three main modules: a shallow feature extraction module, a deep feature extraction module, and an image reconstruction module. Given a low-resolution hyperspectral image... ,in , and These represent the image's height, width, and number of spectral bands, respectively; the corresponding high-resolution image is... Output super-resolution hyperspectral image as , This represents the super-resolution scaling factor. The goal of the model is to learn a function... Achieve from To super-resolution images The mapping makes as close as possible This process can be represented as: ; in, This represents the spatial-spectral mapping function of the proposed network.

[0022] Significant similarity exists between adjacent spectral bands, and this high correlation can reduce the model's ability to perceive spectral differences to some extent. Considering the high dimensionality and strong spectral redundancy of hyperspectral data, the input image is divided into G overlapping spectral groups, i.e. Each spectral group It contains c spectral bands, with o overlapping bands. For each spectral group, a multi-scale dual-attention fusion (MDAF) module is designed to fully explore the correlations and spectral variations between bands. The multi-scale spatial-spectral features of the g-th group can be expressed as: ; in, This represents the feature extraction function of the module. After extraction, the features from each group are concatenated along the spectral dimension to form a local multi-scale feature map. ; in, This represents the spectral dimension concatenation operation. Before inputting the concatenated features into the global network, convolutional layers are applied to extract initial features and effectively map them to a higher hyperspectral dimension, thus providing a solid foundation for optimization of deep feature extraction. This process can be represented as: ; in This is a 1×1 convolution operation. This represents the spatial spectral features obtained by mapping the spectral dimensions.

[0023] Besides local band correlations, hyperspectral images also exhibit global spatial-spectral dependence. To more fully model long-range structural relationships, a Spatial–Spectral Aggregation Transformer Module (S) is designed. 2 AM), from the characteristics Extracting global contextual information across bands. The input features are sequentially fed into N cascaded S... 2 AM, extracting global depth spatial-spectral features This process can be represented as: ; in, Indicates the Nth S 2 AM. Subsequently, local residual learning was used to obtain the deep spatial-spectral feature fusion result. The process can be described as follows: ; During the reconstruction phase, the PixelShuffle algorithm is used to upsample deep features to obtain the target resolution. ; in, This represents the upsampling function, and the spatial size of the output features is... The spectral dimension is C.

[0024] To further improve reconstruction efficiency and high-frequency detail representation, a two-stage feature alignment strategy is introduced. First, the input low-resolution image is spatially upsampled using bicubic interpolation to match the target size. Then, a 1×1 convolutional layer is used to lightweightly adjust the aligned feature channels, achieving efficient residual fusion. Furthermore, the 1×1 convolution enables lightweight feature adaptation, significantly improving efficiency while effectively preserving spatial-spectral correlation characteristics. The final reconstruction process can be summarized as follows: ; ; in, and All are 1×1 convolutional layers. This indicates the output of the final super-resolution hyperspectral image.

[0025] Multi-scale Dual Attention Fusion Module (MDAF) Compared to traditional RGB images, hyperspectral images (HSI) have a wider spectral coverage, and different bands typically exhibit strong spatial consistency. Furthermore, the reflectance intensity of pixels at the same spatial location in different bands reflects the spectral characteristics of an object, which are influenced by material properties. After band grouping, although different spectral groups have high spatial similarity, subtle spectral differences still exist. Most existing hyperspectral super-resolution (HSI SR) methods rely primarily on single 2D or 3D convolution operations for spatial-spectral feature modeling, making it difficult to fully capture complex spatial-spectral correlations at different spatial scales. This embodiment designs a multi-scale dual-attention fusion (MDAF) module to efficiently extract and fuse shallow multi-scale spatial-spectral features in the early stages of the network. Figure 12As shown, the input features are first fed into three parallel convolutional branches, using 3×3, 5×5, and 7×7 kernels respectively, to capture spatial details at different scales and expand the receptive field, thereby mitigating information loss caused by a single scale. Subsequently, features from different scales are fused in pairs to enhance the diversity of feature representation. Let... Given the g-th MDAF input, the features extracted at different scales are: This process can be represented as: ; ; ; in, , , These represent convolution operations with kernel sizes of 3×3, 5×5, and 7×7, respectively.

[0026] To further highlight important features and suppress redundant information, a dual attention mechanism is introduced to optimize feature responses from both channel and spatial dimensions. Specifically, channel attention (CA) adaptively adjusts the importance weights of each spectral channel by modeling the correlation between channels through global average pooling dimensionality reduction, ReLU activation, and the sigmoid function. ; Spatial attention (SA) combines global average pooling and max pooling operations, concatenating their results and applying convolution and sigmoid activation to obtain a spatial weight map, which is used to highlight key regions and capture spatial spectral dependencies. ; Finally, channel and spatial attention features are integrated through 1×1 convolution, and the enhanced output is obtained by using residual connections: ; in, This represents a 1×1 convolution operation. This module represents the output feature of the g-th MDAF. While preserving local spectral consistency, it effectively enhances the multi-scale representation capability of the features.

[0027] Spatial spectrum aggregation Transformer module (S 2 AM): After multi-scale local feature extraction, capturing long-range dependencies between bands and within space is crucial for further improving hyperspectral image reconstruction performance. In recent years, the Transformer architecture based on self-attention has demonstrated superior global modeling capabilities in computer vision tasks. CNNs excel at extracting local spatial-spectral features, while the Transformer structure has a stronger advantage in modeling long-range spatial-spectral dependencies. The synergy between the two is essential for hyperspectral image super-resolution (HSI SR) tasks. Therefore, this embodiment designs a Spatial–Spectral Aggregation Transformer Module (S... 2 AM) to fully explore the complementary features of the spectral and spatial dimensions.

[0028] like Figure 2 As shown, let the features from the previous stage of the convolutional network be... S 2 AM consists of three parts: S-GSA (Spectral Grouping Self-Attention), S-WSA (Windowing Self-Attention), and a 3×3 convolutional layer. S-GSA models the long-range dependencies of spectral groupings to uncover global relationships across bands; S-WSA captures local spatial context through windowing; and the 3×3 layer provides the inductive bias unique to CNNs, compensating for the shortcomings of attention mechanisms in modeling local structures. These three parts work together to fuse spectral and spatial information at different scales. Borrowing from the architecture of most Transformer encoders, S... 2 AM consists of two residual blocks: the first residual block is composed of a LayerNorm layer and a self-attention layer, and the second residual block is composed of a LayerNorm layer and a multilayer perceptron (MLP). The process is represented as follows: ; ; in, This represents the LayerNorm layer function. This represents a multilayer perceptron. This refers to global features within the module.

[0029] a. Spectral Grouping Self-Attention S-GSA Because different spectral bands exhibit both strong correlations and subtle differences, this embodiment designs a spectral-grouped self-attention (S-GSA) mechanism to simultaneously ensure spatial detail recovery and spectral fidelity. For example... Figure 5As shown, this mechanism achieves efficient modeling and dynamic adaptive sampling of hyperspectral channels by combining the complementary properties of deformable convolution and grouped convolution. Specifically, deformable convolution introduces a learnable offset to dynamically adjust the sampling position during convolution calculation, thereby adapting to the non-uniformity of spectral distribution; grouped convolution reduces redundant dependencies between high-dimensional spectral channels through channel grouping and independent convolution calculation, achieving efficient spectral grouping representation. Let the input features be... After deformable convolution, grouped convolution, and two 1×1 pointwise convolutions, the query is mapped to the result. ),key( ) and value ( Taking a single attention head as an example, S-GSA calculation can be expressed as: ; ; ; ; in, This represents a deformable convolution operation. Represents grouped convolution. and For pointwise convolution functions, For the dimension of Q, specifically Finally, the outputs of each attention head are concatenated along the spectral dimension to obtain the output features of the spectral grouping self-attention module.

[0030] b. Spatial Window Self-Attention S-WSA To further enhance spatial dependency modeling capabilities, this embodiment introduces a spatial-window self-attention (S-WSA) mechanism into the spatial-spectral aggregation Transformer module. For example... Figure 6 As shown, by dividing the attention range into local windows in the spatial dimension, the computational complexity is effectively reduced, and the local spatial modeling capability is enhanced by relative position encoding. Specifically, the input features are divided into windows of size M×M in the spatial dimension, and each window is regarded as a "token". The input features are then reshaped into... (in Indicates the number of windows. (Number of pixels contained in each window). This is then transformed into a query vector via linear projection. Key vector Sum value vector ,Right now: ; ; ; in, This is the learnable projection matrix. The window size M is set to 8. For a single attention head, the calculation process is as follows: ; in, For the dimension of Q, specifically B is a learnable relative position offset matrix used to model the spatial relative positional relationships between pixels within the window.

[0031] This embodiment verifies the effectiveness of the proposed HSANet in hyperspectral image super-resolution tasks through comprehensive experiments. Experiments were conducted on four publicly available benchmark datasets: Chikusei, CAVE, Pavia Center, and XiongAn. HSANet was compared quantitatively and qualitatively with eight mainstream hyperspectral image super-resolution methods: Bicubic, GDRRN, SSPSR, MCNet, PDENet, GELIN, MSDformer, and SNLSR. Six commonly used evaluation metrics were used for quantitative comparison: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Spectral Angle Mapping (SAM), Cross-Correlation Coefficient (CC), Root Mean Square Error (RMSE), and Relative Global Root Mean Square Error (ERGAS). PSNR and SSIM are used to evaluate the reconstruction quality of the hyperspectral image in the spatial dimension; SAM is used to evaluate the reconstruction quality in the spectral dimension; CC quantifies the linear correlation between the reconstructed image and the ground truth image; RMSE and ERGAS are used to evaluate the overall quality of the reconstructed image. Experimental results are as follows: (1) Quantitative comparison results (underlined indicates suboptimal, bold indicates optimal) Table 1. Quantitative evaluation with a scale factor of 4 on the Chikusei dataset.

[0032] Table 2 shows the quantitative assessment with a scale factor of 4 on the CAVE dataset.

[0033] Table 3 shows the quantitative assessment with a scale factor of 4 on the Pavia Center dataset.

[0034] Table 4 shows the quantitative evaluation with a scale factor of 4 on the XiongAn dataset.

[0035] (2) Visualization results Figures 7-10This represents the visualization results on the Chikusei, CAVE, PaviaCenter, and XiongAn datasets with a scaling factor of 4.

[0036] (3) Error chart Figures 11-14 This represents the error plots of the Chikusei, CAVE, Pavia Center, and XiongAn datasets at a scaling factor of ×4.

[0037] (4) Average spectral difference curve Figure 15-16 This represents the average spectral difference curves of different datasets under a ×4 scaling factor.

[0038] Experimental results show that the proposed model achieves good super-resolution reconstruction results on four publicly available hyperspectral datasets and exhibits excellent performance on a variety of commonly used objective evaluation metrics.

[0039] The experimental environment is as follows: (1) Hardware environment: The experiment is based on NVIDIA RTX A6000 GPU and implemented using the PyTorch framework.

[0040] (2) Datasets: Four public benchmark datasets, Pavia Center, CAVE, Chikusei, and XiongAn, were used. The specific processing method was as follows: The Chikusei dataset: This dataset was acquired by the Hyperspec-VNIR-C imaging sensor in Chikusei City, Ibaraki Prefecture, Japan. The scenes include agricultural land and urban areas. The spectral range is 363 nm to 1018 nm, containing 128 spectral bands. The spatial resolution of the images is 2517 × 2335 pixels, with a ground sampling distance of 2.5 m. It covers various scenes such as urban areas, rice paddies, forests, and roads, and is commonly used for remote sensing analysis and classification tasks.

[0041] CAVE dataset: Released by Columbia University, this dataset contains 32 hyperspectral images of natural scenes, ranging from 400nm to 700nm, acquired in 10nm steps, and includes 31 spectral bands. The spatial resolution of the images is 512×512 pixels.

[0042] Pavia Center Dataset: This dataset was obtained using the Reflection Optical System Imaging Spectroradiometer (ROSIS) in downtown Pavia, Italy. After noise removal, the spectral range is 430 nm to 860 nm, containing 102 bands. The image spatial resolution is 1096 × 1096 pixels, with a ground sampling distance of 1.3 m; after cropping unnecessary edges, the final image size is 1096 × 715 × 102.

[0043] XiongAn Dataset: This dataset was collected by the Shanghai Institute of Technical Physics, Chinese Academy of Sciences, in Xiong'an New Area (Matiwan Village), China. The spectral range is 400nm to 1000nm, containing 250 bands. The image size is 3750×1580 pixels, with a spatial resolution of 0.5m.

[0044] (3) Network parameter settings: The input hyperspectral image (HSI) is divided into G=8 spectral groups, each group contains c=8 consecutive bands and there are o=2 overlapping bands between adjacent groups; the number of feature map channels C is set to 240, and the continuously stacked S 2 The number of AMs is set to 4. The size of the self-attention window for the spatial dimension is set to 8×8.

[0045] (4) Training parameter settings: The Adam optimizer is used. , The training process is conducted for 150 epochs with a batch size of 32 and an initial learning rate of 1×10⁻⁴. The learning rate decays to 10% of the original rate every 30 epochs.

[0046] Therefore, this invention adopts the above-mentioned method for super-resolution of single hyperspectral images based on a hybrid spatial-spectral aggregation network, which combines the advantages of CNN in local multi-scale feature learning with the ability of Transformer in global dependency modeling from a multi-scale perspective, thereby achieving efficient and coordinated spatial-spectral feature representation.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A single hyperspectral image super-resolution method based on a hybrid spatial-spectral aggregation network, for low-resolution hyperspectral images. ,in , and These represent the image's height, width, and number of spectral bands, respectively; the corresponding high-resolution image is... Its characteristics are, include: Divide the input image into G overlapping spectral groups: ； Each spectral group It contains c spectral bands, and there are 0 overlapping bands; Design a multi-scale dual attention fusion module MDAF to extract multi-scale spatial spectral features for each spectral group and stitch them together in the spectral dimension to form a local multi-scale feature map; Convolutional layers are applied to extract initial features from local multi-scale feature maps, which are then mapped to the hyperspectral dimension to obtain spatial-spectral features. ; Design a Transformer module for spatial spectrum aggregation S 2 AM, spatial spectral features The system extracts global contextual information across bands, sequentially feeding the input features into N cascaded S-bands. 2 AM, extracting global depth spatial-spectral features The deep spatial-spectral feature fusion result is obtained by using local residual learning. ; In the reconstruction stage, the PixelShuffle algorithm is used to upsample deep features to obtain the target resolution; a two-stage feature alignment strategy is introduced to finally obtain a super-resolution hyperspectral image. .

2. The method for super-resolution of a single hyperspectral image based on a hybrid spatial-spectral aggregation network according to claim 1, characterized in that, Multi-scale spatial-spectral features are extracted for each spectral group and concatenated along the spectral dimension to form a local multi-scale feature map. A convolutional layer is then applied to the local multi-scale feature map for initial feature extraction, which is then mapped to the hyperspectral dimension to obtain the spatial-spectral features. The process is as follows: No. The multi-scale spatial spectral features are represented as follows: ； in, This represents the feature extraction function of the module; After extraction, the features from each group are stitched together along the spectral dimension to form a local multi-scale feature map: ； in, This indicates a spectral dimension splicing operation; Before inputting the concatenated features into the global network, convolutional layers are applied to extract initial features and map them to the hyperspectral dimension. The process is represented as follows: ； in This is a 1×1 convolution operation; This represents the spatial spectral features obtained by mapping the spectral dimensions.

3. The method for super-resolution of a single hyperspectral image based on a hybrid spatial-spectral aggregation network according to claim 2, characterized in that, The Multi-Scale Dual Attention Fusion Module (MDAF) contains the following: First, the input features are fed into three parallel convolutional branches, using 3×3, 5×5, and 7×7 convolutional kernels respectively; then, features of different scales are fused in pairs. set up For the first The input to MDAF, and the features extracted at different scales are: The process is represented as follows: ； ； ； in, , , These represent convolution operations with kernel sizes of 3×3, 5×5, and 7×7, respectively. A dual attention mechanism is introduced to optimize feature responses from both channel and spatial dimensions. Channel attention (CA) models the correlation between channels through global average pooling dimensionality reduction, ReLU activation, and the sigmoid function, adaptively adjusting the importance weights of each spectral channel. ； Spatial attention (SA) combines global average pooling and max pooling operations, and then convolves the concatenated result with a sigmoid activation to obtain a spatial weight map. ； Finally, channel and spatial attention features are integrated through 1×1 convolution, and the enhanced output is obtained by using residual connections: ； in, This represents a 1×1 convolution operation. Indicates the first The output features of each MDAF.

4. The single hyperspectral image super-resolution method based on a hybrid spatial-spectral aggregation network according to claim 3, characterized in that, The deep spatial-spectral feature fusion results are obtained. The process is as follows: Design a Transformer module for spatial spectrum aggregation S 2 AM, from spatial spectral features Extract global context information across bands; sequentially feed the input features into N cascaded S... 2 AM, extracting global depth spatial-spectral features The deep spatial-spectral feature fusion result is obtained by using local residual learning. .

5. The single hyperspectral image super-resolution method based on a hybrid spatial-spectral aggregation network according to claim 4, characterized in that, Spatial spectrum aggregation Transformer module S 2 AM content is as follows: S 2 AM consists of three parts: spectral grouping self-attention S-GSA, windowed self-attention S-WSA, and a 3×3 convolutional layer containing two residual blocks. The first residual block consists of a LayerNorm layer and a self-attention layer, and the second residual block consists of a LayerNorm layer and a multilayer perceptron (MLP). The process is represented as follows: ； ； in, This represents the LayerNorm layer function. This represents a multilayer perceptron. This refers to global features within the module.

6. The single hyperspectral image super-resolution method based on a hybrid spatial spectrum aggregation network according to claim 5, characterized in that: The Spectral Grouped Self-Attention (S-GSA) mechanism combines the complementary properties of deformable convolution and grouped convolution to achieve efficient modeling and dynamic adaptive sampling of hyperspectral channels. Deformable convolution introduces a learnable offset to dynamically adjust the sampling position during convolution computation, adapting to the non-uniformity of spectral distribution. Grouped convolution, through channel grouping and independent convolution computation, achieves efficient spectral grouping representation. Let the input features be... After deformable convolution, grouped convolution, and two 1×1 pointwise convolutions, the query is mapped to the result. ,key Sum Taking a single attention head as an example, the S-GSA calculation is expressed as follows: ； ； ； ； in, This represents a deformable convolution operation. Represents grouped convolution. and For pointwise convolution functions, For the dimension of Q, specifically Finally, the outputs of each attention head are concatenated along the spectral dimension to obtain the output features of the spectral grouping self-attention module.

Images