Hyperspectral image super-resolution method based on double-domain gated convolution attention mechanism

Abstract

The invention discloses a hyperspectral image super-resolution method based on a double-domain gating convolution attention mechanism, which can improve the attention of high-frequency information of a hyperspectral image and improve the super-resolution effect after feature fusion. The method comprises a depth feed-forward convolution module, a non-bureau gate control convolution module, a dual-domain gate control attention module and a reconstruction module. According to the method, firstly, a spatial high-frequency information region in an image is filtered by adopting non-local department controlled convolution (NGC), and meanwhile, attention to a global context is kept through a self-adaptive high-frequency component attention mechanism; under the guidance of spatial domain-spectral domain correlation, a hyperspectral image is reconstructed without influencing details, and then spatial high frequency and texture information of the hyperspectral image is reserved through depth feed-forward convolution (DFC) and double-domain convolution attention. Experimental results prove that the algorithm is superior to other common remote sensing image super-resolution algorithms in subjective visual effects and objective evaluation indexes.

Description

Technical Field

[0001] The embodiments of the present application relate to the fields of image processing technology and remote sensing technology based on a specific computing system, and in particular to a hyperspectral image super-resolution method based on a dual-domain gated convolution attention mechanism. Background Art

[0002] Hyperspectral images provide spectral information in hundreds of continuous bands, providing a richer understanding of ground objects than traditional natural or multispectral images. This rich spectral data enables accurate object localization and unmixing in image scenes, making hyperspectral images invaluable for target detection, geological exploration, and medical diagnosis. Hyperspectral images typically have low spatial resolution due to hardware limitations and atmospheric noise. This raises the common challenge of balancing spatial and spectral resolution in hyperspectral images, as low spatial resolution may cause a single pixel to contain multiple target materials, thereby reducing the accuracy of hyperspectral image processing and affecting detection performance. A promising solution is hyperspectral image super-resolution, which improves spatial and spectral resolution by producing high-resolution hyperspectral images from their low-resolution counterparts.

[0003] The rise of deep learning, especially convolutional neural networks, has promoted significant progress in various computer vision algorithms, including super-resolution algorithms for hyperspectral imagery. Compared with traditional methods based on manually designed priors, deep learning methods based on convolutional neural networks are data-driven, allowing the network to learn appropriate priors based on the inherent characteristics of the dataset. However, convolutional neural networks have limitations in learning global image features. In contrast, attention mechanism networks from natural language processing are better at capturing global features, although their implementation in image processing requires a lot of computing resources, which increases the computational requirements of hyperspectral image super-resolution reconstruction tasks.

[0004] In summary, the technical problems of the existing technology are: convolutional neural networks have limitations in learning global image features, and the use of 2D / 3D mixed convolution methods is prone to introduce feature redundancy and model complexity. This leads to overemphasis on local areas and loss of high-frequency information, reducing the effect of hyperspectral image super-resolution. The extraction of effective features will affect the classification results of the classifier. Samples in hyperspectral data generally have the characteristics of high-dimensional features. How to effectively select features for data, remove irrelevant features, and alleviate the "dimensionality disaster" still has certain challenges; secondly, there is an imbalance problem in hyperspectral data sets. How to pay attention to the characteristics of different domains at the same time has become a difficulty and key research content in the field of hyperspectral image processing. Based on the summary of previous work, further research is conducted on existing problems, and a new machine learning method is proposed, so that in the final result, the overall super-resolution effect is improved by retaining and learning high-frequency information. Summary of the invention

[0005] In order to overcome the above technical defects, the embodiment of the present application provides a hyperspectral image super-resolution method based on the dual-domain gated convolutional attention (DGCA) mechanism, which can improve the attention of high-frequency information of hyperspectral images and improve the super-resolution effect after feature fusion. The hyperspectral image super-resolution network based on dual-domain gated attention in the present application includes: feed-forward deep convolution, non-local gated convolution NGC module, dual-domain gated attention module and reconstruction module, wherein the NGC module mainly includes two parts: adaptive high-frequency information processing and gated convolution;

[0006] The specific scheme of the hyperspectral image super-resolution method based on the dual-domain gated convolutional attention mechanism is as follows:

[0007] S1. Extract shallow features of the input remote sensing image through feedforward deep convolution to obtain features Features Perform residual connection and feature enhancement through linear layer before outputting to non-local gated convolution NGC module;

[0008] S2. In the non-local gated convolution NGC module, the input features are transformed by adjusting the size of the deep convolution sliding window. Adaptively scale to high-frequency regions to extract spatially enhanced features Use batch normalization function to process and activate, and then use information threshold to activate feature X based on gate threshold mechanism again. θ Filter the spectral information, perform deep convolution on the different frequency information and fuse them to obtain the fusion feature X (1) θ ;

[0009] S3. In the dual-domain gated attention module, the fusion feature X of the input is obtained based on the spatial attention mechanism and the spectral attention mechanism. (1) θ Perform attention feature learning to obtain learning feature X a ;

[0010] S4. In the reconstruction module, based on the learned feature X a The image is super-reconstructed by bicubic interpolation upsampling algorithm and deconvolution interpolation upsampling algorithm to obtain the reconstructed feature X output , and will reconstruct the feature X output The reconstructed super-resolution image X is obtained by fusing it with the upsampled original remote sensing image hr .

[0011] Further optionally, the scaling distance is determined according to the step size during the scaling process. The scaling process is dynamically determined based on the information in the high-frequency area, that is, the distribution of the high-frequency information in the deep convolution sliding window. If the high-frequency information in the window reaches or exceeds the set threshold, the sub-area with the highest high-frequency information and the largest coverage within the selection range will be used as the new sliding window.

[0012] Further optionally, the above fusion feature X of the input is processed by the spatial attention mechanism and the spectral attention mechanism (1) θ Conduct attention feature learning, including:

[0013] S3.1. Based on the spatial attention mechanism, the fusion feature X of the input (1) θ Perform frequency domain segmentation to obtain the spatial local feature X spa , whose characteristic dimension is Then the spatial local feature X spa Downsampling is performed, and the feature dimension after downsampling is: Then, the downsampled spatial features are threshold segmented by the gated suppression feature coefficient in the gated threshold mechanism to suppress the redundant features in the spatial information, and the segmented features are bicubic interpolated again to obtain the enhanced spatial features;

[0014] S3.2. Based on the spectral attention mechanism, the fusion feature X of the input (1) θ Perform pixel segmentation to obtain spectral features X spe , for the spectral feature X spe After normalization, the spatial features are fused with the downsampled ones to mine the spectral-spatial correlation of the image; then, the linear layer is used to extract and enhance the features, and the redundant features in the spectral dimension are suppressed according to the gated suppression feature coefficient after the features are activated by the activation function. The spectral-spatial correlation of the fused features at different scales is learned by using the dilated convolution to obtain the enhanced spectral features; finally, the enhanced spectral features are fused with the enhanced spatial features to obtain the learning features X a , whose characteristic dimension is

[0015] Further optionally, in the reconstruction process of S4, different interpolation algorithms are used according to different task magnifications, as follows:

[0016] S4.1. When the super-resolution ratio is an even number, the bicubic interpolation upsampling algorithm can significantly enhance the spatial expression ability of the image. The formula is as follows:

[0017] X output =(X a ,s)↑ bic, where ↑ bic is the bicubic interpolation upsampling algorithm, s is the upsampling ratio;

[0018] S4.2. When the super-resolution ratio is an odd number, the deconvolution interpolation upsampling algorithm can effectively fit the difference between the original image features and the learned features. The formula is as follows:

[0019] X output =Deconv(X a ,s), where Deconv is the deconvolution interpolation upsampling algorithm, and s is the upsampling ratio;

[0020] S4.3. The reconstructed super-resolution image is obtained by fusing the upsampled input image with the feature. The formula is as follows:

[0021] X hr =(X lr , s)↑ bic +X output , where X hr is the reconstructed super-resolution image, i.e., the hyperspectral image, (X lr , s)↑ bic is the original remote sensing image after upsampling.

[0022] The embodiments of the present application adopt the above technical solution to achieve the following technical effects:

[0023] This application proposes a new hyperspectral image super-resolution network based on dual-domain gated attention. This method uses non-local gated convolution and extended sliding window to fuse and extract local high-frequency information and global low-frequency information from the image. Different from previous methods, this application introduces a dual-domain gated attention mechanism to focus on high-frequency information in the spatial and spectral domains, which can ensure that the low-frequency information in the image is fully learned while enhancing the details of the reconstructed image. Deep feedforward convolution is used to prevent the network from over-focusing on local information.

[0024] In this application, the obtained hyperspectral data set is first preprocessed and the high-frequency features in the spatial domain data and the spectral domain data are preliminarily preprocessed. The purpose is to simplify the calculation amount in the subsequent steps to distinguish high-frequency information from background features. Secondly, the data in different sliding windows are subjected to a large kernel selection algorithm to dynamically select local areas containing complete regional information while paying attention to the global spectral features. Then, based on a given high-frequency information threshold, high-frequency information and Beijing information are distinguished, and global features and local features are learned and extracted, that is, the DGCA mechanism. Subsequently, the fused feature image is upsampled through the bicubic interpolation and deconvolution algorithm, and a super-resolution hyperspectral image is obtained. This algorithm enhances the complementarity between effective features while ensuring that high-frequency features are not lost, and uses an improved feature selection method to remove redundant features. Compared with the traditional hyperspectral image super-resolution algorithm based on convolutional neural networks, this method has higher clarity and richer image information. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0026] Figure 1 A schematic diagram of the structure of a hyperspectral image super-resolution network based on dual-domain gated attention in this application;

[0027] Figure 2 A schematic diagram of the structure of the non-local gated convolution module (NGC) in this application;

[0028] Figure 3 A flowchart of a hyperspectral image super-resolution method based on dual-domain gated attention in this application;

[0029] Figure 4 This is the visualization and explanation of the local data of the CAVE dataset and different super-resolution magnifications of this application;

[0030] Figure 5 This is a demonstration and explanation of the visualization effect of this application on local data of the HARVARD dataset and at different super-resolution magnifications;

[0031] Figure 6 Compare and illustrate the visualization results of this application on the CAVE dataset with other advanced algorithms;

[0032] Figure 7 This paper compares and explains the visualization results of this application on the HARVARD dataset and other advanced algorithms. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0034] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0035] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0036] First, a novel hyperspectral image super-resolution network based on dual-domain gated attention proposed in the embodiment of the present application is described. Its specific structure is as follows: Figure 1 and Figure 2 As shown:

[0037] The hyperspectral image super-resolution network based on dual-domain gated attention mainly includes: feed-forward deep convolution, non-local gated convolution NGC module, dual-domain gated attention module and reconstruction module, among which the NGC module mainly includes two parts: adaptive high-frequency information processing and gated convolution.

[0038] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the following describes the related technologies. Figure 1 and Figure 2 , see Figure 3 In an embodiment of the present application, a hyperspectral image super-resolution method based on a dual-domain gated convolutional attention mechanism includes:

[0039] S1. Extract shallow features of the input remote sensing image through feedforward deep convolution to obtain features Features After residual connection and feature enhancement through linear layer, the output is sent to the non-local gated convolution NGC module.

[0040] Specifically, remote sensing images are hyperspectral images, which can be divided into two types: low spatial resolution-high spectral resolution images, and high spatial resolution-high spectral resolution images. The feed-forward depthwise convolution can be a separable convolution, that is, a feed-forward depthwise separable convolution, and each feed-forward depthwise separable convolution includes a depthwise convolution and a conventional convolution.

[0041] Assume that the input remote sensing image is The shallow feature extraction specifically includes: grouping the input remote sensing images into batch blocks, and for each batch of data, using feedforward deep convolution to extract shallow features. The calculation formula of the deep convolution is:

[0042]

[0043] Among them, Conv is the convolution operation, k w , k c are the convolution kernel sizes of the feedforward residual convolution operation and the conventional convolution operation in the depthwise convolution. The above convolution kernel configuration can efficiently extract local information while reducing the parameter scale. In order to further enhance the network's expressive power and feature retention capabilities.

[0044] Residual connections are introduced in the convolutional layer to preserve prior features and alleviate the gradient vanishing problem. Residual connections add skip paths in the feature transfer process, allowing shallow information to be directly integrated with deep information, thereby improving the learning effect and training stability of the network.

[0045] After the shallow features are extracted, the linear layer is used to consolidate and strengthen these features. The purpose of this step is to integrate the local features obtained in the convolution operation into a more abstract and robust representation, laying the foundation for the subsequent deep feature extraction or classification tasks. Through this series of steps, the shallow information of the input image can be fully mined and represented.

[0046] S2. In the non-local gated convolution NGC module, the input features are transformed by adjusting the size of the deep convolution sliding window. Adaptively scale to high-frequency regions to extract spatially enhanced features Use batch normalization function to process and activate, and then use information threshold to activate feature X based on gate threshold mechanism again. θ Filter the spectral information, perform deep convolution on the different frequency information and fuse them to obtain the fusion feature X (1) θ .

[0047] Specifically, the input is adaptively scaled to the high-frequency area by adjusting the size of the deep convolution DWC sliding window to capture the texture details and edge sharpness in the image. The formula for dynamic adjustment is as follows:

[0048]

[0049] Among them, k1 is the sliding window size before scaling, and k2 is the sliding window size after scaling.

[0050] During the scaling process, the scaling distance is determined according to the step size. The scaling process is dynamically determined based on the information in the high-frequency area, that is, the distribution of high-frequency information in the deep convolution DWC sliding window. If the high-frequency information in the window reaches or exceeds the set threshold, the sub-area with the highest high-frequency information and the largest coverage within the selection range will be used as a new sliding window. This operation can significantly enhance the pertinence and effectiveness of feature extraction. The extracted features Batch normalization and activation are performed to standardize the feature distribution and improve the nonlinear expression ability of the model. Next, the spatial enhancement features are enhanced through the gating threshold mechanism. Further division is performed into high-frequency features and background features. The formula of the gate threshold mechanism is as follows:

[0051]

[0052] Among them, BatchNorm is the batch normalization function, e is the natural number basis, and τ is the learnable gated suppression feature coefficient. It is initially used as a hyperparameter and dynamically adjusted to the range of [0.1, 0.5] according to the high-frequency information threshold. The high-frequency information in the space determines the texture details and boundary sharpness of the image, while the low-frequency information represents the background information and color information. Threshold differentiation of high-frequency information helps to effectively control the sharpening information of local areas and help the network to perform high-quality fitting learning of texture features. Subsequently, feature fusion is performed on different frequency information separately. The formula is as follows:

[0053]

[0054] Among them, k w is the depthwise separable convolution kernel size, and α is the feature split ratio, which determines the distribution ratio of high-frequency and low-frequency features. To ensure the robustness of the network, the feature split ratio is set to the same value as the gated suppression feature coefficient. This targeted processing not only improves the sharpening effect of the high-frequency area, but also ensures the integrity of the low-frequency background information, thereby helping the network to achieve high-quality fitting learning of information of different frequencies, and ultimately forming a feature representation with stronger expressive power.

[0055] It is easy to know that the spatial attention based on non-local gated convolution not only focuses on the local information in the convolution sliding window, but also obtains the global spatial information to assist the sliding window in extracting features of local information. Specifically: Each feature vector of is extracted through a convolutional layer with a kernel of k1. Before feature extraction, the image is downsampled to make the image size 1 / 4 of the original. The extracted features are expanded through adaptive windows. The amount of high-frequency information in a given sliding window is As the sliding window expands outward, The rate of change (i.e. gradient) Represents the richness of high-frequency information in the region. When the rate of change is less than the given information threshold, that is, When , it is considered that the high-frequency information in the area reaches the threshold, and the kernel size k2 of the enlarged sliding window is determined. The neighborhood information of the image is extracted through the convolution layer with the new kernel size to obtain the spatial enhancement feature right After batch normalization and activation function, we get the activation feature X θ .

[0056] S3. In the dual-domain gated attention module, the fusion feature X of the input is obtained based on the spatial attention mechanism and the spectral attention mechanism. (1) θ Perform attention feature learning to obtain learning feature X a .

[0057] Specifically, the input fusion feature X is obtained through the spatial attention mechanism and the spectral attention mechanism. (1) θ Conduct attention feature learning, including:

[0058] S3.1. Based on the spatial attention mechanism, the fusion feature X of the input (1) θ Perform frequency domain segmentation to obtain the spatial local feature X spa , whose characteristic dimension is Then the spatial local feature X spa Downsampling is performed, and the feature dimension after downsampling is: Then, the downsampled spatial features are threshold segmented by the gated suppression feature coefficient in the gated threshold mechanism to suppress the redundant features in the spatial information, and the segmented features are bicubic interpolated again to obtain the enhanced spatial features;

[0059] It should be noted that the purpose of downsampling is to better aggregate features to enhance the local feature expression capability.

[0060] S3.2. Based on the spectral attention mechanism, the fusion feature X of the input (1)θ Perform pixel segmentation to obtain spectral features X spe , for the spectral feature X spe After normalization, the spatial features are fused with the downsampled ones to mine the spectral-spatial correlation of the image; then, the linear layer is used to extract and enhance the features, and the redundant features in the spectral dimension are suppressed according to the gated suppression feature coefficient after the features are activated by the activation function. The spectral-spatial correlation of the fused features at different scales is learned by using the dilated convolution to obtain the enhanced spectral features; finally, the enhanced spectral features are fused with the enhanced spatial features to obtain the learning features X a , whose characteristic dimension is

[0061] Specifically, during pixel segmentation in the above spectral attention mechanism, the attention weight of each pixel is calculated through the query matrix Q, the key-value matrix K and the value matrix V to fully learn the spatial features of the image. The calculation formulas of the query matrix Q, the key-value matrix K and the value matrix V are as follows:

[0062]

[0063] Among them, f conv is the convolution operation, f softmax is the normalization operation, f fusion For fusion operation.

[0064] S4. In the reconstruction module, based on the learned feature X a The image is super-reconstructed by bicubic interpolation upsampling algorithm and deconvolution interpolation upsampling algorithm to obtain the reconstructed feature X output , and will reconstruct the feature X output The reconstructed super-resolution image X is obtained by fusing it with the upsampled original remote sensing image hr .

[0065] Specifically, the learning features of the input Different interpolation algorithms are used according to different task magnifications:

[0066] S4.1. When the super-resolution ratio is an even number, the bicubic interpolation upsampling algorithm can significantly enhance the spatial expression ability of the image. The formula is as follows:

[0067] X output =(X a ,s)↑ bic , where ↑ bic is the bicubic interpolation upsampling algorithm, s is the upsampling ratio;

[0068] S4.2. When the super-resolution ratio is an odd number, the deconvolution interpolation upsampling algorithm can effectively fit the difference between the original image features and the learned features. The formula is as follows:

[0069] X output =Deconv(X a ,s), where Deconv is the deconvolution interpolation upsampling algorithm, and s is the upsampling ratio;

[0070] S4.3. The reconstructed super-resolution image is obtained by fusing the upsampled input image with the feature. The formula is as follows:

[0071] X hr =(X lr , s)↑ bic +X output , where (X lr , s)↑ bic is the original remote sensing image after upsampling.

[0072] Get X hr To reconstruct a super-resolution image, i.e., a hyperspectral image, according to the method, the reconstructed image learns the high-frequency information of the image based on dual-domain gated convolution while retaining the overall information richness of the image, thereby further enhancing the texture details of the image and the visualization effect after super-resolution reconstruction.

[0073] In order to verify the performance of the hyperspectral image super-resolution method based on the dual-domain gated convolutional attention mechanism, different methods based on different underlying logics were selected as comparison methods to verify the advancement and effectiveness of the proposed method. The visualization results are shown in Figure 2. Figure 4 , Figure 5 , Figure 6 and Figure 7 shown.

[0074] Specifically, Figure 4 Reconstruction effect for CAVE dataset (fourfold super resolution), Figure 4 The pseudo-color images in the middle represent the difference between the reconstruction result and the original image. (a): G original image, (b): grayscale image, (c): GDRRN, (d): DCNN;, (e) MCNet;, (f): SSPSR;, (g): SRDNet, (h): this application.

[0075] Figure 5 Reconstruction effect for Harvard dataset (fourfold super resolution), Figure 5 The pseudo-color images in the middle represent the difference between the reconstruction result and the original image. (a): original image, (b): grayscale image, (c): GDRRN, (d): DCNN, (e): MCNet, (f): SSPSR, (g): SRDNet, (h): this application.

[0076] Figure 6 For this application, the comparison method is presented in the CAVE dataset. Figure 6 It can be seen that this application has richer high-frequency information and peak signal-to-noise ratio than DCNN and GDRRN.

[0077] Figure 7 For this application, the comparison method is presented on the Harvard dataset. Figure 7 It can be seen that this application has richer high-frequency information and peak signal-to-noise ratio than DCNN and GDRRN.

[0078] Specifically, GDRRN stands for Grouped Deep Recursive Residual Network, DCNN stands for Diffused Convolutional Neural Network, MCNet stands for Hybrid 2D / 3D Convolutional Network, SSPSR stands for Spatial Spectral Prior, and SRDNet stands for Dual Domain Hybrid Convolution.

[0079] Five different evaluation indicators are used in this application, namely, peak signal-to-noise ratio (PSNR), structural similarity (SSIM), spectral angle similarity (SAM), relative root mean square error (RRMSE) and root mean square spectral absolute error ratio (ERGAS). The experimental results are shown in Tables 1 and 2, where Table 1 shows the objective evaluation indicators of the hyperspectral image super-resolution method based on the dual-domain gated convolution attention mechanism on the CAVE dataset, and Table 2 shows the objective evaluation indicators of the hyperspectral image super-resolution method based on the dual-domain gated convolution attention mechanism on the Harvard dataset.

[0080] The present application achieved the best results on two hyperspectral image super-resolution datasets representing outdoor scenes and indoor scenes respectively, proving that the method of the present invention is superior to other advanced image super-resolution methods available today.

[0081] Table 1

[0082]

[0083] Table 2

[0084]

[0085] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0086] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A hyperspectral image super-resolution method based on dual-domain gated convolutional attention mechanism, characterized in that: The hyperspectral image super-resolution network based on dual-domain gated attention includes: feed-forward deep convolution, non-local gated convolution NGC module, dual-domain gated attention module and reconstruction module, among which the NGC module mainly includes two parts: adaptive high-frequency information processing and gated convolution; Hyperspectral image super-resolution methods include: S1. Extract shallow features of the input remote sensing image through feedforward deep convolution to obtain features Features Perform residual connection and feature enhancement through linear layer before outputting to non-local gated convolution NGC module; S2. In the non-local gated convolution NGC module, the input features are transformed by adjusting the size of the deep convolution sliding window. Adaptively scale to high-frequency regions to extract spatially enhanced features Use batch normalization function to process and activate to get activation feature X θ , again based on the gated threshold mechanism, the activation feature X is activated by the information threshold θ Spectral information is screened, and then different frequency information is deeply convolved and fused to obtain the fusion feature X (1) θ ; S3. In the dual-domain gated attention module, the fusion feature X of the input is obtained based on the spatial attention mechanism and the spectral attention mechanism. (1) θ Perform attention feature learning to obtain learning feature X a ; S4. In the reconstruction module, based on the learned feature X a The image is super-reconstructed by bicubic interpolation upsampling algorithm and deconvolution interpolation upsampling algorithm to obtain the reconstructed feature X output , and will reconstruct the feature X output The reconstructed super-resolution image X is obtained by fusing it with the upsampled original remote sensing image hr .

2. The hyperspectral image super-resolution method according to claim 1, characterized in that: Remote sensing images are The shallow feature extraction specifically includes: grouping the input remote sensing images into batch blocks, and for each batch of data, using feedforward deep convolution to extract shallow features. The calculation formula of the deep convolution is: Among them, Conv is the convolution operation, k w , k c They are the convolution kernel sizes of the feedforward residual convolution operation and the conventional convolution operation in the depthwise convolution, respectively; Residual connection adds a skip path in the feature transfer process, so that shallow information can be directly integrated with deep information.

3. The hyperspectral image super-resolution method according to claim 1, characterized in that: The formula for dynamically adjusting the size of the depth convolution sliding window is: Where k1 is the sliding window size before scaling, k2 is the sliding window size after scaling, and DWC represents depthwise convolution.

4. The hyperspectral image super-resolution method according to claim 1 or 3, characterized in that: During the scaling process, the scaling distance is determined according to the step size. The scaling process is dynamically determined based on the information in the high-frequency area, that is, the distribution of high-frequency information in the deep convolution sliding window. If the high-frequency information in the window reaches or exceeds the set threshold, the sub-area with the highest high-frequency information and the largest coverage within the selection range will be used as the new sliding window.

5. The hyperspectral image super-resolution method according to claim 1, characterized in that: The formula for the gating threshold mechanism is as follows: Among them, BatchNorm is the batch normalization function, e is the natural number base, and τ is the learnable gated inhibition feature coefficient; The calculation formula for deep convolution and fusion of different frequency information is as follows: Among them, k w is the depthwise separable convolution kernel size, and α is the feature segmentation ratio, which determines the distribution ratio of high-frequency and low-frequency features.

6. The hyperspectral image super-resolution method according to claim 1, characterized in that: The fusion feature X of the input is processed through the spatial attention mechanism and the spectral attention mechanism (1) θ Conduct attention feature learning, including: S3.

1. Based on the spatial attention mechanism, the fusion feature X of the input (1) θ Perform frequency domain segmentation to obtain the spatial local feature X spa , whose characteristic dimension is Then the spatial local feature X spa Downsampling is performed, and the feature dimension after downsampling is: Then, the downsampled spatial features are threshold segmented by the gated suppression feature coefficient in the gated threshold mechanism to suppress the redundant features in the spatial information, and the segmented features are bicubic interpolated again to obtain the enhanced spatial features; S3.

2. Based on the spectral attention mechanism, the fusion feature X of the input (1) θ Perform pixel segmentation to obtain spectral features X spe , for the spectral feature X spe After normalization, the spatial features are fused with the downsampled ones to mine the spectral-spatial correlation of the image; then, the linear layer is used to extract and enhance the features, and the redundant features in the spectral dimension are suppressed according to the gated suppression feature coefficient after the features are activated by the activation function. The spectral-spatial correlation of the fused features at different scales is learned by using the dilated convolution to obtain the enhanced spectral features; finally, the enhanced spectral features are fused with the enhanced spatial features to obtain the learning features X a , whose characteristic dimension is 7. The hyperspectral image super-resolution method according to claim 6, characterized in that: In the above spectral attention mechanism, the attention weight of each pixel is calculated through the query matrix Q, key-value matrix K and value matrix V during pixel segmentation to fully learn the spatial features of the image. The calculation formulas of the query matrix Q, key-value matrix K and value matrix V are as follows: Among them, f conv is the convolution operation, f softmax is the normalization operation, f fusion For fusion operation.

8. The hyperspectral image super-resolution method according to claim 1, characterized in that: In the reconstruction process of S4, different interpolation algorithms are used according to different task magnifications, as follows: S4.

1. When the super-resolution ratio is an even number, the bicubic interpolation upsampling algorithm can significantly enhance the spatial expression ability of the image. The formula is as follows: X output =(X a ,s)↑ bic , where ↑ bic is the bicubic interpolation upsampling algorithm, s is the upsampling ratio; S4.

2. When the super-resolution ratio is an odd number, the deconvolution interpolation upsampling algorithm can effectively fit the difference between the original image features and the learned features. The formula is as follows: X output =Deconv(X a ,s), where Deconv is the deconvolution interpolation upsampling algorithm, and s is the upsampling ratio; S4.

3. The reconstructed super-resolution image is obtained by fusing the upsampled input image with the feature. The formula is as follows: X hr =(X lr , s)↑ bic +X output , where X hr is the reconstructed super-resolution image, i.e., the hyperspectral image, (X lr , s)↑ bic is the original remote sensing image after upsampling.

Images