Multispectral Image Super-Resolution Reconstruction Method Based on Adaptive Spectral Feature Fusion Network

Abstract

This application discloses a multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network, relating to the field of remote sensing image technology. The method includes: acquiring a panchromatic image and its corresponding multispectral image; fusing the panchromatic image and multispectral image through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. Each of the encoder, bottleneck module, and decoder includes a feature enhancement feedforward network. Both the bottleneck module and decoder include a High-Resolution Mechanism (HRM), which includes an Assay Network (ASTM). The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features. By avoiding redundant features in the multispectral image, the quality of the reconstructed multispectral image is improved.

Description

Technical Field

[0001] This application relates to the field of remote sensing image technology, and in particular to a method for super-resolution reconstruction of multispectral images based on an adaptive spectral feature fusion network. Background Technology

[0002] Multispectral imagery is a key tool in the field of remote sensing, enabling the effective acquisition of data across multiple spectral bands and the capture of the unique characteristics of various materials.

[0003] To enhance the spatial resolution of multispectral images, they are often fused with panchromatic images to balance high spatial resolution and rich spectral information. Currently, adaptive panchromatic sharpening neural networks are commonly used to improve panchromatic sharpening performance, effectively preserving the spectral and spatial information of the source images during fusion. However, multispectral images contain data from multiple bands, which may be highly correlated, leading to spectral feature redundancy. Current panchromatic sharpening neural networks need to extract these redundant spectral features, resulting in the reconstructed multispectral image containing these redundant spectral features, thus degrading the quality of the reconstructed multispectral image.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method for super-resolution reconstruction of multispectral images based on an adaptive spectral feature fusion network, which aims to solve the technical problem that the reconstructed multispectral images contain redundant spectral features, thereby leading to a decrease in the quality of the reconstructed multispectral images.

[0006] To achieve the above objectives, this application proposes a multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network. The method includes:

[0007] Acquire the panchromatic image and its corresponding multispectral image;

[0008] A reconstructed spectral image is obtained by fusing the panchromatic image and the multispectral image through a preset spectral feature fusion network. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include a Hidden Resonance Memory (HRM). The HRM includes an Assay Spectrum (ASTM). The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0009] In one embodiment, the bottleneck module further includes an upsampling module;

[0010] The step of fusing the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain the reconstructed spectral image includes:

[0011] Effective spatial features are extracted from the panchromatic image through feature enhancement feedforward processing in the encoder.

[0012] The upsampling module extracts features whose resolution meets a preset expectation from the effective spatial features to obtain spatial features to be merged.

[0013] The bottleneck module fuses the spatial features to be merged with the multispectral image to obtain an initial fused image;

[0014] The HRM is used to extract and fuse features from the initial fused image based on frequency separation to obtain a reconstructed spectral image.

[0015] In one embodiment, the HRM further includes a convolutional layer and a spectral attention module;

[0016] The step of extracting and fusing features from the initial fused image according to frequency separation using the HRM to obtain a reconstructed spectral image includes:

[0017] The HRM separates high-frequency information with bands higher than a preset separation band and low-frequency information with bands lower than or equal to the preset separation band from the initial fused image.

[0018] The high-frequency information is weighted and convolved using the ASTM algorithm to obtain high-frequency features.

[0019] The spectral attention module extracts global and spatial texture features from the low-frequency information to obtain supplementary features.

[0020] The high-frequency features are fused with the supplementary features using the HRM to obtain a reconstructed spectral image.

[0021] In one embodiment, the ASTM includes a predictor and a token mixer, the predictor including a token masking module and a spectral attention weighting module;

[0022] The step of performing weighted convolution processing on the high-frequency information using the ASTM algorithm to obtain high-frequency features includes:

[0023] The high-frequency information is preprocessed in the predictor to obtain an intermediate feature map;

[0024] The token mask module performs spatial dimension feature sorting on the intermediate feature map to obtain the spectral token mask of each intermediate feature in the intermediate feature map.

[0025] The token mixer extracts key spectral features corresponding to the spectral token mask from each of the intermediate features.

[0026] The intermediate features are processed by the spectral attention weighting module to obtain spectral fusion features.

[0027] The key spectral features are fused with the spectral fusion features to obtain high-frequency features.

[0028] In one embodiment, the step of extracting global features and spatial texture features from the low-frequency information using the spectral attention module to obtain supplementary features includes:

[0029] The low-frequency information is upsampled according to a preset ratio using the spectral attention module to obtain global features and spatial texture features.

[0030] Based on the weights of the low-frequency information, supplementary features are selected from the global features and the spatial texture features.

[0031] In one embodiment, the feature enhancement feedforward network includes a gating mechanism and a pixel attention mechanism;

[0032] The step of extracting effective spatial features from the panchromatic image through feature enhancement feedforward network processing in the encoder includes:

[0033] The gating mechanism is used to divide the panchromatic image into redundant spatial features and representative spatial features according to the band.

[0034] The representative spatial features are obtained by performing grouped convolutions and depthwise convolutions on the convolutional layers in the feature enhancement feedforward network.

[0035] By using the convolutional layers in the feature enhancement feedforward network, the redundant spatial features are subjected to deep convolution to obtain supplementary spatial features.

[0036] The pixel attention mechanism is used to fuse the representative features and the supplementary spatial features pixel by pixel to obtain effective spatial features.

[0037] Furthermore, to achieve the above objectives, this application also proposes a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network, wherein the multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network includes:

[0038] The acquisition module is used to acquire panchromatic images and their corresponding multispectral images;

[0039] The reconstruction module is used to fuse the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include a Hidden Resonance Memory (HRM). The HRM includes an Assay Spectrum (ASTM). The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0040] Furthermore, to achieve the above objectives, this application also proposes a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described above.

[0041] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described above.

[0042] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described above.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] Because the preset spectral feature fusion network includes a feature enhancement feedforward network and an ASTM (Adaptive Spectral Token Mixer), after the acquired panchromatic image and the corresponding multispectral image are input into the preset spectral feature fusion network, the feature enhancement feedforward network identifies effective spatial features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spatial features. Similarly, the ASTM identifies effective spectral features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spectral features, thus avoiding excessive redundant features in the output reconstructed multispectral image. By avoiding too many redundant features in the reconstructed multispectral image, the accuracy and detail of the reconstructed multispectral image are improved, thereby enhancing the quality of the reconstructed multispectral image. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating an embodiment of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network provided in this application.

[0048] Figure 2 This is a schematic diagram of the architecture of the preset spectral feature fusion network in the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network of this application;

[0049] Figure 3 This is a flowchart illustrating Embodiment 2 of the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network provided in this application;

[0050] Figure 4 This is a flowchart illustrating Embodiment 3 of the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network provided in this application;

[0051] Figure 5 This is a schematic diagram of the architecture of ASTM in the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network in this application;

[0052] Figure 6 This is a schematic diagram of the module structure of the multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network according to an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network in the embodiments of this application.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0056] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0057] The main solution of this application embodiment is: a remote sensing image processor acquires a panchromatic image and the corresponding multispectral image; the panchromatic image and the multispectral image are fused through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include a High-Resolution Mechanism (HRM). The HRM includes an Advanced Spectral Mechanism (ASTM). The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0058] In this embodiment, for ease of description, the remote sensing image processor will be used as the execution subject in the following description.

[0059] To enhance the spatial resolution of multispectral images, they are often fused with panchromatic images to balance high spatial resolution and rich spectral information. Currently, adaptive panchromatic sharpening neural networks are commonly used to improve panchromatic sharpening performance, effectively preserving the spectral and spatial information of the source images during fusion. However, multispectral images contain data from multiple bands, which may be highly correlated, leading to spectral feature redundancy. Current panchromatic sharpening neural networks need to extract these redundant spectral features, resulting in the reconstructed multispectral image containing these redundant spectral features, thus degrading the quality of the reconstructed multispectral image.

[0060] This application provides a solution in which a feature enhancement feedforward network and ASTM are set in the preset spectral feature fusion network. After the acquired panchromatic image and the corresponding multispectral image are input into the preset spectral feature fusion network, the feature enhancement feedforward network identifies effective spatial features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spatial features. The ASTM identifies effective spectral features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spectral features, thus avoiding the output reconstructed multispectral image from containing too many redundant features. By avoiding too many pairs of redundant features in the reconstructed multispectral image, the accuracy and detail of the reconstructed multispectral image are improved, thereby improving the quality of the reconstructed multispectral image.

[0061] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or remote sensing image processor capable of performing the above functions. The following description uses a remote sensing image processor as an example to illustrate this embodiment and the subsequent embodiments.

[0062] Based on this, embodiments of this application provide a multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network of this application.

[0063] In this embodiment, the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network includes steps S10 to S20:

[0064] Step S10: Obtain the panchromatic image and the corresponding multispectral image;

[0065] It should be noted that panchromatic images are mixed images acquired by remote sensors covering the entire visible light spectrum (generally defined between 0.38-0.76 μm). They are typically single-channel grayscale images. Because panchromatic images are single-band images, they are displayed as grayscale images on the map, lacking color information to show the colors of ground features, thus providing limited spectral information. They generally have high spatial resolution, providing more spatial details and displaying a wealth of spatial information about ground features. Multispectral images, on the other hand, are images acquired by remote sensors across multiple discrete bands (usually within the visible and near-infrared range), typically containing several to a dozen bands. Each band records light radiation information within a specific wavelength range, thus providing rich color information about ground features. However, due to the need to balance spectral and spatial resolution, the spatial resolution is usually lower.

[0066] It is understandable that, since panchromatic images have high spatial resolution and multispectral images have high spectral resolution, it is necessary to acquire both panchromatic and multispectral images in order to make multispectral images have high spatial resolution.

[0067] In practical implementation, panchromatic and multispectral images can be received by a remote sensor and uploaded to a remote sensing image processor. This allows the remote sensing image processor to obtain the multispectral image corresponding to the panchromatic image, facilitating subsequent fusion and reconstruction of the panchromatic and multispectral images using neural networks.

[0068] Step S20: The panchromatic image and the multispectral image are fused through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include an HRM. The HRM includes ASTM. The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0069] It should be noted that the pre-defined spectral feature fusion network is a neural network model used to fuse panchromatic and multispectral images. Its purpose is to combine the advantages of panchromatic and multispectral images to generate a reconstructed spectral image with high spatial resolution and accurate spectral information. The encoder, part of the neural network, is responsible for converting input data (such as a panchromatic image) into a set of feature representations. These representations typically have low dimensionality but contain the main information of the input data. The bottleneck module, usually located in the middle of the neural network, compresses the input data to a lower dimension before restoring it to the original dimension through the decoder. This helps extract the main features of the data and remove redundant information. The decoder in the neural network is responsible for converting the feature representations generated by the encoder back to the original data dimension. The feature enhancement feedforward network is a neural network structure used to identify and enhance effective features. It can identify effective spatial features in the input data and suppress the extraction of redundant spatial features, thereby improving the quality and information content of the data. The HRM (High-frequency Refinement Module) can effectively separate high-frequency information from low-frequency features through fuzzing and downsampling operations, thereby improving processing efficiency. ASTM can effectively identify the most important features while filtering out redundant information, reconstructing high-resolution multispectral images, and improving the efficiency of spectral information utilization. Effective spectral features are those spectral information in spectral data that significantly reflect the characteristics of a target object or phenomenon. These features typically exhibit a clear response in specific spectral bands and can be used to distinguish different substances or identify specific objects. Redundant spectral features contain a large amount of repetitive or irrelevant information in hyperspectral data. This information does not contribute substantially to target detection and classification analysis and may instead increase computational complexity and reduce model performance. Effective spatial features are features in the spatial domain that significantly reflect the spatial structure and morphology of a target object or phenomenon. These features typically include information such as edges, textures, and shapes, and can be used to describe the spatial distribution and interrelationships of objects. Redundant spatial features are spatial features extracted due to the spatial distribution characteristics of ground features and sensor resolution limitations, containing a large amount of repetitive or irrelevant spatial information.

[0070] It is understandable that by introducing ASTM into the preset spectral feature fusion network, effective spectral features that contribute to image reconstruction can be effectively identified, while redundant spectral features that do not significantly improve image quality or even have a negative impact can be suppressed, thereby improving the quality of the reconstructed spectral image. Furthermore, ASTM utilizes learnable spectral token masks and deformable spectral attention mechanisms to effectively focus on important feature maps, optimize the allocation of computational resources, and improve the fusion efficiency of the preset spectral feature fusion network for panchromatic and multispectral images.

[0071] Understandably, by setting a feature enhancement feedforward network in the preset spectral feature fusion network, the effective spatial features in the image can be preserved and enhanced, and redundant spatial information can be reduced. This can further improve the spatial resolution and detail representation of the reconstructed spectral image. In other words, through the feature enhancement feedforward network, the spatial texture information of the panchromatic image can be better transferred to the multispectral image. At the same time, the module adopts a segmentation-enhancement strategy to suppress the extraction of redundant spatial features, promote the flow of valuable signals, and ensure that the complementary information from the multispectral image and the panchromatic image is fully utilized.

[0072] Understandably, a feature enhancement feedforward network is set up in the encoder and decoder parts of the preset spectral feature fusion network to better transfer the spatial texture information of the panchromatic image to the multispectral image. The feature enhancement feedforward network uses a segmentation enhancement strategy to suppress redundant extraction of spatial features and promote the flow of valuable signals. In order to fully focus on the high-frequency information and low-rank characteristics in the spectral data, an HRM (High-frequency Refinement Module) is set up in the preset spectral feature fusion network, and ASTM is set up in the HRM to effectively focus on important feature maps by utilizing learnable spectral token masks and deformable spectral attention, and optimize the allocation of computational resources. Through the integration of these two mechanisms, the preset spectral feature fusion network can effectively identify key information and filter out redundant parts, thereby achieving high-quality reconstruction of multispectral images.

[0073] In the specific implementation, refer to Figure 2Each stage of the encoder mainly consists of a basic module and a convolutional layer. The basic module contains N1 Feature Enhancement Feed-forward Networks (FEFNs). The basic module in the decoder consists of N2 HRMs and N2 FEFNs to reduce the computational burden and number of parameters of the pre-defined spectral feature network, improve computational efficiency, and promote the flow of low-level features. A bottleneck module is introduced between the encoder and decoder. This module includes an upsampling module, a convolutional layer, N3 HRMs, and N3 FEFNs to construct a more hierarchical representation. The input parts of the encoder and decoder are connected to their respective output parts to form residual connections to prevent the loss of deep information. Finally, a convolutional layer is used to process the result of adding the decoder output and the residuals to obtain the final output.

[0074] This embodiment provides a multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network. Since the preset spectral feature fusion network includes a feature enhancement feedforward network and an adaptive spectral feature fusion (ASTM), after the acquired panchromatic image and corresponding multispectral image are input into the preset spectral feature fusion network, the feature enhancement feedforward network identifies effective spatial features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spatial features. Similarly, the ASTM identifies effective spectral features from the panchromatic image and / or multispectral image and suppresses the extraction of redundant spectral features, thus avoiding excessive redundant features in the output reconstructed multispectral image. By avoiding too many redundant features in the reconstructed multispectral image, the accuracy and detail of the reconstructed multispectral image are improved, thereby enhancing the quality of the reconstructed multispectral image.

[0075] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. In addition, the bottleneck module further includes an upsampling module, please refer to... Figure 3 Step S20 also includes steps S21 to S24:

[0076] Step S21: Extract effective spatial features from the panchromatic image through feature enhancement feedforward network processing in the encoder;

[0077] Step S22: Through the upsampling module, extract features from the effective spatial features whose resolution meets the preset expectation to obtain spatial features to be merged;

[0078] Step S23: The bottleneck module fuses the spatial features to be merged with the multispectral image to obtain an initial fused image;

[0079] Step S24: Using the HRM, feature extraction and fusion are performed on the initial fused image according to frequency separation to obtain a reconstructed spectral image.

[0080] It should be noted that the upsampling module is used to upscale low-resolution feature maps to high resolution.

[0081] It is understandable that, since the feature enhancement feedforward network is a deep learning model designed to enhance the effective features in the input data through forward propagation, it typically contains multiple convolutional layers and non-linear activation functions. It can automatically learn and extract important features from the input data while suppressing redundant information. Therefore, the effective spatial features can be extracted from the panchromatic image through the feature enhancement feedforward network in the encoder.

[0082] It is understandable that, since the upsampling module is used to enlarge the low-resolution feature map to a high resolution, the model can recover detailed information through upsampling, making the output feature map more refined and accurate. Therefore, the upsampling module can be used to extract features from the effective spatial features that meet the preset expectation in terms of resolution.

[0083] It is understandable that, since the bottleneck module contains one or more fully connected layers, it can compress high-dimensional features into a low-dimensional space while retaining the most important feature information. It is usually used to reduce feature dimensionality and capture the most important feature information. Therefore, the bottleneck module can be used to fuse the spatial features to be merged with the multispectral image to obtain an initial fused image.

[0084] Understandably, the HRM module can use blurring and downsampling operations to separate high-frequency and low-frequency information, and use a dual-branch structure to process the two types of features. By enhancing high-frequency features, it improves visual contrast and ensures the accuracy of the fusion process, thereby improving the fidelity and quality of the fused image.

[0085] Understandably, by constructing a deep learning model that includes a feature enhancement feedforward network, an upsampling module, a bottleneck module, and an HRM, effective feature extraction and fusion of panchromatic and multispectral images are achieved, ultimately resulting in high-quality reconstructed spectral images. In other words, when processing high-resolution images, it is able to effectively fuse multi-scale feature information and maintain the high resolution of the output image.

[0086] Furthermore, the feature enhancement feedforward network includes a gating mechanism and a pixel attention mechanism; step S21 includes:

[0087] The gating mechanism is used to divide the panchromatic image into redundant spatial features and representative spatial features according to the band.

[0088] The representative spatial features are obtained by performing grouped convolutions and depthwise convolutions on the convolutional layers in the feature enhancement feedforward network.

[0089] By using the convolutional layers in the feature enhancement feedforward network, the redundant spatial features are subjected to deep convolution to obtain supplementary spatial features.

[0090] The pixel attention mechanism is used to fuse the representative features and the supplementary spatial features pixel by pixel to obtain effective spatial features.

[0091] It should be noted that gating mechanisms can control the input and output of information by introducing one or more gates, thereby achieving more efficient sequential data processing. Pixel attention mechanisms, by assigning a weight to each pixel in an image to emphasize or suppress certain pixel features, allow pre-defined spectral feature fusion networks to focus more meticulously on specific regions or objects in the image, thereby improving the accuracy of feature extraction.

[0092] Understandably, the gating mechanism is used to evaluate the information content of the feature map, and grouped convolution and depthwise convolution are used to refine the extraction and optimization of frequency components. At the same time, pixel-level attention is applied to the spatial dimension of the feature map, thereby enhancing the spatial feature representation.

[0093] In specific implementation, refer to Figure 2 Through a gating mechanism, the feature map X of the input feature enhancement feedforward network is divided into two parts, and the input feature map X is further divided into two bands along the channel dimension. These two bands contain β and (1-β) channels respectively, where 0 < β < 1, and β is a pre-defined parameter. Subsequently, two 1×1 convolutional layers (Conv) are independently applied to each feature map. 1×1 ), thus obtaining representative feature X rep and redundant feature X red Next, X rep The features are transformed by applying grouped convolutions and depthwise convolutions, ultimately generating representative features (Y1). This transformation process can be represented as follows:

[0094]

[0095] in, and It is the projection matrix. Input features and output features Let X and Y represent the input and output of the transformation, respectively. For the low-level feature X... red Only depthwise convolutions are used to extract additional redundant features into (supplementary spatial features) Y2 to supplement representative features:

[0096]

[0097] in, It is a projection matrix. Features and These are the input and output of the lower-level branches, respectively.

[0098] Then, a pixel-level attention mechanism is applied to enhance the fineness of the features. This is achieved by multiplying Y1 and Y2 pixel by pixel. Then, the softmax function is used to refine it. The resulting weights are compared with Y. m Perform pixel-by-pixel multiplication to generate the final output (effective spatial features) Y. out This process can be represented as:

[0099] Y m =Y1⊙Y2;μ out =Expand(Softmax(μ c (Y m )))⊙Y m

[0100] in, This represents the intermediate result after pixel-by-pixel multiplication and softmax thinning. Operation μ c (·) makes the mean function applied to the channel dimension, This is the final output; ⊙ indicates element-wise multiplication.

[0101] Based on the first and second embodiments of this application, the same or similar content as the above embodiments in the third embodiment of this application can be referred to the above description and will not be repeated hereafter. The HRM also includes convolutional layers and a spectral attention module; please refer to [further details needed]. Figure 4 Step S24 also includes steps S01 to S04:

[0102] Step S01: Using the HRM, high-frequency information with a band higher than the preset separation band and low-frequency information with a band lower than or equal to the preset separation band are separated from the initial fused image.

[0103] Step S02: Using the ASTM algorithm, perform weighted convolution processing on the high-frequency information to obtain high-frequency features;

[0104] Step S03: Using the spectral attention module, global features and spatial texture features are extracted from the low-frequency information to obtain supplementary features;

[0105] Step S04: The high-frequency features and the supplementary features are fused using the HRM to obtain a reconstructed spectral image.

[0106] It's important to note that the spectral attention module enhances the model's focus on spectral information in the input data, thereby improving its ability to capture spectral features. High-frequency information refers to the high-frequency components contained in an image or signal; these components typically correspond to features such as details, edges, and noise. Low-frequency information refers to the low-frequency components contained in an image or signal; these components typically correspond to features such as smooth regions, large color patches, and overall shape. Complementary features refer to features extracted from low-frequency information; these features are usually related to spatial information such as the overall structure, general outline, and smooth regions of the image.

[0107] It is understandable that by using the high-frequency and low-frequency information separation strategy in HRM, as well as fuzzing and downsampling operations to process the input features and selectively route them to two different branches, the complex spectral information in multispectral and panchromatic images can be effectively managed, and the computational complexity and performance can be effectively balanced. By using fuzzing and downsampling operations, high-frequency information and low-frequency features can be effectively separated, thereby improving processing efficiency.

[0108] Furthermore, the ASTM includes a predictor and a token mixer, the predictor including a token mask module and a spectral attention weighting module; step S02 further includes:

[0109] The high-frequency information is preprocessed in the predictor to obtain an intermediate feature map;

[0110] The token mask module performs spatial dimension feature sorting on the intermediate feature map to obtain the spectral token mask of each intermediate feature in the intermediate feature map.

[0111] The token mixer extracts key spectral features corresponding to the spectral token mask from each of the intermediate features.

[0112] The intermediate features are processed by the spectral attention weighting module to obtain spectral fusion features.

[0113] The key spectral features are fused with the spectral fusion features to obtain high-frequency features.

[0114] It should be noted that the predictor is used to generate the token mask, spectral attention weights, and location embeddings, respectively. The token mixer is used to fuse features from different sources or at different levels to improve the ability of the pre-defined spectral feature fusion network to capture key spectral features.

[0115] Understandably, by applying the token mask module, the spatial dimension features in the intermediate feature map can be effectively sorted, thereby more accurately identifying useful high-frequency features. Furthermore, by extracting and fusing key spectral features, unnecessary computation can be reduced, thus improving the computational efficiency of the entire feature extraction process.

[0116] Furthermore, step S03 also includes:

[0117] The low-frequency information is upsampled according to a preset ratio using the spectral attention module to obtain global features and spatial texture features.

[0118] Based on the weights of the low-frequency information, supplementary features are selected from the global features and the spatial texture features.

[0119] It should be noted that the preset ratio is a magnification or reduction factor pre-set during upsampling or other image processing operations. Spatial texture features are the spatial distribution patterns of pixel values ​​within local regions of an image, such as edges, corners, and textures.

[0120] Understandably, the application of the spectral attention module can effectively upsample low-frequency information, thereby more accurately identifying useful low-frequency features.

[0121] In the specific implementation, given an initial fused image as input... The separation operation can be represented as:

[0122] F h =Conv 3×3 (F), F l =Down2(Blur(F))

[0123] Where Fh and Fl represent high-frequency and low-frequency information, respectively. Conv 3×3 The expression indicates the use of a 3×3 convolution operation. Blu(·) refers to a blurring operation achieved through depthwise separable dilated convolution, while Down2(·) indicates downsampling with a scaling factor of 2. In this method, the high-frequency information processing branch serves as the main path, using N4 ASTMs, where each ASTM incorporates a deformable spectral attention mechanism to process representative spectral features. Simultaneously, the low-frequency information processing branch employs N5 SWABs to integrate low-frequency information. Finally, the HRM performs a weighted fusion of the results from both branches to obtain refined high-frequency information (high-frequency features), mathematically represented as:

[0124]

[0125] in This indicates the output of the HRM. and These are the weights assigned to the high-frequency and low-frequency branches, respectively. This indicates the sequential processing of Fh using N4 ASTM standards. Up2(·) indicates upsampling with a scaling factor of 2, while This indicates that N5 SWABs are used to integrate low-frequency information.

[0126] refer to Figure 5 The overall structure of ASTM mainly consists of a predictor and a token mixer, which combines deformable spectral attention mechanisms and convolution operations. Given input features... (High-frequency information after convolution) is first processed by pointwise convolution and a global refinement module to generate global and local conditions respectively. The local and global conditions are then concatenated to form the input to the predictor, expressed by the following formula:

[0127] P in =Concat(f gr (X),f pwc (X)0

[0128] Here, fgr(·) represents the global refinement operation, which contains a Conv 5×5 A LeakyReLU activation function and a Conv 3×3 f pwc (·) represents pointwise convolution, and Concat represents the concatenation operation. It is the input to the predictor, and then the intermediate feature map P is processed. in Preprocessing is performed to form common input features for the three sub-modules within the predictor. The three sub-modules then utilize the intermediate feature maps to generate a token mask, spectral attention weights, and location embeddings, respectively.

[0129] F s =f pre (P in M = argsort(softmax(μ)); s (Conv 3×3 (F s ))))

[0130] W=σ(Conv 3×3 (μ s (F s )));PE=f pwc (ReLU(f gwc (F s )))

[0131] Where f pre (·) indicates the preprocessing stage, which includes a join operation and a Conv 3×3 And a layer normalization. This represents the shared intermediate feature map in the predictor. The function μ s (·) calculates the mean along the spatial dimension. `argsort(·)` is a PyTorch function used to sort tensors. Vector represents the generated spectral token mask, while σ(·) represents the sigmoid function. This indicates channel attention weights. `Fgec(·)` represents grouped convolution. Furthermore, This indicates positional embedding.

[0132] Subsequently, we sort matrix M according to its weight values ​​and use M to sample X. Next, we divide the αC features with the highest weights into an information feature map. The remaining (1-α)C features are classified as redundant features. Where 0 < α < 1. For Zinf, we apply a deformable spectral attention mechanism for more refined processing. For Zred, we use a simple Conv... 3×3 Next, we perform a spectral reconstruction operation to fuse the results from the previous step, obtaining... Finally, by multiplying the channel attention weight W by Zfus and adding the positional embedding PE, we obtain the final output. The specific equation is:

[0133] M sorted =argsort(M); Z inf Z red =Sampling(M sorted ,X)

[0134] Z fus =Shuffle(DCA(Z) inf ),Conv 3×3 (Z r e d ))

[0135] Z out =Z fus W+PE

[0136] Here, `argsort(·)` represents the sorting function provided by PyTorch, while the `Sampling(·)` function selects elements from `X` based on vector `M`. The `DCA(·)` function represents the deformable spectral attention mechanism. Spectral reconstruction operations are performed by Conv... 1×1 and Conv 3×3 Composition. This spectral token partitioning allows ASTM to better focus on key regions in the feature map. By applying attention to core information regions and using convolutions on redundant parts, the module can more effectively allocate computational resources among different features, thereby improving the balance between complexity and performance.

[0137] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0138] This application also provides a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network. Please refer to [link / reference]. Figure 6 The multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network includes:

[0139] The acquisition module 10 is used to acquire the panchromatic image and the corresponding multispectral image;

[0140] The reconstruction module 20 is used to fuse the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include an HRM. The HRM includes ASTM. The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0141] Optionally, the bottleneck module further includes an upsampling module;

[0142] The reconstruction module 20 is further configured to extract effective spatial features from the panchromatic image through feature enhancement feedforward network processing in the encoder; extract features from the effective spatial features with a resolution that meets a preset expectation through the upsampling module to obtain spatial features to be merged; fuse the spatial features to be merged with the multispectral image through the bottleneck module to obtain an initial fused image; and perform feature extraction and fusion on the initial fused image according to frequency separation through the HRM to obtain a reconstructed spectral image.

[0143] Optionally, the HRM may further include convolutional layers and a spectral attention module;

[0144] The reconstruction module 20 is further configured to separate high-frequency information with bands higher than a preset separation band and low-frequency information with bands lower than or equal to the preset separation band from the initial fused image using the HRM; to perform weighted convolution processing on the high-frequency information using the ASTM to obtain high-frequency features; to extract global features and spatial texture features from the low-frequency information using the spectral attention module to obtain supplementary features; and to fuse the high-frequency features and the supplementary features using the HRM to obtain a reconstructed spectral image.

[0145] Optionally, the ASTM includes a predictor and a token mixer, the predictor including a token mask module and a spectral attention weighting module;

[0146] The reconstruction module 20 is further configured to: preprocess the high-frequency information in the predictor to obtain an intermediate feature map; sort the spatial features of the intermediate feature map using the token mask module to obtain a spectral token mask for each intermediate feature in the intermediate feature map; extract key spectral features corresponding to the spectral token mask from each intermediate feature using the token mixer; perform additive processing on each intermediate feature using the spectral attention weight module to obtain a spectral fusion feature; and fuse the key spectral features with the spectral fusion feature to obtain a high-frequency feature.

[0147] Optionally, the reconstruction module 20 is further configured to upsample the low-frequency information according to a preset ratio through the spectral attention module to obtain global features and spatial texture features; and to select supplementary features from the global features and spatial texture features based on the weight of the low-frequency information.

[0148] Optionally, the feature-enhanced feedforward network includes a gating mechanism and a pixel attention mechanism;

[0149] The reconstruction module 20 is further configured to, through the gating mechanism, divide the panchromatic image into redundant spatial features and representative spatial features according to the bands; through the convolutional layers in the feature enhancement feedforward network, perform grouped convolution and depth convolution on the representative spatial features to obtain representative features; through the convolutional layers in the feature enhancement feedforward network, perform depth convolution on the redundant spatial features to obtain supplementary spatial features; and through the pixel attention mechanism, fuse the representative features and the supplementary spatial features pixel by pixel to obtain effective spatial features.

[0150] The multispectral image super-resolution reconstruction apparatus based on adaptive spectral feature fusion network provided in this application, employing the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network in the above embodiments, can solve the technical problem that the reconstructed multispectral image contains redundant spectral features, leading to a decrease in the quality of the reconstructed multispectral image. Compared with the prior art, the beneficial effects of the multispectral image super-resolution reconstruction apparatus based on adaptive spectral feature fusion network provided in this application are the same as those of the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network provided in the above embodiments, and other technical features in the multispectral image super-resolution reconstruction apparatus based on adaptive spectral feature fusion network are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0151] This application provides a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network. The multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network in the above embodiment 1.

[0152] The following is for reference. Figure 7 This document illustrates a structural schematic diagram of a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network, suitable for implementing embodiments of this application. The multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0153] like Figure 7As shown, a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the multispectral image super-resolution reconstruction device based on the adaptive spectral feature fusion network. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0154] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0155] The multispectral image super-resolution reconstruction device based on adaptive spectral feature fusion network provided in this application, employing the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network in the above embodiments, can solve the technical problem that the reconstructed multispectral image contains redundant spectral features, leading to a decrease in the quality of the reconstructed multispectral image. Compared with the prior art, the beneficial effects of the multispectral image super-resolution reconstruction device based on adaptive spectral feature fusion network provided in this application are the same as those of the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network provided in the above embodiments, and other technical features in this multispectral image super-resolution reconstruction device based on adaptive spectral feature fusion network are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0156] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0157] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0158] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network in the above embodiments.

[0159] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0160] The aforementioned computer-readable storage medium may be included in a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network; or it may exist independently and not be assembled into a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network.

[0161] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network, the device performs the following actions: acquires a panchromatic image and the corresponding multispectral image; fuses the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include a High-Resolution Mechanism (HRM). The HRM includes an Advanced Spectral Method (ASTM). The ASTM is used to identify effective spectral features and suppress the extraction of redundant spectral features. The feature enhancement feedforward network is used to identify effective spatial features and suppress the extraction of redundant spatial features.

[0162] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0164] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0165] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network. This solves the technical problem that the reconstructed multispectral image contains redundant spectral features, leading to a decrease in the quality of the reconstructed multispectral image. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network provided in the above embodiments, and will not be repeated here.

[0166] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described above.

[0167] The computer program product provided in this application can solve the technical problem that the reconstructed multispectral image contains redundant spectral features, which leads to a decrease in the quality of the reconstructed multispectral image. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multispectral image super-resolution reconstruction method based on adaptive spectral feature fusion network provided in the above embodiments, and will not be repeated here.

[0168] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for multispectral image super-resolution reconstruction based on an adaptive spectral feature fusion network, characterized in that, The method includes: Acquire the panchromatic image and its corresponding multispectral image; A reconstructed spectral image is obtained by fusing the panchromatic image and the multispectral image through a preset spectral feature fusion network. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include a High-Resolution Mechanism (HRM). The HRM includes an adaptive spectral feature mixer, a convolutional layer, and a spectral attention module. The adaptive spectral feature mixer includes a predictor and a token mixer. The predictor includes a token mask module and a spectral attention weight module. The bottleneck module also includes an upsampling module. The HRM is a high-frequency feature refinement module. The step of fusing the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain the reconstructed spectral image includes: Effective spatial features are extracted from the panchromatic image through feature enhancement feedforward processing in the encoder. The upsampling module extracts features whose resolution meets a preset expectation from the effective spatial features to obtain spatial features to be merged. The bottleneck module fuses the spatial features to be merged with the multispectral image to obtain an initial fused image; The HRM separates high-frequency information with bands higher than a preset separation band and low-frequency information with bands lower than or equal to the preset separation band from the initial fused image. The high-frequency information is preprocessed in the predictor to obtain an intermediate feature map; The token mask module performs spatial dimension feature sorting on the intermediate feature map to obtain the spectral token mask of each intermediate feature in the intermediate feature map. The token mixer extracts key spectral features corresponding to the spectral token mask from each of the intermediate features. The intermediate features are processed by the spectral attention weighting module to obtain spectral fusion features. The key spectral features are fused with the spectral fusion features to obtain high-frequency features; The spectral attention module extracts global and spatial texture features from the low-frequency information to obtain supplementary features. The high-frequency features are fused with the supplementary features using the HRM to obtain a reconstructed spectral image.

2. The method as described in claim 1, characterized in that, The step of extracting global features and spatial texture features from the low-frequency information using the spectral attention module to obtain supplementary features includes: The low-frequency information is upsampled according to a preset ratio using the spectral attention module to obtain global features and spatial texture features. Based on the weights of the low-frequency information, supplementary features are selected from the global features and the spatial texture features.

3. The method as described in claim 1, characterized in that, The feature enhancement feedforward network includes a gating mechanism and a pixel attention mechanism; The step of extracting effective spatial features from the panchromatic image through feature enhancement feedforward network processing in the encoder includes: The gating mechanism is used to divide the panchromatic image into redundant spatial features and representative spatial features according to the band. The representative spatial features are obtained by performing grouped convolutions and depthwise convolutions on the convolutional layers in the feature enhancement feedforward network. By using the convolutional layers in the feature enhancement feedforward network, the redundant spatial features are subjected to deep convolution to obtain supplementary spatial features. The pixel attention mechanism is used to fuse the representative features and the supplementary spatial features pixel by pixel to obtain effective spatial features.

4. A multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network, characterized in that, The device includes: The acquisition module is used to acquire panchromatic images and their corresponding multispectral images; The reconstruction module is used to fuse the panchromatic image and the multispectral image through a preset spectral feature fusion network to obtain a reconstructed spectral image. The preset spectral feature fusion network includes an encoder, a bottleneck module, and a decoder. The encoder, the bottleneck module, and the decoder all include a feature enhancement feedforward network. The bottleneck module and the decoder both include an HRM. The HRM includes an adaptive spectral feature mixer, a convolutional layer, and a spectral attention module. The adaptive spectral feature mixer includes a predictor and a token mixer. The predictor includes a token mask module and a spectral attention weight module. The bottleneck module also includes an upsampling module. The reconstruction module is further configured to extract effective spatial features from the panchromatic image through feature enhancement feedforward processing in the encoder; extract features from the effective spatial features with a resolution meeting a preset expectation through the upsampling module to obtain spatial features to be merged; fuse the spatial features to be merged with the multispectral image through the bottleneck module to obtain an initial fused image; separate high-frequency information with bands higher than a preset separation band and low-frequency information with bands lower than or equal to the preset separation band from the initial fused image through the HRM; preprocess the high-frequency information in the predictor to obtain an intermediate feature map; and use the token mask module... The process involves: sorting the intermediate feature maps spatially to obtain spectral token masks for each intermediate feature; extracting key spectral features corresponding to the spectral token masks from each intermediate feature using a token mixer; performing additive processing on each intermediate feature using a spectral attention weighting module to obtain spectral fusion features; fusing the key spectral features with the spectral fusion features to obtain high-frequency features; extracting global and spatial texture features from the low-frequency information using a spectral attention module to obtain supplementary features; and fusing the high-frequency features with the supplementary features using an HRM to obtain a reconstructed spectral image. The adaptive spectral feature mixer utilizes learnable spectral token masks and deformable spectral attention mechanisms to focus on effective feature maps, identify effective spectral features, and suppress the extraction of redundant spectral features. The feature enhancement feedforward network transfers the spatial texture information of the panchromatic image to the multispectral image. At the same time, it adopts a segmentation-enhancement strategy to suppress the extraction of redundant spatial features. The HRM is a high-frequency feature refinement module, and the adaptive spectral feature mixer is an adaptive spectral feature mixer.

5. A multispectral image super-resolution reconstruction device based on an adaptive spectral feature fusion network, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described in any one of claims 1 to 3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the multispectral image super-resolution reconstruction method based on an adaptive spectral feature fusion network as described in any one of claims 1 to 3.

Images