Hyperspectral and multispectral image fusion method and system based on Reformer residual iteration

Abstract

The invention discloses a hyperspectral and multispectral image fusion method and system based on Reform residual iteration. The method comprises the following steps: acquiring an image data set of a hyperspectral remote sensing image and a multispectral remote sensing image in the same region and preprocessing the image data set; performing spatial down-sampling on the preprocessed data set to generate a low-resolution hyperspectral image and a low-resolution multispectral image; an image priori network model is constructed, the image priori network model comprises a space priori module and a spectrum priori module, the space priori module captures a long-range spatial dependency relationship through a Reformer layer, and the spectrum priori module extracts spectrum features through a depth separable convolutional layer; establishing a multi-level iteration model, setting a residual connection reconstruction function, and training the model by adopting an optimization algorithm and a space-spectrum joint loss function to obtain a trained model; and finally, generating a hyperspectral image with high spatial resolution. According to the method, the parameter quantity required by fusion can be effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to a hyperspectral and multispectral image fusion method and system based on Reformer residual iteration. Background Art

[0002] Hyperspectral images (HSI), with their broad spectral range and rich band information, can capture the fine spectral characteristics of ground objects, providing an important basis for object identification and classification. However, HSI typically suffers from low spatial resolution, limiting its application for fine-grained object identification. In contrast, multispectral images (MSI) offer higher spatial resolution and can provide more accurate spatial distribution information of ground objects, but their limited number of bands makes them inadequate for complex object classification. With the advancement of remote sensing technology, multi-source remote sensing image fusion techniques have emerged, aiming to address the limitations of single-source imagery by combining the strengths of different remote sensing image types. Traditional image fusion methods such as pan sharpening and convolutional neural network (CNN)-based fusion have been widely used in multi-source remote sensing image fusion, demonstrating promising results. Furthermore, hyperspectral remote sensing technology, due to its multiple bands and strong spectral continuity, has been widely applied in fields such as soil heavy metal pollution monitoring and environmental monitoring.

[0003] Although multi-source remote sensing image fusion technology has, to a certain extent, addressed the limitations of HSI and MSI, existing methods still have numerous shortcomings. Traditional pan-sharpening methods cannot effectively address the negative impact of differences in sensor parameters, easily leading to a decrease in the quality of the fused image. While CNN-based fusion methods can extract spatial features of images, due to the large volume and rich features of remote sensing image data, using only CNNs results in excessive computational effort and is prone to falling into local optimal solutions. Furthermore, most existing deep learning fusion methods optimize the model using a loss function that takes the mean of each band in the spectral dimension. This approach not only amplifies the effects of noise and outliers but also destroys the shape characteristics of the spectral curve, making it impossible to effectively distinguish morphological differences in spectral curves (such as absorption peak positions and reflectance trends). Therefore, the accuracy and efficiency of existing technologies for multi-source remote sensing image fusion still need to be improved. There is an urgent need to develop more efficient and reliable fusion methods and systems that can fully utilize the spatial and spectral information of HSI and MSI to achieve more accurate Earth observation. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a hyperspectral and multispectral image fusion method based on Reformer residual iteration to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a hyperspectral and multispectral image fusion method based on Reformer residual iteration, comprising:

[0006] Acquiring an image dataset of a hyperspectral remote sensing image and a multispectral remote sensing image of the same region, and preprocessing the image dataset to obtain a preprocessed dataset;

[0007] Performing spatial downsampling on the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and segmenting the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset;

[0008] Constructing an image prior network model, the image prior network model includes a spatial prior module and a spectral prior module, the spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer;

[0009] Establishing a multi-level iterative model, setting a residual connection reconstruction function, dynamically adjusting the fusion weight through the residual connection reconstruction function, and training the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model;

[0010] The test dataset is input into the trained model to generate a hyperspectral image with high spatial resolution.

[0011] Preferably, the process of preprocessing the image dataset includes:

[0012] Perform radiometric calibration and atmospheric correction on hyperspectral and multispectral remote sensing images;

[0013] Crop the hyperspectral remote sensing image and the multispectral remote sensing image to the same area size and perform image registration;

[0014] Select bands and perform data cleaning on hyperspectral and multispectral remote sensing images to remove outliers;

[0015] The original DN value of the remote sensing image is converted into reflectance and normalized.

[0016] Preferably, the process of spatially downsampling the preprocessed data set comprises:

[0017] According to the Wald protocol, the hyperspectral remote sensing image and the multispectral remote sensing image are spatially downsampled by corresponding multiples to obtain low-resolution hyperspectral images and low-resolution multispectral images;

[0018] Blurring low-resolution hyperspectral images and low-resolution multispectral images using Gaussian kernels;

[0019] The blurred low-resolution hyperspectral image and low-resolution multispectral image are used as the low-resolution images of the simulation input, and the original hyperspectral remote sensing image is used as the reference image.

[0020] Preferably, the image prior network model includes:

[0021] Initial 2D depth-wise separable convolutional layers and resampling layers to capture shallow local features;

[0022] The Reformer layer is used to capture long-range spatial dependencies through the local sensitive hashing window attention mechanism and extract the spatial features of hyperspectral remote sensing images;

[0023] The 3D depth-wise separable convolutional layer is used to preliminarily extract the spectral features of the hyperspectral remote sensing image and perform deep interaction to fuse multi-level features.

[0024] Preferably, the steps of establishing a multi-level iterative model include:

[0025] A residual connection reconstruction function is set, and the residual connection reconstruction function includes a dynamic residual optimization function and a multimodal feature fusion reconstruction function; wherein the dynamic residual optimization function is used to calculate the spectral projection residual, project the current reconstruction result into the multispectral remote sensing image space through convolution, calculate the difference with the real multispectral remote sensing image, and map the residual back to the hyperspectral remote sensing image space; the multimodal feature fusion reconstruction function combines the spatial residual, spectral residual and depth features, and obtains the final reconstruction result through weighted fusion;

[0026] Construct a joint spatial-spectral loss function.

[0027] Preferably, the joint loss function includes spatial loss and spectral loss, the spatial loss adopts L1 loss function, and the spectral loss dynamically calculates band weights through a self-attention mechanism.

[0028] Preferably, the joint loss function dynamically adjusts the weights of spatial loss and spectral loss through a temperature annealing mechanism.

[0029] In a second aspect, the present invention further provides a hyperspectral and multispectral image fusion system based on Reformer residual iteration, which is used to implement the method described in the first aspect, including:

[0030] A data set acquisition module is used to acquire an image data set of hyperspectral remote sensing images and multispectral remote sensing images of the same area, and preprocess the image data set to obtain a preprocessed data set;

[0031] A downsampling module is used to spatially downsample the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and to segment the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset;

[0032] A model building module is used to build an image prior network model, wherein the image prior network model includes a spatial prior module and a spectral prior module. The spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer.

[0033] An iterative training module is used to establish a multi-level iterative model, set a residual connection reconstruction function, dynamically adjust the fusion weight through the residual connection reconstruction function, and train the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model;

[0034] The prediction module is used to input the test data set into the trained model to generate a hyperspectral image with high spatial resolution.

[0035] In a third aspect, the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0036] In a fourth aspect, the present invention further discloses a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0037] Compared with the prior art, the present invention has the following advantages and technical effects:

[0038] The present invention provides a hyperspectral and multispectral image fusion method based on Reformer residual iteration, comprising the following steps: first, obtaining an image dataset of hyperspectral remote sensing images and multispectral remote sensing images of the same area, and preprocessing the image dataset to obtain a preprocessed dataset; second, spatially downsampling the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and dividing the hyperspectral remote sensing images, low-resolution hyperspectral images and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset; then, constructing an image priori network model, the image priori network model comprising a spatial priori module and a spectral priori module, the spatial priori module capturing long-range spatial dependencies through a Reformer layer, and the spectral prior module extracting spectral features through a depthwise separable convolutional layer; third, establishing a multi-level iterative model, setting a residual connection reconstruction function, dynamically adjusting the fusion weights through the residual connection reconstruction function, and training the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model; finally, inputting the test dataset into the trained model to generate a hyperspectral image with high spatial resolution.

[0039] By combining Reformer with depthwise separable convolution, the present invention can effectively reduce the number of parameters required for fusion, overcoming the problem of large number of parameters in traditional deep learning methods.

[0040] The present invention uses band self-attention weights to match the multi-bands of hyperspectral images with the complexity of actual ground objects. By extracting bands within a certain spatial range and enhancing bands with significant spatial details, it solves the poor physical interpretability of traditional remote sensing image fusion caused by insensitivity to spectral information.

[0041] The spatial-spectral joint loss function proposed in the present invention takes into account both spatial deformation and spectral distortion during model training, realizes information complementarity between hyperspectral and multispectral images, and simultaneously optimizes spatial and spectral dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0043] Figure 1 Flowchart of a hyperspectral and multispectral image fusion method based on Reformer residual iteration according to an embodiment of the present invention;

[0044] Figure 2 Schematic diagram of an image prior network model according to an embodiment of the present invention;

[0045] Figure 3Schematic diagram of the fusion of hyperspectral remote sensing images and multispectral remote sensing images according to an embodiment of the present invention;

[0046] Figure 4 This is a difference comparison diagram made by the existing remote sensing image fusion method according to an embodiment of the present invention;

[0047] Figure 5 This is a comparison chart of the original hyperspectral spectral reflectance of an embodiment of the present invention and the spectral reflectance of the fusion method proposed in the present invention. DETAILED DESCRIPTION

[0048] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0049] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] Example 1

[0051] like Figure 1 As shown, this embodiment provides a hyperspectral and multispectral image fusion method based on Reformer residual iteration, including:

[0052] S1. Acquire an image dataset of a hyperspectral remote sensing image (HSI) and a multispectral remote sensing image (MSI) of the same region, and preprocess the image dataset to obtain a preprocessed dataset;

[0053] Furthermore, the HSI and MSI images were cropped, image registration, band selection, data cleaning and normalization were performed to remove outliers in the images;

[0054] In this example, a mining area and its surroundings were used as the experimental area, and GF-5 hyperspectral remote sensing images and S2A multispectral remote sensing images of the covered area were downloaded. Radiometric calibration and atmospheric correction were performed on the HSI and MSI data, respectively. The two images were cropped to the same area size, image registration and band selection were performed, and images with the same unbiased area were cropped again. Data cleaning of the HSI and MSI was performed using Python to correct or delete erroneous, incomplete, duplicate, or irrelevant parts, thereby improving the quality and usability of the data and ensuring its accuracy and consistency. The original DN value of the remote sensing image was then calculated as reflectivity, and the image was normalized for subsequent model training.

[0055] S2, spatially downsampling the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and segmenting the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset;

[0056] Specifically, according to the Wald protocol, the HSI and MSI are first spatially downsampled by corresponding multiples (for example, a 30m HSI is downsampled to 90m as a low-resolution hyperspectral image, and a 10m MSI is downsampled to 30m as a low-resolution multispectral image). The sensor spectral response function is simulated and blurred using a 3×3 Gaussian kernel to generate a lower-resolution LR-HSI. Similarly, the MSI is processed in the same way to obtain a lower-resolution LR-MSI. The blurring operation reduces high-frequency noise and avoids aliasing during the downsampling process. The blurred LR-HSI and LR-MSI serve as the simulated low-resolution input images, and the HSI serves as the reference image. The HSI (HR-HSI), LR-HSI, and LR-MSI are segmented and used as the training and validation datasets.

[0057] S3. Construct an image prior network model, wherein the image prior network model includes a spatial prior module and a spectral prior module. The spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer.

[0058] Further, if Figure 2 As shown in the figure, an image prior network model is constructed, which consists of spatial prior and spectral prior. The spatial prior is composed of the Reformer layer using the Locality Sensitive Hashing (LSH) window attention mechanism to capture long-range spatial dependencies and extract the spatial features of HSI. The 2D depthwise separable convolution layer extracts the edge and texture details of HSI. The spectral prior is composed of the 3D depthwise separable convolution layer to capture and integrate the spectral information of HSI.

[0059] Specifically, an image prior network model is constructed, using initial 2D depthwise separable convolutional layers and resampling layers to capture shallow local features (such as edges and textures). The Reformer layer captures long-range spatial dependencies through a locality-sensitive hashing (LSH) window attention mechanism. The input image is linearly transformed to generate Queries (Q), Keys (K), and Values ​​(V). Q and K are projected, and hash values ​​are generated using a sign function. The generated hash values ​​are used to assign Q and K to different hash buckets. The attention weights between Q and K in each hash bucket are calculated separately. The obtained attention weights are weighted summed over V to obtain the attention output. The attention of all hash buckets is combined and output, reconstructing the input shape to obtain the deep spatial features of HSI and extracting the spatial relationship between local and global. The 3D depthwise separable convolutional layer initially extracts spectral features in the HSI spectral dimension. Subsequently, deep interaction is performed to fuse the multi-level features obtained from the previous convolutions. Through cascaded feature transformations, the spatial-spectral joint representation is gradually extracted to obtain the spatial and spectral priors of the image, providing prior guidance for subsequent reconstruction optimization.

[0060] S4. Establish a multi-level iterative model, set a residual connection reconstruction function, dynamically adjust the fusion weight through the residual connection reconstruction function, and systematically train the model using the Adam optimization algorithm and the spatial-spectral joint loss function to adjust the network parameters to obtain a trained model;

[0061] Specifically, we first construct the residual connection reconstruction function, which is divided into two parts:

[0062] The first part is the dynamic residual optimization function, which calculates the spectral projection residual, projects the current reconstruction result into the MSI space through convolution, calculates the difference with the true MSI, and maps the residual back to the HSI space to achieve spectral matching between the reconstruction result and the MSI observation, which is expressed by the following formula:

[0063]

[0064] Among them, z(t) is the reconstruction result of the t-th iteration, Ctomsi is the conversion convolution result from HSI to MSI, Ctohsi is the inverse conversion convolution result from MSI to HSI, and δl and ηl are the learnable parameters of the l-th layer.

[0065] The second part is the multimodal feature fusion reconstruction function, which is used to build a fused image. The current reconstruction result is downsampled, compared with the low-resolution HSI, and then upsampled back to the original size, recorded as err1; the reconstruction result of the previous step is projected into the MSI space, the residual with the true MSI is calculated and mapped back to the HSI, recorded as err3; the current result, spatial residual, spectral residual and depth features are weightedly fused to obtain the final reconstruction result recon, which is expressed as the following formula:

[0066]

[0067] Among them, D is the downsampling, U is the upsampling operation, F is the deep feature map in the Reformer layer, and C tomsi is the convolution result from HSI to MSI, C tohsi is the inverse transformation convolution result from MSI to HSI, δ l and η l is the learnable parameter of layer l.

[0068] Going further,

[0069] A joint spatial-spectral loss function training model was constructed. During spectral loss monitoring training, the self-attention method was used to determine the importance of the bands and dynamically calculate the weights of each loss. 3×3, 5×5, and 7×7 Gaussian kernels were used to capture the upper and lower ranges of different spaces. After splicing multi-scale features, weights were generated through a fully connected layer to enhance bands with significant spatial details. This is expressed as follows:

[0070]

[0071] Among them, G x and G y They are the horizontal and vertical edge detection grayscale values, Sobel x and Sobel y are the horizontal and vertical 3×3 matrices of the Sobel operator respectively, and T is the reference image tensor.

[0072]

[0073] Among them, ω is the band attention weight, W and b are the fully connected layer parameters, and f3, f5, and f7 are the average pooling features of different scales.

[0074]

[0075] Among them, C is the number of spectral bands, P is the fused image in the current state, T is the reference image, ⊙ is the element-by-element product, ω c is the band attention weight.

[0076] When the Adam optimization algorithm is used to train the fusion model, the spatial loss function uses the L1 loss function to constrain the network. The absolute error between the fused image and the reference image is calculated to guide the model optimization direction, which is expressed as the following formula:

[0077]

[0078] Among them, P is the fused image in the current state, T is the reference image, and N is the total number of pixels in the image.

[0079] Model training is jointly constrained by spatial loss and spectral loss. The band attention weights are normalized by the sigmoid function. The softmax function is used to add a temperature annealing mechanism to combine the spatial and spectral loss functions to dynamically adjust the multi-task loss weights, which can be expressed as the following formula:

[0080]

[0081] Among them, w k is the multi-task loss weight, T0 is the initial temperature in the temperature annealing mechanism, γ is the annealing coefficient, and epoch is the number of cycle stages.

[0082] Finally, the spectral loss function and the spatial loss function are multiplied by the weight coefficients learned iteratively during the training process to obtain the overall loss function that represents the model training effect, which is expressed as the following formula:

[0083]

[0084] Among them, P is the fused image in the current state, T is the reference image, and w k is the multi-task loss weight.

[0085] S5: Input the image to be fused (test image) into the trained model to generate a high spatial resolution hyperspectral image. Figure 3 shown.

[0086] In summary, this embodiment combines the spatial-spectral characteristics of hyperspectral and multispectral remote sensing images to reconstruct a low-spatial-resolution hyperspectral image into a high-spatial-resolution hyperspectral image. The goal of remote sensing image fusion is to inject details from the high-spatial-resolution image into the low-spatial-resolution image to achieve reconstruction. The image fusion method includes: cleaning and normalization of hyperspectral images (HSI) and multispectral images (MSI), Gaussian blur downsampling, and image segmentation to obtain training samples; constructing a spatial attention Reformer and a deep separable convolution fusion model. The Reformer uses the locality sensitive hashing (LSH) window attention mechanism to establish long-range spatial dependencies, extract the spatial features of training samples, and capture local and global spatial relationships; deep separable convolution is used for spatial dimensionality reduction, 3D convolution extracts spectral features, 2D convolution extracts spatial features, and images are segmented based on a sliding window strategy to construct training sample pairs with spatial-spectral correspondences; a reconstruction function is established to dynamically combine the residual information of spatial features and spectral features, and the proximal gradient algorithm is used to solve the optimization problem of the HSI and MSI fusion model. The iterative process of the proximal gradient algorithm is expanded into a multi-stage network, and the network is trained using a spatial-spectral joint loss function. The fusion model is repeated in each iteration, and the spatial attention mechanism and spectral convolution operations are used alternately, combined with multi-stage optimization. The present invention captures the spatial correlation of images by introducing Reformer, and adds self-attention band weight loss and depthwise separable convolution to achieve high-quality fusion reconstruction of HSI-MSI images, while maintaining computational efficiency and making full use of the information complementarity of the two imaging modalities.

[0087] In order to demonstrate the superiority of the fusion effect of this embodiment in terms of spatial and spectral fidelity, a comparison is made with the commonly used remote sensing image fusion methods at this stage. The results are as follows: Figure 4 shown. Figure 4 The first row shows the experimental results obtained by each fusion method and the original hyperspectral reference image. The second row shows the difference map between the experimental results of each fusion method and the original hyperspectral reference image. The darker the difference map, the smaller the difference between the experimental results and the reference image. It can be clearly seen from the figure that the difference map of the method proposed in this embodiment has the darkest color, indicating that the experimental results of the fusion method proposed in this embodiment perform best.

[0088] The difference in bands between the fused image and the original hyperspectral image is as follows: Figure 5 As shown, it can be seen that the reflectivity morphology between the two images is similar and basically overlaps.

[0089] In addition, this experimental example uses peak signal-to-noise ratio (PSNR), correlation coefficient (CC), global comprehensive error index (ERGAS), spectral angle (SAM), and structural similarity (SSIM) as quantitative evaluation indicators. The quantitative evaluation results compared with other methods are shown in Table 1. In terms of PSNR, CC, and SSIM, the method proposed in this example performs better than other fusion methods, and ERGAS and SAM are also suboptimal.

[0090] The number of parameters required for the training process of this embodiment and the deep learning fusion model is shown in Table 2. The number of parameters required for this embodiment is 7.2869% of that of the comparison model, which reduces the complexity and time of model training.

[0091] Table 1

[0092]

[0093] Table 2

[0094]

[0095] Example 2

[0096] Based on the same inventive concept, this embodiment further provides a hyperspectral and multispectral image fusion system based on Reformer residual iteration, which is used to implement the method described in the first embodiment, including:

[0097] A data set acquisition module is used to acquire an image data set of hyperspectral remote sensing images and multispectral remote sensing images of the same area, and preprocess the image data set to obtain a preprocessed data set;

[0098] A downsampling module is used to spatially downsample the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and to segment the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset;

[0099] A model building module is used to build an image prior network model, wherein the image prior network model includes a spatial prior module and a spectral prior module. The spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer.

[0100] An iterative training module is used to establish a multi-level iterative model, set a residual connection reconstruction function, dynamically adjust the fusion weight through the residual connection reconstruction function, and train the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model;

[0101] The prediction module is used to input the test data set into the trained model to generate a hyperspectral image with high spatial resolution.

[0102] The hyperspectral and multispectral image fusion system based on Reformer residual iteration provided in this embodiment has all the advantages of the hyperspectral and multispectral image fusion method based on Reformer residual iteration provided in the first embodiment.

[0103] Example 3

[0104] This embodiment further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0105] Example 4

[0106] This embodiment further discloses a computer program product, including a computer program, which implements the steps of the method described in the first embodiment when executed by a processor.

[0107] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A hyperspectral and multispectral image fusion method based on Reformer residual iteration, characterized in that: The following steps are involved: Acquiring an image dataset of a hyperspectral remote sensing image and a multispectral remote sensing image of the same region, and preprocessing the image dataset to obtain a preprocessed dataset; Performing spatial downsampling on the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and segmenting the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset; Constructing an image prior network model, the image prior network model includes a spatial prior module and a spectral prior module, the spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer; Establishing a multi-level iterative model, setting a residual connection reconstruction function, dynamically adjusting the fusion weight through the residual connection reconstruction function, and training the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model; The test dataset is input into the trained model to generate a hyperspectral image with high spatial resolution.

2. The method according to claim 1, characterized in that The process of preprocessing the image dataset includes: Perform radiometric calibration and atmospheric correction on hyperspectral and multispectral remote sensing images; Crop the hyperspectral remote sensing image and the multispectral remote sensing image to the same area size and perform image registration; Select bands and perform data cleaning on hyperspectral and multispectral remote sensing images to remove outliers; The original DN value of the remote sensing image is converted into reflectance and normalized.

3. The method according to claim 1, characterized in that The process of spatially downsampling the preprocessed dataset includes: According to the Wald protocol, the hyperspectral remote sensing image and the multispectral remote sensing image are spatially downsampled by corresponding multiples to obtain low-resolution hyperspectral images and low-resolution multispectral images; Blurring low-resolution hyperspectral images and low-resolution multispectral images using Gaussian kernels; The blurred low-resolution hyperspectral image and low-resolution multispectral image are used as the low-resolution images of the simulation input, and the original hyperspectral remote sensing image is used as the reference image.

4. The method according to claim 1, wherein The image prior network model includes: Initial 2D depth-wise separable convolutional layers and resampling layers to capture shallow local features; The Reformer layer is used to capture long-range spatial dependencies through the local sensitive hashing window attention mechanism and extract the spatial features of hyperspectral remote sensing images; The 3D depth-wise separable convolutional layer is used to preliminarily extract the spectral features of the hyperspectral remote sensing image and perform deep interaction to fuse multi-level features.

5. The method according to claim 1, wherein The steps to build a multi-level iterative model include: A residual connection reconstruction function is set, and the residual connection reconstruction function includes a dynamic residual optimization function and a multimodal feature fusion reconstruction function; wherein the dynamic residual optimization function is used to calculate the spectral projection residual, project the current reconstruction result into the multispectral remote sensing image space through convolution, calculate the difference with the real multispectral remote sensing image, and map the residual back to the hyperspectral remote sensing image space; the multimodal feature fusion reconstruction function combines the spatial residual, spectral residual and depth features, and obtains the final reconstruction result through weighted fusion; Construct a joint spatial-spectral loss function.

6. The method according to claim 1, characterized in that The joint loss function includes spatial loss and spectral loss. The spatial loss adopts the L1 loss function, and the spectral loss dynamically calculates the band weights through the self-attention mechanism.

7. The method according to claim 1, characterized in that The joint loss function dynamically adjusts the weights of spatial loss and spectral loss through a temperature annealing mechanism.

8. A hyperspectral and multispectral image fusion system based on Reformer residual iteration, used to implement the method according to any one of claims 1 to 7, characterized in that: include: A data set acquisition module is used to acquire an image data set of hyperspectral remote sensing images and multispectral remote sensing images of the same area, and preprocess the image data set to obtain a preprocessed data set; A downsampling module is used to spatially downsample the preprocessed dataset to generate low-resolution hyperspectral images and low-resolution multispectral images, and to segment the hyperspectral remote sensing images, low-resolution hyperspectral images, and low-resolution multispectral images in the preprocessed dataset into a training dataset and a test dataset; A model building module is used to build an image prior network model, wherein the image prior network model includes a spatial prior module and a spectral prior module. The spatial prior module captures long-range spatial dependencies through a Reformer layer, and the spectral prior module extracts spectral features through a depthwise separable convolutional layer. An iterative training module is used to establish a multi-level iterative model, set a residual connection reconstruction function, dynamically adjust the fusion weight through the residual connection reconstruction function, and train the model using an optimization algorithm and a spatial-spectral joint loss function to obtain a trained model; The prediction module is used to input the test data set into the trained model to generate a hyperspectral image with high spatial resolution.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images