Space-time fusion method for remote sensing image

Abstract

The invention belongs to the technical field of remote sensing image processing, and particularly relates to a time-space fusion method for a remote sensing image, which comprises the following steps: converting a high-time low-spatial resolution image at a T0 moment and a low-spatial high-temporal resolution image at a T1 moment into a differential coarse image M12; an image obtained by splicing the differential coarse image M12 and the high-space low-time-resolution image at the T1 moment is input to a differential network layer; inputting the high-space low-time-resolution image at the T1 moment into a feature extraction network to extract features of space details; and the output feature map of the difference network layer and the output feature map of the feature extraction network are fused to obtain a high-space high-time-resolution image at the T1 moment, the problem that the fused image is blurred can be solved, the proportion of time and space feature extraction and learning is adaptively balanced, and a high-time high-space remote sensing image is generated.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and in particular relates to a spatiotemporal fusion method for remote sensing images. Background Art

[0002] At present, the acquisition of remote sensing images faces the trade-off between temporal resolution and spatial resolution. Traditional remote sensing sensors, such as SeaWiFS, MODIS, and AVHRR, can provide images with high temporal resolution, but the spatial resolution is relatively low, resulting in insufficient image details and difficulty in meeting the needs of high-precision applications. On the contrary, sensors such as ASTER, Landsat, and Sentinel-2 can provide images with high spatial resolution, but have long revisit cycles, low temporal resolution, and are easily affected by weather factors such as clouds, resulting in a limited number of available images.

[0003] In order to solve the contradiction between temporal resolution and spatial resolution, spatiotemporal fusion methods have emerged. These methods aim to generate synthetic images with both high temporal resolution and high spatial resolution by fusing remote sensing images of different times and spatial resolutions. Existing methods include non-hybrid-based methods, weight function-based methods, Bayesian-based methods, hybrid methods, and learning-based methods. Among them, learning-based methods, especially deep learning-based methods, have attracted much attention due to their powerful data processing and feature extraction capabilities.

[0004] Although deep learning-based remote sensing spatiotemporal fusion methods have made significant progress in performance, there are still some key issues that need to be addressed. First, for areas with rich spatial details, a single and simple network layer may not be able to fully extract the spatial details in the image, resulting in blurred images in the fusion results. This is mainly due to the limitations of deep learning models in processing complex spatial features and their inability to effectively capture and preserve the fine structures in the image. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a spatiotemporal fusion method for remote sensing images, comprising:

[0006] S1: Convert the high temporal and low spatial resolution image at time T0 and the low spatial and high temporal resolution image at time T1 into a differential coarse image M12;

[0007] S2: Input the image obtained by stitching the difference coarse image M12 and the high spatial and low temporal resolution image at time T1 into the difference network layer; the difference network layer includes: three parallel hole convolution layers, a regular convolution layer, a first attention module and a second attention module;

[0008] S3: Inputting the high spatial and low temporal resolution image at time T1 into a feature extraction network to extract features of spatial details; the feature extraction network comprises: a spatial displacement multi-layer perceptron module and a common convolutional layer connected in series;

[0009] S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and high temporal resolution image at time T1, wherein the three parallel hole convolution layers are used to extract the features of the input image of the differential network layer in different receptive fields, and the output features of the three parallel hole convolution layers are spliced ​​and input into the regular convolution layer for convolution to obtain the first intermediate feature; the first attention module is used to perform an attention operation on the first intermediate feature to obtain the second intermediate feature; the first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are spliced ​​and input into the second attention module for attention operation to obtain the output feature of the differential network layer.

[0010] Another aspect of the present invention provides a computer-readable storage medium storing a program, wherein when the program is executed by a processor, the method for spatiotemporal fusion of remote sensing images is implemented.

[0011] The present invention has at least the following beneficial effects

[0012] The present invention introduces a feature extraction network, which adopts a spatial displacement multi-layer perceptron module and a common convolutional layer in series, and can more effectively extract spatial detail features in a high spatial and low temporal resolution image at time T1. This design not only improves the ability to maintain spatial resolution, but also enhances the model's ability to recognize and process complex spatial structures. The design of the differential network layer cleverly combines three parallel hole convolutional layers, a conventional convolutional layer, and two attention modules. The hole convolutional layer extracts features of different receptive fields through different expansion rates, which helps to capture subtle changes in time series images. The introduction of the attention module further strengthens the model's attention to important features, improves the extraction accuracy of time series information, and reduces the possibility of spectral distortion. The method achieves effective capture of time-varying information through the generation of a differential coarse image M12 and the processing of a differential network layer. At the same time, the feature extraction network focuses on the extraction of spatial details. The combination of the two enables the model to adaptively balance the extraction of temporal features and spatial features, thereby accurately reflecting temporal changes while maintaining high spatial resolution. Due to the use of advanced deep learning architecture, including hole convolution, attention mechanism, etc., the method of the present invention shows stronger robustness and generalization when processing different data sets. This means that no matter what type of remote sensing image is faced, the method can provide high-quality spatiotemporal fusion results. However, through reasonable network structure design (such as parallel hole convolution layer, attention module, etc.), and efficient feature fusion strategy, the overall computational efficiency is optimized. While ensuring the fusion quality, the computational cost is also reduced, which is conducive to promotion and deployment in practical applications. In summary, the remote sensing image spatiotemporal fusion method provided by the present invention shows significant advantages in improving spatiotemporal resolution, enhancing the ability to extract spatial details and capture temporal changes, and improving the robustness and generalization of fusion results, etc., and provides new ideas and technical means for remote sensing image processing and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the model framework flow chart of the present invention;

[0014] Figure 2 Schematic diagram of the framework flow of the global attention module model of the present invention;

[0015] Figure 3 It is a schematic diagram of the framework flow of the spatial displacement multilayer perceptron of the present invention;

[0016] Figure 4 This is a comparison chart of the effects of the present invention on the CIA dataset and other models;

[0017] Figure 5 This is a comparison chart of the effects of the present invention on the LGC dataset and other models;

[0018] Figure 6 This is a comparison chart of the effects of the present invention and other models on the AHB dataset. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0020] See also Figure 1 The present invention provides a spatiotemporal fusion method for remote sensing images, comprising:

[0021] S1: Convert the high temporal and low spatial resolution image at time T0 and the low spatial and high temporal resolution image at time T1 into a differential coarse image M12;

[0022] S2: Input the image obtained by stitching the difference coarse image M12 and the high spatial and low temporal resolution image at time T1 into the difference network layer; the difference network layer includes: three parallel hole convolution layers, a regular convolution layer, a first attention module and a second attention module;

[0023] S3: Inputting the high spatial and low temporal resolution image at time T1 into a feature extraction network to extract features of spatial details; the feature extraction network comprises: a spatial displacement multi-layer perceptron module and a common convolutional layer connected in series;

[0024] S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and high temporal resolution image at time T1, wherein the three parallel hole convolution layers are used to extract the features of the input image of the differential network layer in different receptive fields, and the output features of the three parallel hole convolution layers are spliced ​​and input into the regular convolution layer for convolution to obtain the first intermediate feature; the first attention module is used to perform an attention operation on the first intermediate feature to obtain the second intermediate feature; the first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are spliced ​​and input into the second attention module for attention operation to obtain the output feature of the differential network layer.

[0025] In this embodiment, a remote sensing image fusion model is first constructed, and the constructed remote sensing image fusion model is trained, and the high temporal and low spatial resolution image M1 at time T0, the low spatial and high temporal resolution image L1 at time T1, and the high spatial and low temporal resolution image M2 at time T1 are fused with the trained remote sensing image model to obtain a high spatial and high temporal resolution image at time T1; the remote sensing image fusion model includes: a differential network layer and a feature extraction network.

[0026] When training the remote sensing image fusion model, Landsat remote sensing images and MODIS remote sensing images are obtained and preprocessed, and the preprocessed remote sensing images are divided into training set, verification set and test set; the MODIS images (M1 and M2) of the reference time and the prediction time are enlarged to the same size as the Landsat image (L1) of the prediction time through bilinear interpolation, and then the enlarged low spatial resolution images M1 and M2 are converted into a differential coarse image M12; the loss functions used to train the remote sensing image fusion model include:

[0027]

[0028] Among them, λ1=1, λ2=0.5, N represents the sample size. PL1 represents the feature map obtained by EF Net (feature extraction network), and PL2 represents the result map obtained after fusion of EF Net and D Net (differential network layer). Indicates that huber loss is applied to PL1 to supervise the output of the feature extraction network and enhance the model's ability to extract spatial details; This means that huber loss is applied to PL2, which can better preserve the brightness and color of the final result. It means that MS-SSIM loss is applied to PL2, which can preserve the edges and details of the image and focus on the visual quality of the generated image. These two loss functions are used in combination to supervise the visual quality of the final fused image and ensure the consistency and integrity of the image. hutber The function combines all the advantages of l1 and l2, enhances the robustness of l2's outliers, and reduces the sensitivity to outliers. When the error is large, using l1 can reduce the impact of outliers and make the training more robust.

[0029]

[0030] Where α = β j =λ j =1, j=1, ..., M, where M represents the number of pixels;

[0031] (Indicates image brightness)

[0032] (Indicates image contrast)

[0033] (Indicates image structure)

[0034] Among them, x represents the predicted image, y represents the original image, μ x , μ y , σ x , σ y Represents the mean and standard deviation of images x and y, respectively, xy represents the covariance of the two images. In addition, c1 and c2 are constants, and c3 is equal to half of c2; The predicted image x is the feature map obtained by the feature extraction network, and y is the Landsat image (L1) at time T0; and The predicted image x is the result obtained by fusing EF Net and D Net (differential network layer), and y is the Landsat image (L2) at time T1.

[0035] Preprocess the remote sensing images. The CIA, LGC, and AHB datasets all contain six bands, and each band has undergone the same atmospheric correction to ensure the consistency of the dataset. We crop all Landsat images to a size of 1200×1200×1, and MODIS images to a size of 75×75×1. Two pairs of Landsat-MODIS images at each adjacent moment in the same dataset are input into the model as a set of experiments, of which the training set accounts for 60% for training, the validation set accounts for 20% for validating the model, and the test set accounts for 20% for testing the fusion results. In the training and validation stages, each input image is divided into small blocks of 240×240×1. In the test stage, the input image is cropped into smaller fragments of 75×75×1 pixels.

[0036] Preferably, the step S1 comprises: subtracting the low spatial and high temporal resolution image at time T1 from the high temporal and low spatial resolution image at time T0 pixel by pixel to obtain a coarse differential image M12.

[0037] Preferably, the dilation rates of the three parallel dilated convolutional layers are 2, 4 and 8 respectively, the activation function used is the ReLU activation function, and the convolution kernel size of the dilated convolutional layer is 3×3.

[0038] Preferably, the conventional convolutional layer comprises a series of 3×3 convolutions and a ReLU activation function.

[0039] Preferably, the first attention module adopts a global attention module; the second attention module includes three sub-attention modules connected in series and a 1×1 convolution; each sub-attention module includes: a global attention module connected in series, a 3×3 convolution and a ReLU activation function.

[0040] Preferably, the size of the input feature R of the global attention module is defined as (C, H, W), where C represents the number of channels, and H and W represent the width and height. The feature processing flow of the global attention module is as follows:

[0041] S101: Perform 1×1 convolution on feature R to obtain feature r1 of size (1, H, W); Reshape feature R to obtain feature r2 of size (C, H*W);

[0042] S102: Reshape feature r1 to obtain feature r3 of size (1, H*W);

[0043] S103: Perform a softmax operation on feature r3 to obtain feature r4 with a size of (H*W, 1);

[0044] S104: Perform matrix multiplication on feature r4 and feature r2 to obtain feature r5 of size (C, 1);

[0045] S105: Reshape feature r5 to obtain feature r6 of size (C, 1, 1);

[0046] S106: Perform 1×1 convolution on feature r6 to obtain feature r7 of size (C / r, 1, 1);

[0047] S107: Perform LayerNorm and ReLU activation functions on feature r7 in turn to obtain feature r8 of size (C / r, 1, 1); perform 1×1 convolution on feature r8 to obtain feature r9 of size (C, 1, 1);

[0048] S108: Add feature r9 and feature R to obtain the output feature of the global attention module.

[0049] Preferably, the common convolution layer includes a series of 3×3 convolutions, a ReLU activation function, a 3×3 convolution, a ReLU activation function, and a 1×1 convolution.

[0050] Preferably, the spatial displacement multilayer perceptron module includes a series of 3×3 convolutions, a ReLU activation function, a 3×3 dilated convolution with a dilation rate of 2, a ReLU activation function, a 3×3 dilated convolution with a dilation rate of 4, a ReLU activation function, and a spatial displacement multilayer perceptron;

[0051] Define the size of the input feature Y of the spatial displacement multilayer perceptron as (c, h, w), where c represents the number of channels, h and w represent the width and height, and the feature processing flow of the spatial displacement multilayer perceptron is as follows:

[0052] S201: Input feature Y into the multi-layer perceptron MLP for processing to obtain feature y1 with a size of (w, h, 3c);

[0053] y1=MLP(Y)

[0054] S202: Split feature y1 into three parts along the channel dimension through the Split operation to obtain feature y 21 =y1[:,:,1:c], feature y 22 =y1[:,:,c+1:2c] and feature y 23 =y1[:,:,2c+1:3c];

[0055] S203: for feature y 21 Perform Shift1 operation to obtain feature y 211 ; for feature y 22 Perform Shift2 operation to obtain feature y 221 ; for feature y 23 Perform Shift3 operation to get feature y 231 ;

[0056]

[0057]

[0058] S204: feature y 211 , feature y 221 and feature y 231 The output feature y3 of the spatial displacement multilayer perceptron is obtained by performing attention operation through the Split Attention function:

[0059] y3=SA(y 211 ,y 221 ,y 231 )

[0060] Among them, SA(.) represents the SplitAttention function.

[0061] Another aspect of the present invention provides a computer-readable storage medium storing a program, wherein when the program is executed by a processor, the method for spatiotemporal fusion of remote sensing images is implemented.

[0062] Experimental Simulation,This embodiment rigorously verifies the robustness and universality of the,proposed model using public remote sensing datasets from three different,geographical regions;

[0063] See also Figure 4 , Figure 4 This is a comparison chart of the fusion results of different models on the CIA dataset. The CIA dataset contains images from the Colimbary Irrigation Area (CIA) (New South Wales, Australia, 34.0034°E, 145.0675°S). This area contains 17 pairs of MODIS-Landsat cloud-free images taken from October 2001 to May 2002. It is an irrigation system dominated by rice. All MODIS images are from MODIS Terra MOD09GA Collection 5 data, and all Landsat images are from Lansat-7 ETM+. Due to the relatively small area of ​​irrigated farmland in the region, the spatial heterogeneity of the CIA dataset is strong. Among them, Figure 4 (a) and (b) represent the MODIS image and Landsat image of the CIA dataset, respectively. Figure 4 (c) to (i) show the fusion results of STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN and ECPW-STFN models on the CIA dataset. Figure 4 (j) in the middle shows the fusion result diagram of the present invention on the CIA dataset.

[0064] See also Figure 5 , which is a comparison chart of the fusion results of different models on the LGC dataset. The LGC dataset covers the Lower Ghedir Basin (LGC) images in northern New South Wales, Australia (149.2815°E, 29.0855°S), and contains 14 pairs of MODIS-Landsat cloud-free image pairs from April 2004 to April 2005. All MODIS images are from MODISTerra MOD09GA Collection 5 data, and all Landsat images are from Landsat-5TM. Due to the significant changes in land cover types caused by the flood event in mid-December 2004, this area is considered as a research dataset with strong temporal dynamics. Among them, Figure 5 (a) and (b) represent the MODIS image and Landsat image of the LGC dataset, respectively. Figure 5 (c) to (i) show the fusion results of STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN and ECPW-STFN models on the LGC dataset. Figure 5 (j) in the middle shows the fusion result diagram of the present invention on the LGC dataset.

[0065] See also Figure 6, which is a comparison chart of the fusion results of different models on the AHB dataset. The AHB dataset includes 27 pairs of cloud-free MODIS-Landsat image pairs from May 30, 2013 to December 6, 2018, covering the Al Horqin Banner (43.3619°N, 119.0375°E) area in the central part of Inner Mongolia, Northeast China. All MODIS images are from MODIS TerraMOD09GA Collection 5 data, and all Landsat images are from Landsat-8OLI data. Since agriculture and animal husbandry are the main industries in the region, there are many circular pastures and farmlands in the dataset images. Due to the long period of the dataset, the regional phenological changes are significant and have great heterogeneity. Among them, Figure 6 (a) and (b) represent the MODIS image and Landsat image of the AHB dataset, respectively. Figure 6 (c) to (i) show the fusion results of STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN and ECPW-STFN models on the AHB dataset. Figure 6 (j) in the middle shows the fusion result diagram of the present invention on the AHB data set.

[0066] Depend on Figure 4 , Figure 5 , Figure 6 As shown, it can be seen that the fusion result map of this scheme is closer to the Landsat image of the predicted date in terms of overall color and spectrum compared with other models. In terms of overall spatial details, our result map is clearer.

[0067] In summary, the present invention introduces a feature extraction network, which adopts a spatial displacement multi-layer perceptron module and a common convolutional layer in series, and can more effectively extract spatial detail features in a high spatial and low temporal resolution image at time T1. This design not only improves the ability to maintain spatial resolution, but also enhances the model's ability to recognize and process complex spatial structures. The design of the differential network layer cleverly combines three parallel hole convolutional layers, conventional convolutional layers, and two attention modules. The hole convolutional layer extracts features of different receptive fields through different expansion rates, which helps to capture subtle changes in time series images. The introduction of the attention module further strengthens the model's attention to important features, improves the extraction accuracy of time series information, and reduces the possibility of spectral distortion. This method achieves effective capture of time-varying information through the generation of a differential coarse image M12 and the processing of a differential network layer. At the same time, the feature extraction network focuses on the extraction of spatial details. The combination of the two enables the model to adaptively balance the extraction of temporal features and spatial features, thereby accurately reflecting time changes while maintaining high spatial resolution. Due to the use of advanced deep learning architecture, including hole convolution, attention mechanism, etc., the method of the present invention shows stronger robustness and generalization when processing different data sets. This means that no matter what type of remote sensing image is faced, the method can provide high-quality spatiotemporal fusion results. However, through reasonable network structure design (such as parallel hole convolution layer, attention module, etc.), and efficient feature fusion strategy, the overall computational efficiency is optimized. While ensuring the fusion quality, the computational cost is also reduced, which is conducive to promotion and deployment in practical applications. In summary, the remote sensing image spatiotemporal fusion method provided by the present invention shows significant advantages in improving spatiotemporal resolution, enhancing the ability to extract spatial details and capture temporal changes, and improving the robustness and generalization of fusion results, etc., and provides new ideas and technical means for remote sensing image processing and application.

[0068] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A spatiotemporal fusion method for remote sensing images, characterized in that: include: S1: Convert the high temporal and low spatial resolution image at time T0 and the low spatial and high temporal resolution image at time T1 into a differential coarse image M12; S2: Input the image obtained by stitching the difference coarse image M12 and the high spatial and low temporal resolution image at time T1 into the difference network layer; the difference network layer includes: three parallel hole convolution layers, a regular convolution layer, a first attention module and a second attention module; S3: Inputting the high spatial and low temporal resolution image at time T1 into a feature extraction network to extract features of spatial details; the feature extraction network comprises: a spatial displacement multi-layer perceptron module and a common convolutional layer connected in series; S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and high temporal resolution image at time T1, wherein the three parallel hole convolution layers are used to extract the features of the input image of the differential network layer in different receptive fields, and the output features of the three parallel hole convolution layers are spliced ​​and input into the regular convolution layer for convolution to obtain the first intermediate feature; the first attention module is used to perform an attention operation on the first intermediate feature to obtain the second intermediate feature; the first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are spliced ​​and input into the second attention module for attention operation to obtain the output feature of the differential network layer.

2. A spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The step S1 includes: subtracting the low spatial and high temporal resolution image at time T1 from the high temporal and low spatial resolution image at time T0 pixel by pixel to obtain a differential coarse image M12.

3. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The dilation rates of the three parallel dilated convolutional layers are 2, 4, and 8 respectively, and the activation function used is the ReLU activation function.

4. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The conventional convolutional layer includes a series of 3×3 convolutions and a ReLU activation function.

5. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The first attention module adopts a global attention module; The second attention module includes three sub-attention modules connected in series and a 1×1 convolution; each sub-attention module includes: a global attention module connected in series, a 3×3 convolution and a ReLU activation function.

6. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: Define the size of the input feature R of the global attention module as (C, H, W), where C represents the number of channels, H and W represent the width and height, and the feature processing flow of the global attention module is as follows: S101: Perform 1×1 convolution on feature R to obtain feature r1 of size (1, H, W); Reshape feature R to obtain feature r2 of size (C, H*W); S102: Reshape feature r1 to obtain feature r3 with a size of (1, H*W); S103: Perform a softmax operation on feature r3 to obtain feature r4 with a size of (H*W, 1); S104: Perform matrix multiplication on feature r4 and feature r2 to obtain feature r5 of size (C, 1); S105: Reshape feature r5 to obtain feature r6 of size (C, 1, 1); S106: Perform 1×1 convolution on feature r6 to obtain feature r7 of size (C / r, 1, 1); S107: Perform LayerNorm and ReLU activation functions on feature r7 in turn to obtain feature r8 of size (C / r, 1, 1); perform 1×1 convolution on feature r8 to obtain feature r9 of size (C, 1, 1); S108: Add feature r9 and feature R to obtain the output feature of the global attention module.

7. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The common convolution layer includes a series of 3×3 convolutions, a ReLU activation function, a 3×3 convolution, a ReLU activation function, and a 1×1 convolution.

8. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that: The spatial displacement multilayer perceptron module includes a series of 3×3 convolutions, a ReLU activation function, a 3×3 dilated convolution with a dilation rate of 2, a ReLU activation function, a 3×3 dilated convolution with a dilation rate of 4, a ReLU activation function, and a spatial displacement multilayer perceptron; Define the size of the input feature Y of the spatial displacement multilayer perceptron as (c, h, w), where c represents the number of channels, h and w represent the width and height, and the feature processing flow of the spatial displacement multilayer perceptron is as follows: S201: Input feature Y into the multi-layer perceptron MLP for processing to obtain feature y1 with a size of (w, h, 3c); y1=MLP(Y) S202: Split feature y1 into three parts along the channel dimension to obtain feature y 21 =y1[:,:,1:c], feature y 22 =y1[:,:,c+1:2c] and feature y 23 =y1[:,:,2c+1:3c]; S203: for feature y 21 Perform Shift1 operation to obtain feature y 211 ; for feature y 22 Perform Shift2 operation to get feature y 221 ; for feature y 23 Perform Shift3 operation to get feature y 231 ; S204: feature y 211 , feature y 221 and feature y 231 The output feature y3 of the spatial displacement multilayer perceptron is obtained by performing attention operation through the Split Attention function: y3=SA(y 211 ,and 221 ,and 231 ) Among them, SA(.) represents the Split Attention function.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, a spatiotemporal fusion method for remote sensing images as described in any one of claims 1 to 6 is implemented.

Images