A spatiotemporal fusion method for remote sensing images

Abstract

This invention belongs to the field of remote sensing image processing technology, specifically relating to a spatiotemporal fusion method for remote sensing images. The method includes: converting a high temporal-low spatial resolution image at time T0 and a low spatial-high temporal resolution image at time T1 into a coarse differential image M12; inputting the image obtained by stitching the coarse differential image M12 and the high spatial-low temporal resolution image at time T1 into a differential network layer; inputting the high spatial-low temporal resolution image at time T1 into a feature extraction network to extract spatial detail features; and fusing the output feature map of the differential network layer and the output feature map of the feature extraction network to obtain a high spatial-high temporal resolution image at time T1. This invention can solve the problem of blurred fused images, adaptively balance the proportion of temporal and spatial feature extraction and learning, and generate high temporal-high spatial remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a spatiotemporal fusion method for remote sensing images. Background Technology

[0002] Currently, remote sensing image acquisition faces a trade-off between temporal and spatial resolution. Traditional remote sensing sensors, such as SeaWiFS, MODIS, and AVHRR, can provide high temporal resolution images, but their spatial resolution is relatively low, resulting in insufficient image detail and making it difficult to meet the needs of high-precision applications. Conversely, sensors such as ASTER, Landsat, and Sentinel-2 can provide high spatial resolution images, but they have long revisit periods, low temporal resolution, and are susceptible to weather factors such as clouds, resulting in a limited number of usable images.

[0003] To resolve the conflict between temporal and spatial resolution, spatiotemporal fusion methods have emerged. These methods aim to generate synthetic images with both high temporal and spatial resolution by fusing remote sensing images from different times and spatial resolutions. Existing methods include non-fusion-based methods, weight function-based methods, Bayesian methods, hybrid methods, and learning-based methods. Among these, learning-based methods, especially deep learning-based methods, have attracted significant attention due to their powerful data processing and feature extraction capabilities.

[0004] Despite significant performance advancements in deep learning-based remote sensing spatiotemporal fusion methods, several key challenges remain. First, for regions rich in spatial detail, a single, simple network layer may fail to adequately extract this detail, leading to blurred images in the fusion result. This is primarily due to the limitations of deep learning models in handling complex spatial features, hindering their ability to effectively capture and preserve fine structures within images. Summary of the Invention

[0005] To address the problems existing in the background art, the present invention provides a spatiotemporal fusion method for remote sensing images, comprising:

[0006] S1: Convert the high temporal-low spatial resolution image at time T0 and the low spatial-high temporal resolution image at time T1 into a coarse differential image M12;

[0007] S2: The image obtained by stitching together the coarse difference image M12 and the high spatial low temporal resolution image at time T1 is input into the differential network layer; the differential network layer includes: three parallel dilated convolutional layers, a regular convolutional layer, a first attention module and a second attention module;

[0008] S3: Input the high spatial low temporal resolution image at time T1 into the feature extraction network to extract spatial detail features; the feature extraction network includes: a cascaded spatial displacement multilayer perceptron module and a regular convolutional layer;

[0009] S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and temporal resolution image at time T1. The three parallel dilated convolutional layers are used to extract features of the input image of the differential network layer in different receptive fields. The output features of the three parallel dilated convolutional layers are concatenated and then input into a regular convolutional layer for convolution to obtain the first intermediate feature. The first attention module is used to perform attention operation on the first intermediate feature to obtain the second intermediate feature. The first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are concatenated and then input into the second attention module for attention operation to obtain the output feature of the differential network layer.

[0010] Another aspect of the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned spatiotemporal fusion method for remote sensing images.

[0011] The present invention has at least the following beneficial effects

[0012] This invention introduces a feature extraction network that employs a spatial displacement multilayer perceptron module cascaded with ordinary convolutional layers. This network more effectively extracts spatial detail features from high spatial resolution but low temporal resolution images at time T1. This design not only improves spatial resolution preservation but also enhances the model's ability to recognize and process complex spatial structures. The differential network layer cleverly combines three parallel dilated convolutional layers, regular convolutional layers, and two attention modules. The dilated convolutional layers extract features from different receptive fields using varying dilation rates, helping to capture subtle changes in temporal images. The introduction of attention modules further strengthens the model's focus on important features, improving the accuracy of temporal information extraction and reducing the possibility of spectral distortion. This method effectively captures temporal change information through the generation of the differential coarse image M12 and the processing by the differential network layer. Simultaneously, the feature extraction network focuses on spatial detail extraction. The combination of these two aspects allows the model to adaptively balance the extraction of temporal and spatial features, thus accurately reflecting temporal changes while maintaining high spatial resolution. By employing an advanced deep learning architecture, including dilated convolutions and attention mechanisms, the method of this invention exhibits stronger robustness and generalization when processing different datasets. This means that the method can provide high-quality spatiotemporal fusion results regardless of the type of remote sensing image. Furthermore, through reasonable network structure design (such as parallel dilated convolutional layers and attention modules) and efficient feature fusion strategies, the overall computational efficiency is optimized. While ensuring fusion quality, computational costs are also reduced, which is beneficial for promotion and deployment in practical applications. In summary, the spatiotemporal fusion method for remote sensing images provided by this invention demonstrates significant advantages in improving spatiotemporal resolution, enhancing spatial detail extraction and temporal change capture capabilities, and improving the robustness and generalization of fusion results, providing new ideas and technical means for remote sensing image processing and applications. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the model framework process of the present invention;

[0014] Figure 2 This is a schematic diagram of the framework flow of the global attention module model of the present invention;

[0015] Figure 3 This is a schematic diagram of the framework of the spatial displacement multilayer sensor of the present invention;

[0016] Figure 4 This is a comparison chart of the performance of this invention with other models on the CIA dataset;

[0017] Figure 5 This is a comparison chart of the performance of this invention with other models on the LGC dataset;

[0018] Figure 6 This is a comparison chart of the performance of this invention with other models on the AHB dataset. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0020] Please see Figure 1 This invention provides a spatiotemporal fusion method for remote sensing images, comprising:

[0021] S1: Convert the high temporal-low spatial resolution image at time T0 and the low spatial-high temporal resolution image at time T1 into a coarse differential image M12;

[0022] S2: The image obtained by stitching together the coarse difference image M12 and the high spatial low temporal resolution image at time T1 is input into the differential network layer; the differential network layer includes: three parallel dilated convolutional layers, a regular convolutional layer, a first attention module and a second attention module;

[0023] S3: Input the high spatial low temporal resolution image at time T1 into the feature extraction network to extract spatial detail features; the feature extraction network includes: a cascaded spatial displacement multilayer perceptron module and a regular convolutional layer;

[0024] S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and temporal resolution image at time T1. The three parallel dilated convolutional layers are used to extract features of the input image of the differential network layer in different receptive fields. The output features of the three parallel dilated convolutional layers are concatenated and then input into a regular convolutional layer for convolution to obtain the first intermediate feature. The first attention module is used to perform attention operation on the first intermediate feature to obtain the second intermediate feature. The first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are concatenated and then input into the second attention module for attention operation to obtain the output feature of the differential network layer.

[0025] In this embodiment, a remote sensing image fusion model is first constructed, and the constructed remote sensing image fusion model is trained. The high temporal low spatial resolution image M1 at time T0, the low spatial high temporal resolution image L1 at time T1, and the high spatial low temporal resolution image M2 at time T1, along with the trained remote sensing image fusion model, are used to obtain the high spatial high temporal resolution image at time T1. The remote sensing image fusion model consists of a differential network layer and a feature extraction network.

[0026] When training the remote sensing image fusion model, Landsat and MODIS remote sensing images are acquired and preprocessed. The preprocessed remote sensing images are divided into training, validation, and test sets. The MODIS images (M1 and M2) at the reference and prediction times are enlarged to the same size as the Landsat image (L1) at the prediction time using bilinear interpolation. Then, the enlarged low spatial resolution images M1 and M2 are converted into a coarse difference image M12. The loss function used to train the remote sensing image fusion model includes:

[0027]

[0028] Where λ1 = 1, λ2 = 0.5, and N represents the sample size. PL1 represents the feature map obtained by EF Net (feature extraction network), and PL2 represents the result map obtained after fusing EF Net and D Net (differential network layer). This indicates that Huber loss is applied to PL1 to supervise the output of the feature extraction network and enhance the model's ability to extract spatial details. This indicates that applying Huber loss to PL2 can better preserve the brightness and color of the final result. This indicates that applying MS-SSIM loss to PL2 preserves image edges and details while focusing on the visual quality of the generated image. These two loss functions are used together to supervise the visual quality of the final fused image and ensure image consistency and integrity. Where l hutber The function combines all the advantages of L1 and L2, enhancing the robustness of L2 to outliers and reducing its sensitivity to outliers. When the error is large, using L1 can reduce the impact of outliers, making the training more robust.

[0029]

[0030] Where α = β j =λ j =1, j=1, ..., M, where M represents the number of pixels;

[0031] (Indicates image brightness)

[0032] (Indicates image contrast)

[0033] (Representing image structure)

[0034] Where x represents the predicted image, y represents the original image, and μ x μ y , σ x , σ y Let σ represent the mean and standard deviation of the x and y values ​​of the image, respectively. xy This represents the covariance of the two images. Furthermore, c1 and c2 are constants, and c3 is equal to half of c2. In the prediction image x, the feature map obtained by the feature extraction network is y, which is the Landsat image (L1) at time T0. and The predicted image x is the result obtained by fusing EF Net and D Net (differential network layer), and y is the Landsat image (L2) at time T1.

[0035] The remote sensing images were preprocessed. The CIA, LGC, and AHB datasets each contain six bands, and each band underwent the same atmospheric correction to ensure dataset consistency. All Landsat images were cropped to 1200×1200×1 pixels, and MODIS images were cropped to 75×75×1 pixels. Two pairs of Landsat-MODIS images from each adjacent time step within the same dataset were used as a single experimental input to the model. The training set comprised 60% of the dataset for training, the validation set for validating the model, and the test set for testing the fusion results. During training and validation, each input image was segmented into 240×240×1 pixel blocks. During testing, the input images were cropped into smaller 75×75×1 pixel fragments.

[0036] Preferably, step S1 includes: subtracting the low spatial high temporal resolution image at time T1 and the high temporal low spatial resolution image at time T0 pixel by pixel to obtain a differential coarse image M12.

[0037] Preferably, the three parallel dilated convolutional layers have dilation rates of 2, 4, and 8, respectively, use the ReLU activation function, and have a kernel size of 3×3.

[0038] Preferably, the conventional convolutional layer comprises cascaded 3×3 convolutions and a ReLU activation function.

[0039] Preferably, the first attention module is a global attention module; the second attention module includes three cascaded sub-attention modules and a 1×1 convolution; each sub-attention module includes: a cascaded global attention module, a 3×3 convolution and a ReLU activation function.

[0040] Preferably, the size of the input feature R of the global attention module is defined as (C, H, W), where C represents the number of channels, and H and W represent the width and height. The feature processing flow of the global attention module is as follows:

[0041] S101: Perform a 1×1 convolution on feature R to obtain feature r1 with size (1, H, W); perform reshape on feature R to obtain feature r2 with size (C, H*W);

[0042] S102: Reshape feature r1 to obtain feature r3 with size (1, H*W);

[0043] S103: Perform a softmax operation on feature r3 to obtain feature r4 with size (H*W, 1);

[0044] S104: Perform matrix multiplication on feature r4 and feature r2 to obtain feature r5 of size (C, 1);

[0045] S105: Reshape feature r5 to obtain feature r6 with size (C, 1, 1);

[0046] S106: Perform a 1×1 convolution on feature r6 to obtain feature r7 with size (C / r, 1, 1);

[0047] S107: Process feature r7 sequentially with LayerNorm and ReLU activation functions to obtain feature r8 of size (C / r, 1, 1); perform 1×1 convolution on feature r8 to obtain feature r9 of size (C, 1, 1);

[0048] S108: Add feature r9 and feature R together to obtain the output feature of the global attention module.

[0049] Preferably, the ordinary convolutional layer includes a series of 3×3 convolutions, ReLU activation functions, 3×3 convolutions, ReLU activation functions, and 1×1 convolutions.

[0050] Preferably, the spatial displacement multilayer perceptron module includes a series of 3×3 convolutions, ReLU activation functions, 3×3 dilated convolutions with a dilation rate of 2, ReLU activation functions, 3×3 dilated convolutions with a dilation rate of 4, ReLU activation functions, and a spatial displacement multilayer perceptron.

[0051] Define the dimensions of the input feature Y of the spatial displacement multilayer perceptron as (c, h, w), where c represents the number of channels, and h and w represent the width and height. The feature processing flow of the spatial displacement multilayer perceptron is as follows:

[0052] S201: Input the feature Y into a multilayer perceptron (MLP) for processing to obtain the feature y1 with size (w, h, 3c);

[0053] y1 = MLP(Y)

[0054] S202: Divide feature y1 into 3 parts along the channel dimension using a Split operation to obtain feature y. 21 =y1[:,:,1:c]、characteristic y 22 =y1[:,:,c+1:2c] and feature y 23 =y1[:,:,2c+1:3c];

[0055] S203: For feature y 21 Perform the Shift1 operation to obtain feature y 211 ; for feature y 22 Perform the Shift2 operation to obtain feature y 221 ; for feature y 23 Perform a Shift3 operation to obtain feature y 231 ;

[0056]

[0057]

[0058] S204: [The following appears to be a separate, unrelated sentence: "Imagine feature y..."] 211 , feature y 221 and feature y 231 The output feature y3 of the spatial displacement multilayer perceptron is obtained by performing attention operations using the Split Attention function:

[0059] y3=SA(y 211 y 221 y 231 )

[0060] Here, SA(.) represents the SplitAttention function.

[0061] Another aspect of the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned spatiotemporal fusion method for remote sensing images.

[0062] Experimental simulations were conducted to rigorously verify the robustness and universality of the proposed model using publicly available remote sensing datasets from three different geographical regions.

[0063] Please see Figure 4 , Figure 4 This image shows a comparison of the fusion results of different models on the CIA dataset, which contains images from the Colim Barry Irrigation Area (CIA) (New South Wales, Australia, 34.0034°E, 145.0675°S). This region contains 17 pairs of cloud-free MODIS-Landsat images, taken between October 2001 and May 2002, representing a rice-growing irrigation system. All MODIS images are from MODIS Terra MOD09GA Collection 5, and all Landsat images are from Lansat-7 ETM+. Due to the relatively small irrigated area of ​​farmland in this region, the CIA dataset exhibits strong spatial heterogeneity. Figure 4 In the image, (a) and (b) represent MODIS and Landsat images from the CIA dataset, respectively. Figure 4 In the diagram, (c) to (i) represent the fusion results of the STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN, and ECPW-STFN models on the CIA dataset, respectively. Figure 4 In the diagram (j), the fusion result of the present invention is shown in the CIA dataset.

[0064] Please see Figure 5 This image shows a comparison of the fusion results of different models on the LGC dataset, which covers images of the Lower Guederle Basin (LGC) in northern New South Wales, Australia (149.2815°E, 29.0855°S). It includes 14 pairs of cloud-free MODIS-Landsat images from April 2004 to April 2005. All MODIS images are from MODISTerra MOD09GA Collection 5, and all Landsat images are from Landsat-5TM. This region is considered a dataset with strong temporal dynamics due to significant changes in land cover type caused by a flood event in mid-December 2004. Figure 5 In the image, (a) and (b) represent the MODIS image and the Landsat image from the LGC dataset, respectively. Figure 5 In the diagram, (c) to (i) represent the fusion results of the STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN, and ECPW-STFN models on the LGC dataset, respectively. Figure 5 In the diagram (j), the fusion result of the present invention is shown in the LGC dataset.

[0065] Please see Figure 6This image shows a comparison of the fusion results of different models on the AHB dataset. The AHB dataset comprises 27 pairs of cloudless MODIS-Landsat images from May 30, 2013 to December 6, 2018, covering the Arukorqin Banner (43.3619°N, 119.0375°E) in central Inner Mongolia, Northeast China. All MODIS images are from MODIS TerraMOD09GA Collection 5, and all Landsat images are from Landsat-8OLI. Since agriculture and animal husbandry are the main industries in this region, there are many circular pastures and farmlands in the dataset images. Due to the long period of the dataset, regional phenological variations are significant, exhibiting considerable heterogeneity. Figure 6 In the image, (a) and (b) represent the MODIS image and the Landsat image from the AHB dataset, respectively. Figure 6 In the diagram, (c) to (i) represent the fusion results of the STARFM, ESTARFM, FSDAF, STFDCNN, STFNet, STFMCNN, and ECPW-STFN models on the AHB dataset, respectively. Figure 6 In the diagram (j), the fusion result of the present invention is shown in the AHB dataset.

[0066] Depend on Figure 4 , Figure 5 , Figure 6 As shown, the fusion result image of this scheme is closer to the Landsat image of the predicted date in terms of overall color and spectrum compared to other models. In terms of overall spatial detail, our result image is clearer.

[0067] In summary, this invention introduces a feature extraction network that employs a spatial displacement multilayer perceptron module connected in series with ordinary convolutional layers. This allows for more effective extraction of spatial detail features from high spatial resolution but low temporal resolution images at time T1. This design not only improves spatial resolution preservation but also enhances the model's ability to recognize and process complex spatial structures. The differential network layer design cleverly combines three parallel dilated convolutional layers, regular convolutional layers, and two attention modules. The dilated convolutional layers extract features from different receptive fields using varying dilation rates, helping to capture subtle changes in temporal images. The introduction of attention modules further strengthens the model's focus on important features, improving the accuracy of temporal information extraction and reducing the possibility of spectral distortion. This method effectively captures temporal change information through the generation of the differential coarse image M12 and the processing by the differential network layer. Simultaneously, the feature extraction network focuses on spatial detail extraction; the combination of these two aspects allows the model to adaptively balance the extraction of temporal and spatial features, thus accurately reflecting temporal changes while maintaining high spatial resolution. By employing an advanced deep learning architecture, including dilated convolutions and attention mechanisms, the method of this invention exhibits stronger robustness and generalization when processing different datasets. This means that the method can provide high-quality spatiotemporal fusion results regardless of the type of remote sensing image. Furthermore, through reasonable network structure design (such as parallel dilated convolutional layers and attention modules) and efficient feature fusion strategies, the overall computational efficiency is optimized. While ensuring fusion quality, computational costs are also reduced, which is beneficial for promotion and deployment in practical applications. In summary, the spatiotemporal fusion method for remote sensing images provided by this invention demonstrates significant advantages in improving spatiotemporal resolution, enhancing spatial detail extraction and temporal change capture capabilities, and improving the robustness and generalization of fusion results, providing new ideas and technical means for remote sensing image processing and applications.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A spatiotemporal fusion method for remote sensing images, characterized in that, include: S1: Convert the high temporal-low spatial resolution image at time T0 and the low spatial-high temporal resolution image at time T1 into a coarse differential image M12; S2: The image obtained by stitching together the coarse difference image M12 and the high spatial low temporal resolution image at time T1 is input into the differential network layer; the differential network layer includes: three parallel dilated convolutional layers, a regular convolutional layer, a first attention module and a second attention module; The first attention module employs a global attention module; the second attention module comprises three cascaded sub-attention modules and a 1×1 convolution; each sub-attention module includes: a cascaded global attention module, a 3×3 convolution, and a ReLU activation function; S3: Input the high spatial low temporal resolution image at time T1 into the feature extraction network to extract spatial detail features; the feature extraction network includes: a cascaded spatial displacement multilayer perceptron module and a regular convolutional layer; The ordinary convolutional layer includes a series of 3×3 convolutions, ReLU activation functions, 3×3 convolutions, ReLU activation functions, and 1×1 convolutions; The spatial displacement multilayer perceptron module includes a series of 3×3 convolutions, ReLU activation functions, 3×3 dilated convolutions with a dilation rate of 2, ReLU activation functions, 3×3 dilated convolutions with a dilation rate of 4, ReLU activation functions, and a spatial displacement multilayer perceptron. Define the input features of a spatial displacement multilayer perceptron. The size is , Indicates the number of channels. and If width and height are represented, the feature processing flow of the spatial displacement multilayer perceptron is as follows: S201: Features The input is processed by a multilayer perceptron (MLP) to obtain a size of Features ; S202: Features The features are obtained by splitting the data into three parts along the channel dimension using a Split operation. ,feature and characteristics ; S203: Features Shift1 operation to obtain features ; for features Shift2 operation to obtain features ; for features Shift3 operation to obtain features ; S204: Features ,feature and characteristics The output features of the spatial displacement multilayer perceptron are obtained by performing attention operations using the Split Attention function. : in, Represents the Split Attention function; S4: The output feature map of the differential network layer and the output feature map of the feature extraction network are fused to obtain a high spatial and temporal resolution image at time T1. The three parallel dilated convolutional layers are used to extract features of the input image of the differential network layer in different receptive fields. The output features of the three parallel dilated convolutional layers are concatenated and then input into a regular convolutional layer for convolution to obtain the first intermediate feature. The first attention module is used to perform attention operation on the first intermediate feature to obtain the second intermediate feature. The first intermediate feature, the second intermediate feature and the output feature map of the feature extraction network are concatenated and then input into the second attention module for attention operation to obtain the output feature of the differential network layer.

2. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that, Step S1 includes: subtracting the low spatial high temporal resolution image at time T1 and the high temporal low spatial resolution image at time T0 pixel by pixel to obtain the differential coarse image M12.

3. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that, The three parallel dilated convolutional layers have void ratios of 2, 4, and 8, respectively, and the ReLU activation function is used.

4. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that, The conventional convolutional layer consists of cascaded 3×3 convolutions and a ReLU activation function.

5. The spatiotemporal fusion method for remote sensing images according to claim 1, characterized in that, Define the input features of the global attention module The size is , Let H represent the number of channels, and H and W represent the width and height. The feature processing flow of the global attention module is as follows: S101: Features Performing a 1×1 convolution yields a size of Features ; Features Reshape to obtain the size Features ; S102: Features Reshape to obtain the size Features ; S103: Features Performing a softmax operation yields a size of Features ; S104: Features and characteristics Performing matrix multiplication yields a value of size 1. Features ; S105: Features Reshape to obtain the size Features ; S106: Features Performing a 1×1 convolution yields a size of Features ; S107: Features LayerNorm and ReLU activation functions are applied sequentially to obtain a size of Features ; Features Performing a 1×1 convolution yields a size of Features ; S108: Features and characteristics The features are summed to obtain the output features of the global attention module.

6. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the spatiotemporal fusion method for remote sensing images as described in any one of claims 1-5.

Images