A spatio-temporal fusion method and system for remote sensing images

Abstract

This invention belongs to the field of remote sensing image processing technology and discloses a method and system for spatiotemporal fusion of remote sensing images. This method introduces Mamba modules in both the spatial and frequency domains to enhance the extraction and modeling capabilities of multi-scale features and structural information. In the spatial domain, the Mamba module is used to capture the local structure and geometric details of the image; in the frequency domain, the Mamba module is introduced for feature extraction of the high-frequency subbands after wavelet transform, thereby more fully exploring the contextual dependencies and structural features of high-frequency information. Through channel attention and spatial attention mechanisms, dual-domain interactive fusion is achieved, improving the spatial clarity and spectral consistency of the fused image. It possesses strong generalization and adaptability, fully utilizing information from both the spatial and frequency domains, and improving the accuracy and efficiency of acquiring remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and particularly relates to a method and system for spatiotemporal fusion of remote sensing images. Background Technology

[0002] With the rapid development of remote sensing technology and Earth observation methods, humanity's ability to observe the Earth's surface has significantly improved. Various satellite sensors continuously acquire remote sensing imagery data of different scales, frequencies, and types, which are widely used in numerous fields such as land use change monitoring, agricultural yield estimation, ecological environment assessment, hydrological and meteorological forecasting, and disaster assessment. However, remote sensing imagery still faces a prominent problem in practical applications: existing remote sensing satellites struggle to simultaneously achieve high spatial and temporal resolution, resulting in data that cannot fully meet the demands for high-frequency, detailed analysis of dynamic changes in ground features.

[0003] In terms of spatial resolution, sensors like the Landsat series, such as the Landsat 8, can achieve a spatial resolution of up to 30 meters, while the Sentinel-2 achieves 10 meters, enabling them to finely represent land cover types and their detailed changes. However, these sensors have long revisit periods, typically around 5 to 16 days, and are heavily influenced by cloud cover and atmospheric conditions. This further reduces the frequency of acquiring cloudless images, limiting their ability to continuously monitor rapidly changing processes.

[0004] In contrast, sensors such as MODIS have the advantage of high temporal resolution, enabling daily or even higher-frequency Earth observations, and are widely used in large-scale time-series analyses such as climate monitoring and vegetation index calculation. However, their spatial resolution is relatively low. For example, MODIS surface reflectance products are mostly 250-meter or 500-meter, which is insufficient to support tasks with high spatial resolution requirements, such as fine-grained land cover classification and urban detail extraction.

[0005] To address this trade-off between spatial and temporal resolution, spatiotemporal fusion technology for remote sensing images has emerged. This technology aims to utilize remote sensing image data from different sensors, fusing high spatial resolution images with high temporal resolution images to generate a sequence of remote sensing images with high spatial and temporal resolution at any given time point. In this way, researchers and operational users can obtain near-all-weather, high-frequency, high-resolution remote sensing data at a lower cost, providing a clearer and more continuous observational perspective on the dynamic processes of the Earth's surface.

[0006] Since 2000, the academic community has proposed various algorithms for spatiotemporal fusion of remote sensing images. Among them, the STARFM (Spatial and Temporal Adaptive Reflectance Fusion Model) is a representative example. This method models the spatiotemporal relationship between high- and low-resolution image pairs and predicts future high-resolution images based on weighted similarity pixel weights, achieving for the first time effective spatiotemporal fusion based on optical images. Subsequently, extended models such as ESTARFM, ISTAFM, and FSDAF have been proposed to improve fusion accuracy and applicability.

[0007] However, STARFM and its extended models still have certain limitations in practical applications. First, when dealing with heterogeneous and complex surface scenes (such as urban building clusters, mountain vegetation, farmland boundaries, etc.), problems such as large fusion errors, blurred edges, or temporal discontinuities are prone to occur. Second, the models rely heavily on the spatial similarity of neighboring pixels, failing to fully explore higher-dimensional temporal series information and image structural features. In addition, for time points lacking reference images, the model's predictive ability is weak, making it difficult to guarantee the stability and reliability of long-term series data.

[0008] In recent years, with the development of artificial intelligence, deep learning, and big data technologies, the academic community has gradually attempted to introduce advanced technologies such as neural networks, image super-resolution reconstruction, and time series modeling into the spatiotemporal fusion of remote sensing images. For example, some studies have used convolutional neural networks (CNNs) to extract multi-scale spatial features of images and combined them with recurrent neural networks (RNNs) and Transformers to model the dynamic changes in the time dimension, achieving significant improvements in fusion accuracy. Furthermore, some studies have begun to focus on multi-source, multi-modal fusion problems, that is, combining optical images, radar images, thermal infrared images, and other types of data for collaborative fusion, attempting to overcome the limitations of traditional single-sensor fusion strategies. These new methods have shown stronger robustness and generalization ability in practical remote sensing applications, especially in areas with severe cloud cover or scarce observational data, demonstrating significant application potential.

[0009] However, the existing technologies have the following shortcomings: (1) Traditional algorithms have low fusion accuracy when dealing with heterogeneous landscapes or rapidly changing scenes, and it is difficult to capture complex spatiotemporal features; (2) The fusion results are not robust enough due to cloud pollution and data loss; (3) Although deep learning-based fusion methods improve accuracy, they do not make sufficient use of frequency domain information, resulting in limited spectral consistency and detail reconstruction capabilities; (4) The computational complexity is high, making it difficult to meet the needs of large-scale or near-real-time applications.

[0010] It is evident that existing remote sensing imagery methods struggle to fully utilize spatial and frequency domain information and achieve a balance between accuracy and efficiency. Summary of the Invention

[0011] To overcome the shortcomings and defects mentioned in the background art above, this application provides a method and system for spatiotemporal fusion of remote sensing images.

[0012] Firstly, this application provides a method for spatiotemporal fusion of remote sensing images, including:

[0013] S1: Acquire remote sensing images and process them to obtain high-resolution reference images and residual images;

[0014] S2: Extract spatial domain features of the high-resolution reference image and residual image based on the deep convolution module and the spatial Mamba module;

[0015] S3: Perform wavelet transform on the high-resolution reference image and the residual image respectively to generate low-frequency sub-bands and high-frequency sub-bands, and extract features from the low-frequency sub-bands and high-frequency sub-bands. Then, add each sub-band of the high-resolution reference image and the residual image one by one and perform inverse wavelet transform to obtain the frequency domain features of the high-resolution reference image and the residual image.

[0016] S4: The spatial domain features and the frequency domain features are fused to obtain the fused features;

[0017] S5: The high-resolution reference image and the fused features are stitched together and enhanced to generate the final fused image.

[0018] In a second aspect, this application provides a remote sensing image spatiotemporal fusion system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] This application provides a spatiotemporal fusion method for remote sensing images, introducing Mamba modules in both the spatial and frequency domains to enhance the extraction and modeling capabilities of multi-scale features and structural information. In the spatial domain, the Mamba module is used to capture the local structure (such as texture and edges) and geometric details (such as shape and contour) of the image. In the frequency domain, the Mamba module is introduced for feature extraction of the high-frequency subbands after wavelet transform, thereby more fully exploring the contextual dependencies and structural features of high-frequency information. Through channel attention and spatial attention mechanisms, dual-domain interactive fusion is achieved, improving the spatial clarity and spectral consistency of the fused image. It possesses strong generalization and adaptability, fully utilizing information from both the spatial and frequency domains, and improving the accuracy and efficiency of remote sensing image acquisition. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a remote sensing image spatiotemporal fusion method provided in the application embodiment;

[0023] Figure 2 This is the overall architecture of a remote sensing image spatiotemporal fusion method provided in the application embodiment;

[0024] Figure 3 This is a schematic diagram of spatial domain feature extraction provided by the present invention;

[0025] Figure 4 This is one of the schematic diagrams of frequency domain feature extraction provided by the present invention;

[0026] Figure 5 This is the second schematic diagram of frequency domain feature extraction provided by the present invention;

[0027] Figure 6 This is a schematic diagram of the feature fusion process provided by the present invention;

[0028] Figure 7 This is a schematic diagram of the image reconstruction and enhancement processing flow provided by the present invention. Detailed Implementation

[0029] To facilitate understanding of the present invention, the present invention will be described more fully and in detail below with reference to the accompanying drawings and preferred embodiments, but the scope of protection of the present invention is not limited to the following specific embodiments.

[0030] Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by those skilled in the art. The technical terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the scope of the invention.

[0031] Unless otherwise specified, all raw materials, reagents, instruments and equipment used in this invention can be purchased from the market or prepared by existing methods.

[0032] Please see Figure 1 This application provides a method for spatiotemporal fusion of remote sensing images, including:

[0033] S1: Acquire remote sensing images and process them to obtain high-resolution reference images and residual images.

[0034] In this step, residual calculation is performed on the high-resolution reference image and the low-resolution predicted image to obtain the residual image. The input part of the remote sensing image in the algorithm only requires two images, namely the high-resolution reference image and the residual image.

[0035] S2: Extract spatial domain features of the high-resolution reference image and residual image based on the deep convolution module and the spatial Mamba module.

[0036] In this step, both the high-resolution reference image and the residual image undergo feature extraction via a deep convolutional module and a spatial Mamba module. The extracted features then enter a spatial attention mechanism module to enhance the representation of salient regions and suppress background redundancy. Finally, the features output from the two modules are summed, and the features output from the high-resolution reference image and the residual image after the spatial domain portion are also summed.

[0037] S3: Perform wavelet transform on the high-resolution reference image and the residual image respectively to generate low-frequency sub-bands and high-frequency sub-bands. After extracting features from the low-frequency sub-bands and high-frequency sub-bands, add each sub-band of the high-resolution reference image and the residual image one by one and perform inverse wavelet transform to obtain the frequency domain features of the high-resolution reference image and the residual image.

[0038] In this step, wavelet transforms are first performed on the high-resolution reference image and the residual image respectively to generate low-frequency sub-bands (LL) and high-frequency sub-bands (LH, HL, HH). Next, feature extraction is performed on the generated low-frequency and high-frequency sub-bands to capture the detailed features of the high-resolution reference image and the variation features of the residual image. Finally, the four sub-bands of the high-resolution reference image and the residual image are added one by one and then subjected to an inverse wavelet transform.

[0039] S4: The spatial domain features and the frequency domain features are fused to obtain the fused features.

[0040] In this step, the spatial domain features and frequency domain features are first concatenated and differentially processed to achieve spatial-frequency feature interaction. The key features are then enhanced by global pooling and attention mechanisms, and finally the fused features are output.

[0041] S5: The high-resolution reference image and the fused features are stitched together and enhanced to generate the final fused image.

[0042] In this step, since the high-resolution reference image has more detailed features and the preliminary fusion result (spatial-frequency interaction output result) contains change information, the two are stitched together and passed through the channel attention module and the spatial attention module to enhance the information in the channel dimension and spatial dimension. Finally, the image is decoded to generate the fused image.

[0043] The aforementioned spatiotemporal fusion method for remote sensing images introduces Mamba modules in both the spatial and frequency domains to enhance the extraction and modeling capabilities of multi-scale features and structural information. In the spatial domain, the Mamba module captures local structures (such as texture and edges) and geometric details (such as shape and contour) of the image. In the frequency domain, the Mamba module is introduced for feature extraction of the high-frequency subbands after wavelet transform, thereby more fully exploring the contextual dependencies and structural features of high-frequency information. Through channel attention and spatial attention mechanisms, dual-domain interactive fusion is achieved, improving the spatial clarity and spectral consistency of the fused image. It possesses strong generalization and adaptability, fully utilizing information from both the spatial and frequency domains, and improving the accuracy and efficiency of remote sensing image acquisition.

[0044] The steps of the above-mentioned remote sensing image spatiotemporal fusion method are described below with a complete example:

[0045] First, input image data. Taking Landsat imagery (high-resolution reference image) and Modis imagery (low-resolution prediction image) as examples, perform residual analysis on both to obtain the residual image Res_image. The high-resolution reference image is Pre_image. The input model has a size of 256×256 and 6 bands.

[0046] Furthermore, the high-resolution reference image and the residual image are independently input into the spatial domain module, and feature extraction and enhancement are performed on the two inputs respectively; the spatial domain module consists of a deep convolution module, a spatial Mamba module, and a spatial attention mechanism module;

[0047] After both inputs have been processed by the complete spatial domain module, their spatial domain features are obtained respectively;

[0048] Finally, the two spatial domain features are fused by adding them element by element to obtain the final spatial domain fused feature representation.

[0049] In other words, in this embodiment, both Res_image and Pre_image undergo feature extraction via a deep convolutional module and a spatial Mamba module. The extracted features are then fed into a spatial attention mechanism module to enhance the representation of salient region features and suppress background redundancy interference. Finally, the features output by the two modules are summed, and the features output by the high-resolution reference image and the residual image after the spatial domain portion are also summed.

[0050] In this embodiment, the depthwise convolution module mainly consists of three convolutional blocks. The first convolutional block consists of a convolutional layer and a ReLU activation function layer in sequence, the second convolutional block consists of a convolutional layer and a ReLU activation function layer in sequence, and the third convolutional block consists of a convolutional layer and a ReLU activation function layer in sequence. The convolutional kernel size in the convolutional layer is 3, the stride is 1, and the padding is 1.

[0051] In this embodiment, the first branch of the spatial Mamba module is a learnable descriptive convolution, which has strong adaptability and can adjust the receptive field according to the input data to adapt to features of different scales and orientations. The second branch consists of batch normalization, a spatial selection module, and efficient channel attention. The spatial selection module enhances the spatial feature representation ability, allowing the model to focus on key regions and improve classification and detection accuracy. It has low computational cost. The efficient channel attention module has low computational cost, preserves cross-channel correlation, and does not lose information.

[0052] Furthermore, the input image is subjected to wavelet transform, yielding a low-frequency component LL and three high-frequency components HL, LH, and HH, which represent approximate image information, vertical details, horizontal details, and diagonal details, respectively. Next, feature extraction is performed: features of the low-frequency components are extracted through convolution and activation functions, while features of the high-frequency components are processed using the Mamba module. Finally, the components of the high-resolution reference image are added to the components of the residual image, and then an inverse wavelet transform is performed.

[0053] The wavelet transform uses the Haar basis, and the input image F (1×6×w×h) is decomposed in both horizontal and vertical directions. First, F is decomposed row by row.

[0054] For any position F(x,y), x∈{1,2,…,w}, y∈{1,2,…,h}.

[0055] Low-frequency range:

[0056]

[0057] High-frequency section:

[0058]

[0059] Among them, F L (x,y) and F H The dimensions of (x,y) are all

[0060] Then each column is further broken down.

[0061] The decomposition of the low-frequency component satisfies the following relationship:

[0062]

[0063] The decomposition of the vertical high-frequency component satisfies the following relationship:

[0064]

[0065] The decomposition of the horizontal high-frequency component satisfies the following relationship:

[0066]

[0067] The decomposition of the diagonal high-frequency components satisfies the following relationship:

[0068]

[0069] After decomposition, the low-frequency feature F is obtained. LL (x,y) and high-frequency features F HL (x,y),F LH (x,y) and F HH (x, y), whose size is (x, y)

[0070] The inverse wavelet transform first recovers the low-frequency and high-frequency components of each column, and then recovers the low-frequency and high-frequency components of each row. The column direction is then restored, satisfying the following formula:

[0071]

[0072] In the formula, F' L (x, 2y - 1) and F' L (x, 2y) represent the low-frequency components in the column direction recovered at (x, 2y-1) and (x, 2y), respectively, and F' H (x, 2y - 1) and F' H (x, 2y) represent the high-frequency components in the column direction recovered at (x, 2y-1) and (x, 2y), respectively, and F' LL (x,y) represents the low-frequency feature at (x,y) obtained after wavelet transform, F' HL (x,y), F' LH (x,y) and F' HH (x,y) represent the vertical high-frequency features, horizontal high-frequency features, and diagonal high-frequency features at (x,y) obtained after wavelet transform, respectively.

[0073] To restore the row direction, satisfy the following formula:

[0074]

[0075]

[0076] In the formula, F' L (2x-1,y),F' L (2x,y) and F' L (x,y) represent the low-frequency components of the row direction recovered at (2x-1,y), (2x,y), and (x,y), respectively, and F' H (2x-1,y),F' H(2x,y) and F' H (x,y) represent the high-frequency components of the row direction recovered at (2x-1,y), (2x,y), and (x,y), respectively. F F This represents the image obtained after the inverse wavelet transform.

[0077] Furthermore, the input spatial and frequency features are first concatenated and then processed through a convolutional layer with a ReLU activation function. Next, global average pooling and global max pooling operations are performed to extract global information. Subsequently, the outputs of these two paths are each processed through a Sigmoid activation function to generate attention weights, which are then multiplied element-wise with the original features to highlight important features. The results of these multiplications are then fused using addition to obtain channel-weighted fused features. Finally, the spatial and frequency features are subtracted and processed again through a convolutional layer with a ReLU activation function to obtain differential enhancement features. These differential enhancement features are then concatenated with the channel-weighted fused features to output the fused features.

[0078] Furthermore, the output of the spatial-frequency interaction is stitched together with the high-resolution reference image, and then fed into the channel attention and spatial attention modules to enhance the expressive power of image features. Channel attention effectively preserves key information and suppresses redundant features by modeling the importance weights of features in each channel; spatial attention focuses on identifiable regions in the image, making the network pay more attention to spatially significant and densely packed targets, thereby improving the accuracy of image fusion and reconstruction. Finally, the output image is processed through three convolutional layers and a ReLU function layer.

[0079] The above-mentioned spatiotemporal fusion method of remote sensing images is described below with a specific experiment:

[0080] Step 1: Remote sensing image data input and processing:

[0081] like Figure 2 As shown, residual calculation is performed on the high-resolution reference image L0 and the low-resolution prediction image M1 to obtain the residual image. The input part of the remote sensing image in the algorithm only requires two images, namely the high-resolution reference image and the residual image.

[0082] Step 2: Spatial Domain Feature Extraction:

[0083] like Figure 3As shown, spatial domain feature extraction is performed. Both the high-resolution reference image and the residual image undergo feature extraction via a deep convolutional module and a spatial Mamba module. The extracted features are then fed into a spatial attention mechanism module to enhance the representation of salient regions and suppress background redundancy interference. Finally, the features output from the two modules are summed, and the features output from the high-resolution reference image and the residual image after the spatial domain processing are also summed.

[0084] Step 3: Frequency domain feature extraction:

[0085] like Figure 4 As shown, frequency domain feature extraction is performed. First, wavelet transform is applied to the high-resolution reference image and the residual image respectively to generate low-frequency sub-bands (LL) and high-frequency sub-bands (LH, HL, HH). Then, feature extraction is performed on the generated low-frequency and high-frequency sub-bands, such as... Figure 5 As shown, the detailed features of the high-resolution reference image and the variation features of the residual image are captured. Finally, the four sub-bands of the high-resolution reference image and the residual image are added one by one and then subjected to inverse wavelet transform.

[0086] Step 4: Spatial-Frequency Domain Collaborative Fusion:

[0087] like Figure 6 As shown, the spatial domain features and frequency domain features are first concatenated and differentially processed to achieve spatial-frequency feature interaction. The key features are then enhanced by global pooling and attention mechanisms, and finally the fused features are output.

[0088] Step 5: Image Reconstruction and Enhancement

[0089] like Figure 7 As shown, since the high-resolution reference image has more detailed features, and the preliminary fusion result (spatial-frequency interaction output result) contains variation information, the two are stitched together and then processed through a channel attention module and a spatial attention module to enhance the information in the channel and spatial dimensions. Finally, the fused image is generated by decoding using three layers of convolution and the ReLU activation function.

[0090] This application also provides a remote sensing image spatiotemporal fusion system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described method. This remote sensing image spatiotemporal fusion system can implement various embodiments of the above-described remote sensing image spatiotemporal fusion method and achieve the same beneficial effects; further details are omitted here.

[0091] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for spatiotemporal fusion of remote sensing images, characterized in that, include: S1: Acquire remote sensing images and process them to obtain high-resolution reference images and residual images; S2: Extract spatial domain features of the high-resolution reference image and residual image based on the deep convolution module and the spatial Mamba module; S3: Perform wavelet transform on the high-resolution reference image and the residual image respectively to generate low-frequency sub-bands and high-frequency sub-bands, and extract features from the low-frequency sub-bands and high-frequency sub-bands. Then, add each sub-band of the high-resolution reference image and the residual image one by one and perform inverse wavelet transform to obtain the frequency domain features of the high-resolution reference image and the residual image. S4: The spatial domain features and the frequency domain features are fused to obtain the fused features; S5: The high-resolution reference image and the fused features are stitched together and enhanced to generate the final fused image; S1 includes: Acquire a high-resolution reference image and a low-resolution predicted image from remote sensing images, and calculate the residual between the high-resolution reference image and the low-resolution predicted image to obtain a residual image. S2 includes: The high-resolution reference image and the residual image are independently input into the spatial domain module, and feature extraction and enhancement are performed on the two inputs respectively; the spatial domain module consists of a deep convolution module, a spatial Mamba module, and a spatial attention mechanism module; After both inputs have been processed by the complete spatial domain module, their spatial domain features are obtained respectively; Finally, the two spatial domain features are fused by adding them element by element to obtain the final spatial domain fused feature representation; S3 includes: Wavelet transform is performed on the high-resolution reference image and the residual image to generate a low-frequency component LL and three high-frequency components HL, LH, and HH; wherein the low-frequency component LL represents the approximate information of the image, the high-frequency component HL represents the vertical details of the image, the high-frequency component LH represents the horizontal details of the image, and the high-frequency component HH represents the diagonal details of the image. The low-frequency component LL is extracted using convolution and activation functions, while the high-frequency component is extracted using the Mamba module. Additionally, the components of the high-resolution reference image and the components of the residual image are added together, and then inverse wavelet transform is performed to obtain the frequency domain features of the high-resolution reference image and the residual image.

2. The remote sensing image spatiotemporal fusion method according to claim 1, characterized in that, The steps of the wavelet transform are as follows: Wavelet transform is implemented using the Haar basis for the input image. ( According to the decomposition in both horizontal and vertical directions, among which, Indicates the width of the image. Indicates the height of the image; First, the input image F is decomposed line by line as follows: For any position , , The decomposition of the low-frequency component satisfies the following relationship: ； The decomposition of the high-frequency component satisfies the following relationship: ； in Indicates low-frequency components. Representing high-frequency components, both have dimensions of [missing information]. ; Then each column is further decomposed as follows: The decomposition of the low-frequency component satisfies the following relationship: The decomposition of the vertical high-frequency component satisfies the following relationship: The decomposition of the horizontal high-frequency component satisfies the following relationship: The decomposition of the diagonal high-frequency components satisfies the following relationship: After decomposition, low-frequency features are obtained. and high frequency characteristics , and Their sizes are all .

3. The remote sensing image spatiotemporal fusion method according to claim 1, characterized in that, The steps of the inverse wavelet transform are as follows: When performing inverse wavelet transform, the low-frequency and high-frequency components of each column are recovered first, and then the low-frequency and high-frequency components of each row are recovered; wherein, the column direction is recovered, satisfying the following formula: ； ； ； ； In the formula, and Let (x, 2y-1) and (x, 2y) represent the low-frequency components in the column direction recovered at (x, 2y-1) and (x, 2y), respectively. and These represent the high-frequency components in the column direction recovered at (x, 2y-1) and (x, 2y), respectively. This indicates that the low-frequency feature at (x,y) is obtained after wavelet transform. , and These represent the vertical high-frequency features, horizontal high-frequency features, and diagonal high-frequency features at (x,y) obtained after wavelet transform, respectively. To restore the row direction, satisfy the following formula: ； ； ； ； ； In the formula, , and Let represent the low-frequency components of the row direction recovered at (2x-1,y), (2x,y), and (x,y), respectively. , and Let represent the high-frequency components of the row direction at (2x-1,y), (2x,y), and (x,y), respectively. This represents the image obtained after the inverse wavelet transform.

4. The remote sensing image spatiotemporal fusion method according to claim 1, characterized in that, S4 includes: After concatenating spatial and frequency features, preliminary features are obtained by using a convolutional layer combined with the ReLU activation function. The preliminary processed features are subjected to global average pooling and global max pooling operations respectively to extract global information; The outputs of global average pooling and global max pooling are each processed by a Sigmoid activation function to generate attention weights, which are then multiplied by the original features element-wise. The results of these multiplications are then fused by addition to obtain channel-weighted fused features. Finally, the spatial features and frequency features are subtracted and processed again by a convolutional layer combined with a ReLU activation function to obtain differential enhancement features. These differential enhancement features are then concatenated with the channel-weighted fused features to output the fused features.

5. The remote sensing image spatiotemporal fusion method according to claim 1, characterized in that, S5 includes: The high-resolution reference image and the fused features are stitched together, and the stitching result is input into the channel attention module and the spatial attention module to enhance the information in the channel dimension and spatial dimension. The output of each module is decoded to generate the fused image.

6. A remote sensing image spatiotemporal fusion system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5.

Images