Multispectral imaging satellite data spatio-temporal fusion method, medium, equipment, product

Abstract

The application discloses a multispectral imaging satellite data space-time fusion method, medium, equipment and product, relates to the technical field of image processing, and comprises the following steps: adding noise to satellite data serving as a label through a diffusion process, and constructing a data pair of two kinds of satellite data; an encoder-decoder architecture based on a U-Net structure is used to construct a noise predictor; the data with noise is input into a noisy feature encoder for multi-scale feature extraction, and time step information of the diffusion process is embedded in the noisy feature encoder; the data pair is input into a clean feature encoder for feature fusion and multi-scale feature extraction; at each scale, difference features between the noisy features and the clean features are calculated, and the difference features are input into a decoder for restoration to obtain predicted noise data; and the data with noise is denoised to obtain clean data. The predicted noise can stably guide the reverse denoising process, and the quality and reliability of the finally generated image are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to spatiotemporal fusion methods, media, devices, and products for multispectral imaging satellite data. Background Technology

[0002] Remote sensing satellites are crucial tools for Earth's surface observation, with different systems offering advantages in spatial resolution, temporal resolution, and spectral range. Landsat-8 boasts good spectral consistency and long-term observation capabilities, with a spatial resolution of 30 meters and a revisit period of approximately 16 days. Sentinel-2 offers even higher spatial resolution (10 to 20 meters) and a shorter revisit period (5 days), making it more sensitive in observing rapidly changing regions. However, constrained by sensor design and orbital parameters, a single remote sensing satellite cannot simultaneously acquire high spatial resolution and high temporal frequency imagery, limiting its application in highly dynamic monitoring scenarios.

[0003] The fusion of multi-source remote sensing images to obtain remote sensing products with high temporal resolution and high spatial detail has become an important research direction for spatiotemporal data reconstruction. Various spatiotemporal fusion models have emerged, such as the STARFM model based on linear weighting and spatiotemporal weights, the ESTARFM model to enhance its robustness, the FSDAF method using weighted window regression, and the Fit-FC fusion framework to adapt to dynamic changes. These methods have improved the quality of fused images to some extent, but most still rely on pre-defined spatial-temporal assumptions. When faced with complex ground structures, drastic temporal changes, or large time intervals, their fusion accuracy and generalization ability are limited. In recent years, deep learning methods have been widely applied to remote sensing image super-resolution, change detection, and temporal prediction tasks, and have gradually been introduced into remote sensing image fusion research. However, while existing methods perform well in areas with relatively uniform distribution of multiple ground types and gentle temporal changes, they are prone to problems such as blurred spatial details, misaligned boundaries, and spectral distortion under conditions of drastic ground changes, large image time differences, or significant data noise. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing image fusion methods in modeling nonlinear temporal variations and spectral consistency, and to propose a spatiotemporal fusion method for multispectral imaging satellite data, comprising the following steps:

[0005] S1. Acquire two types of multispectral imaging satellite data, add noise to the satellite data used as tags through a diffusion process to obtain noisy data, and construct a data pair between the two types of satellite data;

[0006] S2. Construct a noise predictor. The noise predictor adopts an encoder-decoder architecture based on the U-Net structure. The encoder includes a noisy feature encoder and a clean feature encoder.

[0007] S3. Input the noisy data into the noisy feature encoder. First, extract the features of the noisy image from the noisy data. Then, perform multi-scale feature extraction on the features of the noisy image to obtain multi-scale features of the noisy image. Embed the time step information of the diffusion process into the noisy feature encoder. Input the data pairs of the two satellite data into the clean feature encoder. First, fuse the data pairs to obtain clean image features. Then, perform multi-scale feature extraction on the clean image features to obtain multi-scale clean image features.

[0008] At each scale, the difference features between the features of the noisy image and the features of the clean image are calculated, and the difference features are sent to the decoder for reconstruction to obtain the predicted noise data.

[0009] S4. Based on the predicted noisy data, denoise the noisy data to obtain clean data.

[0010] Furthermore, the noisy data can be expressed by the formula:

[0011]

[0012] in, Indicates time step Noisy data, This represents the retention factor accumulated during forward diffusion. This represents the raw satellite data. It is standard Gaussian noise.

[0013] Furthermore, features of noisy images are extracted from noisy data through convolution operations.

[0014] Furthermore, a multi-resolution feature fusion network is used to fuse data pairs from the two satellites. When the two satellites are Landsat-8 and Sentinel-2 remote sensing satellite images, the fusion process of the multi-resolution feature fusion network for the data pairs is as follows:

[0015] The 60-meter band image in the Sentinel-2 imagery is upsampled to a 30-meter scale and then stitched with the 30-meter band image. The stitched features are then input into a multi-level attention feature extraction module. The output features of the multi-level attention feature extraction module are residually concatenated with the features upsampled from the 60-meter band image and passed through a convolutional layer to obtain the Sentinel-2 image features. ;

[0016] Landsat-8 imagery with resolutions of 45 meters and 90 meters was upsampled to a 30-meter scale and compared with feature... The two satellite images are stitched together, and their stitched features are input into a multi-level attention feature extraction module. The output features of the multi-level attention feature extraction module are residually concatenated with the features upsampled from the 90-meter band image and then passed through a convolutional layer to obtain clean image features.

[0017] Furthermore, the multi-level attention feature extraction module includes two convolutional layers and three ARDBs. The input features of the multi-level attention feature extraction module pass through the first convolutional layer and the three ARDBs in sequence. The output features of the three ARDBs are concatenated and then input into the second convolutional layer to obtain the output features of the multi-level attention feature extraction module.

[0018] ARDB consists of an attention mechanism module and residual dense blocks.

[0019] Furthermore, the noisy feature encoder includes n DGRBs and n-1 downsampling modules. The first n-1 noisy feature coding levels consist of a cascaded DGRB and a downsampling module, while the last noisy feature coding level contains a single DGRB. The features of the noisy image... Features are obtained sequentially through n noisy feature encoding levels. ;

[0020] The clean image feature encoder consists of n residual blocks and n-1 downsampling modules. The first n-1 clean image feature encoding levels consist of a residual block and a downsampling module connected in series. The last clean image feature encoding level contains a residual block. Features are obtained sequentially through n clean image feature encoding levels. ;

[0021] The decoder consists of n DGRBs and n-1 upsampling modules. The first n-1 decoding levels each consist of a cascaded DGRB and an upsampling module. The last decoding level contains a single DGRB. The input to the first decoding level is... and Feature splicing, the first The input of each decoding level is and The feature splicing, among which , This is the output of the (i-1)th decoding level;

[0022] DGRB consists of an MLP part, two convolutional parts, and a channel weight generation part. The time step information is processed by the MLP to generate scaling and bias coefficients for the affine transformation. and DGRB input features After the first convolutional part, the features are obtained. ,feature and Multiply and then Adding together yields the features ,feature After the second convolutional part, the features are obtained. ,feature With features Adding together yields the features ,feature The channel weights are obtained after the channel weight generation process, and the features are... The output features of DGRB are obtained by multiplying the channel weights.

[0023] Furthermore, a denoising diffusion implicit model is used as the sampling method to denoise the noisy data and obtain clean data. The specific sampling equation is shown in the formula:

[0024]

[0025] in, , Representing time steps , The sample, , Representing time steps , The cumulative noise attenuation factor For the predicted noise, For time step The adjustment factor is z, where z is the random noise term added during sampling.

[0026] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described spatiotemporal fusion method for multispectral imaging satellite data.

[0027] The present invention also proposes an electronic device, including a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to execute the above-described spatiotemporal fusion method for multispectral imaging satellite data.

[0028] The present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described spatiotemporal fusion method for multispectral imaging satellite data.

[0029] The beneficial effects of the technical solution provided by this invention are:

[0030] The noise predictor of this invention adopts an encoder-decoder architecture based on a U-Net structure. The encoder includes a noisy feature encoder and a clean feature encoder. The noisy feature encoder embeds time step information, accurately reflecting the nonlinear temporal changes of the features, enabling the model to dynamically adjust when dealing with different levels of noise. The noisy feature encoder and the clean feature encoder extract noisy and clean features at different scales. The clean feature encoder fully integrates two remote sensing images. The joint input of the two images can improve the spatiotemporal integrity and detail richness of the generated image without increasing the observation burden, and fully model the spectral consistency of the two images from different sources. The difference features are obtained by subtracting the noisy features and clean features at different scales. The predicted noise can stably guide the reverse denoising process, improving the quality and reliability of the final generated image. Attached Figure Description

[0031] Figure 1 This is a flowchart of the spatiotemporal fusion method for multispectral imaging satellite data according to an embodiment of the present invention;

[0032] Figure 2 This is a structural diagram of the noise predictor according to an embodiment of the present invention;

[0033] Figure 3 This is a structural diagram of the multi-resolution feature fusion network according to an embodiment of the present invention;

[0034] Figure 4 This is a structural diagram of the Differential Guided Modulation Residual Block (DGRB) according to an embodiment of the present invention;

[0035] Figure 5 This is an experimental result diagram of the first dataset in the embodiment of the present invention;

[0036] Figure 6 This is an experimental result diagram of the second dataset in the embodiment of the present invention;

[0037] Figure 7 This is a block diagram of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0039] The flowchart of the spatiotemporal fusion method for multispectral imaging satellite data in this embodiment of the invention is as follows: Figure 1 Specifically, it includes the following steps:

[0040] S1. Acquire two types of multispectral imaging satellite data, add noise to the satellite data used as tags through a diffusion process to obtain noisy data, and construct a data pair between the two types of satellite data.

[0041] This invention utilizes images from two multispectral imaging satellites, Landsat-8 and Sentinel-2. After acquiring Landsat-8 and Sentinel-2 image data for the same region and date, a unified preprocessing operation is required. Due to differences in revisit periods between different sensors and factors such as cloud cover, it is generally difficult to obtain image pairs on the same date. Here, Landsat-8 and Sentinel-2 image data from adjacent dates are acquired to construct a unified input format required by the model and to address the problem of lacking real label data during the training phase. Because the original remote sensing images differ in spatial resolution, temporal distribution, and image size, directly using them for model training will lead to inconsistent feature extraction, semantic mismatch, and missing label data. To solve these problems, the original high-resolution images are first downsampled. The original Landsat-8 and Sentinel-2 images are reduced in spatial resolution to one-third of their original resolution, thus allowing the original Landsat-8 high-resolution image to be used as label data during the training phase. The model is trained to recover high-resolution images from low-quality images by using the original image as the target image (ground truth) and the downsampled image as the simulated low-quality input.

[0042] Simultaneously, to achieve spatial alignment of multi-source images, Landsat-8 and Sentinel-2 images need to be resampled separately to unify their resolution to the standard value at the target scale. Methods such as bilinear interpolation or cubic convolution interpolation are used for image resampling to ensure the continuity of spatial details during scaling. Spatial cropping or padding of the images ensures strict alignment in pixel size, image width and height, and spatial coverage, forming a unified input matrix. The image pairs constructed through this step have consistent spatial size and resolution and represent the same geographic area, allowing them to be directly used as model input and ensuring structural matching in subsequent feature fusion and prediction processes.

[0043] Noisy image samples required for the diffusion model are constructed using the original Landsat-8 high-resolution imagery at the target time. Through the forward process of the diffusion model, Gaussian noise is progressively injected into the original imagery, generating multiple images with varying noise intensities. This process simulates the path of an image gradually degrading from a true distribution to a noisy distribution, providing a supervisory target for the model.

[0044] For the input raw high-resolution Landsat-8 image The model uses a specific noise scheduling strategy to gradually add Gaussian noise at each time step t, generating a series of intermediate images. , ,..., Until the image approaches a completely random Gaussian noise image. This process can be represented by a formula:

[0045]

[0046] in, Indicates time step Noisy data, This represents the original high-resolution Landsat-8 image. Standard Gaussian noise, It is the retention factor accumulated during the forward diffusion process. ,in These are preset noise scheduling parameters.

[0047] In the initial stage, only a small amount of noise needs to be added, resulting in minimal changes to the image. However, as time progresses, the image becomes increasingly irregular. If only a small amount of noise is added in the initial manner, the diffusion amplitude will gradually decrease or even disappear. Therefore, during forward diffusion, the noise intensity needs to be gradually increased over time to ensure that the image maintains a uniform randomization effect throughout the diffusion process. This mechanism ensures that the forward diffusion process does not excessively damage the image structure in the early stages, while still effectively removing details from the image in the later stages, achieving an ideal random Gaussian distribution. This is crucial for the subsequent inverse denoising process, because if the image fails to reach a completely random state, the denoising process may struggle to recover the image's true spatiotemporal characteristics.

[0048] S2. Construct a noise predictor. The noise predictor adopts an encoder-decoder architecture based on U-Net, where the encoder includes a noisy feature encoder and a clean feature encoder. The noisy feature encoder embeds time-step information, enabling the model to dynamically adjust when handling different levels of noise. The two encoders extract multi-scale features from the noisy image and the simulated clean image, respectively. At each scale, the difference between the two sets of features is calculated, and the difference features are fed into the decoder for reconstruction. The decoder gradually recovers the structure of the noisy image through a backsampling operation and outputs the noise prediction result at the current time step.

[0049] The noisy feature encoder consists of n difference-guided residual blocks (DGRBs) and n-1 downsampling modules. The first n-1 noisy feature coding levels consist of a cascaded DGRB and a downsampling module, while the last noisy feature coding level contains a single DGRB. The noisy image features... Features are obtained sequentially through n noisy feature encoding levels. .

[0050] The clean image feature encoder consists of n residual blocks and n-1 downsampling modules. The first n-1 clean image feature encoding levels consist of a residual block and a downsampling module connected in series. The last clean image feature encoding level contains a residual block. Features are obtained sequentially through n clean image feature encoding levels. .

[0051] The decoder consists of n DGRBs and n-1 upsampling modules. The first n-1 decoding levels each consist of a cascaded DGRB and an upsampling module. The last decoding level contains a single DGRB. The input to the first decoding level is... and Feature splicing, the first The input of each decoding level is and The feature splicing, among which , This is the output of the (i-1)th decoding level.

[0052] DGRB is based on the ResBlock residual block architecture. It aggregates spatial information of differences through convolution and global average pooling to extract channel-level statistics. A lightweight fully connected network (MLP) is used to generate channel attention weights, using the difference information as the modulation core to enhance the network's perception of local noise. DGRB consists of an MLP part, two convolutional parts, and a channel weight generation part. The time-step information is processed by the MLP to generate scaling and bias coefficients for affine transformation. and DGRB input features After the first convolutional part, the features are obtained. ,feature and Multiply and then Adding together yields the features ,feature After the second convolutional part, the features are obtained. ,feature With features Adding together yields the features ,feature The channel weights are obtained after the channel weight generation process, and the features are... The output features of DGRB are obtained by multiplying the channel weights.

[0053] DGRB Structure Diagram Reference Figure 4 The time step information t embedded in the noisy feature encoder is used by an MLP to generate scaling and bias coefficients for an affine transformation. and This enables the network to adaptively adjust the feature distribution at different time steps, achieving temporal modulation. DGRB's input features... Features are obtained after a 3×3 convolution and group normalization (GroupNorm). , and Multiply and then Adding together yields the features ,feature The features are obtained by sequentially applying the SiLU activation function, a 3×3 convolution, GroupNorm, and the SiLU activation function. ,feature and characteristics Adding together yields the features ,feature The channel weights are obtained by sequentially passing through a 3×3 convolution, SiLU activation function, adaptive average pooling, a linear layer, SiLU activation function again, a linear layer, and a sigmoid function. The channel weights and features are then processed. Multiplying them yields the output features of DGRB.

[0054] A structural diagram of the noise predictor of the present invention is shown in the reference diagram. Figure 2 The noisy feature encoder first extracts features from the input image through a convolution operation. The noisy feature encoder of the present invention has four coding levels. The first three coding levels consist of a DGRB and a downsampling module connected in series, and the fourth coding level includes a DGRB. Features of different scales are output from the four coding levels respectively. , , , .

[0055] The clean feature encoder first fuses data pairs from two satellites using a multi-resolution feature fusion network to obtain features. The clean feature encoder of this invention has four coding levels. The first three coding levels consist of a residual block and a downsampling module connected in series, and the fourth coding level includes a residual block. The four coding levels output features at different scales. , , , .

[0056] A multi-resolution feature fusion network is used to fuse data pairs from two satellites. When the two satellite data are Landsat-8 and Sentinel-2 remote sensing satellite images, the fusion process of the multi-resolution feature fusion network for the data pairs is referenced. Figure 3 Specifically:

[0057] The 60-meter band data from the Sentinel-2 imagery is upsampled to a 30-meter scale and then stitched with the 30-meter band imagery. The stitched features are then input into a multi-level attention feature extraction module. The output features of this module are residually concatenated with the upsampled features from the 60-meter band imagery and passed through a convolutional layer to obtain the Sentinel-2 image features. .

[0058] Landsat-8 imagery with resolutions of 45 meters and 90 meters was upsampled to a 30-meter scale and compared with feature... The two satellite images are stitched together, and their stitched features are input into a multi-level attention feature extraction module. The output features of the multi-level attention feature extraction module are residually concatenated with the features upsampled from the 90-meter band image and then passed through a convolutional layer to obtain clean image features.

[0059] The multi-level attention feature extraction module includes two convolutional layers and three attention-coupled residual dense blocks (ARDBs). The input features of the multi-level attention feature extraction module pass through the first convolutional layer and the three ARDBs in sequence. The output features of the three ARDBs are concatenated and then input into the second convolutional layer to obtain the output features of the multi-level attention feature extraction module.

[0060] ARDB consists of an attention mechanism module and residual dense blocks.

[0061] The features of the clean feature encoder at the corresponding scale are subtracted from the features of the noisy feature encoder to obtain the difference features. These difference features are then fed into the decoder for reconstruction. The decoder gradually recovers the noisy image structure through inverse upsampling and outputs the noise prediction result at the current time step. The decoder in this embodiment includes four DGRBs and three upsampling modules. The first three decoding levels consist of a cascaded DGRB and an upsampling module, while the last decoding level contains a single DGRB. The input to the first decoding level is... and The feature concatenation, the output of the first decoding level is The input to the second decoding level is and The output of the second decoding layer is the feature concatenation. The input to the third decoding level is and The features are concatenated, and the output of the third decoding level is... The input to the fourth decoding level is and The features are concatenated, and the output of the fourth decoding level is... . Figure 2 In this context, C represents the feature concatenation operation. The output of the fourth decoding level... This refers to the predicted noise.

[0062] S3. Input the noisy data into the noisy feature encoder for multi-scale feature extraction to obtain the features of the noisy image at multiple scales, and embed the time step information of the diffusion process into the noisy feature encoder; input the data pairs of the two satellite data into the clean feature encoder, first fuse the data pairs to obtain clean image features, and then perform multi-scale feature extraction on the clean image features to obtain multi-scale clean image features.

[0063] At each scale, the difference features between the features of the noisy image and the features of the clean image are calculated. The feature differences between the corresponding levels are used as an approximate expression of the noise representation. The difference features at all scales are passed to the decoder in sequence. Through decoding, the predicted noisy image with the same size as the original image is gradually restored, and the predicted noise data is obtained.

[0064] S4. Based on the predicted noise data, the noisy data is denoised to obtain clean data. The predicted noise data is used as input to the back-diffusion denoising process. This invention employs the Denoising Diffusion Implicit Model (DDIM) as the sampling method. The key idea of ​​DDIM sampling is to use a non-Markovian sampling strategy to reduce the number of iterations without sacrificing image quality. The specific sampling equation is shown in the formula:

[0065]

[0066] in, , Representing time steps , The sample, , Representing time steps , The cumulative noise attenuation factor For the predicted noise, For time step The adjustment factor is z, where z is the random noise term added during sampling.

[0067] By constructing a noisy-clean encoder architecture to model noise feature differences and introducing a time modulation mechanism to enhance the ability to express diffusion dynamics, the network model proposed in this invention has strong robustness and high fidelity. It can accurately estimate the image degradation path under conditions of image quality loss or drastic changes in ground features, and achieve high-quality reconstruction of the target image.

[0068] To verify the effectiveness of the method of this invention, two datasets were used. The first dataset contained 24 cloudless scenes from 2018, of which 11 were from Landsat-8 and 13 were from Sentinel-2. The second dataset contained 23 cloudless scenes from 2019, of which 10 were from Landsat-8 and 13 were from Sentinel-2. Using the ground truth (GT) as a reference, the performance of the following five models was compared: STARFM-SI (Remote Sensing Spatiotemporal Fusion Algorithm: Spatiotemporal Adaptive Reflectivity Fusion Model), ATPRMK (Area-to-point regression kriging), GANSTFM (A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network), MARSTFN (Deep Learning-Based Spatiotemporal Fusion Architecture of Landsat 8 and Sentinel-2 Data for 10 mSeries Imagery) and the method of this invention (Proposed).

[0069] The experimental results for the first dataset and the second dataset are shown below. Figure 5 and Figure 6As shown in the magnified view, the STARFM-SI predicted image in the first dataset exhibits spectral distortion, and the ATPRF image is relatively blurry. The deep learning-based methods GANSTFM, MARSTFN, and the method proposed in this invention outperform other methods in terms of spatial resolution and spectral restoration. The experimental results on the second dataset are similar to those on the first dataset, again showing that deep learning-based methods perform relatively better. However, among several deep learning methods, the method proposed in this invention does not require real labeled images throughout the denoising process, relying entirely on the restoration path learned by the model for image reconstruction. Even with significant dynamic changes in ground features or incomplete image quality, it can still stably output structurally complete and detail-rich fused images, achieving high-quality reconstruction of the target image.

[0070] In one exemplary embodiment, a computer-readable storage medium is included, which stores a computer program that, when executed by a processor, implements the above-described spatiotemporal fusion method for multispectral imaging satellite data.

[0071] Please see Figure 7 In one exemplary embodiment, the device further includes an electronic device including at least one processor, at least one memory, and at least one communication bus.

[0072] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the aforementioned spatiotemporal fusion method for multispectral imaging satellite data.

[0073] In one exemplary embodiment, a computer program product is proposed, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described spatiotemporal fusion method for multispectral imaging satellite data.

[0074] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A spatiotemporal fusion method for multispectral imaging satellite data, characterized in that, Includes the following steps: S1. Acquire two types of multispectral imaging satellite data, add noise to the satellite data used as tags through a diffusion process to obtain noisy data, and construct a data pair between the two types of satellite data; S2. Construct a noise predictor. The noise predictor adopts an encoder-decoder architecture based on the U-Net structure. The encoder includes a noisy feature encoder and a clean feature encoder. S3. Input the noisy data into the noisy feature encoder. First, extract the features of the noisy image from the noisy data. Then, perform multi-scale feature extraction on the features of the noisy image to obtain multi-scale features of the noisy image. Embed the time step information of the diffusion process into the noisy feature encoder. The time step information is processed by MLP to generate scaling coefficients and bias coefficients of affine transformation to adaptively adjust the feature distribution of the noisy image. Input the data pairs of the two satellite data into the clean feature encoder. First, fuse the data pairs to obtain clean image features. Then, perform multi-scale feature extraction on the clean image features to obtain multi-scale clean image features. At each scale, the difference features between the features of the noisy image and the features of the clean image are calculated, and the difference features are sent to the decoder for reconstruction to obtain the predicted noise data. S4. Based on the predicted noisy data, denoise the noisy data to obtain clean data; The noisy feature encoder comprises n DGRBs and n-1 downsampling modules. The first n-1 noisy feature coding levels consist of a cascaded DGRB and a downsampling module, while the last noisy feature coding level contains a single DGRB. The noisy image features... Features are obtained sequentially through n noisy feature encoding levels. ; DGRB consists of an MLP part, two convolutional parts, and a channel weight generation part. The time step information is processed by the MLP to generate scaling and bias coefficients for the affine transformation. and DGRB input features After the first convolutional part, the features are obtained. ,feature and Multiply and then Adding together yields the features ,feature After the second convolutional part, the features are obtained. ,feature With features Adding together yields the features ,feature The channel weights are obtained after the channel weight generation process, and the features are... The output features of DGRB are obtained by multiplying the channel weights.

2. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 1, characterized in that, Noisy data can be expressed by the following formula: in, Indicates time step Noisy data, This represents the retention factor accumulated during forward diffusion. This represents the raw satellite data. It is standard Gaussian noise.

3. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 1, characterized in that, Features of noisy images are extracted from noisy data using convolution operations.

4. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 1, characterized in that, A multi-resolution feature fusion network is used to fuse data pairs from two satellites. When the two satellite data are Landsat-8 and Sentinel-2 remote sensing satellite images, the fusion process of the multi-resolution feature fusion network for the data pairs is as follows: The 60-meter band image in the Sentinel-2 imagery is upsampled to a 30-meter scale and then stitched with the 30-meter band image. The stitched features are then input into a multi-level attention feature extraction module. The output features of the multi-level attention feature extraction module are residually concatenated with the features upsampled from the 60-meter band image and passed through a convolutional layer to obtain the Sentinel-2 image features. ; Landsat-8 imagery with resolutions of 45 meters and 90 meters was upsampled to a 30-meter scale and compared with feature... The two satellite images are stitched together, and their stitched features are input into a multi-level attention feature extraction module. The output features of the multi-level attention feature extraction module are residually concatenated with the features upsampled from the 90-meter band image and then passed through a convolutional layer to obtain clean image features.

5. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 4, characterized in that, The multi-level attention feature extraction module includes two convolutional layers and three ARDBs. The input features of the multi-level attention feature extraction module pass through the first convolutional layer and the three ARDBs in sequence. The output features of the three ARDBs are concatenated and then input into the second convolutional layer to obtain the output features of the multi-level attention feature extraction module. ARDB consists of an attention mechanism module and residual dense blocks.

6. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 1, characterized in that, The clean image feature encoder consists of n residual blocks and n-1 downsampling modules. The first n-1 clean image feature encoding levels consist of a residual block and a downsampling module connected in series. The last clean image feature encoding level contains a residual block. Features are obtained sequentially through n clean image feature encoding levels. ; The decoder consists of n DGRBs and n-1 upsampling modules. The first n-1 decoding levels each consist of a cascaded DGRB and an upsampling module. The last decoding level contains a single DGRB. The input to the first decoding level is... and Feature splicing, the first The input to each decoding level is and The feature splicing, among which , This is the output of the (i-1)th decoding level.

7. The spatiotemporal fusion method for multispectral imaging satellite data according to claim 1, characterized in that, A denoising diffusion implicit model is used as the sampling method to denoise the noisy data and obtain clean data. The specific sampling equation is shown in the formula: in, , Representing time steps , The sample, , Representing time steps , The cumulative noise attenuation factor For the predicted noise, For time step The adjustment factor is z, where z is the random noise term added during sampling.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

9. An electronic device, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to perform the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-7.

Images