High-temporal-spatial-resolution optical image reconstruction method and system based on multi-source remote sensing data fusion

Abstract

The invention discloses a high-temporal-spatial-resolution optical image reconstruction method and system based on multi-source remote sensing data fusion, and aims to solve the problem of temporal-spatial information loss of an optical remote sensing image due to cloud layer shielding, sensor fault and the like in the prior art. The method comprises the following steps: 1, obtaining and preprocessing multi-source remote sensing data and constructing a data pair; 2, constructing and training an optical image reconstruction model, wherein the model comprises a generator for receiving multi-source data as input and a discriminator; and 3, reconstructing the optical image without the time phase by using the trained generator, and generating a complete continuous optical image time sequence with high temporal-spatial resolution. According to the method, the vacancy of optical image data can be effectively filled up, the generated image has high spatial and spectral precision, data with different resolutions are effectively processed through a multi-input network structure, the influence of geometric registration errors is reduced, and reliable data guarantee is provided for remote sensing application depending on continuous optical observation.

Description

Technical Field

[0001] The present invention belongs to the technical fields of remote sensing technology and earth observation technology, and particularly relates to a method and system for reconstructing high spatio-temporal resolution optical images based on multi-source remote sensing data fusion. Background Art

[0002] Remote sensing images, as important data sources for earth observation, play an irreplaceable role in a wide range of applications. In particular, time-series optical images have important value in various application scenarios such as surface phenology surveys, disaster prediction, and vegetation monitoring.

[0003] However, current optical imaging technologies still have inherent limitations, mainly affected by clouds. Due to the high reflectivity of clouds in the optical band, optical signals are difficult to penetrate clouds, often resulting in image distortion, decreased clarity, and even complete occlusion of ground object information. According to statistics, the cloud coverage rate is relatively high globally and on land. Large-scale and long-term cloud cover will lead to serious loss of temporal phase information of optical images. Especially in cloudy and rainy areas, it is difficult to obtain continuous and cloud-free optical image sequences. In addition, problems such as sensor failures may also lead to a decline in image quality or data loss. These problems have seriously hindered the application research relying on continuous optical remote sensing image time series, such as regional crop pest and disease monitoring and evaluation.

[0004] Therefore, there is an urgent need for a method that can effectively fill in the missing optical remote sensing data and reconstruct the time-gapped images to obtain complete and continuous high spatio-temporal resolution optical time series data. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for reconstructing high spatio-temporal resolution optical images based on multi-source remote sensing data fusion, aiming to overcome the problems of spatio-temporal information loss caused by cloud occlusion, sensor failures, etc. in existing optical remote sensing images, generate high spatio-temporal resolution and complete and continuous optical time series data, and provide high-quality data guarantee for remote sensing applications such as regional-scale vegetation monitoring, disaster warning, and crop disease monitoring and evaluation.

[0006] In the first aspect, the present invention provides a method for reconstructing high spatio-temporal resolution optical images based on multi-source remote sensing data fusion, which includes:

[0007] Using the existing cloud-free satellite optical data before and after the target time phase as reference data, and the synthetic aperture radar data and moderate resolution imaging spectrometer data closest to the target time phase as the first auxiliary data and the second auxiliary data respectively, input them into the optical image reconstruction model to generate the satellite optical data of the target time phase.

[0008] The optical image reconstruction model adopts a generative adversarial network structure. The reference data and the first auxiliary data are concatenated and then input into the first encoding branch of the optical image reconstruction model. The first encoding branch performs layer-by-layer downsampling feature extraction on the input data through residual convolution. The second auxiliary data is input into the second encoding branch of the optical image reconstruction model. The second encoding branch performs feature extraction on the input data through residual convolution. The output features of the first encoding branch and the second encoding branch are concatenated and then input into the decoder. The decoder performs layer-by-layer upsampling on the input features through residual transposed convolution. The outputs of the residual convolutions in each layer of the first encoding branch are concatenated and fused with the inputs of the residual transposed convolutions in each layer of the decoder through skip connections.

[0009] Preferably, the total loss function of the optical image reconstruction model includes adversarial loss, L1 norm loss, and spectral angle mapping loss. The spectral angle mapping loss takes the mean of the angles between the spectral vectors of each pixel in the generated satellite optical data and the real satellite optical data.

[0010] Preferably, each residual convolution block in the first encoding branch includes a two-dimensional convolutional layer, a batch normalization layer, and a leaky rectified linear unit (Leaky ReLU), and uses a residual connection structure to perform layer-by-layer encoding on the reference data and the first auxiliary data.

[0011] Preferably, the output of the last residual transposed convolution in the decoder adjusts the number of feature channels through multiple residual convolutions and one convolution, and then passes through a hyperbolic tangent activation function to obtain the satellite optical data of the target time phase.

[0012] Preferably, the reference data, the first auxiliary data, and the second auxiliary data are cropped to form image patches and then input into the optical image reconstruction model; the image reconstruction result patches output by the optical image reconstruction model are concatenated to form the satellite optical data of the target time phase.

[0013] Preferably, the discriminator in the optical image reconstruction model generates a confidence level for the input image patches, including a series of convolutional blocks, a flattening layer, and a fully connected layer.

[0014] Preferably, the optical image reconstruction model is trained using an image dataset. The image dataset is obtained by preprocessing the satellite optical data, synthetic aperture radar data, and moderate resolution imaging spectroradiometer data. The preprocessing process includes: truncating the pixel values in the image data that exceed the valid range. Data normalization processing. Using a sliding window to crop the image data to obtain the image patches that make up the dataset. Expanding the sample size of the dataset through data augmentation.

[0015] Second aspect, the present invention provides a high spatio-temporal resolution optical image reconstruction system, which is used to execute the foregoing high spatio-temporal resolution optical image reconstruction method. The high spatio-temporal resolution optical image reconstruction system includes a data acquisition module, a preprocessing module, and a reconstruction module. The data acquisition module is used to obtain synthetic aperture radar data, moderate resolution imaging spectrometer data, and cloud-free satellite optical data before and after the target phase. The preprocessing module is used to preprocess the satellite optical data, synthetic aperture radar data, and moderate resolution imaging spectrometer data. The reconstruction module includes a generator and a discriminator. The generator includes an encoder and a decoder. The encoder includes two encoding branches. The first encoding branch includes a plurality of residual convolutional blocks connected in series. The second encoding branch includes a residual convolutional block. The output features of the first encoding branch and the second encoding branch are concatenated and then input into the decoder. The decoder includes a plurality of residual deconvolutional blocks, a plurality of residual convolutional blocks, a convolutional layer, and an activation function connected in series. The output of the residual convolutional block in the first encoding branch of the corresponding layer is jump-connected to the input of the residual deconvolutional block in the encoder.

[0016] Third aspect, the present invention provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. The memory stores the computer program; the processor executes the foregoing high spatio-temporal resolution optical image reconstruction method.

[0017] Fourth aspect, the present invention provides a readable storage medium, which stores a computer program; when the computer program is executed by a processor, it is used to implement the foregoing high spatio-temporal resolution optical image reconstruction method.

[0018] The beneficial effects of the present invention are as follows:

[0019] 1. The present invention can effectively reconstruct the missing optical remote sensing images caused by reasons such as cloud occlusion and sensor failure, fill the data gap, generate high spatio-temporal resolution and complete continuous optical time series data, and provide reliable data guarantee for various remote sensing applications relying on continuous optical observations.

[0020] 2. The present invention combines the advantages of multi-source remote sensing data (including optical and SAR), adopts a specially designed optical image reconstruction model, and introduces spectral angle mapping loss into the loss function of the model, effectively processes multi-source heterogeneous data, and significantly improves the spatial and spectral accuracy of the reconstructed images, especially in complex or data missing areas.

[0021] 3. The multi-input network structure design adopted by the present invention can effectively process data with different resolutions, avoid information loss or distortion caused by resampling in traditional methods, reduce the impact of geometric registration error on the model performance, and enhance the robustness and applicability of the method. Description of the Drawings

[0022] Figure 1 It is the flowchart of Embodiment 1 of the present invention.

[0023] Figure 2 It is the structural diagram of the optical image reconstruction model in Embodiment 1 of the present invention.

[0024] Figure 3 It is a partial generation of image blocks and real image blocks in the Ningbo Experimental Area in Embodiment 1 of the present invention.

[0025] Figure 4 It is the result map of high spatio-temporal resolution optical time series data generated in the Ningbo Experimental Area from May 1 to November 25, 2021 in Embodiment 1 of the present invention. Specific Embodiments

[0026] The following further elaborates in detail a specific embodiment of the present invention with reference to the accompanying drawings.

[0027] Embodiment 1

[0028] As Figure 1 shown, a high spatio-temporal resolution optical image reconstruction method based on multi-source remote sensing data fusion includes data acquisition, preprocessing and data pair construction, optical image reconstruction model construction and training, and optical image reconstruction and time series generation. In this embodiment, taking the Ningbo Experimental Area as an example, using the acquired Sentinel-2 (Sentinel-2) optical data, Sentinel-1 (Sentinel-1) SAR (Synthetic Aperture Radar) data, and MODIS (Moderate-Resolution Imaging Spectroradiometer) optical data, by constructing and training an optical image reconstruction model, the reconstruction of missing optical images is realized, and complete and continuous high spatio-temporal resolution optical time series data is generated. The specific steps are as follows:

[0029] Step 1: Perform the acquisition, preprocessing and data pair construction of multi-source remote sensing data. Acquire multi-source remote sensing image data within the Ningbo Experimental Area, including Sentinel-2 optical data, Sentinel-1 SAR data, and MODIS optical data. The corresponding selected band information for each data is shown in Table 1.

[0030] Table 1

[0031]

[0032] Preprocess the acquired original data. The specific process of preprocessing is as follows:

[0033] (1) Outlier processing is performed. To ensure the training stability of the model, the pixel values in the original image data that exceed the valid range are truncated. Specifically, the pixel values of Sentinel-2 images are truncated to the range of 0–10,000; the pixel values of Sentinel-1 SAR images are truncated to the range of -50–1; the pixel values of MODIS images are truncated to the range of -100–16,000.

[0034] (2) Data normalization processing is performed. All the image data after outlier processing are normalized so that their mean is 0 and the standard deviation is 1, which is convenient for the convergence and training of the model.

[0035] (3) The image is cropped using a sliding window to construct image patches suitable for model input. Among them, for high-resolution Sentinel-2 and Sentinel-1 SAR images, a sliding window of 250×250 pixels is used for cropping, and the stride is set to 200 pixels to increase the data volume and the overlap between windows. For MODIS images, a sliding window of 5×5 pixels matching its resolution is used for cropping, and the stride is set to 4 pixels.

[0036] (4) A data augmentation strategy is adopted. To improve the robustness and generalization ability of the model, the cropped data are randomly flipped up and down and left and right, and the data set is amplified to four times the original data.

[0037] Through the above preprocessing process, time-matched data pairs for model training, validation, and testing are finally constructed. The time-matched data pairs include SAR image patches and MODIS optical image patches as auxiliary inputs to the model, Sentinel-2 optical image patches (Sentinel-2 images adjacent before and after the target time phase and cloud-free) that are temporally close to the target time phase and cloud-free as reference inputs to the model, and cloud-free Sentinel-2 optical image patches corresponding to the target time phase as outputs of the model.

[0038] Step 2: Construct and train an optical image reconstruction model; the optical image reconstruction model adopts a generative adversarial network model with the main body of MSF-ECGAN, as Figure 2 shown. The optical image reconstruction model includes a generator and a discriminator.

[0039] The generator receives the reference data Sentinel-2 image patches, the auxiliary data SAR image patches, and MODIS image patches as inputs, and outputs the corresponding optical image reconstruction result patches. The generator adopts an improved encoder-decoder structure (Encoder-Decoder), including an encoder and a decoder. The encoder includes two encoding branches. The first encoding branch includes six residual convolutional blocks (ResConvBlock) connected in series. The second encoding branch includes one residual convolutional block. The input of the first encoding branch is the concatenated features of the reference data Sentinel-2 image patches and the auxiliary data SAR image patches. The input of the second encoding branch is the auxiliary data MODIS image patches. The decoder includes six residual deconvolutional blocks (ResDeconvBlock) connected in series, three residual convolutional blocks (ResConvBlock), a 1×1 convolutional layer, and a Tanh activation function. The output of the residual convolutional block in the first encoding branch of the corresponding layer is skip-connected to the input of the residual deconvolutional block in the encoder. The output features of the first encoding branch and the second encoding branch are concatenated and then input into the decoder.

[0040] In the first encoding branch of the encoder, for the input reference data Sentinel-2 and auxiliary data SAR (the total number of input channels is 18), first, six residual convolutional blocks (ResConvBlock) connected in series are used for layer-by-layer downsampling feature extraction. Each residual convolutional block includes a two-dimensional convolutional layer, a batch normalization layer (BatchNorm), and an activation function (LeakyReLU), and uses a residual connection structure to encode the concatenated features of the Sentinel-2 image patches and the SAR image patches layer by layer, extracting multi-scale deep features.

[0041] In the second encoding branch of the encoder, after the auxiliary data MODIS image patches (input channels are 6) are extracted by an independent residual convolutional block (input-output channels are 6→6, kernel size is 3, stride is 1, padding is 1), before entering the decoder, they are concatenated with the features output by the first encoding branch (1152 channels), and the number of channels of the concatenated features is 1158.

[0042] In the decoder part, there are 6 residual deconvolution blocks (ResDeconvBlock). Each residual deconvolution block (ResDeconvBlock) uses deconvolution operations to upsample the high-dimensional features extracted by the encoding part layer by layer to gradually restore the spatial resolution. At the same time, skip connections are used to splice and fuse the features of each layer of the encoder with the upsampling results of the corresponding layers of the decoder to combine and utilize feature information of different scales, retaining more effective details and spatial structures. At the end of the decoder, three serial residual convolution blocks and a final 1×1 convolution layer adjust the number of feature channels and pass through the Tanh activation function to finally output the optical image reconstruction result block.

[0043] The optical image reconstruction result block output by the generator and the real target Sentinel-2 optical image block (8 bands) are respectively spliced in the channel dimension with the corresponding SAR image block (2 bands) to generate the corresponding spliced image blocks. The spliced image block generated based on the optical image reconstruction result block is a fake spliced block; the spliced image block generated based on the real target Sentinel-2 optical image block is a real spliced block.

[0044] The discriminator receives the spliced image block as input, and the total input channel number is 10. The discriminator includes 7 cascaded convolution blocks. Each convolution block includes a cascaded two-dimensional convolution layer (Conv2d), a batch normalization layer (BatchNorm), and an activation function (LeakyReLU). The input channels and output channels of the convolution block are in turn: 10→20, 20→40, 40→80, 80→40, 40→20, 20→10, 10→5; the corresponding convolution kernel sizes, strides, and paddings are: (3, 2, 1), (3, 2, 1), (3, 2, 1), (2, 2, 0), (2, 2, 0), (2, 2, 0), (2, 2, 0). The extracted features are then flattened through a flattening layer (Flatten) and input into a fully connected layer, and finally a confidence score indicating whether the input image block is "fake" (generated by the generator) or "real" (real data) is output.

[0045] During the training process, a variety of loss functions are comprehensively used to optimize the optical image reconstruction model in an end-to-end manner. The total loss function is composed of a weighted combination of adversarial loss, L1 loss, and spectral angle (SAM) mapping loss, and its form is shown in formula (1):

[0046] (1)

[0047] where is the adversarial loss, and the least squares GAN loss is adopted, as shown in formula (2):

[0048] L LSCGAN = 1 2 [ D ( real , anc ) − 1 ] 2 + 1 2 [ D ( G ( ref , anc ), anc )] 2 (2)

[0049] is the pixel loss, using the L1 norm, as shown in formula (3):

[0050] (3)

[0051] is the spectral angle mapping loss, used to measure the spectral similarity between the generated image and the real image, as shown in formula (4):

[0052] (4)

[0053] D represents the discriminator, and G represents the generator; represents the real image, represents the generated image, represents the auxiliary image, represents the reference image; H, W, and C respectively represent the height, width, and number of channels of the image block; and are the weight coefficients of the L1 loss and the SAM loss respectively, and the optimal weight coefficients are determined through repeated experiments and set to = 100, = 10.

[0054] In the model training stage, the constructed time-matched data pairs are divided into a training set, a validation set, and a test set. In this embodiment, 9432 data from the total data set (a total of 11394 data) are used for the training set, 1038 data are used for the validation set, and 924 data are used for the test set. During the training process, the data batch size is set to 8. The Adam optimizer is used to optimize the parameters of both the generator and the discriminator, and the initial learning rate is set to 0.0002, and the momentum parameters 、 。

[0055] Step 3: Use the trained optical image reconstruction model, take the corresponding auxiliary data image blocks (SAR and MODIS) and reference data image blocks (Sentinel-2 at different time phases) of the time to be reconstructed as inputs, and the generator can output the reconstructed optical image block of this time phase. Repeat this process for all missing time phases in the time series, and splice the reconstructed image blocks to finally obtain a complete and continuous optical image time series with high spatio-temporal resolution.

[0056] The test results in the Ningbo experimental area show that the reconstruction method provided in this embodiment has achieved good accuracy. Evaluated using common image quality assessment indicators, the obtained mean absolute error (MAE) is 0.0131, the root mean square error (RMSE) is 0.0005, the peak signal-to-noise ratio (PSNR) is 36.2754 dB, the spectral angle mapper (SAM) is 0.0518, and the structural similarity index (SSIM) is 0.9431. As Figure 3 shown are some generated image patches and real image patches in the Ningbo experimental area. The generated image results are highly consistent with the real images in terms of spatial structure, spatial texture, and ground object spectra. As Figure 4 shown are the results of continuous high spatio-temporal resolution optical time series data obtained in the Ningbo experimental area from May 1 to November 25, 2021. These results indicate that the reconstruction method provided in this embodiment can generate optical images highly similar to real images and has high spatial and spectral fidelity.

[0057] Embodiment 2

[0058] A high spatio-temporal resolution optical image reconstruction system for implementing the high spatio-temporal resolution optical image reconstruction method provided in Embodiment 1. The high spatio-temporal resolution optical image reconstruction system includes a data acquisition module, a preprocessing module, and a reconstruction module. The data acquisition module is used to obtain synthetic aperture radar data, moderate resolution imaging spectrometer data, and satellite optical data of the missing target phase.

[0059] The preprocessing module is used to preprocess the satellite optical data, synthetic aperture radar data, and moderate resolution imaging spectrometer data. The reconstruction module includes a generator and a discriminator. The generator includes an encoder and a decoder. The encoder includes two encoding branches. The first encoding branch includes six residual convolutional blocks connected in series.

[0060] The input channels and output channels of the 6 residual convolutional blocks (ResConvBlock) in the first encoding branch are in sequence: 18→36, 36→72, 72→144, 144→288, 288→576, 576→1152; the corresponding convolutional kernel sizes, strides, and paddings are in sequence: (3, 2, 1), (3, 2, 2), (3, 2, 1), (2, 2, 0), (2, 2, 0), (2, 2, 1).

[0061] The second encoding branch includes one residual convolutional block. The output features of the first encoding branch and the second encoding branch are concatenated and then input into the decoder. The decoder includes six residual transposed convolutional blocks, three residual convolutional blocks, a 1×1 convolutional layer, and an activation function connected in series.

[0062] The input and output channels of six residual deconvolution blocks (ResDeconvBlock) are as follows in sequence: 1158→579, 866→578, 577→433, 360→288, 126→90, 32→16; the corresponding deconvolution kernel sizes, strides, and paddings are: (4, 1, 0), (2, 2, 0), (2, 2, 0), (2, 2, 0), (3, 2, 2), (2, 2, 0).

[0063] The input and output channels of three sequential residual convolution blocks are as follows in sequence: 50→24, 24→16, 16→8, the convolution kernel size is 3, the stride is 1, and the padding is 1.

[0064] The input and output channels of the 1×1 convolution layer are 8→8, the convolution kernel size is 1, the stride is 1, and the padding is 0.

[0065] In the first encoding branch of the corresponding layer, there is a skip connection between the output of the residual convolution block and the input of the residual deconvolution block in the encoder.

[0066] Embodiment 3

[0067] An electronic device. Specifically, the electronic device includes a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the method described in Embodiment 1 is implemented.

[0068] Among them, the memory may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (Non-volatile Memory), such as at least one disk memory. Through at least one communication interface (which can be wired or wireless), a communication connection is realized between this system network element and at least one other network element. The Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0069] The bus can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0070] Among them, the memory is used to store a program. After receiving an execution instruction, the processor executes the program. The method executed by the device defined by the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor or implemented by the processor.

[0071] A processor may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0072] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, reference can be made to the foregoing method embodiments, which will not be elaborated here.

[0073] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program code.

[0074] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any technician familiar with the technical field of the present invention can still modify the technical solutions described in the foregoing embodiments or easily think of changes, or perform equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for reconstructing high spatio-temporal resolution optical images based on multi-source remote sensing data fusion, characterized in that, The method includes: Using the existing cloud-free satellite optical data before and after the target time phase as reference data, and the synthetic aperture radar data and the moderate resolution imaging spectrometer data closest to the target time phase as the first auxiliary data and the second auxiliary data respectively, inputting them into the optical image reconstruction model to generate the satellite optical data of the target time phase; The optical image reconstruction model adopts a generative adversarial network structure; the reference data and the first auxiliary data are spliced and then input into the first encoding branch in the optical image reconstruction model; the first encoding branch performs layer-by-layer downsampling feature extraction on the input data through residual convolution; the second auxiliary data is input into the second encoding branch in the optical image reconstruction model; the second encoding branch performs feature extraction on the input data through residual convolution; the output features of the first encoding branch and the second encoding branch are spliced and then input into the decoder; the decoder performs layer-by-layer upsampling on the input features through residual transposed convolution; the outputs of each layer of residual convolution in the first encoding branch are spliced and fused with the inputs of each layer of residual transposed convolution in the decoder through skip connections.

2. The high spatio-temporal resolution optical image reconstruction method according to claim 1, wherein: The total loss function of the optical image reconstruction model includes adversarial loss, L1 norm loss, and spectral angle mapping loss; the spectral angle mapping loss takes the mean of the angles between the spectral vectors of each pixel in the generated satellite optical data and the real satellite optical data.

3. The high spatio-temporal resolution optical image reconstruction method according to claim 1, characterized in that: Each residual convolution block in the first encoding branch includes a two-dimensional convolutional layer, a batch normalization layer, and a leaky rectified linear activation function, and uses a residual connection structure to perform layer-by-layer encoding on the reference data and the first auxiliary data.

4. The high spatio-temporal resolution optical image reconstruction method according to claim 1, wherein: The output of the last layer of residual transposed convolution in the decoder is adjusted for the number of feature channels through multiple layers of residual convolution and one layer of convolution, and then passes through a hyperbolic tangent activation function to obtain the satellite optical data of the target time phase.

5. The high spatio-temporal resolution optical image reconstruction method according to claim 1, characterized in that: The reference data, the first auxiliary data, and the second auxiliary data are cropped to form image patches and then input into the optical image reconstruction model; the optical image reconstruction result patches output by the optical image reconstruction model are spliced to form the satellite optical data of the target time phase.

6. The high spatio-temporal resolution optical image reconstruction method according to claim 1, wherein: The discriminator in the optical image reconstruction model generates a confidence level for the input image patches, including a series of convolutional blocks, a flattening layer, and a fully connected layer.

7. The high spatio-temporal resolution optical image reconstruction method according to claim 1, characterized in that: The optical image reconstruction model is trained using an image dataset; the image dataset is obtained by preprocessing the satellite optical data, synthetic aperture radar data, and moderate resolution imaging spectrometer data; The process of the preprocessing includes: truncating the pixel values in the image data that exceed the valid range; performing data normalization processing; using a sliding window to crop the image data to obtain the image patches that make up the dataset; expanding the number of samples in the dataset through data augmentation.

8. A high spatio-temporal resolution optical image reconstruction system, characterized in that: For performing the high spatio-temporal resolution optical image reconstruction method described in claim 1; the high spatio-temporal resolution optical image reconstruction system includes a data acquisition module, a preprocessing module, and a reconstruction module; the data acquisition module is used to obtain synthetic aperture radar data, medium-resolution imaging spectrometer data, and cloud-free satellite optical data before and after the target phase; the preprocessing module is used to preprocess the satellite optical data, synthetic aperture radar data, and medium-resolution imaging spectrometer data; the reconstruction module includes a generator and a discriminator; the generator includes an encoder and a decoder; the encoder includes two encoding branches; the first encoding branch includes a plurality of residual convolutional blocks connected in series; the second encoding branch includes a residual convolutional block; the output features of the first encoding branch and the second encoding branch are concatenated and then input into the decoder; the decoder includes a plurality of residual transposed convolutional blocks, a plurality of residual convolutional blocks, a convolutional layer, and an activation function connected in series; the output of the residual convolutional block in the first encoding branch at the corresponding layer is jump-connected to the input of the residual transposed convolutional block in the encoder.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: The memory stores a computer program; the processor executes the high spatio-temporal resolution optical image reconstruction method described in any one of claims 1-7.

10. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by the processor, it is used to implement the high spatio-temporal resolution optical image reconstruction method described in any one of claims 1-7.

Images