Lake algae bloom spatio-temporal fusion method and system
By constructing a generative adversarial network model with dual discriminators and collaborative supervision, and combining spatial and temporal discriminators, the contradiction between high temporal resolution and high spatial resolution in remote sensing monitoring of algal blooms in lakes was resolved, generating high-quality remote sensing image sequences and achieving high precision and continuity in dynamic monitoring of algal blooms in lakes.
Patent Information
- Application Number
- CN202610789589.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-03
- Publication Date
- 2026-08-25
AI Technical Summary
Existing remote sensing technologies for monitoring algal blooms in lakes cannot simultaneously meet the requirements of high temporal and spatial resolution, resulting in blurred internal structures or time-series breaks in algal blooms. Existing deep learning methods are prone to over-smoothing under pixel-level loss constraints, lacking explicit constraints in the temporal dimension and the preservation of water body information.
A generative adversarial network model was constructed by employing a dual discriminator collaborative supervision mechanism. By combining spatial and temporal discriminators and optimizing the generator through a joint loss function, high spatial resolution and temporal continuity remote sensing images of lake algal blooms were generated. MODIS and Sentinel-2 images were then used for data fusion.
It significantly improves the spatial realism and temporal continuity of the fused images, generating a continuous remote sensing image sequence with both high temporal and spatial resolution, providing reliable data support for the dynamic monitoring of lake algal blooms.
Smart Images
Figure CN122637247A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of water environment monitoring technology, specifically to a method and system for spatiotemporal fusion of remote sensing images of algal blooms in lakes. Background Technology
[0002] Monitoring algal blooms in lakes places dual demands on remote sensing data: high temporal resolution (to capture the rapid evolution of algal blooms) and high spatial resolution (to characterize the internal spatial structure of algal blooms). However, existing single sensors cannot simultaneously meet both requirements: High temporal resolution sensors (such as MODIS and VIIRS) can achieve daily or higher revisit frequencies, but their spatial resolution is typically ≥250 meters, resulting in severe blurring of the fine structure of algal blooms within the lake and an inability to distinguish spatial differences between nearshore and lake center areas, and between shallow and deep water areas. High spatial resolution sensors (such as Sentinel-2 MSI and Landsat OLI) can achieve spatial resolutions of 10-30 meters, clearly presenting the spatial morphology of algal blooms, but their revisit cycles are long (5-10 days for Sentinel-2 and 16 days for Landsat) and are severely affected by weather conditions. The actual effective observation frequency is far lower than the theoretical value, causing severe breaks in the time series and making it impossible to support dynamic analysis of the entire life cycle of algal blooms.
[0003] Current deep learning-based fusion methods achieve image reconstruction through end-to-end learning, but they are prone to over-smoothing under pixel-level loss (L1 / L2) constraints, leading to the loss of high-frequency textures. Deep learning-based fusion methods (such as SRGAN and STF) improve texture generation quality through adversarial training, but existing research has the following common shortcomings: the optimization objective focuses on single-phase accuracy, lacking explicit constraints on the dynamic coherence of generating complete time series; there is a lack of specific design for water bodies, resulting in insufficient retention of information such as red edges that are sensitive to targets such as algae; and the discriminator structure is simple, constraining only from the spatial dimension and failing to ensure the rationality of evolution in the temporal dimension at the same time.
[0004] Therefore, it is necessary to provide a new method for spatiotemporal fusion of remote sensing images of algal blooms in lakes. Summary of the Invention
[0005] Based on the aforementioned problems in the existing technology, the purpose of this invention is to provide a spatiotemporal fusion method and system for remote sensing images of algal blooms in lakes. Through a dual discriminator collaborative supervision mechanism, the generated results are simultaneously constrained from two complementary dimensions: spatial authenticity and temporal continuity. This effectively improves the spatial authenticity and temporal continuity of the fused images, providing reliable data support for high-precision, high-frequency dynamic monitoring of algal blooms in lakes.
[0006] To achieve the above objectives, the technical solution adopted by this invention is: a spatiotemporal fusion method for remote sensing images of lake algal blooms, comprising:
[0007] S1, acquire multi-source satellite imagery data and perform preprocessing. The multi-source satellite imagery data includes high temporal resolution satellite imagery and high spatial resolution satellite imagery. S2, Construct a dual-discriminator collaborative supervised generative adversarial network model, wherein the generative adversarial network model includes a generator, a spatial discriminator, and a temporal discriminator; S3, the parameters of the adversarial generative network model are optimized using a joint loss function, which includes content reconstruction loss, perceptual loss and adversarial loss; S4: Input the high temporal resolution time series images to be fused into the trained generative adversarial network model, and output the high spatial resolution fused image of the target date.
[0008] Furthermore, the acquisition and preprocessing of multi-source satellite imagery data includes: Acquire high temporal resolution MODIS satellite imagery and high spatial resolution Sentinel-2 satellite imagery. MODIS imagery provides high temporal resolution observations with daily revisits, while Sentinel-2 imagery provides high spatial resolution observations with a spatial resolution of 10-30m. Spatial matching was performed on MODIS and Sentinel-2 images. The MODIS images were unified to a spatial resolution of 20m through bilinear resampling and aligned with the Sentinel-2 images. Spectral matching was performed on MODIS and Sentinel-2 images. Based on the spectral response functions of MODIS and Sentinel-2, six common bands with similar center wavelengths were selected for fusion. The common bands include blue light band, green light band, red light band, near-infrared band, shortwave infrared 1 band, and shortwave infrared 2 band.
[0009] Furthermore, the construction of the dual-discriminator collaborative supervised generative adversarial network model includes: A generator is constructed, which adopts an encoder-decoder symmetrical structure. The generator includes a feature enhancement module and skip connections. The generator takes high temporal resolution satellite imagery as input and outputs a high spatial resolution fused imagery of the target date. A spatial discriminator is constructed using the PatchGAN architecture. The input to the spatial discriminator is a single-frame generated high-resolution image and the corresponding low-resolution conditional image, and the output is a local receptive field realism score matrix. A temporal discriminator is constructed. The temporal discriminator takes short temporal segments containing generated images as input, evaluates the temporal coherence of the short temporal segments, and judges whether the temporal evolution trajectory is smooth and reasonable.
[0010] Furthermore, the generator includes an encoder, a feature enhancement module, a skip connection, and a decoder; The encoder is used for multi-scale feature extraction and has a total of 5 encoding layers. Each layer consists of a convolution operation with a kernel size of 5×5, a LeakyReLU nonlinear activation function, and a downsampling with a stride of 2. The number of convolution kernels increases layer by layer. The feature enhancement module is connected between the encoder and the decoder, and consists of two convolutional layers with a kernel size of 5×5 and a channel count of 512, and a LeakyReLU activation function. It adds the input features to the residual mapping element-wise through residual connections. The skip connection directly transmits the low-level features output from each level of the encoder to the feature reconstruction layer of the corresponding scale of the decoder, thereby achieving the fusion of multi-scale information. The decoder is symmetrical to the encoder and has a total of 5 decoding layers. Each layer includes upsampling operation, feature fusion and nonlinear activation.
[0011] Furthermore, the spatial discriminator adopts the PatchGAN architecture. The input consists of a single-frame generated high-resolution image and the corresponding low-resolution MODIS conditional image. The output is an N×N discriminant matrix, where each element corresponds to a local receptive field realism score of the input image. The spatial discriminator is used to evaluate whether the local texture and edges of the generated image are natural; to evaluate whether the spectral distribution of the generated image conforms to the statistical characteristics of the real high-resolution remote sensing image; and to constrain the cross-scale physical consistency between the generated details and the low-resolution input to prevent the generation of unreasonable high-frequency artifacts.
[0012] Furthermore, the formula for calculating the joint loss function is as follows:
[0013] in, For the joint loss function, The weighting coefficients corresponding to the content reconstruction loss function. For content reconstruction loss function, The weight coefficients corresponding to the perceptual loss function. For the perceptual loss function, To counteract the weight coefficients corresponding to the loss function, To counteract the loss function.
[0014] Furthermore, the formula for calculating the content reconstruction loss function is as follows:
[0015]
[0016] Where H is the height of the image, W is the width of the image, C is the number of bands in the image, and N is the total number of pixels in the image. For content reconstruction loss function, This is a true high-resolution Sentinel-2 image. The predicted image output by the generator; The formula for calculating the perceptual loss function is as follows:
[0017] in, For the perceptual loss function, Let be the total number of elements in the feature map of the i-th layer. This is a true high-resolution Sentinel-2 image. The predicted image output by the generator. Let be the feature mapping function of the i-th layer of the pre-trained feature extraction network; The formula for calculating the adversarial loss function is as follows:
[0018] in, To counteract the loss function, To obtain the expectation of the generated image distribution, X represents the high-resolution Sentinel-2 image generated by the generator, and X represents the low-resolution MODIS image as input. This represents the probability that the discriminator will judge the generated image, i.e., the probability that it considers it to be a real image.
[0019] A spatiotemporal fusion system for remote sensing images of lake algal blooms, applied to the aforementioned spatiotemporal fusion method for remote sensing images of lake algal blooms, the system comprising: The data acquisition module is used to acquire and preprocess multi-source satellite image data, which includes high temporal resolution satellite imagery and high spatial resolution satellite imagery. The modeling module is used to construct a dual-discriminator collaborative supervised generative adversarial network model, which includes a generator, a spatial discriminator, and a temporal discriminator. The model training optimization module is used to optimize the parameters of the adversarial generative network model using a joint loss function, which includes content reconstruction loss, perceptual loss, and adversarial loss. The fusion output module is used to input high temporal resolution time-series images to be fused into the trained generative adversarial network model and output high spatial resolution fused images of the target date.
[0020] Embodiments of the present invention also provide a network-side server, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described spatiotemporal fusion method for remote sensing images of algal blooms in lakes.
[0021] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for spatiotemporal fusion of remote sensing images of lake algal blooms.
[0022] The beneficial effects of this invention are as follows: The spatiotemporal fusion method for lake algal bloom remote sensing images of this invention employs a dual-discriminator collaborative supervision mechanism, simultaneously introducing spatial and temporal discriminators to constrain the generated results from two complementary dimensions: spatial realism and temporal continuity. This effectively overcomes the limitations of existing methods that constrain only from a single dimension, significantly improving the overall quality of the fused images. The generator in the generative adversarial network model uses an encoder-decoder structure with residual learning and skip connections, achieving a balance between high-level semantic extraction and low-level detail preservation. Subpixel convolution effectively eliminates checkerboard artifacts during the upsampling process, resulting in images with better spatial continuity than traditional deconvolution methods. This generates continuous remote sensing image sequences with both high temporal and spatial resolution, providing data support for studying the development process of algal blooms. Attached Figure Description
[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0024] In the picture: Figure 1 A flowchart of a method for spatiotemporal fusion of remote sensing images of lake algal blooms provided in the first embodiment of the present invention; Figure 2 The structure diagram of the dual-discriminator collaborative supervised generative adversarial network model provided in the first embodiment of the present invention; Figure 3 Visible light images of MODIS raw images, model-generated images, and Sentinel real images of Chaohu Lake, Taihu Lake, and Dianchi Lake provided for the first embodiment of the present invention. Figure 4 A schematic diagram of the module of the lake algal bloom remote sensing image spatiotemporal fusion system provided for the second embodiment of the present invention; Figure 5 This is a schematic diagram of the network-side server provided according to the third embodiment of the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] First implementation method: The first embodiment of the present invention provides a spatiotemporal fusion method for remote sensing images of algal blooms in lakes, comprising: acquiring and preprocessing multi-source satellite image data, the multi-source satellite image data including high temporal resolution satellite images and high spatial resolution satellite images; constructing a dual-discriminator collaborative supervised generative adversarial network model, the generative adversarial network model including a generator, a spatial discriminator, and a temporal discriminator; optimizing the parameters of the adversarial generative network model using a joint loss function, the joint loss function including content reconstruction loss, perceptual loss, and adversarial loss; inputting the high temporal resolution time-series image to be fused into the trained generative adversarial network model, and outputting a high spatial resolution fused image for the target date. The spatiotemporal fusion method for lake algal bloom remote sensing images of this invention employs a dual-discriminator collaborative supervision mechanism, simultaneously introducing spatial and temporal discriminators to constrain the generated results from two complementary dimensions: spatial realism and temporal continuity. This effectively overcomes the limitations of existing methods that constrain only from a single dimension, significantly improving the overall quality of the fused images. The generator in the generative adversarial network model utilizes an encoder-decoder structure with residual learning and skip connections, achieving a balance between high-level semantic extraction and low-level detail preservation. Subpixel convolution effectively eliminates checkerboard artifacts during upsampling, resulting in images with superior spatial continuity compared to traditional deconvolution methods. This generates continuous remote sensing image sequences with both high temporal and spatial resolution, providing data support for studying the development process of algal blooms.
[0027] The following details the implementation of the spatiotemporal fusion method for lake algal bloom remote sensing images in this embodiment. The following details are provided for ease of understanding and are not essential for implementing this solution. The specific process of this embodiment is as follows: Figure 1 As shown.
[0028] Step S1: Acquire multi-source satellite imagery data and perform preprocessing. The multi-source satellite imagery data includes high temporal resolution images and high spatial resolution images.
[0029] Specifically, the acquisition and preprocessing of multi-source satellite imagery data includes the following steps: Step S11: Acquire MODIS high temporal resolution satellite imagery and Sentinel-2 high spatial resolution satellite imagery. MODIS imagery provides high temporal resolution observations that are revisited daily, while Sentinel-2 imagery provides high spatial resolution observations with a spatial resolution of 10-30m.
[0030] Step S12: Spatial matching is performed on the MODIS image and the Sentinel-2 image. The MODIS image is unified to a spatial resolution of 20m through bilinear resampling and aligned with the Sentinel-2 image.
[0031] Step S13: Perform spectral matching between MODIS image and Sentinel-2 image. Based on the spectral response functions of MODIS and Sentinel-2, select 6 common bands with similar center wavelengths for fusion. The common bands include blue light band, green light band, red light band, near-infrared band, shortwave infrared 1 band and shortwave infrared 2 band.
[0032] Specifically, the wavelengths are blue (MODIS Band 3 corresponds to Sentinel-2 B2), green (MODIS Band 4 corresponds to B3), red (MODIS Band 1 corresponds to B4), near-infrared (MODIS Band 2 corresponds to B8), short-wave infrared 1 (MODIS Band 6 corresponds to B11), and short-wave infrared 2 (MODIS Band 7 corresponds to B12). The wavelength fusion parameters are shown in Table 1 below.
[0033] Table 1 Wavelength fusion parameters
[0034] Step S2: Construct a dual-discriminator collaborative supervised generative adversarial network model, which includes a generator, a spatial discriminator, and a temporal discriminator.
[0035] Specifically, such as Figure 2 As shown, the construction of the dual-discriminator collaborative supervised generative adversarial network model includes the following steps: Step S21: Construct a generator. The generator adopts an encoder-decoder symmetrical structure. The generator includes a feature enhancement module and skip connections. The generator takes high temporal resolution satellite imagery as input and outputs a high spatial resolution fused imagery of the target date.
[0036] Specifically, the generator includes an encoder, a feature enhancement module, skip connections, and a decoder.
[0037] The encoder, used for multi-scale feature extraction, consists of five coding layers. Each layer comprises a 5×5 convolutional kernel, a LeakyReLU non-linear activation function, and a downsampling step of 2. The number of convolutional kernels increases layer by layer to enhance the ability to represent high-level semantic features. The parameters of each layer of the encoder are shown in Table 2 below.
[0038] Table 2. Parameters of each layer of the encoder
[0039] The feature enhancement module, connected between the encoder and decoder, consists of two 5×5 convolutional layers with 512 channels and a LeakyReLU activation function. It adds the input features to the residual mapping element-wise via residual connections. The feature enhancement module alleviates the vanishing gradient problem in deep networks and enhances the ability to capture high-frequency detail information (such as water texture and algal bloom boundaries).
[0040] Skip connections directly transmit low-level features (edges, textures, local structures) from each level of the encoder to the corresponding scale feature reconstruction layer of the decoder, achieving multi-scale information fusion and improving the spatial continuity and structural integrity of the reconstructed image. Skip connection structures can compensate for the loss of small-scale spatial information during downsampling.
[0041] The decoder is symmetrical to the encoder, with a total of 5 decoding layers. Each layer includes upsampling operations, feature fusion, and nonlinear activation. To avoid the checkerboard effect caused by traditional deconvolution, this embodiment uses subpixel convolution for upsampling: first, a high-channel-count feature map is generated in the low-resolution space, and then the channel information is mapped to the spatial dimension through channel rearrangement, thereby efficiently improving the resolution. The settings of each decoding layer are shown in Table 3 below.
[0042] Table 3 Decoding layer settings parameters of the decoder
[0043] Step S22: Construct a spatial discriminator. The spatial discriminator adopts the PatchGAN architecture. The input of the spatial discriminator is a high-resolution image generated from a single frame and the corresponding low-resolution conditional image. The output is a local receptive field realism score matrix.
[0044] Specifically, the spatial discriminator adopts the PatchGAN architecture. The input is a single-frame generated high-resolution image (or a real Sentinel-2 image) and the corresponding low-resolution MODIS conditional image. The output is an N×N discriminator matrix, where each element corresponds to a local receptive field realism score of the input image. The spatial discriminator is used to evaluate whether the local texture and edges of the generated image are natural and whether there is blur or structural distortion; to evaluate whether the spectral distribution of the generated image conforms to the statistical characteristics of the real high-resolution remote sensing image; and to constrain the cross-scale physical consistency between the generated details and the low-resolution input to prevent the generation of unreasonable high-frequency artifacts.
[0045] Step S23: Construct a temporal discriminator. The temporal discriminator takes short temporal segments containing generated images as input, evaluates the temporal coherence of the short temporal segments, and determines whether the temporal evolution trajectory is smooth and reasonable.
[0046] Specifically, the temporal discriminator ensures the dynamic rationality of the generated imagery in the temporal dimension. The design of the temporal discriminator is based on the following: when there is no drastic disturbance to the surface, water bodies and their surface biological features exhibit strong temporal autocorrelation over short timescales, and reflectance changes between adjacent observation dates show a gradual characteristic. The temporal discriminator assesses the temporal coherence of the segment: a true segment should exhibit a smooth transition and strong inter-frame correlation. This constraint forces the generator to learn and generate images that can embed the true temporal evolution trajectory.
[0047] Step S3: Optimize the parameters of the adversarial generative network model using a joint loss function, which includes content reconstruction loss, perceptual loss, and adversarial loss.
[0048] Specifically, the generation results are constrained from three levels: pixel precision, spatial structure, and overall distribution. A joint loss function is constructed, and the formula for calculating the joint loss function is as follows:
[0049] in, For the joint loss function, The weighting coefficients corresponding to the content reconstruction loss function. For content reconstruction loss function, The weight coefficients corresponding to the perceptual loss function. For the perceptual loss function, To counteract the weight coefficients corresponding to the loss function, To counteract the loss function.
[0050] Content reconstruction loss, perceptual loss, and adversarial loss constrain the generation process from different perspectives, complementing each other and jointly guiding the model to generate high-quality, high-resolution Sentinel-2 images. The generator learns the mapping relationship from low-resolution input to high-resolution Sentinel-2 images under multiple constraints by minimizing its total loss function.
[0051] Content reconstruction loss measures the difference between the generated image and the real high-resolution reference image at the pixel level. It is one of the most fundamental and important supervision signals during generator training. The formula for calculating the content reconstruction loss function is as follows:
[0052]
[0053] Where H is the height of the image, W is the width of the image, C is the number of bands in the image, and N is the total number of pixels in the image. For content reconstruction loss function, This is a true high-resolution Sentinel-2 image. The predicted image output by the generator.
[0054] Content reconstruction loss directly constrains the numerical accuracy of the generated image, ensuring it approximates the true Sentinel-2 image at every spatial location and spectral band. In remote sensing image reconstruction and spatiotemporal fusion tasks, image pixel values have clear physical meaning and directly participate in subsequent quantitative analysis processes, such as remote sensing index calculation, surface reflectance inversion, and change detection.
[0055] While relying solely on pixel-level content reconstruction loss can ensure a high degree of numerical consistency between the generated image and the real image, it still easily leads to problems such as blurred spatial structure and insufficient texture details in high-resolution remote sensing image reconstruction tasks. This is because pixel-level loss mainly focuses on the numerical error of individual pixels and is insufficient to fully characterize the structural relationships and semantic information of ground features at a larger spatial scale. Perceptual loss, by measuring the difference between the generated image and the real image in a high-level feature space, guides the model to focus on the overall structure, texture patterns, and morphological features of ground features, thus effectively compensating for the shortcomings of pixel-level loss in structural representation. The formula for calculating the perceptual loss function is:
[0056] in, For the perceptual loss function, Let be the total number of elements in the feature map of the i-th layer. This is a true high-resolution Sentinel-2 image. The predicted image output by the generator. Let be the feature mapping function of the i-th layer of the pre-trained feature extraction network.
[0057] While content reconstruction loss and perceptual loss can constrain the reconstruction accuracy of generated images at the pixel value level and feature structure level, respectively, relying solely on these two types of losses may still lead to differences between generated images and real Sentinel-2 images in terms of overall distribution and spatial statistical characteristics, manifesting as insufficient texture detail or an overall smoothness. To further improve the realism and distribution consistency of generated images, the model uses adversarial loss. Through an adversarial learning process between the generator and discriminator, the generated images gradually approximate real high-resolution Sentinel-2 images in terms of overall spatial structure, texture statistical characteristics, and spectral distribution. The formula for calculating the adversarial loss function is:
[0058] in, To counteract the loss function, To obtain the expectation of the generated image distribution, X represents the high-resolution Sentinel-2 image generated by the generator, and X represents the low-resolution MODIS image as input. This represents the probability that the discriminator will judge the generated image, i.e., the probability that it considers it to be a real image.
[0059] Step S4: Input the high temporal resolution time series images to be fused into the trained generative adversarial network model, and output the high spatial resolution fused image of the target date.
[0060] Specifically, the high temporal resolution MODIS images to be fused are input into a trained generative adversarial network model, which outputs a fused image sequence with high spatiotemporal resolution. For each target date, multiple adjacent MODIS images are taken as input, and a generator outputs a high-resolution Sentinel-2 equivalent image of the target date.
[0061] The spatiotemporal fusion method for lake algal bloom remote sensing images of this invention employs a dual-discriminator collaborative supervision mechanism, simultaneously introducing spatial and temporal discriminators to constrain the generated results from two complementary dimensions: spatial realism and temporal continuity. This effectively overcomes the limitations of existing methods that constrain only from a single dimension, significantly improving the overall quality of the fused images. The generator in the generative adversarial network model utilizes an encoder-decoder structure with residual learning and skip connections, achieving a balance between high-level semantic extraction and low-level detail preservation. Subpixel convolution effectively eliminates checkerboard artifacts during upsampling, resulting in images with superior spatial continuity compared to traditional deconvolution methods. This generates continuous remote sensing image sequences with both high temporal and spatial resolution, providing data support for studying the development process of algal blooms.
[0062] Example 1 To verify the applicability and robustness of the dual-discriminator collaborative supervised generative adversarial network model constructed in this invention under different lake conditions, three typical eutrophic lakes—Chaohu Lake, Taihu Lake, and Dianchi Lake—were selected as research objects, and a unified model training and fusion experiment was conducted.
[0063] For each lake, a representative date was selected, and this date met the following conditions: a cloudless or partially cloudless Sentinel-2 image of the same day was used as a verification ground truth; and multiple adjacent cloudless MODIS images within the preceding and following time windows were used as the input sequence. The specific dates selected for the three lakes were: Chaohu Lake, February 25, 2022; Taihu Lake, December 5, 2021; and Dianchi Lake, April 15, 2023.
[0064] The fused image generated for each lake was compared pixel-by-pixel with the real Sentinel-2 image from the same day across the entire lake. The following metrics were used for quantitative evaluation: Relative Root Mean Square Error (RRMSE) to assess pixel-level accuracy; Correlation Coefficient (CC) to assess spectral fidelity; Structural Similarity Index (SSIM) to assess the degree of spatial structure preservation; Feature Similarity (FR) to assess feature-level consistency; and Spectral Angle Mapping (SAM) to assess the degree of spectral distortion. The evaluation results are shown in Tables 4 and 5 below.
[0065] Table 4. RRMSE and CC evaluation results for reconstructed images in each band.
[0066] Table 5. Evaluation results of SSIM, FR, and SAM for reconstructed images in each band.
[0067] Experimental results show that the method of this invention achieves excellent performance in band fusion of three typical lakes: RRMSE is generally below 10%, correlation coefficient (CC) exceeds 0.84, structural similarity index (SSIM) exceeds 0.84, feature similarity index (FR) exceeds 0.84, and spectral angle mapping (SAM) angle error is less than 4 degrees. These results fully verify the effectiveness and robustness of the method of this invention in lake algal bloom monitoring scenarios.
[0068] like Figure 3 As shown, three typical lakes—Chaohu, Taihu, and Dianchi—were selected as research subjects. A representative MODIS image for one day was chosen from each lake, and a high-resolution image was generated using the proposed spatiotemporal fusion model. The selected dates were: Chaohu, February 25, 2022; Taihu, December 5, 2021; and Dianchi, April 15, 2023. Figure 3The images shown are visible light images of the original MODIS image, model-generated image, and Sentinel real image of Chaohu Lake, Taihu Lake, and Dianchi Lake, respectively.
[0069] Second implementation method: like Figure 4 As shown, the second embodiment of the present invention provides a spatiotemporal fusion system for remote sensing images of algal blooms in lakes. The system includes: a data acquisition module 201, a modeling module 202, a model training and optimization module 203, and a fusion output module 204.
[0070] Specifically, the data acquisition module 201 is used to acquire and preprocess multi-source satellite imagery data, including high temporal resolution satellite imagery and high spatial resolution satellite imagery; the modeling module 202 is used to construct a dual-discriminator collaborative supervised generative adversarial network (GAN) model, which includes a generator, a spatial discriminator, and a temporal discriminator; the model training and optimization module 203 is used to optimize the parameters of the adversarial GAN model using a joint loss function, which includes content reconstruction loss, perceptual loss, and adversarial loss; and the fusion output module 204 is used to input the high temporal resolution time-series imagery to be fused into the trained GAN model and output a high spatial resolution fused imagery for the target date.
[0071] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.
[0072] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.
[0073] The third embodiment of the present invention relates to a network-side server, such as... Figure 4 As shown, it includes at least one processor 302; and a memory 301 communicatively connected to at least one processor 302; wherein the memory 301 stores instructions executable by at least one processor 302, the instructions being executed by at least one processor 302 to enable at least one processor 302 to perform the above-described data processing method.
[0074] The memory 301 and processor 302 are connected via a bus, which may include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 302 and memory 301 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 302 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 302.
[0075] Processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 301 can be used to store data used by processor 302 during operation.
[0076] The fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the spatiotemporal fusion method of remote sensing images of lake algal blooms in the first embodiment.
[0077] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0078] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.
[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for spatiotemporal fusion of remote sensing images of algal blooms in lakes, characterized in that, include: S1, acquire multi-source satellite imagery data and perform preprocessing. The multi-source satellite imagery data includes high temporal resolution satellite imagery and high spatial resolution satellite imagery. S2, Construct a dual-discriminator collaborative supervised generative adversarial network model, wherein the generative adversarial network model includes a generator, a spatial discriminator, and a temporal discriminator; S3, the parameters of the adversarial generative network model are optimized using a joint loss function, which includes content reconstruction loss, perceptual loss and adversarial loss; S4: Input the high temporal resolution time series images to be fused into the trained generative adversarial network model, and output the high spatial resolution fused image of the target date.
2. The method for spatiotemporal fusion of remote sensing images of lake algal blooms according to claim 1, characterized in that, The acquisition and preprocessing of multi-source satellite image data includes: Acquire high temporal resolution MODIS satellite imagery and high spatial resolution Sentinel-2 satellite imagery. MODIS imagery provides high temporal resolution observations with daily revisits, while Sentinel-2 imagery provides high spatial resolution observations with a spatial resolution of 10-30m. Spatial matching was performed on MODIS and Sentinel-2 images. The MODIS images were unified to a spatial resolution of 20m through bilinear resampling and aligned with the Sentinel-2 images. Spectral matching was performed on MODIS and Sentinel-2 images. Based on the spectral response functions of MODIS and Sentinel-2, six common bands with similar center wavelengths were selected for fusion. The common bands include blue light band, green light band, red light band, near-infrared band, shortwave infrared 1 band, and shortwave infrared 2 band.
3. The spatiotemporal fusion method for remote sensing images of lake algal blooms according to claim 1, characterized in that, The construction of the dual-discriminator collaborative supervised generative adversarial network model includes: A generator is constructed, which adopts an encoder-decoder symmetrical structure. The generator includes a feature enhancement module and skip connections. The generator takes high temporal resolution satellite imagery as input and outputs a high spatial resolution fused imagery of the target date. A spatial discriminator is constructed using the PatchGAN architecture. The input to the spatial discriminator is a single-frame generated high-resolution image and the corresponding low-resolution conditional image, and the output is a local receptive field realism score matrix. A temporal discriminator is constructed. The temporal discriminator takes short temporal segments containing generated images as input, evaluates the temporal coherence of the short temporal segments, and judges whether the temporal evolution trajectory is smooth and reasonable.
4. The spatiotemporal fusion method for remote sensing images of lake algal blooms according to claim 3, characterized in that, The generator includes an encoder, a feature enhancement module, a skip connection, and a decoder; The encoder is used for multi-scale feature extraction and has a total of 5 encoding layers. Each layer consists of a convolution operation with a kernel size of 5×5, a LeakyReLU nonlinear activation function, and a downsampling with a stride of 2. The number of convolution kernels increases layer by layer. The feature enhancement module is connected between the encoder and the decoder, and consists of two convolutional layers with a kernel size of 5×5 and a channel count of 512, and a LeakyReLU activation function. It adds the input features to the residual mapping element-wise through residual connections. The skip connection directly transmits the low-level features output from each level of the encoder to the feature reconstruction layer of the corresponding scale of the decoder, thereby achieving the fusion of multi-scale information. The decoder is symmetrical to the encoder and has a total of 5 decoding layers. Each layer includes upsampling operation, feature fusion and nonlinear activation.
5. The spatiotemporal fusion method for remote sensing images of lake algal blooms according to claim 3, characterized in that, The spatial discriminator adopts the PatchGAN architecture. The input is a single-frame generated high-resolution image and the corresponding low-resolution MODIS conditional image. The output is an N×N discriminator matrix. Each element corresponds to a local receptive field realism score of the input image. The spatial discriminator is used to evaluate whether the local texture and edges of the generated image are natural; and to evaluate whether the spectral distribution of the generated image conforms to the statistical characteristics of the real high-resolution remote sensing image. Constrain the generation of details to maintain cross-scale physical consistency with low-resolution input, preventing the generation of unreasonable high-frequency artifacts.
6. The spatiotemporal fusion method for remote sensing images of lake algal blooms according to claim 1, characterized in that, The formula for calculating the joint loss function is as follows: in, For the joint loss function, The weighting coefficients corresponding to the content reconstruction loss function. For content reconstruction loss function, The weight coefficients corresponding to the perceptual loss function. For the perceptual loss function, To counteract the weight coefficients corresponding to the loss function, To counteract the loss function.
7. The spatiotemporal fusion method for remote sensing images of lake algal blooms according to claim 6, characterized in that, The formula for calculating the content reconstruction loss function is as follows: Where H is the height of the image, W is the width of the image, C is the number of bands in the image, and N is the total number of pixels in the image. For content reconstruction loss function, This is a true high-resolution Sentinel-2 image. The predicted image output by the generator; The formula for calculating the perceptual loss function is as follows: in, For the perceptual loss function, Let be the total number of elements in the feature map of the i-th layer. This is a true high-resolution Sentinel-2 image. The predicted image output by the generator. Let be the feature mapping function of the i-th layer of the pre-trained feature extraction network; The formula for calculating the adversarial loss function is as follows: in, To counteract the loss function, To obtain the expectation of the generated image distribution, X represents the high-resolution Sentinel-2 image generated by the generator, and X represents the low-resolution MODIS image as input. This represents the probability that the discriminator will judge the generated image, i.e., the probability that it considers it to be a real image.
8. A spatiotemporal fusion system for remote sensing images of lake algal blooms, characterized in that, The system, applied to the spatiotemporal fusion method of remote sensing images of lake algal blooms according to any one of claims 1-7, comprises: The data acquisition module is used to acquire and preprocess multi-source satellite image data, which includes high temporal resolution satellite imagery and high spatial resolution satellite imagery. The modeling module is used to construct a dual-discriminator collaborative supervised generative adversarial network model, which includes a generator, a spatial discriminator, and a temporal discriminator. The model training optimization module is used to optimize the parameters of the adversarial generative network model using a joint loss function, which includes content reconstruction loss, perceptual loss, and adversarial loss. The fusion output module is used to input high temporal resolution time-series images to be fused into the trained generative adversarial network model and output high spatial resolution fused images of the target date.
9. A network-side server, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the spatiotemporal fusion method of lake algal bloom remote sensing images as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal fusion method of remote sensing images of lake algal blooms as described in any one of claims 1 to 7.