Deep learning based multi-source remote sensing data fusion and wetland drought monitoring method

Abstract

The application provides a kind of multi-source remote sensing data fusion and wetland drought monitoring method based on deep learning, including obtaining multi-source remote sensing data covering target area, and the multi-source remote sensing data is spatially resampled, cloud and shadow mask and missing value interpolation processing is handled, to form time series consistent input data set;The input data set after pre-processing is cut according to the window size and sliding step of pre-set, obtain a plurality of spatial tiles, and each index is stacked as an input tensor according to the band;The input tensor is input into the deep learning model trained in advance, and the predicted drought index tile is output;The drought index tile is spliced to generate a seamless panoramic drought index image, and is exported as a geographic reference raster format.The application realizes high-precision, seamless, robust wetland drought monitoring through multi-source remote sensing data fusion, special deep learning network structure, intelligent tile splicing and optimization strategy.

Description

Technical Field

[0001] This application belongs to the field of remote sensing image processing technology, and in particular relates to a method for multi-source remote sensing data fusion and wetland drought monitoring based on deep learning. Background Technology

[0002] Wetland ecosystems play a crucial role in regional climate regulation and biodiversity conservation. Hydrological connectivity is a key concept describing the flow and retention of water, matter, and energy among water bodies such as rivers, lakes, and wetlands. Hydrological connectivity is significant in hydrology, ecology, and water resource management, influencing water quality, ecological functions, and biodiversity. It is affected by various factors, including topography, soil type, land use patterns, and climate change. Human activities, such as water conservancy projects and urbanization, often significantly alter hydrological connectivity, thereby impacting the stability of the ecological environment and the sustainable use of water resources.

[0003] However, existing methods have many problems in wetland drought monitoring, such as low spatial resolution, inability to comprehensively reflect hydrological and ecological processes, lack of wetland-specific models, and splicing seams and noise.

[0004] Therefore, a new scientific drought monitoring method is urgently needed to address the above shortcomings.

[0005] The foregoing statements are for informational purposes only and are not intended to provide background information in connection with this application. Unless otherwise stated herein, the content described in this section is not prior art to the rest of this application. Summary of the Invention

[0006] This invention proposes a deep learning-based method for multi-source remote sensing data fusion and wetland drought monitoring. Through multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, it achieves high-precision, seamless, and robust wetland drought monitoring, overcoming the shortcomings of existing methods.

[0007] According to a first aspect of the embodiments of this application, a method for multi-source remote sensing data fusion and wetland drought monitoring based on deep learning is provided, comprising the following steps: Acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators, and static topographic factors. Spatial resampling, cloud and shadow masking, and missing value imputation are performed on the multi-source remote sensing data to form a time-series consistent input dataset. The preprocessed input dataset is divided into multiple spatial tiles according to the preset window size and sliding step size. The indicators are stacked into input tensors according to the bands, and the corresponding effective data masks are generated at the same time. The input tensor is fed into a pre-trained deep learning model, which outputs a predicted drought index tile. The drought index tiles were stitched together using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which was then exported as a georeferenced raster format.

[0008] In some embodiments of this application, the deep learning model sequentially includes: The pyramid convolution module is used to extract multi-scale spatial features from the input tensor; The Transformer encoder module is used to fuse global contextual information for multi-scale spatial features; The attention-enhanced decoder module is used to upsample the fused features and apply channel attention and spatial attention to reconstruct a high-resolution feature map.

[0009] In some embodiments of this application, the pyramid convolution module includes four convolutional layers with a kernel size of 3×3, strides of 1, 2, 2, and 2, and output channels of 128, 256, 512, and 1024, respectively.

[0010] In some embodiments of this application, the input of the Transformer encoder module projects the features to 256 dimensions through a 1×1 convolutional layer and adds sine-cosine position encoding; the Transformer encoder module contains three encoder layers, each with 8 attention heads, and the feedforward network has a dimension of 1024.

[0011] In some embodiments of this application, the attention enhancement decoder module performs upsampling through three levels of transposed convolution, and after each level of upsampling, the channel attention module and the spatial attention module are sequentially connected in series; the channel attention module uses a multilayer perceptron structure with a reduction ratio of 16, and the spatial attention module uses a 7×7 convolution kernel to generate a spatial weight map.

[0012] In some embodiments of this application, the missing value imputation process uses the LightGBM model to perform pixel-by-pixel regression prediction and imputation, and the imputed time series data is smoothed by Savitzky-Golay filtering.

[0013] In some embodiments of this application, when training a deep learning model, the loss function used is a masked mean squared error loss function. The calculation formula is: ; in For predicted values, For the target value, For mask weights, 1.0 × 10⁻8 ; The Bayesian optimization algorithm is used to automatically optimize the hyperparameters of the deep learning model. The hyperparameters include at least the learning rate, hidden layer dimension, and weight decay coefficient. The optimization objective is to maximize the coefficient of determination R² on the validation set.

[0014] In some embodiments of this application, a feathering weighted algorithm based on the Hanning window is used for stitching to generate seamless panoramic drought index images, including: Based on the tile side length φ and the sliding step size s Calculate the overlap width The calculation formula is: ; Construction length is The one-dimensional Hanning window weight vector, with a weight of 1 in the middle, gradually decreasing to 0 towards both ends; The one-dimensional Hanning window weight vector is extended into a two-dimensional feathered weight matrix with the same size as the tile. During the stitching process, the prediction result of each tile is multiplied by its corresponding two-dimensional feathering weight matrix, calculated and accumulated on a pixel-by-pixel basis, and finally the overlapping area is normalized to eliminate seams.

[0015] According to a second aspect of the embodiments of this application, a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system is provided, comprising: The data acquisition and preprocessing module is used to acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators, and static topographic factors. The module also performs spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset. The tile segmentation and tensor construction module is used to segment the preprocessed input dataset according to the preset window size and sliding step size to obtain multiple spatial tiles, and stack the indicators according to the band into input tensors, while generating the corresponding effective data mask. The deep fusion processing module, deployed on a GPU server, is used to load a pre-trained deep learning model, input the input tensor into the pre-trained deep learning model, and output the predicted drought index tiles. The feathering and output module is used to stitch drought index tiles using a feathering weighting algorithm based on the Hanning window, generating seamless panoramic drought index images and exporting them as georeferenced raster format.

[0016] According to a third aspect of the embodiments of this application, a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device is provided, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the storage unit to execute the executable instructions to complete the multi-source remote sensing data fusion and wetland drought monitoring method.

[0017] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon; the computer program is executed by a processor to implement a method for multi-source remote sensing data fusion and wetland drought monitoring.

[0018] The method and system for wetland drought monitoring based on deep learning proposed in this application include: acquiring multi-source remote sensing data covering the target area; performing spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset; dividing the preprocessed input dataset according to a preset window size and sliding step to obtain multiple spatial tiles; stacking each index by band into an input tensor and generating corresponding effective data masks; inputting the input tensor into a pre-trained deep learning model to output predicted drought index tiles; and stitching the drought index tiles using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which is then exported as a georeferenced raster format.

[0019] This application achieves high-precision, seamless, and robust wetland drought monitoring by using multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 The diagram illustrates the steps of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring method according to an embodiment of this application. Figure 2 The diagram shows a step diagram of tile splicing according to an embodiment of this application; Figure 3 The diagram shows a flowchart of a multi-source remote sensing data fusion and wetland drought monitoring method according to an embodiment of this application; Figure 4 The diagram shows a tile segmentation schematic according to an embodiment of this application; Figure 5 The diagram shows a schematic of a deep learning-based network structure according to an embodiment of this application; Figure 6 The diagram shows a schematic representation of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system according to an embodiment of this application. Figure 7 The diagram shows a system deployment architecture according to an embodiment of this application; Figure 8 The diagram shows a schematic representation of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device according to an embodiment of this application. Detailed Implementation

[0021] Regarding this application, wetland ecosystems play a crucial role in regional climate regulation and biodiversity conservation, and hydrological connectivity is influenced by a variety of factors, including topography, soil type, land use patterns, and climate change.

[0022] However, existing methods have the following limitations in wetland drought monitoring: Insufficient monitoring accuracy: Traditional ground monitoring stations are sparse, making it difficult to capture the spatiotemporal heterogeneity of water within wetlands.

[0023] Index singularity: Existing remote sensing drought indices (such as VCI and TVDI) cannot comprehensively reflect the hydrological-ecological coupling process of wetlands.

[0024] Weak generalization ability of models: Existing deep learning models lack specialized structural designs for wetland high moisture backgrounds and dynamic water bodies.

[0025] Low spatial resolution: Most fusion methods are still based on linear weighting or low-resolution (0.5–1 km) data, which cannot meet the needs of refined wetland management.

[0026] Data processing defects: There are seam problems in the tile splicing, and the noise of missing pixels has not been effectively processed.

[0027] To address the aforementioned technical problems, this invention aims to provide a multi-source remote sensing data fusion method for wetlands. By employing an end-to-end network with a pyramid convolution-Transformer-spatiotemporal attention mechanism and a Hann window feathering strategy, a comprehensive drought index is generated with high precision, enabling monthly-scale wetland drought monitoring.

[0028] This application provides a method and system for multi-source remote sensing data fusion and wetland drought monitoring, including acquiring multi-source remote sensing data covering a target area, and performing spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset; dividing the preprocessed input dataset according to a preset window size and sliding step size to obtain multiple spatial tiles, and stacking each index by band into an input tensor, while generating corresponding effective data masks; inputting the input tensor into a pre-trained deep learning model to output predicted drought index tiles; stitching the drought index tiles using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, and exporting it as a georeferenced raster format.

[0029] This application achieves high-precision, seamless, and robust wetland drought monitoring by using multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods.

[0030] It has the following technical effects: 1. High-precision nonlinear mapping: High-dimensional nonlinear transformation from multi-source indicators to the comprehensive drought index is achieved through deep networks, thereby improving monitoring accuracy.

[0031] 2. Seamless stitching and texture preservation: The Hann window feathering algorithm effectively eliminates tile stitching seams and maintains image edge consistency.

[0032] 3. Strong robustness: Even with a large amount of missing data, the combination of mask loss and interpolation methods can still maintain high R² performance.

[0033] 4. Automation and scalability: Bayesian optimization avoids human bias, and the model structure supports rapid migration to other regions without the need to reconstruct the network.

[0034] 5. Engineering-friendly: The system supports GPU acceleration (such as RTX 4090), has a fast inference speed (0.8 seconds / tile), and has efficient deployment and expansion capabilities.

[0035] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0036] Example 1 Figure 1 The diagram illustrates the steps of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring method according to an embodiment of this application.

[0037] like Figure 1 As shown, a method for multi-source remote sensing data fusion and wetland drought monitoring based on deep learning is provided, including the following steps: S1: Acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators, and static topographic factors. Spatial resampling, cloud and shadow masking, and missing value imputation are performed on the multi-source remote sensing data to form a time-series consistent input dataset. S2: The preprocessed input dataset is divided into multiple spatial tiles according to the preset window size and sliding step size, and the indicators are stacked into input tensors according to the bands, while generating the corresponding effective data mask. S3: Input the input tensor into a pre-trained deep learning model and output the predicted drought index tiles; S4: The drought index tiles are stitched together using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which is then exported as a georeferenced raster format.

[0038] This application achieves high-precision, seamless, and robust wetland drought monitoring through multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies.

[0039] In S1, the missing value imputation process uses the LightGBM model to perform pixel-by-pixel regression prediction and imputation, and the imputed time series data is smoothed by Savitzky-Golay filtering.

[0040] Preferably, the adaptive Savitzky-Golay filter formula is used for smoothing time series data and processing time series noise. The window is an odd number and adaptively selected according to the proportion of missing data (e.g., {5,7,9,11}), and the polynomial order is determined according to the principle of minimizing verification error.

[0041] For the temporal data Yt of each pixel, the filtered value The result is given by the local polynomial least squares fit within the window: ; Where m is the width of half the window (if window=7, then m=3); C i The convolution coefficients are dependent on the polynomial order; N is the normalization factor.

[0042] The window size in S2 is 384×384 pixels, and the sliding step size is 192 pixels to achieve a 50% overlap. The effective data mask is a binary mask, with effective pixels set to 1 and invalid pixels set to 0.

[0043] In the preferred implementation, dynamic mask synthesis is used when generating the corresponding effective data mask. Specifically, the intersection of the effective regions of each band is used as the final mask, and only the position where all input bands are effective is recorded as effective, so as to ensure the uniformity of the input tensor data quality.

[0044] In a preferred implementation, for a pixel location (x, y) with K input bands, the final effective mask M total The formula for (x, y) is: ; Where, M(x,y) {0,1} is the validity mask for the k-th band at this location (1 is valid, 0 is invalid).

[0045] In addition, step S2 performs Z-score normalization on the input data for each band before constructing the input tensor.

[0046] The deep learning model includes the following structure in sequence: The pyramid convolution module is used to extract multi-scale spatial features from the input tensor; The Transformer encoder module is used to fuse global contextual information for multi-scale spatial features; The attention-enhanced decoder module is used to upsample the fused features and apply channel attention and spatial attention to reconstruct a high-resolution feature map.

[0047] In a preferred implementation, the pyramid convolution module contains four convolutional layers with a kernel size of 3×3, strides of 1, 2, 2, and 2, and output channels of 128, 256, 512, and 1024, respectively.

[0048] In a preferred implementation, the input of the Transformer encoder module projects the features to 256 dimensions through a 1×1 convolutional layer and adds sine-cosine position encoding; the Transformer encoder module contains three encoder layers, each with 8 attention heads, and the feedforward network has a dimension of 1024.

[0049] In other preferred implementations, the multi-scale features {F1, F2, F3, F4} extracted by the pyramid convolution module are fused using an adaptive weighted fusion formula.

[0050] During the decoder upsampling process, the number of channels for each scale feature is first aligned to C using a 1×1 convolution, and then bilinear upsampling is performed to achieve the same spatial size. , recorded as Features from different scales (After upsampling to the same size) a weighted sum is performed to obtain the fused features. : ; Among them, weight The weights can be fixed (set based on experience) or adaptive ( , After broadcasting according to channel and spatial dimensions, and Multiply, and then sum to obtain ;when When using adaptive weights, they are generated through a small attention network: ; In this process, the MLP maps the concatenated vectors to 4 dimensions (consistent with the scale number), and then obtains the result via Softmax. , The scalar weights are shared across the entire image (or window), varying with the sample size but not spatially. The change is that GAP is a global average pooling. .

[0051] In a preferred embodiment, the attention enhancement decoder module performs upsampling through three levels of transposed convolution, and after each level of upsampling, the channel attention module and the spatial attention module are sequentially connected in series; the channel attention module uses a multilayer perceptron structure with a reduction ratio of 16, and the spatial attention module uses 7×7 convolution kernels to generate a spatial weight map.

[0052] When training deep learning models, the loss function used is the masked mean squared error loss function. The calculation formula is: ; in For predicted values, For the target value, For mask weights, 1.0 × 10⁻ 8 ; The Bayesian optimization algorithm is used to automatically optimize the hyperparameters of the deep learning model. The hyperparameters include at least the learning rate, hidden layer dimension, and weight decay coefficient. The optimization objective is to maximize the coefficient of determination R² on the validation set.

[0053] Figure 2 The diagram shows a step-by-step illustration of tile splicing according to an embodiment of this application.

[0054] like Figure 2 As shown, a feathering weighted algorithm based on the Hanning window is used in S4 to stitch together seamless panoramic drought index images, including: S41: Based on the tile side length φ and sliding step sizes Calculate the overlap width The calculation formula is: ; In the preferred implementation, the calculated overlap width The new overlap width is adaptively modified based on the terrain complexity.

[0055] The overlap width is adaptively determined based on the standard deviation of the terrain. If adaptive overlap is used for sliding windows, then the update step size is... If used only for feathering weights, the step size remains the base value, and the feathering window length is taken as... , The calculation formula is: ; in, This is the base overlap width (e.g., 192 pixels); The adjustment coefficient ( ); This is the standard deviation of the current elevation within the tile, where the elevation is consistent with the image resolution and is calculated within the tile window. Limited to , , and take the nearest even number.

[0056] S42: Construction length is the overlap width The one-dimensional Hanning window weight vector, with a weight of 1 in the middle, gradually decreasing to 0 towards both ends; S43: Extend the one-dimensional Hanning window weight vector into a two-dimensional feathered weight matrix with the same size as the tile; S44: During splicing, the prediction result of each tile is multiplied by its corresponding two-dimensional feathering weight matrix, calculated and accumulated on a pixel-by-pixel basis, and finally the overlapping area is normalized to eliminate seams.

[0057] The embodiments of this application achieve high-precision, seamless, and robust wetland drought monitoring through multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods in terms of resolution, fusion capability, and noise processing.

[0058] To illustrate in detail the multi-source remote sensing data fusion and wetland drought monitoring method of this application, the following is a description of the specific implementation process.

[0059] Figure 3 The diagram shows a flowchart of a multi-source remote sensing data fusion and wetland drought monitoring method according to an embodiment of this application.

[0060] like Figure 3As shown in the figure, the process of the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring method in the implementation embodiment is as follows: Step 1: Acquire multi-source remote sensing images covering the target wetland area, including at least optical vegetation index, thermal infrared temperature, hydro-meteorological indicators, and static topographic factors; Step 2: Use bilinear interpolation to resample the image obtained in Step 2 to a resolution of 30m and reproject it to a uniform geographic coordinate system EPSG: 4326 to ensure that all input tensors have consistent dimensions. Step 3: Use Fmask to detect cloud shadows and snowfall pixels, generate a quality control layer and set it to Nodata; Step 4: Use the LightGBM model to reconstruct the missing values ​​pixel by pixel to ensure that each indicator forms a continuous monthly scale sequence; Step 5: Slice the tile by a size of 384×384 pixels and a step size of 192 pixels, and generate a 0 / 1 mask for the remaining tiles; Step 6: Stack all indicators except the target feature into a tensor in band order. Z-score normalization is performed on each channel; Step 7: Input X into a four-layer pyramid convolutional network to obtain multi-scale features {F1, F2, F3, F4}, where the scale of the deepest layer F4 is [not specified]. ; Step 8: Project F4 to 256 dimensions using 1×1 convolution and flatten it into a sequence, add sine-cosine position encoding and input it into a three-layer Transformer encoder with 8 heads, and output the fused feature T; Step 9: Upsample T to its original resolution using a three-stage transpose convolution, then enhance it sequentially with channel attention (reduction ratio 16) and spatial attention (7×7 convolution), and finally output the predicted image through a 1×1 convolution. ; Step 10: Define the mean squared error with mask: ),in For predicted values, For the target value, For mask weights, 1.0 × 10⁻ 8 ; Step 11: The AdamW optimizer is used to update parameters, and the learning rate is dynamically adjusted in conjunction with the learning rate scheduler (factor=0.5, patience=4). An early stopping mechanism is introduced to improve training stability. Automatic Mixed Precision (AMP) is enabled during model training to improve computational efficiency, and gradient clipping is used to prevent gradient explosion. The clipping norm threshold is set to 1.0. Step 12: Use the Bayesian optimization algorithm within the preset search space of hyperparameters such as learning rate, hidden dimension, number of multiple heads, and weight decay to automatically find the optimal value with the maximum value of R² on the validation set as the objective function; Step 13: Output the tiles using a Hann window weight matrix of the same length as the overlap width. Feathering and weighting are performed and the images are stitched together to form a panoramic image. ; Step 14: When exporting the predicted image, save it in a georeferenced raster format (such as GeoTIFF), use LZW lossless compression, set the Nodata value to NaN, and write the projection, resolution, and generation time metadata.

[0061] To further explain, steps 1-4 pertain to the data source and preprocessing process.

[0062] (1) Data Collection and Sources: ① Vegetation and Hydrological Remote Sensing Indices: Including indicators such as NDVI, NDII, MSI, and TCW, the data source is Landsat series satellite imagery, with a spatial resolution of 30m and a time scale of monthly. ② Meteorological and Drought Indices: These include indicators such as SPEI, SPI, TCI, ESI, SMAI, and RAI. Data sources are publicly available meteorological remote sensing and reanalysis data products such as MODIS / NASA GLDAS / ERA5, with spatial resolutions ranging from 500 m to 1 km and a monthly temporal scale. To maintain complete spatial scale consistency with vegetation remote sensing imagery, bilinear interpolation to a resolution of 30 m is uniformly used to ensure consistency in the dimensions of all input tensors, meeting the input requirements of subsequent models. ③ Static Topographic Indicator (TNI): Based on global high-precision DEM data, the original spatial resolution is 30 m, and it is processed and unified to a 30 m grid of remote sensing imagery as a static feature.

[0063] (2) Cloud mask: CFMask; water mask uses NDWI>0.4 as an auxiliary.

[0064] (3) Missing value imputation: LightGBM was used to imput missing values ​​globally; the entire remote sensing image sequence was processed by SG filtering (window=7, poly=2).

[0065] Steps 5-6 pertain to the tile segmentation and input tensor construction process.

[0066] Figure 4 The diagram shows a tile segmentation schematic according to an embodiment of this application.

[0067] like Figure 4 As shown, the unshaded area represents a single tile region with a size of 384 × 384 pixels; the shaded area represents the overlapping area between adjacent tiles with an overlap width of 192 pixels (i.e., 50% overlap). (1) Tile segmentation: The window size is 384×384 pixels, and the step size is 192 pixels. The sliding segmentation generates spatial tile data; the effective data mask is a binary mask, with effective pixels set to 1 and invalid pixels set to 0.

[0068] (2) Dynamic window generation: The starting position of the window is automatically adjusted according to the actual shape of the study area to ensure that all boundary effective pixels are completely covered.

[0069] (3) Dynamic mask synthesis: The intersection of the effective regions of each band is used as the final mask. Only the position where all input bands are effective is recorded as effective, so as to ensure the uniformity of the input tensor data quality.

[0070] (4) Input tensor construction: To ensure the stability of model training and improve prediction accuracy, all input metrics are standardized using Z-score to eliminate the differences in dimensionality between different data dimensions and improve the consistency of feature representation.

[0071] Steps 7-13 pertain to the network architecture and implementation process.

[0072] Figure 5 The diagram shows a schematic of a deep learning-based network structure according to an embodiment of this application.

[0073] like Figure 5 As shown, ① represents multi-source features; ② represents tensor stacking; ③ represents the PFEM module; ④ represents the 1×1 Convolution; ⑤ represents the Transformer module; ⑥ represents the decoding module; ⑦ represents the CBAM module; and ⑧ represents the 1×1 convolutional output layer.

[0074] (1) Pyramid Feature Extraction Module (PFEM): 4 layers of Conv-BN-ReLU, 3×3 convolution kernel, stride=1→2→2→2, output channels are 128→256→512→1024 in sequence, all using SAME padding.

[0075] (2) Transformer module: Conv1×1 projected to 256 dimensions → three-layer TransformerEncoder (heads=8, ff_dim=1024, dropout=0.1); add PosEnc (Max_len=5000); add Dropout layer with a probability of 0.1.

[0076] (3) Spatiotemporal attention mechanism decoder CBAM module (Convolutional Block AttentionModule): ConvTranspose2d×3, upsampling factor 2-2-2; each layer is followed by ReLU, and ChannelAttention (reduction ratio r=16) and Spatial Attention (convolutional kernel k=7) are sequentially connected, and finally a 1×1 Conv is generated to generate the predicted image Ŷ.

[0077] (4) Loss function: Optimizer: AdamW (lr=1e-4, wd=1e-4); Learning rate scheduler: ReduceLROnPlateau (factor=0.5, patience=4).

[0078] (5) Model performance evaluation: A strict 5-fold cross-validation strategy is adopted to evaluate the model performance. The data is randomly divided into 5 subsets, and 4 subsets are used alternately to train the model and 1 subset is used for model validation to ensure the generalization performance and reliability of the model.

[0079] (6) Training strategy: AMP + cumulative gradient = 4 + gradient clipping = 1.0; K-Fold = 5; early stopping patience = 10, factor = 0.3.

[0080] (7) Hyperparameter optimization: The search space is as described in claim 17, with ≥50 iterations and an objective R².

[0081] Steps 13-14: Feathering and Exponential Output: (1) Overlap width calculation: Let the tile side length be L, the sliding step size be s, the overlap width be o = L – s, and the half-width of the single-sided overlap be... .

[0082] (2) Construction of a one-dimensional Hann window: Generate a one-dimensional Hann vector of length o with weights of 0 at both ends and a heart neighborhood close to 1.

[0083] (3) One-dimensional weight expansion: Establish a weight vector of length L. Initialize it to 1; assign the first half of hann to the left interval [0, h), and the second half to the right interval [...]. , The middle interval remains 1, so that the center weight of the tile is always 1, and the edge gradually transitions along the overlapping area.

[0084] (4) Generation of two-dimensional feathered weights: The two-dimensional feathered weight matrix is ​​obtained through outer product: ; The size is L×L; when the tile is located at the image boundary, W is cropped according to the actual coverage area.

[0085] (5) Weighted accumulation and normalization: For the tile output image Y and its effective pixel mask W, calculate and accumulate per pixel; after traversing all tiles, at... Position normalization is performed. =ΣY / ΣW.

[0086] At The position is assigned an invalid value (nodata). Feathering only takes effect in overlapping areas; the weight in non-overlapping areas remains 1 to maintain texture and edge consistency. (6) Image Export: Export Y_fused as a georeferenced raster format (such as GeoTIFF) with invalid value markers, lossless image compression and spatial reference.

[0087] System specifications: GPU: RTX4090×1; CPU: 96 cores; Memory: 512GB; Inference rate: 0.8s / tile.

[0088] Other improvements in this application specifically addressing the moisture or hydrological data of wetlands, and the corresponding steps are as follows: 1. Data layer (input and quality control).

[0089] In data acquisition and preprocessing (steps 1–4), wetland sensitive channels (such as NDII / MSI / TCW and SSMI / RAI / TCI, etc.) are incorporated and unified to the pixel-level spatial scale; Water / cloud masks can be constructed for quality control, and dynamic water masks can be generated using adaptive thresholds to reduce false signals caused by water fluctuations. Interpolation and temporal smoothing are performed on missing data to suppress breaks and noise propagation caused by dynamic water bodies.

[0090] 2. Model layer (network structure oriented towards wetland processes).

[0091] In feature extraction and encoding (steps 7–9), a pyramid convolutional feature extraction module (PFEM) + Transformer encoding is used to fuse multi-scale texture and spatiotemporal / hysteresis correlation to adapt to the nonlinear coupling of hydrology-vegetation-heat. In the decoding and reconstruction (steps 9–13), CBAM (channel-spatial attention) is introduced to enhance the details of the water-land interface and patches, and suppress false contrast in high water content backgrounds; Both training and inference employ a masked loss / inference strategy to ensure that deep or invalid pixels do not affect parameter updates and output.

[0092] 3. Engineering layer (seamless, stable output over large areas).

[0093] In the inference and stitching (steps 13–14), an overlapping sliding window + Hann window feathering is used, with only the overlapping area being weighted and the center weight kept at 1, to achieve a seamless stitching effect while preserving the texture. The output stage performs ΣY / ΣW normalization on the accumulated results and writes out the georeferenced raster (such as GeoTIFF), carrying invalid value markers and spatial references to ensure the stability and usability of the results at a resolution of 10–30 m (preferably 30 m).

[0094] This application proposes improvements to the data layer, model layer, and engineering layer, taking into account the characteristics of wetlands with "high moisture background and dynamic water bodies," and these improvements are implemented in the corresponding steps.

[0095] In other descriptions, the present invention has made improvements and implemented measures to address the deficiencies of the prior art pointed out above.

[0096] 1. Regarding the problem: lack of specific designs for wetland areas.

[0097] Key improvements of this invention: Wetland sensitive channel and dynamic water / cloud mask are introduced on the data side; Multi-scale Convolutional Pyramid (PFEM) + Transformer encoding + CBAM attention is used on the model side; and masked loss and inference mask are used in training / inference to suppress deep water / invalid cell interference.

[0098] Corresponding steps: Steps 1–4 (Data acquisition and preprocessing: channel organization, masking, missing test reconstruction), Steps 7–13 (Network structure and attention decoding), Steps 8–10 (Mask loss / inference).

[0099] 2. Regarding the problem: Most integrated indices remain at the level of linear weighting, which makes it difficult to characterize the nonlinear coupling of wetlands.

[0100] Key improvements of this invention: It employs a deep fusion network to achieve pixel-level nonlinear feature coupling (PFEM+Transformer) and uses attention decoding to enhance the water-land boundary and patch texture.

[0101] Corresponding steps: Steps 7–13 (deep nonlinear fusion of feature extraction / encoding / decoding).

[0102] 3. Regarding the problem: the spatial resolution is mostly at the kilometer level (about 0.5–1 km), which is difficult to meet the requirements of refined management.

[0103] Key improvements of this invention: Unifying the input scale to the pixel level, combining overlapping sliding window inference with Hann window feathering to form a seamless output process at the 30 m level.

[0104] Corresponding steps: Steps 1–4 (resampling and alignment), Steps 13–14 (sliding window inference and feathering), Step 14 (exponential derivation).

[0105] 4. Regarding the problem: noise propagation at tile splicing seams and incomplete measurement pixels.

[0106] Key improvements of this invention: Hanning window gradient weighting is applied only in the overlapping area, with the center weight always set to 1, to achieve no visible seams; the entire process combines missing measurement interpolation and masked loss, and uses ΣY / ΣW normalization and nodata marking to control noise propagation.

[0107] Corresponding steps: Steps 13–14 (feathering stitching), Steps 1–4 (reconstruction of missing measurements), Steps 8–10 (with mask loss), Step 14 (ΣY / ΣW normalization and derivation).

[0108] The deep learning-based multi-source remote sensing data fusion and wetland drought monitoring method in this application has the following beneficial effects: (1) Improved accuracy of nonlinear mapping: High-dimensional nonlinear inverse transformation from multi-source index to comprehensive index is learned through deep network learning.

[0109] (2) Seam elimination and texture preservation: The Hanning window feathering algorithm is introduced to perform gradient overlay only on the overlapping area, completely eliminating the tile seams; (3) Enhanced robustness: The mask mean square error and LightGBM interpolation work together to maintain a high R² even when there are a lot of missing values ​​in the original data; (4) Automatic hyperparameter optimization: Bayesian optimization is adopted, and the combination is automatically converged to the optimal combination within the iteration, avoiding human experience bias; (5) Strong adaptability: The architecture of multi-source features ensures that it can be migrated to other regions simply by replacing the input data path, without the need to redesign the network structure, and has good engineering deployment and scalability.

[0110] In summary, the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring method of this application includes acquiring multi-source remote sensing data covering the target area, and performing spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset; dividing the preprocessed input dataset according to a preset window size and sliding step size to obtain multiple spatial tiles, and stacking each index by band into an input tensor, while generating corresponding effective data masks; inputting the input tensor into a pre-trained deep learning model to output predicted drought index tiles; and stitching the drought index tiles using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which is then exported as a georeferenced raster format.

[0111] This application achieves high-precision, seamless, and robust wetland drought monitoring by using multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods.

[0112] Example 2 This embodiment provides a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system. For details not disclosed in this embodiment, please refer to the specific implementation details of the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring schemes in other embodiments.

[0113] Figure 6 The diagram shows a schematic representation of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system according to an embodiment of this application.

[0114] like Figure 6 As shown, the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system includes: The data acquisition and preprocessing module 10 is used to acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators and static topographic factors. The module also performs spatial resampling, cloud and shadow masking and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset. The tile segmentation and tensor construction module 20 is used to segment the preprocessed input dataset according to the preset window size and sliding step size to obtain multiple spatial tiles, and stack each index according to the band into an input tensor, while generating the corresponding effective data mask. The deep fusion processing module 30, deployed on a GPU server, is used to load a pre-trained deep learning model, input the input tensor into the pre-trained deep learning model, and output the predicted drought index tiles. The feathering and stitching and output module 40 is used to stitch drought index tiles using a feathering weighting algorithm based on the Hanning window, generate seamless panoramic drought index images, and export them as georeferenced raster format.

[0115] Figure 7 The diagram shows a system deployment architecture according to an embodiment of this application.

[0116] In a preferred implementation, the system deployment architecture of this application embodiment is as follows: Figure 7 As shown, ① is the data acquisition module; ② is the preprocessing module; ③ is the deep fusion module (GPU server); ④ is the result processing module; ⑤ is the data storage module; and ⑥ is the visualization and publishing module. The arrows indicate the data flow direction.

[0117] The deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system according to the embodiments of this application includes a data acquisition and preprocessing module 10 for acquiring multi-source remote sensing data covering the target area, and performing spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset; a tile segmentation and tensor construction module 20 for segmenting the preprocessed input dataset according to a preset window size and sliding step size to obtain multiple spatial tiles, and stacking each index by band into an input tensor, while generating corresponding effective data masks; a deep fusion processing module 30 deployed on a GPU server for loading a pre-trained deep learning model, inputting the input tensor into the pre-trained deep learning model, and outputting predicted drought index tiles; and a feathering stitching and result output module 40 for stitching the drought index tiles using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, and exporting it as a georeferenced raster format.

[0118] This application achieves high-precision, seamless, and robust wetland drought monitoring by using multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods.

[0119] Example 3 This embodiment provides a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device. For details not disclosed in this embodiment, please refer to the specific implementation of the multi-source remote sensing data fusion and wetland drought monitoring methods or systems in other embodiments.

[0120] Figure 8 The diagram shows a structural schematic of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400 according to an embodiment of this application.

[0121] like Figure 8 As shown, the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400 includes: a storage unit 402 for storing executable instructions; and a processing unit 401 for connecting to the storage unit 402 to execute the executable instructions to complete the multi-source remote sensing data fusion and wetland drought monitoring method.

[0122] Those skilled in the art will understand that the illustration Figure 8 This is merely an example of a deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400, and does not constitute a limitation on the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400 may also include input / output devices, network access devices, buses, etc.

[0123] The processing unit 401 (Central Processing Unit, CPU) can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processing unit 401 can be any conventional processor. The processing unit 401 is the control center of the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400, connecting all parts of the device through various interfaces and lines.

[0124] Storage unit 402 can be used to store computer-readable instructions. Processing unit 401 implements various functions of the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400 by running or executing the computer-readable instructions or modules stored in storage unit 402 and calling the data stored in storage unit 402. Storage unit 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400, etc. In addition, storage unit 402 may include hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0125] If the module integrating deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device 400 is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, they can implement the steps of the various method embodiments described above.

[0126] Example 5 This embodiment provides a computer-readable storage medium on which a computer program is stored; the computer program is executed by a processor to implement the multi-source remote sensing data fusion and wetland drought monitoring method in other embodiments.

[0127] The deep learning-based multi-source remote sensing data fusion and wetland drought monitoring device and storage medium of this application include: acquiring multi-source remote sensing data covering the target area; performing spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset; dividing the preprocessed input dataset according to a preset window size and sliding step size to obtain multiple spatial tiles; stacking each index by band into an input tensor, and generating corresponding effective data masks; inputting the input tensor into a pre-trained deep learning model to output predicted drought index tiles; and stitching the drought index tiles using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which is then exported as a georeferenced raster format.

[0128] This application achieves high-precision, seamless, and robust wetland drought monitoring by using multi-source remote sensing data fusion, a dedicated deep learning network structure, and intelligent tile splicing and optimization strategies, thus overcoming the shortcomings of existing methods.

[0129] Those skilled in the art will understand that the terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” as used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0130] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0131] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0132] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for multi-source remote sensing data fusion and wetland drought monitoring based on deep learning, characterized in that, Includes the following steps: Acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators, and static topographic factors. Then, perform spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset. The preprocessed input dataset is divided into multiple spatial tiles according to a preset window size and sliding step size. The indicators are stacked into input tensors according to the bands, and corresponding effective data masks are generated. The input tensor is fed into a pre-trained deep learning model, which outputs the predicted drought index tiles. The drought index tiles are stitched together using a feathering weighting algorithm based on the Hanning window to generate a seamless panoramic drought index image, which is then exported as a georeferenced raster format.

2. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 1, characterized in that, The deep learning model includes, in sequence: The pyramid convolution module is used to extract multi-scale spatial features from the input tensor; The Transformer encoder module is used to perform global context information fusion on the multi-scale spatial features; The attention-enhanced decoder module is used to upsample the fused features and apply channel attention and spatial attention to reconstruct a high-resolution feature map.

3. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 2, characterized in that, The pyramid convolution module contains four convolutional layers with a kernel size of 3×3, strides of 1, 2, 2, and 2, and output channels of 128, 256, 512, and 1024, respectively.

4. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 2, characterized in that, The input of the Transformer encoder module projects the features to 256 dimensions through a 1×1 convolutional layer and adds sine-cosine position encoding; the Transformer encoder module contains three encoder layers, each with 8 attention heads, and the feedforward network has a dimension of 1024.

5. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 2, characterized in that, The attention enhancement decoder module performs upsampling through three levels of transposed convolution, and after each level of upsampling, the channel attention module and the spatial attention module are sequentially connected in series. The channel attention module uses a multilayer perceptron structure with a reduction ratio of 16, and the spatial attention module uses 7×7 convolution kernels to generate a spatial weight map.

6. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 1, characterized in that, The missing value imputation process uses the LightGBM model for pixel-by-pixel regression prediction and imputation, and the imputed time series data is smoothed by Savitzky-Golay filtering.

7. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 1, characterized in that, When training the deep learning model, the loss function used is the masked mean squared error loss function. The calculation formula is: ； in For predicted values, For the target value, For mask weights, 1.0 × 10⁻ 8 ; The hyperparameters of the deep learning model are automatically optimized using a Bayesian optimization algorithm. The hyperparameters include at least the learning rate, hidden layer dimension, and weight decay coefficient. The optimization objective is to maximize the coefficient of determination R² on the validation set.

8. The method for multi-source remote sensing data fusion and wetland drought monitoring according to claim 1, characterized in that, The process employs a feathering weighted algorithm based on the Hanning window to stitch together seamless panoramic drought index images, including: Based on the tile side length φ and the sliding step size s Calculate the overlap width The calculation formula is: ; Construction length is The one-dimensional Hanning window weight vector, with a weight of 1 in the middle, gradually decreasing to 0 towards both ends; The one-dimensional Hanning window weight vector is extended into a two-dimensional feathered weight matrix with the same size as the tile. During the splicing process, the prediction result of each tile is multiplied by its corresponding two-dimensional feathering weight matrix, calculated and accumulated on a pixel-by-pixel basis, and finally the overlapping area is normalized to eliminate seams.

9. A deep learning-based multi-source remote sensing data fusion and wetland drought monitoring system, characterized in that, include: The data acquisition and preprocessing module is used to acquire multi-source remote sensing data covering the target area. The multi-source remote sensing data includes at least optical vegetation index, thermal infrared surface temperature, hydro-meteorological indicators, and static topographic factors. The module performs spatial resampling, cloud and shadow masking, and missing value imputation on the multi-source remote sensing data to form a time-series consistent input dataset. The tile segmentation and tensor construction module is used to segment the preprocessed input dataset according to a preset window size and sliding step size to obtain multiple spatial tiles, and stack each index into an input tensor according to the band, while generating the corresponding effective data mask. The deep fusion processing module, deployed on a GPU server, is used to load a pre-trained deep learning model, input the input tensor into the deep learning model, and output the predicted drought index tiles. The feathering and stitching and result output module is used to stitch the drought index tiles using a feathering weighting algorithm based on the Hanning window, generate a seamless panoramic drought index image, and export it as a georeferenced raster format.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon; the computer program is executed by a processor to implement the method as described in any one of claims 1-8.

Images