Soil humidity space downscaling method based on auto-encoder

Abstract

The invention provides a soil humidity spatial downscaling method based on an auto-encoder. The method comprises the following steps: acquiring low-resolution soil humidity data and high-resolution auxiliary variable data; an auto-encoder framework comprising an encoder and a decoder is constructed, the encoder is a full convolutional neural network to generate high-resolution soil humidity data, and the decoder comprises an information extraction module used for extracting high-resolution texture information and an aggregation module used for ensuring scale consistency; a frame is trained through a total objective function formed by minimizing aggregation consistency loss and texture constraint loss, so that the model learns downscaling mapping without depending on a high-resolution label; and finally, the trained encoder is used as a model to be applied. According to the method, the reliability of microwave data and detail information of optical data are effectively fused through a self-supervised learning mechanism, and the consistency in the scale conversion process is ensured while the spatial resolution of the soil humidity data is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of Earth science data processing technology, specifically to a spatial downscaling method for soil moisture based on an autoencoder. Background Technology

[0002] Soil moisture is a key state variable in the land surface hydrological cycle, playing a crucial role in surface energy distribution, climate prediction, drought monitoring, and agricultural irrigation management. Remote sensing technology, especially microwave remote sensing (such as L-band and C-band), has become the primary means of large-scale, dynamic monitoring of soil moisture due to its ability to penetrate clouds and sparse vegetation and its sensitivity to changes in soil moisture. However, existing mainstream microwave remote sensing-derived soil moisture products (such as SMAP and SMOS) are limited by sensor physics, resulting in typically coarse spatial resolution (tens of kilometers), which is insufficient to meet the urgent need for high spatial resolution soil moisture data in applications such as watershed hydrological simulation, precision agriculture, and ecological research. Therefore, developing effective spatial downscaling techniques to transform low-resolution soil moisture data into high-resolution data is an important research direction in the field of Earth science and remote sensing data processing.

[0003] Currently, methods for spatial downscaling of soil moisture can be mainly classified into the following categories: Spatial interpolation methods: These methods (such as bilinear interpolation and Kriging interpolation) directly perform mathematical interpolation operations on low-resolution soil moisture data to obtain high-resolution grid data. Their drawback is that they do not introduce any additional information about surface spatial heterogeneity; they are essentially a smoothing process and cannot recover the true details and textures of the surface space. The spatial information gain of the downscaling results is limited.

[0004] Spatiotemporal fusion method: This method typically requires both high spatial resolution-low temporal resolution data and high temporal resolution-low spatial resolution data. It generates high spatiotemporal resolution data by establishing pixel relationships at temporal overlap points. However, this method heavily relies on the stability of the relationship between the two data sources, making model construction complex. It also demands extremely high quality input data and spatiotemporal matching accuracy, making it susceptible to interference from factors such as cloud cover and sensor differences in practical applications, resulting in insufficient robustness.

[0005] Downscaling based on process models and data assimilation: This method assimilates remote sensing observation data into a land surface process model or hydrological model, decomposing low-resolution information onto a high-resolution grid through the model's physical mechanisms. Although it has a physical basis, its effectiveness is highly dependent on the complexity of the selected model and the accuracy of the parameterization scheme, resulting in high computational costs. Furthermore, its applicability in different regions is often limited by the inherent limitations of the model itself.

[0006] Machine learning-based downscaling methods have been widely used in recent years. They mainly fall into two paradigms: one uses high-resolution "ground truth" data (such as site observations or even higher-resolution products) as labels to train a model for regional application; however, reliable and sufficiently comprehensive "ground truth" data is often difficult to obtain. The other directly establishes statistical relationships between soil moisture and various high-resolution auxiliary variables (such as optical remote sensing indices, topographic data, etc.) at a low-resolution scale, and then applies these relationships to the high-resolution auxiliary variables to obtain downscaling results. However, relationships learned at low-resolution scales may not be fully applicable to high-resolution scales, exhibiting scale dependence and insufficient model generalization ability, leading to deviations in the spatial consistency or physical plausibility of the downscaling results.

[0007] In summary, existing technologies either struggle to effectively introduce and maintain high-resolution spatial detail, or they have stringent data requirements, complex models with poor universality, or they face the dilemma of "label dependence" or "scale transformation distortion" in machine learning-based approaches. Therefore, there is an urgent need to develop a new spatial downscaling method for soil moisture that can effectively integrate multi-source information without relying on high-precision labeled data, and can ensure consistency and detail enhancement during scale transformation. Summary of the Invention

[0008] The purpose of this invention is to provide a soil moisture spatial downscaling method based on an autoencoder, which solves the problems of existing technologies that rely on high-precision label data, are difficult to effectively integrate multi-source information, and are difficult to ensure consistency and detail enhancement during the scaling process.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A spatial downscaling method for soil moisture based on an autoencoder includes the following steps: S1. Obtain low spatial resolution soil moisture data, high spatial resolution first auxiliary variable data, and high spatial resolution second auxiliary variable data for the target area; S2. Construct an autoencoder downscaling framework, which includes an encoder and a decoder; The encoder is a fully convolutional neural network, used to receive input including the low spatial resolution soil moisture data, the high spatial resolution first auxiliary variable data and the high spatial resolution second auxiliary variable data, and output the high spatial resolution soil moisture data. The decoder includes an information extraction module and an aggregation module. The information extraction module is connected to the output of the encoder and is used to extract, from the high spatial resolution soil moisture data output by the encoder, first texture information corresponding to the high spatial resolution first auxiliary variable data and second texture information corresponding to the high spatial resolution second auxiliary variable data, respectively. The aggregation module is connected to the output of the encoder and is used to resample and aggregate the high spatial resolution soil moisture data output by the encoder to the resolution of the low spatial resolution soil moisture data to obtain reconstructed low spatial resolution soil moisture data. S3. Using the low spatial resolution soil moisture data, the high spatial resolution first auxiliary variable data, and the high spatial resolution second auxiliary variable data as input, train the autoencoder downscaling framework. During training, the overall objective function consisting of the first loss term, the second loss term, and the third loss term is minimized; Wherein, the first loss term is the error between the reconstructed low spatial resolution soil moisture data and the input low spatial resolution soil moisture data; the second loss term is the error between the extracted first texture information and the input high spatial resolution first auxiliary variable data; and the third loss term is the error between the extracted second texture information and the input high spatial resolution second auxiliary variable data. S4. Using the trained encoder as a downscaling model, input the low spatial resolution soil moisture data to be downscaled and its corresponding high spatial resolution first and second auxiliary variable data, and output the high spatial resolution soil moisture data.

[0010] Furthermore, the low spatial resolution soil moisture data is a soil moisture product based on microwave remote sensing; the high spatial resolution first auxiliary variable data is the Normalized Difference Vegetation Index (NDVI) data based on optical remote sensing; and the high spatial resolution second auxiliary variable data is the Shortwave Infrared Soil Moisture Index (SIMI) data based on optical remote sensing.

[0011] Furthermore, the network structure of the encoder includes: In the first part, through multiple convolution operations and average pooling downsampling operations, the spatial size of the feature map is gradually reduced and the number of feature channels is increased; In the latter half, through multiple convolution operations and upsampling operations based on bilinear interpolation, the spatial size of the feature map is gradually restored and the number of feature channels is reduced, ultimately outputting single-channel high spatial resolution soil moisture data; The encoder has a skip connection between the front half and the rear half.

[0012] Furthermore, the final output layer of the encoder uses the Sigmoid activation function.

[0013] Furthermore, the information extraction module includes a first lightweight convolutional neural network and a second lightweight convolutional neural network; The first lightweight convolutional neural network takes the high spatial resolution soil moisture data output by the encoder and the low spatial resolution soil moisture data input as input, and outputs the first texture information. The second lightweight convolutional neural network takes the high spatial resolution soil moisture data output by the encoder and the low spatial resolution soil moisture data input as input, and outputs the second texture information.

[0014] Furthermore, the first lightweight convolutional neural network and the second lightweight convolutional neural network each contain four layers of convolutional operations.

[0015] Furthermore, the overall objective function is expressed as: ; in: This represents the first loss term; This represents the second loss term; This refers to the third loss term. and This represents the weight parameter, and 0 < <1, 0< <1.

[0016] Furthermore, the first loss item Second loss item and the third loss item All are mean absolute errors.

[0017] As can be seen from the above technical solutions, the present invention has the following technical advantages compared with the prior art: 1. This invention designs an autoencoder framework that includes an aggregation module and an information extraction module. It utilizes the consistency constraints of low-resolution data itself and the texture constraints between low-resolution data and high-resolution auxiliary variables for training, completely eliminating the dependence on hard-to-obtain high-resolution real label data and solving the bottleneck problem of label scarcity in traditional machine learning methods. 2. The aggregation module in the decoder of this invention maintains consistency with the original low-resolution input after forced high-resolution output aggregation, ensuring the fidelity of the downscaling process at the macro scale from the algorithm level, and effectively avoiding systematic deviations or distortions caused by scale jumps. 3. The encoder of this invention can learn the deep nonlinear relationship between reliable moisture information of microwave soil moisture products and rich spatial texture of optical auxiliary variables from end to end, thereby generating downscaling results that maintain moisture accuracy and have high spatial detail, achieving complementary advantages; 4. This invention employs a U-Net structure with skip connections as the encoder, effectively combining features from both deep and shallow layers; the lightweight information extraction module in the decoder is cleverly designed to prevent model degradation. Experiments show that this method significantly outperforms traditional interpolation, statistical, and supervised machine learning methods in terms of site verification accuracy, spatial detail recovery realism, and time series stability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the steps of the soil moisture spatial downscaling method based on an autoencoder of the present invention. Figure 2 This is a schematic diagram of the self-encoder downscaling framework of the present invention; Figure 3 A scatter plot comparing the results of different downscaling methods with ground-measured data for verification. Figure 4 Comparison of the spatial distribution of soil moisture obtained by different downscaling methods; Figure 5 This is a comparison chart of the original low-resolution data and high-resolution data obtained by different downscaling methods in the time series. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0020] This embodiment uses the Qinghai-Tibet Plateau region as an example to explain in detail the implementation process of the soil moisture spatial downscaling method based on an autoencoder described in this invention. Figure 1 As shown, the soil moisture spatial downscaling method based on autoencoders specifically includes the following steps: S1. Data Collection and Preprocessing: Multi-source remote sensing data is collected and preprocessed to provide input data for the method of this invention. Specific steps are as follows: S11. Obtaining Soil Moisture Data: Daily SMAP Level-3 soil moisture products (SPL3SMP version) for the non-freezing period (May to October each year) of the study area (Tibetan Plateau) from 2017 to 2019 were obtained. The original spatial resolution of this product is 36 km. For ease of subsequent processing, the daily ascending and descending trajectory data were averaged to obtain low-spatial-resolution soil moisture data representing the daily conditions, denoted as... .

[0021] S12. Obtain high-resolution auxiliary variable data: Obtain daily MODIS surface reflectance products (MCD43A4, spatial resolution 500 meters) for the same period, and calculate optical remote sensing indices closely related to the spatial heterogeneity of soil moisture based on these products: Normalized Difference Vegetation Index : ,in and These represent the near-infrared and red light reflectance of MODIS, respectively. The obtained data are denoted as... .

[0022] Shortwave infrared soil moisture index : ,in and These represent the shortwave infrared reflectance of MODIS, respectively. The obtained data are denoted as... .

[0023] S13, Data Alignment and Preprocessing: ... The data was resampled to a 500-meter resolution using the nearest neighbor method to align it with auxiliary variables on the spatial grid. This step was only to unify the grid for model input; the data itself still contained information at the 36-kilometer scale. and Perform a projection transformation to ensure that the geographic coordinate system and spatial extent are completely consistent with the resampled soil moisture data.

[0024] To fill in the missing values ​​in the optical data caused by cloud obstruction, a linear interpolation method on the time series is used to obtain a spatiotemporally continuous auxiliary variable dataset.

[0025] S14. Data Block Segmentation: To adapt to the input of the convolutional neural network and transform large-scale regional data into batch training samples, the data blocks processed above are segmented into smaller blocks. , , The three channels of data are cropped into multiple overlapping data blocks of 896×640 pixels each. In practice, appropriate redundant areas should be retained at the edges of the blocks during cropping to mitigate the boundary effects caused by subsequent convolution operations.

[0026] S2. Downscaling Framework Construction: The autoencoder downscaling framework constructed in this embodiment is as follows: Figure 2 As shown, it mainly consists of two parts: an encoder and a decoder.

[0027] Encoder Design: The encoder is a deep fully convolutional neural network based on an improved U-Net architecture, responsible for learning the mapping from low-resolution input to high-resolution soil moisture details, while promoting residual learning through multiple skip connections. Specifically, it includes: Downsampling path (shrinkage path): Consists of multiple convolutional modules cascaded together. Each module contains two consecutive 3×3 convolution operations, followed by a batch normalization layer and a ReLU activation function, and then downsampling through a 2×2 average pooling layer. This process is repeated, gradually halving the feature map size while gradually doubling the number of feature channels (e.g., from the initial 64 channels to 128 channels) to extract multi-level, abstract features. Ultimately, the feature map size is reduced to 1 / 64 of the input. Upsampling path (expansion path): Symmetrical to the downsampling path, each layer first performs bilinear interpolation upsampling (magnification by 2x) on the input feature map to replace deconvolution, effectively avoiding the "chessboard effect." After upsampling, feature maps of the same scale as those from the downsampling path are concatenated via skip connections to promote the recovery of detailed information. The concatenated features are then subjected to two 3×3 convolutions (including batch normalization and ReLU). This process is repeated until the feature map size is restored to the input size; Output layer: Finally, a 1×1 convolution is used to compress the number of feature channels to 1, and a Sigmoid activation function is applied to constrain the network output value to a physically reasonable range of [0,1], representing the predicted soil moisture at a resolution of 500 meters, denoted as . .

[0028] Decoder design: The decoder is not used directly for the final prediction, but provides self-supervised constraints for the training of the encoder. It consists of an information extraction module and an aggregation module.

[0029] The information extraction module comprises two independent lightweight convolutional neural networks (CNN_A and CNN_B), respectively used for extracting information from... Extracting and , Related spatial texture information. To prevent modules from overfitting the encoder output and losing their constraints, each lightweight CNN uses... and original input (First upsampled to 500 meters) cascaded as input. The introduction of this helps the module focus on The newly added high-frequency details. Each lightweight CNN contains only four 3×3 convolutional layers, a simple structure that ensures it extracts common spatial texture patterns. Specifically, CNN_A output CNN_B output .

[0030] The aggregation module A simple average pooling operation was performed to aggregate the spatial resolution from 500 meters back to the original 36 kilometers, resulting in a reconstructed low-resolution soil moisture image. .

[0031] S3. Downscaling Framework Training: The goal of the training is to optimize the encoder parameters so that it can generate a soil moisture map that is both macroscopically consistent with the original low-resolution data and incorporates spatial details of high-resolution auxiliary variables.

[0032] Loss function design: The total loss function It consists of three weighted parts: ; : Represents the aggregation consistency loss. Calculation With the original input The mean absolute error (MAE) between the values ​​is minimized. Minimizing this ensures that the downscaling results maintain consistency in magnitude and spatial pattern with authoritative SMAP products at low-resolution scales.

[0033] and : Represents texture constraint loss. Calculated separately. and , and Minimizing both MAE forces the encoder to embed spatial texture details similar to real SIMI and NDVI data when generating high-resolution soil moisture.

[0034] Weight parameters: and These are hyperparameters used to balance the importance of different loss terms. In this embodiment, verification and optimization are performed using a small amount of measured data from 2019 at various sites to determine... =0.2, =0.8. This indicates that the training process focuses more on constraining the details of optical textures.

[0035] Training process: Input the prepared data blocks from S1 into the network. Use the Adam optimizer to minimize... The model is trained iteratively to target the desired model. After training, only the encoder portion is retained as the final soil moisture downscaling model.

[0036] S4. Downscaling Application and Validation: For new input data ( , , By simply inputting it into a trained encoder, high spatial resolution (500 meters) soil moisture data can be directly output. After trimming redundant boundaries from each predicted data block and stitching them together, a high-resolution daily soil moisture product for the entire study area can be obtained.

[0037] Effect verification: Site-scale validation: Validation was conducted using measured data from 20 soil moisture stations at a depth of 5 cm on the Qinghai-Tibet Plateau (non-freezing period, 2017-2018). For example... Figure 3 As shown, compared with the original 36km SMAP product, the downscaled 500m soil moisture data obtained by the method of this invention significantly reduced the root mean square error (RMSE) of the data with respect to station observations from 0.107m³ / m³ to 0.073m³ / m³, and the unbiased root mean square error (ubRMSE) from 0.079m³ / m³ to 0.067m³ / m³, while maintaining a high correlation coefficient (R) above 0.85. The scatter plot shows that the downscaled data is more concentrated around the 1:1 line, demonstrating a significant improvement in accuracy.

[0038] Spatial effect comparison: such as Figure 4 As shown, this paper presents a comparison of downscaling results from different methods in a region of the western Tibetan Plateau on August 21, 2018. It is evident that the method of this invention, while maintaining the overall wet and dry trends of the original SMAP data, generates rich, continuous, and reasonable spatial details. These details highly match the surface features (such as river channels and vegetation cover differences) revealed by the contemporaneous true-color (RGB) imagery. In contrast, the cubic interpolation method yields overly smooth results with no new details; the principal component analysis method shows a false dry area in the lower left corner of the region that is inconsistent with the SMAP trend; and the downscaling method based on random forest exhibits spatial discontinuities and a significant patch effect.

[0039] Time series analysis: such as Figure 5 As shown, a representative pixel was selected to plot a time series. The soil moisture downscaled using the method of this invention is highly consistent with the time variation curve of the original SMAP product, and at the same time, it shows a more reasonable daily spatial heterogeneity fluctuation, proving its reliability in the spatiotemporal dimension.

[0040] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An autoencoder-based soil moisture spatial downscaling method, characterized in that, The method comprises the following steps: S1, obtaining low spatial resolution soil moisture data, high spatial resolution first auxiliary variable data and high spatial resolution second auxiliary variable data of a target area; S2, constructing an autoencoder downscaling framework, the autoencoder downscaling framework comprising an encoder and a decoder; The encoder is a full convolutional neural network, configured to receive an input comprising the low spatial resolution soil moisture data, the high spatial resolution first auxiliary variable data and the high spatial resolution second auxiliary variable data, and output high spatial resolution soil moisture data; The decoder comprises an information extraction module and an aggregation module; the information extraction module is connected to an output end of the encoder, configured to extract first texture information corresponding to the high spatial resolution first auxiliary variable data and second texture information corresponding to the high spatial resolution second auxiliary variable data from the high spatial resolution soil moisture data output by the encoder, respectively; The aggregation module is connected to the output end of the encoder, configured to resample and aggregate the high spatial resolution soil moisture data output by the encoder to the resolution of the low spatial resolution soil moisture data, to obtain reconstructed low spatial resolution soil moisture data; S3, training the autoencoder downscaling framework with the low spatial resolution soil moisture data, the high spatial resolution first auxiliary variable data and the high spatial resolution second auxiliary variable data as input; During the training process, a total objective function composed of a first loss term, a second loss term and a third loss term is minimized; The first loss term is the error between the reconstructed low spatial resolution soil moisture data and the input low spatial resolution soil moisture data; the second loss term is the error between the extracted first texture information and the input high spatial resolution first auxiliary variable data; and the third loss term is the error between the extracted second texture information and the input high spatial resolution second auxiliary variable data; S4, using the trained encoder as a downscaling model, inputting low spatial resolution soil moisture data to be down-scaled and corresponding high spatial resolution first and second auxiliary variable data, and outputting high spatial resolution soil moisture data.

2. The autoencoder-based soil moisture spatial downscaling method of claim 1, wherein, The low spatial resolution soil moisture data is a soil moisture product based on microwave remote sensing; the high spatial resolution first auxiliary variable data is normalized difference vegetation index NDVI data based on optical remote sensing; and the high spatial resolution second auxiliary variable data is shortwave infrared soil moisture index SIMI data based on optical remote sensing.

3. The autoencoder-based soil moisture spatial downscaling method of claim 1, wherein, The network structure of the encoder comprises: The front half gradually reduces the spatial size of the feature map and increases the number of feature channels through multiple convolution operations and average pooling down-sampling operations; The back half gradually recovers the spatial size of the feature map and reduces the number of feature channels through multiple convolution operations and bilinear interpolation-based up-sampling operations, and finally outputs single-channel high spatial resolution soil moisture data; The encoder is provided with a skip connection between the front half and the back half.

4. The autoencoder-based soil moisture spatial downscaling method of claim 3, wherein, The final output layer of the encoder uses a sigmoid activation function.

5. The autoencoder-based soil moisture spatial downscaling method of claim 1, wherein, The information extraction module comprises a first light convolutional neural network and a second light convolutional neural network; The first light convolutional neural network takes the high-spatial-resolution soil moisture data output by the encoder and the input low-spatial-resolution soil moisture data as inputs, and outputs the first texture information. The second light convolutional neural network takes the high-spatial-resolution soil moisture data output by the encoder and the input low-spatial-resolution soil moisture data as inputs, and outputs the second texture information.

6. The autoencoder-based soil moisture spatial downscaling method of claim 5, wherein, The first light convolutional neural network and the second light convolutional neural network each comprise four layers of convolutional operations.

7. The autoencoder-based soil moisture spatial downscaling method of claim 1, wherein, The total target function is represented as: ； wherein: represents the total objective function; represents the first loss term; represents the second loss term; represents the third loss term, and represents a weight parameter, and 0 <1, 0 <1.

8. The autoencoder-based soil moisture spatial downscaling method of claim 1, wherein, The first loss term The second loss term and the third loss term are all mean absolute errors.

Images