A method for spatio-temporal fusion of global land evapotranspiration based on deep learning

Abstract

The application provides a kind of global land evapotranspiration spatio-temporal fusion method based on deep learning, first, multi-source evapotranspiration product and surface features are integrated, and weighted average evapotranspiration product is used as training target;Through attention mechanism fusion feature, then through space, time encoder and decoder for model pre-training;Then, on the basis of pre-training model, with flux station observation data as training target, model fine-tuning is carried out.The application includes spatial geographic features, time sequence features and surface attribute features into the fusion process of global land evapotranspiration product, provides a global high-resolution continuous multi-year land evapotranspiration benchmark product, has good data adaptability and robustness, excellent regression prediction ability and strong generalization ability, significantly improves the precision of global land evapotranspiration estimation, reduces the uncertainty of product.The model can also be conveniently applied to the spatio-temporal estimation and fusion of other remote sensing, hydrological and meteorological data.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing inversion estimation and relates to a spatiotemporal fusion method for global land evapotranspiration based on deep learning. Background Technology

[0002] Terrestrial evapotranspiration is the primary pathway for water to return from land to the atmosphere, accounting for approximately two-thirds of terrestrial precipitation. It primarily occurs through vegetation transpiration, soil evaporation, and canopy interception evaporation. Climate change will exacerbate the hydrological cycle and alter terrestrial evapotranspiration, thereby impacting ecosystem responses to global climate. Accurately estimating the spatiotemporal distribution of terrestrial evapotranspiration is crucial for understanding global terrestrial water, carbon, and energy cycles, as well as environmental change. Flux station-based observations provide direct, precise, and continuous observations of carbon, water, and energy fluxes, representing the most effective method for revealing the interactions between the terrestrial biosphere and atmosphere at the ecological scale. However, these methods are limited by the spatial coverage of flux stations, while satellite-derived global terrestrial evapotranspiration products exhibit significant variability and uncertainty due to complex feedback mechanisms and heterogeneity in topography and climate.

[0003] Combining ground-based flux station and remote sensing satellite observation systems with flux equations makes it possible to estimate land evapotranspiration at multiple spatiotemporal scales. Existing literature and research show that global land evapotranspiration products exhibit relatively good consistency globally, but show significant differences regionally. These products have considerable spatial and temporal uncertainty, each with its own advantages and limitations. While each product provides valuable information, no single global land evapotranspiration product can provide the most accurate estimate for all biological communities. This situation is more pronounced in regions where multiple climate models coexist. It is natural to consider combining multiple global land evapotranspiration estimation products with surface flux station observations, and using land surface characteristics as auxiliary information, to establish a multivariate spatiotemporal fusion model with global land surface characteristics. This would provide a benchmark product for global land evapotranspiration estimation, thereby reducing the uncertainty in global land evapotranspiration estimation.

[0004] Deep learning algorithms can accurately fit the complex nonlinear relationship between input data and a given target, extracting multi-scale, multi-level abstract features that are less affected by observations such as sensor type and spatial scale. They possess good data adaptability and robustness, excellent classification and regression prediction capabilities, and high prediction accuracy, showing great potential in the field of remote sensing inversion estimation. Autoencoders are an important component of deep learning, capable of learning data encoding in an unsupervised manner, and are widely used in natural language processing, speech signal processing, and computer vision. Successful cases using autoencoder models demonstrate their promising future in remote sensing. In 2015, a study proposed a water quality assessment method based on a sparse autoencoder model, achieving optimal recognition rates and suitable for future intelligent management. In 2016, research introduced autoencoder models into terrain classification of polarimetric synthetic aperture radar images, significantly improving classification accuracy. Other studies have used autoencoder models to identify geological hazards, finding that autoencoder models outperform state-of-the-art classifiers in both efficiency and accuracy. In summary, autoencoder models are suitable for high-dimensional data compression and de-wrapping tasks in remote sensing. However, in the above-mentioned convolutional autoencoder model, the intermediate layer generally only plays the role of feature transmission, that is, passing the spatial abstract features extracted by the encoder to the decoder. Its ability to process the temporally related features of remote sensing images is limited. Summary of the Invention

[0005] This invention proposes a deep learning-based spatiotemporal fusion method for global land evapotranspiration, leveraging the feature extraction capabilities of convolutional autoencoders and the temporal modeling capabilities of recurrent neural networks. It integrates ground flux station observation data, global land evapotranspiration products derived from remote sensing satellites, and auxiliary information on land surface attributes. Spatial geographic features, temporal features, and surface attribute features are incorporated into the global land evapotranspiration fusion process to establish a spatiotemporal fusion neural network model (STF). Experimental results show that the global land evapotranspiration product fused by STF can accurately fit the complex nonlinear relationship between feature parameters and land evapotranspiration. It extracts multi-scale, multi-level abstract features from multi-source data that are less affected by sensor type and spatial scale, exhibiting good data adaptability and robustness, excellent classification and regression prediction capabilities, and strong generalization ability. This significantly improves the accuracy of global land evapotranspiration estimation, reduces product uncertainty, and provides a benchmark product for global land evapotranspiration estimation. This model can be conveniently and effectively applied to the spatiotemporal estimation and inversion of other remote sensing, hydrological, and meteorological data.

[0006] The technical solution adopted in this invention is: a method for spatiotemporal fusion of global land evapotranspiration based on deep learning. Its characteristics include the following steps:

[0007] Step 1: Process different global land evapotranspiration products into data with consistent spatiotemporal resolution, and generate two sets of land evapotranspiration products for continuous years at the global scale using simple averaging and weighted averaging methods respectively.

[0008] Step 2: Process the multi-source land surface attribute information related to land evapotranspiration into data with the same spatiotemporal resolution as global land evapotranspiration products, generating a multi-year global sample set of auxiliary land surface attribute information. Split the ground flux station observation data into training and validation sample sets for multi-stage training, adjustment, and evaluation of the STF model parameters.

[0009] Step 3: Steps 3 through 7 constitute the pre-training phase of STF. STF is pre-trained using the training sample set from Step 3, with the fused global-scale multi-year global land evapotranspiration products as the training target. The auxiliary information of land surface attributes is treated as a multi-dimensional feature subspace. An attention mechanism is used to weight each feature subspace, achieving the selection and fusion of multi-dimensional land surface attribute features. Then, nonlinear transformations and batch normalization transformations are applied to the fused features to enhance their expressive power.

[0010] Step 4: Input the result from Step 3 into the spatial encoder. The spatial encoder is a convolutional module that includes convolution operations, activation functions, gating structures, and batch normalization transformations. It can incorporate the geographical proximity effect contained in the input features into the modeling process, better preserving spatial structure and progressively extracting and fusing features from local to global. The spatial encoder is used to reduce the high-dimensional features of the input and output low-dimensional encoded features.

[0011] Step 5: Use the low-dimensional encoded features obtained in Step 4 as input to the time encoder. The time encoder is a recurrent neural network with gated recurrent units (GRU). This lightweight temporal network structure selectively stores and transmits information through update gates and reset gates, thereby effectively mining temporal information in sequence data.

[0012] Step 6: The decoder is a deconvolution module. The result of Step 5 is input into the decoder, and dimensionality enhancement is performed through deconvolution. After nonlinear transformation by the activation function, residuals are concatenated with the features from the spatial encoder in Step 4 through skip connections. Then, after gating and batch normalization transformation, an output with the same dimension as the model input is obtained.

[0013] Step 7: The spatial encoder, temporal encoder, and decoder can be viewed as several hidden layers between the input and output layers. A loss function is calculated to measure the distance between the model's predictions and the training objective. This distance score is used as feedback to adjust and update the weights of each hidden layer. This parameter optimization process is implemented using the backpropagation algorithm. For each training iteration, the loss score decreases as the weights are updated. The STF model with the minimum loss is selected when the output is as close as possible to the ideal target of the trained network.

[0014] Step 8: Obtain the pre-trained STF model from Step 7, and use the training set of ground flux station observation data as the training target to fine-tune the model. The model fine-tuning stage shares the model structure with the model pre-training stage. That is, the model parameters of the pre-trained model using the fused global land evapotranspiration products are used to initialize the model in the model fine-tuning stage, thereby constructing the final STF model.

[0015] Step 9: Input the sample set of auxiliary information on land surface attributes into the STF model constructed in Step 8 for prediction, and obtain a multi-year global-scale land evapotranspiration dataset. Use the validation sample set to evaluate the accuracy of the model. Attached Figure Description

[0016] Figure 1 This is a flowchart of an embodiment of the present invention;

[0017] Figure 2 Information diagram of global land evapotranspiration products and two sets of fused products used in embodiments of the present invention;

[0018] Figure 3 This is a map showing the estimation accuracy of global land evapotranspiration sites used in embodiments of the present invention.

[0019] Figure 4 This is a precision graph of the STF estimation results in the training and validation sets according to an embodiment of the present invention;

[0020] Figure 5 This is a multi-year anomaly line graph of global land evapotranspiration in an embodiment of the present invention.

[0021] Figure 6 This is a line graph showing the latitudinal distribution of global land evapotranspiration in an embodiment of the present invention.

[0022] Figure 7 This is a line graph showing the multi-year variation trend of global land evapotranspiration in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings. The examples described are intended to facilitate understanding of the present invention and are not intended to limit the present invention.

[0024] This invention, based on the feature extraction capabilities of convolutional autoencoders and the temporal modeling capabilities of recurrent neural networks, provides a deep learning-based method for spatiotemporal fusion of global land evapotranspiration. By integrating ground flux station observation data, global land evapotranspiration products derived from remote sensing satellites, and auxiliary information on land surface attributes, it incorporates spatial geographic features, temporal features, and surface attribute features into the global land evapotranspiration fusion process, establishing a spatiotemporal fusion model. (See attached image.) Figure 1 The specific steps of this embodiment of the invention are as follows:

[0025] Step 1: In this embodiment of the invention, seven commonly used global land evaporation products were selected. These products were processed into data with consistent spatiotemporal resolution, spanning from 1982 to 2015, with a monthly temporal resolution and a spatial resolution of 0.5°. Two sets of global-scale land evaporation products, avg and rea, were generated for consecutive years using both simple averaging and weighted averaging methods. The weighted averaging method was based on the research of Lu et al. (Reference: Lu J, Wang G, Chen T, et al. A harmonized global land evaporation dataset from model-based products covering 1980–2017[J]. Earth Syst. Sci. Data, 2021, 13(12): 5879-5898.). Product information can be found at [link to product information]. Figure 2 .

[0026] Step 2: Considering the sources of land evapotranspiration, this embodiment of the invention selects precipitation (pre), temperature (tmp), normalized difference vegetation index (NDVI), relative humidity (rhm), surface soil moisture (ssm), cloud cover (cld), and potential evapotranspiration (pet) as auxiliary information of land surface properties and the basis for estimating land evapotranspiration.

[0027] `pre`, `tmp`, `cld`, and `pet` are derived from the CRU TS v. 4.06 monthly dataset produced by the UK National Centre for Atmospheric Sciences, one of the most widely used climate datasets, with a spatial resolution of 0.5°. `rhm` is calculated from the temperature (tmp) and vapor pressure (vap) in the CRU TS v. 4.06 monthly dataset. The calculation formula is: , Where svap is the saturated vapor pressure, vap is the vapor pressure, and tmp is the temperature.

[0028] NDVI is derived from the GIMMS NDVI3g.V1 dataset produced by the National Oceanic and Atmospheric Administration (NOAA), which is currently the longest-running NDVI data product with a spatial resolution of 1 / 12° and a temporal resolution of 15 days. SSM is obtained from the GLEAMv3.6a dataset, with a spatial resolution of 0.25° and a temporal resolution of monthly. These multi-source surface remote sensing monitoring data cover the period from 1981 to 2015, covering a global area. The NDVI dataset was processed into a monthly dataset using the maximum value synthesis method, and the NDVI and SSM were upscaled to a spatial resolution of 0.5°. After preprocessing, the multi-source surface remote sensing monitoring data related to global land evapotranspiration were processed to achieve consistent spatiotemporal resolution, generating a multi-year global-scale sample set F of auxiliary information on surface attributes.

[0029] Monthly flux station observation datasets were obtained from FluxNet. This dataset covers every representative terrestrial ecosystem globally, comprising 149 valid station datasets and 17,724 monthly station data points, spanning the period from 1991 to 2014. Station location information (longitude, latitude) and latent heat flux were extracted from each station dataset to determine the station's location on a global 0.5° grid, based on the formula: The latent heat flux is converted into evapotranspiration, where LE is the latent heat flux and ET is the evapotranspiration. It is 2.45 Then, the total monthly evapotranspiration dataset (in millimeters per month) is obtained from the monthly evapotranspiration dataset (in millimeters per day).

[0030] The STF model needs to predict the future from the past in time, and the data cannot be randomly shuffled. It must be ensured that the test data is later than the training data. Therefore, in this embodiment of the invention, the monthly evapotranspiration total dataset of flux stations is split into a training sample set (Train) and a validation sample set (Valid) according to time order, which are used to adjust and evaluate the STF model parameters. The time span of Train is 1991.01-2010.02, and the time span of Valid is 2010.03-2014.12.

[0031] Step 3: Input F and Train into the STF model for pre-training, and use rea as the training target. The shape of F is [B0, C0, W0, H0, T0], where B0 is the input batch size, C0 is the number of channels, W0 is the data width, H0 is the data height, and T0 is the time dimension. This step includes the following four processes: (1) Change the shape of F to [B0×W0×H0×T0,C0]; (2) Weight each feature subspace through the attention mechanism to achieve the filtering and fusion of multi-dimensional feature information. The attention mechanism is represented as follows: , , k, q, v are vectors generated based on the input features; (3) Perform nonlinear transformation on the fused features and restore the shape of the features to [B0, W0, H0, T0, C0]. The activation function used in the nonlinear transformation is the Rectified Linear Unit (ReLU), which can effectively prevent gradient explosion and gradient vanishing problems and better realize gradient descent and backpropagation; (4) Perform batch normalization transformation on the features to obtain multidimensional feature parameter sequence sample F1. The batch normalization method is an important guarantee for the good performance of deep neural networks.

[0032] Step 4: The autoencoder is an important part of deep learning and can learn data encoding in an unsupervised learning manner. The autoencoder in the STF model is divided into three parts: spatial encoder, temporal encoder and decoder. Each part can be regarded as a hidden layer between the input layer and the output layer. The spatial encoder is a convolution module that includes convolution operation, activation function, gating structure and batch normalization transformation. It can incorporate the geographical proximity effect contained in the input features into the modeling process, better preserve the spatial structure, and gradually extract and fuse features from local to global. The spatial encoder is used to reduce the high-dimensional features of the input and output low-dimensional encoded features. It includes the following four processes: (1) Change the shape of F1 in step 3 to [B0×T0, C0, W0, H0], use the activation function parameterized corrected linear unit (PReLU) to perform nonlinear transformation on F1, and perform dilated convolution on the result after nonlinear transformation to obtain F. 21 (2) Simultaneously, F1 is dilated by convolution to obtain F. 22 (3) Use a gating structure to control F 21 and F 22Feature concatenation is performed to obtain F3, preventing gradient explosion and gradient vanishing problems; (4) Finally, the concatenated features are batch normalized using the sigmoid function to maintain the nonlinear characteristics of the network, and their shape is changed to [B1, C1, W1, H1, T1] to obtain the low-dimensional encoded features F of the output. 4。

[0033] Step 5: Input the low-dimensional encoded features F4 obtained in Step 4 into the time encoder. The time encoder is a recurrent neural network (RNN) with gated recurrent units (GRU). Since its proposal in 2014, this method has become a highly regarded RNN gated architecture. This lightweight temporal network structure can effectively transmit and express information in long-term sequences by selectively storing and transmitting information through update gates and reset gates without causing useful information from long ago to be forgotten. It can also solve the gradient vanishing and exploding problems in RNN. After F4 enters the time encoder, it includes the following process: (1) Change the shape of F4 to [B1×W1×H1, T1, C1] and input it into GRU to obtain the output features h of the last layer of GRU. t (3) For h t Perform a full connection and change its shape to [B2, C2, W2, H2, T2]; (4) Batch normalize to obtain the final output feature F5.

[0034] Step 6: This step is used for decoding. The decoder in STF is a deconvolution module, which includes the following process: (1) Change the shape of the low-dimensional feature output F4 in step 4 and the time encoder output feature F5 in step 5 to [input batch size × time dimension, number of channels, width, height] respectively; (2) Perform deconvolution operation on F5 to obtain the output result with the same dimension as the high-dimensional input of the spatial encoder, and use the PReLU activation function to perform nonlinear transformation on the deconvolution result to obtain F6; (3) F6 and F4 are residually connected by skip connections, and their shapes are restored to [B5,C5,W5,H5,T5] after passing through a gated structure; (4) The PReLU activation function is then used to perform nonlinear transformation to obtain the model output result O.

[0035] Step 7: To evaluate the performance of the STF model, the distance between the model's prediction and the training target is measured by calculating the root mean square error (RMSE) of the loss function. This RMSE is used as a feedback signal in the backpropagation algorithm to adjust and update the weights of each hidden layer. For each training cycle, the RMSE decreases as the weights are updated. The STF model with the minimum loss is selected when the output is as close as possible to the ideal target of the trained network.

[0036] Step 8: Obtain the pre-trained STF model from Step 7, and use the training set of ground flux station observation data as the training target to fine-tune the model. The model fine-tuning stage shares the model structure with the model pre-training stage. That is, the model parameters of the pre-trained model using the fused global land evapotranspiration products are used to initialize the model in the model fine-tuning stage, thereby obtaining the final STF model.

[0037] Step 9: Input the surface attribute auxiliary information sample set F into the STF model constructed in Step 8 for prediction, and obtain the global-scale multi-year land evapotranspiration dataset STF_rea, with a time span of 1982-2015 and a time resolution of monthly. The spatial resolution is 0.5°. The model accuracy is evaluated using a validation sample set.

[0038] Step 10: Input F and Train into the STF model for pre-training, using avg as the training target. Following steps 3-9 sequentially, a global-scale, multi-year land evapotranspiration dataset, STF_avg, is obtained, spanning 1982-2015, with a monthly temporal resolution and a spatial resolution of 0.5°. The model accuracy is evaluated using a validation sample set.

Claims

1. A spatiotemporal fusion method for global land evapotranspiration based on deep learning, characterized in that, Includes the following steps: Step 1: The fusion method, based on the feature extraction capability of convolutional neural networks and the temporal modeling capability of recurrent neural networks, introduces attention mechanisms, gated recurrent units, dilated convolutions, and residual connections on the basis of convolutional autoencoders to incorporate spatial geographic features, temporal features, and surface attribute features into the data fusion process, including: (1) Different global land evapotranspiration products are processed into data with consistent spatiotemporal resolution, and a global land evapotranspiration sample set for consecutive years is generated according to the weighted average method. (2) The multi-source land surface attribute information is processed into data with the same spatiotemporal resolution as the global land evapotranspiration sample set for continuous years, and a global land surface attribute auxiliary information sample set for continuous years is generated. (3) The ground flux station observation data are split into a training sample set and a validation sample set; (4) After inputting the training sample set and the land surface attribute auxiliary information sample set into the model, the features of the land surface attribute auxiliary information sample set are fused through an attention mechanism. (5) Input the result of step (4) into the spatial encoder and extract deep abstract features through the spatial encoder. The spatial encoder includes dilated convolution, activation function, gating structure and batch normalization transformation. (6) Input the deep abstract features into the time encoder, and process the timing information through the time encoder. The time encoder includes a gated loop unit, which uses update gate and reset gate to selectively store and transmit information. (7) Decode the features using a decoder and then perform residual splicing of the decoded features and the output of the spatial encoder using a skip connection; (8) Calculate the loss function RMSE and adjust and update the model parameters through the backpropagation algorithm; (9) Select the model with the minimum loss; Step 2: Combine multiple global land evapotranspiration estimation products and surface flux station observation data, using land surface features as auxiliary information. Use the global continuous multi-year land evapotranspiration sample set and surface flux station observation data as the targets for the pre-training stage and the fine-tuning stage, respectively. Through multi-stage training, use the model parameters pre-trained from the global continuous multi-year land evapotranspiration sample set to initialize the model in the fine-tuning stage. Then, use the surface flux station observation data to fine-tune the model, thereby establishing a global land evapotranspiration spatiotemporal fusion model with land surface features. The sample set of auxiliary information on land surface attributes is input into the spatiotemporal fusion model to obtain a baseline product for estimating global land evapotranspiration.

Images