A hydrological variable prediction method based on multi-task learning and hybrid expert model

Abstract

The application provides a hydrological variable prediction method based on multi-task learning and a mixed expert model, relates to the technical field of hydrological environment prediction, and comprises the following steps: training a preset hydrological variable prediction model through surface and atmospheric variable data corresponding to multiple prediction tasks, and predicting soil moisture and evapotranspiration of multiple spatial grid points in a future time period based on the trained hydrological variable prediction model. The multi-task feature extraction module designed in the application extracts shared features from data corresponding to different prediction tasks based on a multi-head self-attention mechanism, then configures independent expert networks and gate networks for each task, performs task-specific feature enhancement, and thus completes each prediction task according to the enhanced features. The multi-task feature extraction module can extract features with more comprehensive time information and more complete hydrological processes from multiple input task data, thereby improving the stability and accuracy of the prediction of soil moisture and evapotranspiration in advance for multiple days.

Description

Technical Field

[0001] This invention relates to the field of hydrological environment prediction technology, and more specifically, to a method for predicting hydrological variables based on multi-task learning and hybrid expert models. Background Technology

[0002] Soil moisture and evapotranspiration are crucial for water resource management, monitoring, and forecasting of agricultural, hydrological, and meteorological systems. Current methods combine Transformer and LSTM to predict these two variables, but these approaches only explore relatively short forecast lead times and rely heavily on extensive historical data to simulate runoff and flood trends during training, neglecting the dynamic relationships between variables in hydrological processes. However, learning these dynamic relationships is essential for models to more comprehensively simulate hydrological processes and reduce uncertainties arising from the interactions of different variables in multi-day forecasts.

[0003] In fields such as computer vision and natural language processing, multi-task learning is often used to help models model the relationships between tasks or variables. Multi-task learning uses task input data to help models learn the relationships between them, aiming to optimize multiple objectives simultaneously within a single deep learning model.

[0004] Existing research on multi-task learning in the field of hydrology is relatively superficial, merely modifying the model's output header to enable it to output multiple tasks. It fails to incorporate an information interaction module for multi-task learning into the model, which may lead to insufficient simulation of relationships between tasks and incomplete time-series learning.

[0005] Therefore, there is an urgent need for a method that can extract features with more comprehensive time information and more complete hydrological processes from multiple input task data, so as to improve the stability and accuracy of forecasting soil moisture and evapotranspiration several days in advance. Summary of the Invention

[0006] The purpose of this invention is to provide a hydrological variable prediction method based on multi-task learning and a hybrid expert model to improve the aforementioned problems. To achieve this objective, the technical solution adopted by this invention is as follows: This application provides a method for predicting hydrological variables based on multi-task learning and a hybrid expert model, including: Obtain historical datasets, which include first data for multiple days from multiple spatial grid points. The first data includes surface and atmospheric variable data corresponding to multiple prediction tasks, including soil moisture prediction and evapotranspiration prediction. A pre-defined hydrological variable prediction model is trained based on the historical dataset. The hydrological variable prediction model includes a multi-task feature extraction module and a backbone prediction network. The multi-task feature extraction module is used to extract a first feature tensor from the first data based on a multi-head self-attention mechanism. The multi-task feature extraction module configures an expert network and a gating network for each prediction task. The expert network and the gating network are used to perform feature enhancement on the first feature tensor to obtain a second feature tensor. One second feature tensor corresponds to one prediction task. The backbone prediction network is built based on LSTM and is used to output a sequence of predicted values ​​for spatial grid points in future time periods based on the second feature tensor. Based on the trained hydrological variable prediction model, soil moisture and evapotranspiration at multiple spatial grid points are predicted for future time periods.

[0007] The beneficial effects of this invention are as follows: The multi-task feature extraction module designed in this invention extracts shared features from data corresponding to different prediction tasks based on a multi-head self-attention mechanism. Then, it configures an independent expert network and gating network for each task to perform task-specific feature enhancement, thereby completing each prediction task based on the enhanced features. The multi-task feature extraction module can extract features with more comprehensive temporal information and more complete hydrological processes from input data from multiple tasks, thereby improving the stability and accuracy of soil moisture and evapotranspiration forecasts several days in advance.

[0008] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a hydrological variable prediction method based on multi-task learning and a hybrid expert model, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a Transformer according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a multi-head attention structure in an embodiment of the present invention; Figure 4This is a schematic diagram of the structure of the multi-task feature extraction module of a hydrological variable prediction method based on multi-task learning and hybrid expert models as described in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the use of the RMSE metric to evaluate the prediction accuracy of four benchmark models before and after using the multi-task feature extraction module in an embodiment of the present invention. Figure 6 The use of R in the embodiments of the present invention 2 The diagram illustrates the predictive performance of four benchmark models before and after using the multi-task feature extraction module, using the KGE metric. Figure 7 This is a schematic diagram showing the comparison of the average RMSE predicted by the processed model and the original model in five regions in this embodiment of the invention for 2020. Figure 8 This is a schematic diagram comparing the utilization rate of input features by the processed model and the original model when performing ET and SM seven-day advance forecasts in three regions in an embodiment of the present invention. Figure 9 This is a schematic diagram of the processed LSTM model and the original LSTM model's predicted sequence and the actual observation sequence for one year at a randomly selected location in North America, as described in an embodiment of the present invention. Figure 10 This is a schematic diagram illustrating the diffusion of the RMSM set for the two variables SM and ET predicted by the processed model and the original model in an embodiment of the present invention. Figure 11 This is a schematic diagram illustrating the correlation coefficients between the prediction results of the processed model and the original model on the last day of the test year 2020 (SM and ET) and the most relevant input features of their respective variables, in an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0012] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0013] Example 1: It is important to note that the hydrological cycle closely links the interactions between the atmosphere, lithosphere, biosphere, and anthropocene, and is inextricably linked to human activities and socio-economic development. With the climate change caused by global warming in recent years, the global water cycle is experiencing significant spatiotemporal variability, leading to numerous climate problems. Soil moisture (SM) and soil evapotranspiration (ET) play crucial roles in the global water cycle, significantly impacting it. Soil moisture couples atmospheric processes with surface conditions, while soil evapotranspiration involves multiple cycles, including water, energy, and carbon (photosynthesis). SM and ET are key variables in water resource management, numerical weather prediction, extreme event monitoring, and agricultural irrigation; therefore, accurate and multi-step advance prediction of their trends is crucial for decision-making in these fields. However, SM and ET are influenced by various factors such as precipitation, soil properties, and topography, making multi-day advance predictions subject to significant uncertainty.

[0014] Recently, due to the significant advantages of deep learning in approximation capabilities and automatically capturing nonlinear and non-stationary relationships in high-dimensional data, many researchers have begun using deep learning models to predict soil moisture (SM) and evapotranspiration (ET). Deep learning models break the limitations of mathematical statistics and physical models, enabling the construction of simplified models that ignore complex parameters. Commonly used models include convolutional neural networks (CNNs), support vector machines (SVMs), extreme learning machines (ELMs), random forests, recurrent neural networks (RNNs), and long short-term memory (LSTM) models. Notably, recurrent neural networks (RNNs) excel at capturing temporal information and model order dependencies in time-series data, which aligns well with the dynamic simulation characteristics of hydrological variables such as SM and ET. The long short-term memory (LSTM) model is an improved variant of the recurrent neural network (RNN), effectively addressing the common gradient vanishing and exploding problems in RNNs by introducing gating mechanisms and memory units. Compared to traditional RNNs, LSTMs can not only capture more long-term dependencies in time series data but also learn input-output relationships more accurately. This makes LSTMs superior to other neural networks in multi-day advance forecasting.

[0015] Extensive research has been conducted on the application of Long Short-Term Memory (LSTM) networks in soil moisture (SM) and evapotranspiration (ET) prediction. However, LSTMs themselves lack the ability to resolve prediction uncertainties, causing their prediction accuracy to decline rapidly with increasing lead time. Furthermore, due to the autoregressive or recurrent neural network-based structure of LSTMs, two non-adjacent locations (or units) cannot be directly connected, and their stepwise data processing strategy means that the current state is calculated only based on previous states, failing to directly access distant historical data. Many studies have shown that LSTMs generally cannot learn from long-term patterns regardless of the information retained in the memory state. Therefore, LSTMs are likely unable to learn significantly from the long-term, periodic patterns of hydrological systems in modeling, which also limits their prediction accuracy, especially in multi-day lead time forecasts.

[0016] Attention mechanisms represent a significant breakthrough in neural networks, effectively addressing the selective focus problem in information processing and providing a more comprehensive perspective for time series prediction tasks. The Transformer, a novel sequence modeling architecture based on attention mechanisms, has gained widespread acceptance due to its support for parallel computation and efficient modeling of short-term and long-term dependencies. Unlike Recurrent Neural Networks (RNNs), the Transformer, through its multi-head attention mechanism, can directly and completely access and learn the entire time series. Furthermore, the Transformer can simultaneously predict multiple steps of the output sequence, a challenging problem for traditional RNNs (including Long Short-Term Memory networks (LSTM)). This characteristic allows the Transformer to outperform traditional RNNs and LSTMs in multi-step prediction of hydrological variables. However, because both the encoder and decoder of the Transformer employ multi-head self-attention mechanisms, the computational space complexity is high, and its ability to perceive local information features is weak. This makes the model susceptible to outliers. Meanwhile, in the field of hydrology, the next state of most hydrological variables (such as soil moisture (SM) and evapotranspiration (ET)) is highly dependent on the current time step, and Transformer, by extending its view to the entire time series, may ignore this characteristic. Therefore, existing studies have combined Transformer and LSTM to leverage the characteristics of both to improve model performance. However, these studies have only explored relatively short prediction lead times and mainly relied on large amounts of historical data to simulate the changing trends of runoff and floods during training, neglecting the dynamic relationships between variables in the hydrological process.

[0017] However, learning the dynamic relationships between variables in hydrological processes is crucial for models to more comprehensively simulate hydrological processes and reduce uncertainties arising from the interaction of different variables in multi-day forecasts. In fields such as computer vision and natural language processing, multi-task learning is often used to help models simulate relationships between tasks or variables. Multi-task learning helps models learn the relationships between task input data, aiming to optimize multiple objectives simultaneously within a single deep learning model. In the field of hydrology, some studies have also used multi-task learning to improve the prediction accuracy and stability of models. Multi-task learning can improve the model's simulation of seasonal patterns, indicating that it can enhance the learning ability of hydrological deep learning models in terms of time periodicity. However, existing research is quite superficial, merely changing the model's output header to enable it to output multiple tasks. It does not incorporate a multi-task learning information interaction module into the model, which may lead to insufficient simulation of relationships between tasks and incomplete learning of time series data.

[0018] To address the aforementioned technical problems, this embodiment provides a method for predicting hydrological variables based on multi-task learning and a hybrid expert model.

[0019] For details, see Figure 1 The figure shows that the method includes steps S1, S2 and S3.

[0020] S1. Obtain historical datasets, which include first data for multiple days of history from multiple spatial grid points. The first data includes surface and atmospheric variable data corresponding to multiple prediction tasks. The prediction tasks include soil moisture prediction and evapotranspiration prediction. It is understood that this embodiment uses the LandBench 1.0 benchmark dataset. This dataset provides global, multi-resolution, diurnal-scale reanalysis data of surface and atmospheric variables. Each data sample corresponds to a 1°×1° spatial grid point on the Earth's surface and contains time-series observation records of that grid point over consecutive T days. Each record consists of 15 feature variables, as shown in Table 1: Table 1

[0021] Furthermore, the specific impacts of each feature variable in Table 1 on the prediction of soil moisture and evapotranspiration are as follows: Among the feature variables used for soil moisture prediction, the focus is on those affecting soil moisture balance and storage, specifically including: Second layer soil temperature: Temperature directly affects the phase change and transport rate of soil moisture; Surface solar radiation downwards: Solar radiation is the main driving force of the surface energy balance, affecting soil evaporation and vegetation transpiration; Surface thermal radiation downwards: affects net surface radiation and energy distribution; Rainfall: The direct source of soil moisture.

[0022] Air temperature at 2 meters: affects evaporation capacity and soil temperature.

[0023] Specific humidity: Atmospheric humidity affects evaporation potential.

[0024] Surface air pressure: indirectly affects the air's water vapor carrying capacity and weather systems.

[0025] Clay content, sand content, and silt content: determine the soil's water-holding capacity and hydraulic conductivity.

[0026] Soil water holding capacity: directly determines the maximum water storage capacity of the soil.

[0027] Vegetation type: Vegetation affects canopy interception, transpiration and soil moisture consumption.

[0028] Among the characteristic variables used for soil moisture prediction, the focus is on controlling for the energy and atmospheric dynamics of evaporation and transpiration. Specific variables involved include: surface variables: Second layer soil temperature: Soil thermal conditions affect evaporation energy.

[0029] Solar radiation on the Earth's surface points downwards: the primary source of energy required for evapotranspiration.

[0030] Surface thermal radiation downwards: affects the net energy available for evaporation.

[0031] 2-meter air temperature: directly affects saturated vapor pressure and is a key parameter for calculating potential evapotranspiration.

[0032] Specific humidity: Air humidity is one of the gradient factors driven by evaporation.

[0033] u10, v10 (U / V components of wind): Wind speed affects water vapor transport and boundary layer conditions, and is crucial to evaporation rate.

[0034] Surface air pressure: together with temperature and humidity, determines the physical state of the air.

[0035] Vegetation type: This is the key to distinguishing between soil evaporation and vegetation transpiration, with significant differences in transpiration coefficients among different vegetation types.

[0036] Soil water holding capacity: When soil moisture is limited, it affects the actual evapotranspiration.

[0037] Furthermore, in this embodiment, data from 2000 to 2019 is selected as the training set, with the first 80% used for training and the last 20% used for validation, and the data for the entire year of 2020 is used as the independent test set.

[0038] To eliminate the influence of different variable units, global Z-score standardization is performed on each feature variable X_i in the record, which can be expressed as: ; in, and Let be the mean and standard deviation of the i-th feature in the training set across all time and space, respectively. After global standardization of the feature variables, the standardized data are sampled using a sliding window with a length of T days (which can be adjusted according to the prediction lead time). A single sample can be represented as a triple. ,in, Indicates the number of spatial grid points sampled; This represents the continuous time step, in days, which is the size of the sliding window; This represents the number of feature variables. If the data from a single sample is constructed as a tensor, then the shape of the tensor is... The first dimension Represents different geographical locations (grid points) within a batch; the second dimension The third dimension represents a continuous time series for each grid point. This represents the values ​​of 15 features observed at each time point. Multiple sampling is performed to construct multiple tensors, forming tensor batches, which serve as the direct input to the model.

[0039] S2. A pre-defined hydrological variable prediction model is trained based on the historical dataset. The hydrological variable prediction model includes a multi-task feature extraction module and a backbone prediction network. The multi-task feature extraction module is used to extract a first feature tensor from the first data based on a multi-head self-attention mechanism. The multi-task feature extraction module configures an expert network and a gating network for each prediction task. The expert network and the gating network are used to perform feature enhancement on the first feature tensor to obtain a second feature tensor. One second feature tensor corresponds to one prediction task. The backbone prediction network is constructed based on LSTM and is used to output a sequence of predicted values ​​for spatial grid points in the future time period based on the second feature tensor. One sequence of predicted values ​​corresponds to one prediction task. S3. Based on the trained hydrological variable prediction model, predict soil moisture and evapotranspiration for multiple spatial grid points in the future time period.

[0040] It is understandable that, such as Figure 4As shown, this embodiment constructs a multi-task feature extraction module (MTMEG) that combines multi-task learning and multi-expert gating. MTMEG aims to extract data with more comprehensive temporal information and a more complete understanding of hydrological processes from multiple input task data using the Transformer mechanism and multi-task learning methods. For example, if soil moisture (SM) and evapotranspiration (ET) are used as model prediction tasks, MTMEG will automatically learn the long-term trends of SM and ET themselves, as well as their interactions, and output them as features to the LSTM-based prediction model. This improves the accuracy and stability of its SM and ET forecasts several days in advance. Specifically, MTMEG first uses a Transformer layer to perform global time-series feature extraction on the input data of all tasks. This is the first information interaction, which aims to initially learn and integrate the relationships and mutual influence of predicted hydrological variables, and output them as features. The extracted features are then input as shared features into expert networks. Each expert network corresponds to a specific task and is used to predict a specific hydrological variable. In the expert network, a second interaction is performed using a cross-attention mechanism to combine the task-specific input data and shared features. This second interaction favors the different hydrological variables corresponding to each expert network, aiming to emphasize the importance of different hydrological variables in the interaction process of all predictor variables. Finally, a gating mechanism is used to fuse the features output by all expert networks, completing a third information interaction. This third interaction integrates the features of different hydrological variables through weighted fusion, aiming to enhance the interaction between positively correlated variables and reduce the association between negatively correlated variables. Through these three information interactions, MTMEG transforms the data originally directly input to the LSTM model into features with more comprehensive time-series information and a more complete hydrological interaction process, thereby helping the LSTM model to learn and simulate hydrological processes more effectively.

[0041] Furthermore, it can be understood that the multi-task feature extraction module MTMEG constructed in this embodiment includes a shared-transformer module, a multi-expert module (Task Transformer), and a gated fusion module. The core of the shared-transformer and multi-expert modules is a Transformer block based on different attention mechanisms. The main characteristic of the Transformer is that it utilizes the attention mechanism to process and learn the entire input sequence at once, effectively capturing long-term dependencies in the sequence, which is superior to LSTM based on recursive strategies. The Transformer can also capture the complex temporal variations of hydrological data, thus performing better than traditional LSTM models when making multi-day predictions. A traditional Transformer is as follows: Figure 2 As shown.

[0042] Specifically, in this embodiment, the preprocessed input tensor Input∈R^(S×T×X) is first fed into the MTMEG module constructed in this embodiment. This module transforms the input tensor into enhanced features rich in long-term dependencies and inter-task interaction information through three levels of information exchange. For an input tensor Input∈R^(S×T×X), it first enters the Share-Transformer of the MTMEG module.

[0043] Furthermore, since the Transformer itself lacks time-series awareness, a sine-cosine positional encoding PE∈R^(T×d_model) is first added to the input, enabling the model to perceive the order of data in the time series. The encoded feature is Input_pe=Input+PE. After positional encoding, shared features are extracted. Input_pe is fed into a multi-head self-attention layer. This layer can capture the global dependencies between all time steps T and all features X in parallel, especially the potential hydrological relationships between soil moisture (SM) and evapotranspiration (ET) related variables. The output of this layer is denoted as F_attn. Then, F_attn is fed into a feed-forward network (FFN). The feed-forward network typically consists of two linear transformation layers and an intermediate activation function (such as ReLU), responsible for performing nonlinear transformations and integration on the features output by the multi-head self-attention layer to enhance the model's expressive power. Furthermore, in the above-mentioned shared feature extraction, the standard design of Transformer is followed, introducing residual connections and layer normalization. Specifically, the output of the multi-head self-attention layer is represented as: F_attn=LayerNorm(Input_pe+SelfAttention(Input_pe)); The output of the feedforward network is represented as: F_shared=LayerNorm(F_attn+FFN(F_attn)); Residual connections and layer normalization help stabilize training and improve model performance.

[0044] Finally, the sharing module outputs a shared feature F_shared∈R^(S×T×D_shared), where D_shared is the dimension of the shared feature. This shared feature F_shared contains the general spatiotemporal patterns and inter-task interaction information learned from all inputs.

[0045] It is understandable that the Share Transformer designed in this embodiment uses positional encoding in the shared module. Because the Transformer itself does not include the relative position information of data within the sequence, positional encoding is needed to inform it of the relative positions of the data within the sequence. In this embodiment, a positional encoding method based on sine and cosine functions is adopted. This is achieved by adjusting the sequence length `max_len` and the feature dimension... The calculation is performed to generate a three-dimensional tensor PE. The specific calculation process is as follows: ; ; in, Indicates the position in the sequence. This represents an index representing a feature dimension. In this way, each location is assigned a unique code, enabling the model to better distinguish different locations within the sequence. Location encoding gives each time step specific location information, allowing the Transformer to better understand the temporal dependencies in time series data. This not only helps the model identify and utilize long-term trends, such as seasonal variations and annual fluctuations, but also allows for rapid adjustments to forecasts in the event of extreme events. For example, during rainy seasons or drought periods, location encoding can help the model more accurately predict anomalous changes in soil moisture and evapotranspiration, thereby improving the accuracy and reliability of forecasts.

[0046] The Share Transformer uses a multi-head self-attention mechanism, which automatically identifies relevant information in any part of a sequence when processing sequence information. Traditional self-attention mechanisms take a dimensional input... The key and query, and the dimension as The value is calculated by multiplying the query by all keys and then dividing each dot product by . Then, a soft maximum function is applied to obtain the weights of the values. The formula is as follows: ; in, and These are the query matrix, key matrix, and value matrix, respectively, and softmax is the normalized exponential function.

[0047] In the multi-head self-attention mechanism, the input queries, keys, and values ​​are projected h times through different linear projections. For each projected version of the queries, keys, and values, the attention function is executed in parallel to obtain... The output value of the dimension. For example... Figure 3As shown, these values ​​are concatenated and projected again to obtain the final value. The formula is expressed as follows: ; ; in, To indicate the total number of attention heads, Concat represents the concatenation operation. Representing the The subspace (i.e., the first) Attention calculation results (for each attention head). and The first The first Each query, key, and value is composed of complete... and The matrix is ​​obtained through further, independent linear projections. These projections allow each attention head to focus on learning information from different feature subspaces.

[0048] The sharing module handles two tasks: soil moisture and evapotranspiration. The data from these tasks are concatenated to form the input sequence. Compared to traditional single-head self-attention, the multi-head self-attention mechanism allows the sharing module to more comprehensively consider various aspects of the input sequence, including the complex relationships between different time steps and the interaction between soil moisture and evapotranspiration. Following the sharing module, this embodiment uses self-attention to further enhance the positive transfer relationships among the extracted features. This completes the first information interaction between different variables. The features processed by the sharing module are called shared features.

[0049] The MTMEG module in this embodiment also includes a multi-expert module (Task-Transformer). This module receives two inputs: the position-encoded original feature tensor Input_pe and the shared features F_shared. Each expert in the multi-expert module transforms the position-encoded original feature tensor Input_pe through a task-corresponding trainable query linear projection layer to obtain the query tensor. During training, the parameters of this projection layer automatically learn and emphasize the feature dimensions in the original input that are strongly correlated with the target task (such as SM or ET), thereby achieving implicit information focusing.

[0050] In this embodiment, the multi-expert module comprises two expert networks, corresponding to the prediction tasks of SM and ET respectively. The core of each expert network is a cross-attention layer. Its operation is as follows: Query generation: Input_pe is linearly transformed through a trainable query linear projection layer corresponding to the task to obtain the query tensor. During training, the parameters of this projection layer are automatically learned and the weights are adjusted, so that the SM expert network focuses more on the original feature dimensions strongly correlated with soil moisture (such as precipitation and soil temperature), while the ET expert network focuses more on the original feature dimensions strongly correlated with evapotranspiration (such as solar radiation and wind speed), thereby achieving implicit information focusing.

[0051] Key and value generation: The shared feature F_shared is linearly transformed through two different trainable linear projection layers (key projection layer and value projection layer) to obtain the key tensor and value tensor.

[0052] Cross-attention computation: Based on the multi-head cross-attention mechanism, the query tensor is used to perform attention computation on the key tensor and value tensor to achieve the function of retrieving the most relevant information to the current task from shared features.

[0053] It's worth noting that during the query matrix generation process, when the original task data is passed through a learnable linear projection layer to generate the query vector Q_task, the weights of this projection layer are optimized during training. The projection layer of the SM expert network automatically learns to assign higher weights to features with a high physical correlation to soil moisture (such as TP precipitation and STL2 soil temperature). Thus, the generated query vector Q_SM emphasizes the information corresponding to those features. Similarly, the projection layer of the ET expert network learns to emphasize different features (such as SSRD radiation and U10 / V10 wind speed). This emphasis-learning process is completed end-to-end through training and adaptation.

[0054] Both expert networks for the two tasks use a trainable linear projection layer to transform the complete original task input into a targeted query vector. The only difference lies in the weight distribution learned by the two projection layers after training, due to the different task objectives. This results in their generated query vectors focusing on different parts of the shared features.

[0055] Furthermore, in the multi-expert module, each task (SM prediction task and ET prediction task) is equipped with an independent expert network (Task-Transformer). The core of each expert network is a Transformer block based on multi-head cross-attention. Its complete processing flow is as follows (taking the expert network corresponding to the SM prediction task as an example): First, the query, key, and value required for cross-attention are generated. The original input data of the current task (denoted as Input_task) is transformed through a query linear projection layer to obtain the query tensor Q_task. The shared features F_shared output by the sharing module are transformed through key linear projection layers and value linear projection layers respectively to obtain the key tensor K_share and the value tensor V_share. Then, multi-head cross-attention calculation is performed, and Q_task, K_share, and V_share are input into the multi-head cross-attention layer. This layer calculates the similarity between Q_task and K_share, generates attention weights, and uses these to perform a weighted summation on V_share, realizing the function of retrieving the most relevant information to the current task from the shared features. This step outputs the initial attention features F_cross_attn. Then, the output of the previous step is residually concatenated with the original input of the task, followed by layer normalization. This operation can be represented as: F_norm1= LayerNorm(Input_task+F_cross_attn); This helps stabilize training and preserve the original information.

[0056] Furthermore, the normalized features F_norm1 are input into a feedforward neural network. This network typically consists of two linear transformation layers and an intermediate activation function (such as ReLU), responsible for performing nonlinear transformations and enhancements on the features to improve the model's expressive power. This step outputs the features F_ffn.

[0057] The outputs F_ffn and F_norm1 of the feedforward network are again residually connected and layer normalized to obtain the final output of the expert network, which is expressed as: F_task = LayerNorm(F_norm1 + F_ffn); In this way, the unique characteristics of the task are obtained.

[0058] If it is an expert network serving the SM prediction task, the final output is denoted as F_task_SM∈R^(S×T×D_task).

[0059] Similarly, the expert network serving the ET prediction task has the exact same structure. Its input is the original input data Input_ET corresponding to the ET task and the shared feature F_shared. After the same process described above, the output is the task-specific feature F_task_ET.

[0060] Furthermore, the task-specific features F_task_SM and F_task_ET are both fed into the gating fusion module of the MTMEG module. The input of the gating fusion module is denoted as {F_task_SM, F_task_ET, ...}. Each task corresponds to a lightweight gating network (usually a linear layer + Softmax), namely the SM gating network and the ET gating network.

[0061] Each gated network calculates the contribution weight of each task-specific feature output by the multi-expert module to its corresponding prediction task, i.e., the gating weight g_SM or g_ET of the task-specific feature in the gated network. The calculation of g_SM is expressed as: g_SM=Softmax(W_gate_SM Concat(F_task_SM,F_task_ET)+b_gate_SM); Where W_gate_SM is the learnable weight matrix in the SM gated network, and b_gate_SM is the learnable bias vector in the SM gated network; The calculation of g_ET is expressed as: g_ET=Softmax(W_gate_ET Concat(F_task_SM,F_task_ET)+b_gate_ET); Where W_gate_ET is the learnable weight matrix in the ET gated network, and b_gate_ET is the learnable bias vector in the ET gated network.

[0062] Understandably, W_gate is a learnable weight matrix in the gated network, used to perform a linear transformation on the concatenated features, and its dimension is (2, 2). D_task), where 2 represents the output dimension (corresponding to the weights of the two task-specific features), 2 D_task represents the input dimension (the feature dimension after concatenating F_task_SM and F_task_ET). By learning this matrix, the network can determine how to map the concatenated high-dimensional features onto a two-dimensional weight vector, thereby evaluating the relative importance of the two expert features to the current prediction task. b_gate_SM is the learnable bias vector in the gated network, added to the linear transformation. Its dimension is consistent with the output dimension, used to provide flexibility in model fitting and allow for basic offsets of the weights under specific conditions.

[0063] After obtaining the gating weights for each task-specific feature, the calculated gating weights are used to perform a weighted summation of the task-specific features to obtain the fused enhanced features. Taking the SM gating network as an example, this can be represented as follows: F_fused_SM=g_SM[1] F_task_SM+g_SM[2] F_task_ET; Among them, g_SM[1] and g_SM[2] are the gating weights of task-specific features F_task_SM and F_task_ET in the SM gating network, respectively; Similarly, for the evapotranspiration (ET) prediction task, the corresponding ET gating network follows the exact same computation process as the SM gating network, but uses its own independent parameters and generates specific fusion weights for ET prediction.

[0064] It is understandable that the input, calculation steps, and formulas of the SM-gated network and the ET-gated network are completely identical. However, the two gating networks have their own independent learnable parameter W_gate_. and b_gate_ During training, different weight allocation strategies will be learned. Furthermore, although the inputs to the two gated networks are the same, their parameter matrices W_gate_ The dimensional design ensures that different positions in the transformed vector correspond to the importance of different task features. After Softmax, g_SM and g_ET will have different weight distributions, reflecting the respective tasks' preferences for different expert features.

[0065] This mechanism can adaptively enhance the flow of information between positively correlated tasks and suppress negative migration.

[0066] Preferred, such as Figure 4As shown, each gated network receives two inputs: all task-specific features (such as F_task_SM, F_task_ET) from the multi-expert module output, and the original context information, which is the preprocessed input tensor (or a shallow representation thereof) from the MTMEG module. This provides the gated network with the most direct, untransformed global context, assisting it in making more accurate feature fusion decisions. Taking the SM gated network as an example, feature concatenation and integration are performed first. The two inputs are integrated. In this embodiment, the features output by the multi-expert module are first concatenated, and then the concatenated features are fused again with the original input (or the original input projected through a lightweight linear layer) (e.g., concatenated again or added) to form the comprehensive input feature X_gate_SM of the gated network, as shown below: X_combined=Concat(F_task_SM,F_task_ET); X_gate_SM=Concat(X_combined,Projection(Input)); Here, Projection represents an optional linear projection layer.

[0067] Furthermore, the SM-gated network calculates weights based on its comprehensive input features X_gate_SM, as follows: g_SM=Softmax(W_gate_SM·X_gate_SM + b_gate_SM); Similarly, the ET-gated network uses its own parameters and the combined input features X_gate_ET to calculate the weights, as follows: g_ET=Softmax(W_gate_ET·X_gate_ET+b_gate_ET); Thus, based on the obtained weights, the subsequent feature fusion is performed according to the steps described above.

[0068] The output of the gating network corresponding to the prediction task is the final enhanced feature tensor F_fused∈R^(S×T×D_fused) for that prediction task. This feature not only preserves the temporal patterns extracted from the long sequence, but also encodes the hydrological process interaction information between multiple tasks.

[0069] Understandably, the shared module captures hydrological process information on the individual long-term dependence of soil moisture and evapotranspiration, as well as their mutual influence, through the Share Transformer. However, the extracted features are mostly general features across different variables, which may interfere with the deep learning of variable features by independent task output heads. To enable different output heads to learn richer corresponding variable features, this embodiment designs a cross-task expert multi-gating mechanism. The cross-task expert module in this mechanism uses a Cross-Transformer, responsible for cross-domain queries between shared features and individual tasks. The Cross-Transformer originates from the Task Query Transformer, which aims to exchange information across different tasks, while the former focuses on extracting information applicable to specific tasks from shared features. In the Cross-Transformer, this embodiment does not perform positional encoding on the input. Because the input features are already encoded in the shared module, there is no need to repeat the operation to avoid increasing computational overhead. In addition, repeated use of positional encoding may introduce noise, which will cause greater interference to tasks that need to handle complex uncertainties, such as multi-day forecasts.

[0070] The Cross-Transformer linearly transforms the task-specific raw input data into a query, and interacts with the key-value pairs after linear transformation using a multi-head cross-attention mechanism with shared features. The mathematical expression for each head is as follows: ; in, and The first The query tensor, key tensor, and value tensor of each attention head. This represents the current task's inquiry into shared features from its own perspective. Different heads use different projection parameters to extract different aspects or attributes from the raw data. It is an index representation of shared features, used for queries Matching is performed. Each head has a different key projection matrix, which means that shared features are indexed and divided into multiple different key information patterns for each head to query. This refers to the content or values ​​to be aggregated within the shared features. The attention weights will ultimately affect... The above generates the output. Each head has an independent value projection matrix, allowing each head to extract and reassemble different types of content information from shared features.

[0071] Features extracted through multi-head cross-attention reduce general features applicable to all tasks compared to shared features, while increasing task-specific information. To further enhance the task-specific nature of the features, this embodiment allows each expert to serve only one task. In this embodiment, soil moisture and evapotranspiration forecasts are performed simultaneously, thus using only two experts. The expert serving soil moisture outputs features rich in soil moisture-specific information, while the expert serving evapotranspiration forecasts also outputs features rich in evapotranspiration-specific information. The second information interaction is the interaction between the task and the shared features, aiming to generate features rich in specific information for each task and also containing inter-task interaction information through the expert network.

[0072] This embodiment provides a separate gating network for each task. These gating networks are simply linear transformations of the input and include a softmax layer, as shown below: ; This represents the gating network corresponding to each task. Each task-specific feature is represented by a gating network. The purpose of using gating networks is to flexibly adjust the participation of different experts in the features used by each prediction head, which is ideal for parameter interaction in multi-task learning. During the prediction phase, the gating network adds feature information from other related tasks to the output head of each task based on the closeness of the task relationship. In this embodiment, soil moisture and evapotranspiration are considered as a combination of tasks with positive transfer relationships. Therefore, the gating mechanism adds a proportion of relevant evapotranspiration information to the output head features of soil moisture prediction. This helps the output head of soil moisture to learn the information that may affect soil moisture from evapotranspiration data, thereby improving prediction accuracy and reducing prediction uncertainty. If there are tasks with negative transfer relationships in the task combination being processed, the gating mechanism can maximize the separation of negative tasks. When the output head makes multi-day forecasts, the gating mechanism can simulate variable interactions that approximate real atmospheric conditions, thereby helping the model learn the uncertainty of their mutual influence and improving the accuracy and stability of multi-day forecasts. This completes the third information interaction, which is a reinforcement interaction between features specific to different tasks.

[0073] Furthermore, after the MTMEG module outputs the enhanced feature F_fused, it is input into the backbone prediction network. The backbone prediction network is based on LSTM and can be a standard LSTM, EDLSTM, FAMLSTM, or a hybrid LSTM-Transformer model. The LSTM unit uses its gating mechanism to further process the enhanced feature, learn its internal temporal dynamics, and finally output the predicted value sequence.

[0074] Understandably, if the enhanced feature F_fused_SM corresponding to the SM prediction task is input into the backbone prediction network, the backbone prediction network will output an SM prediction value sequence, including the prediction value sequence for each grid point for the next 1 day, 3 days, or 7 days. For example, for the SM task, the output prediction value sequence Y_pred_SM∈R^(S×3) represents the prediction value sequence for SM prediction for the next 3 days; similarly, the ET prediction value sequence Y_pred_ET can be obtained.

[0075] Furthermore, in this embodiment, the steps for model training and performance evaluation include: In the training configuration, the loss function is the multi-task mean squared error loss, expressed as: Loss=α MSE_SM+(1-α) MSE_ET; Here, MSE_SM and MSE_ET represent the mean squared error losses of the model on the soil moisture (SM) prediction task and the evapotranspiration (ET) prediction task, respectively. They quantify the prediction errors of the model on the SM and ET tasks and are the direct targets driving the optimization of model parameters. α is a hyperparameter used to balance the relative importance of the SM and ET tasks in the total loss. In this embodiment, α=0.5 means that the losses of the two tasks are given completely equal weights and simply averaged. In hydrological multi-task prediction, SM and ET are considered equally important, and model training aims to optimize these two objectives simultaneously and in a balanced manner.

[0076] The optimizer used is the Adam optimizer, with an initial learning rate of 0.001.

[0077] Early stopping (Patience=10) is used to prevent overfitting.

[0078] The evaluation process specifically includes: This embodiment uses the Landbench reanalysis dataset to evaluate the MTMEG proposed in this embodiment. To verify its effectiveness, this embodiment compares the performance of traditional Long Short-Term Memory (LSTM) networks and several state-of-the-art LSTM-based models in the hydrological field, including FAMLSTM, EDLSTM, SAMLSTM, and LSTM-Transformer models, with and without the MTMEG module for feature enhancement.

[0079] Predictions were made on an independent 2020 test set. For each grid point and each lead time (1 / 3 / 7 days), three metrics were calculated between the predicted value sequence Y_pred and the true value sequence Y_true: root mean square error (RMSE), coefficient of determination (R²), and Kling-Gupta efficiency coefficient (KGE). RMSE measures absolute error. The coefficient of determination measures linear correlation. The Kling-Gupta efficiency coefficient comprehensively evaluates the correlation, bias, and variability fit.

[0080] Furthermore, it is understood that this embodiment proposes an MTMEG feature extraction module, aiming to improve the multi-day advance prediction performance of LSTM-based deep learning networks in hydrological variable prediction. To verify the practicality and generalization of the MTMEG module, this embodiment selects three benchmark models to compare the prediction performance before and after adding MTMEG. These three benchmark models represent three mainstream directions for improving LSTM in the hydrological field: enhancing feature selection capabilities by adding an attention mechanism to the LSTM model (FAMLSTM); extending the functionality of LSTM by using an encoder-decoder structure (EDLSTM); and forming a hybrid model by combining LSTM with other models (LSTM-Transformer).

[0081] It is worth noting that in existing technologies, attention mechanisms are added to balance the importance of different features, thereby enhancing the ability of LSTM to capture key features. Attention mechanisms can be used as a preprocessing step to enhance the feature representation of data, or as a post-processing step to refine the model's predictive output. The baseline model used in the backbone prediction network of this embodiment is the Feedforward Attention-Long Short-Term Memory (FAM-LSTM) model. This model leverages FAM's ability to extract key features when processing large amounts of information, dynamically weighting temporal features to improve LSTM's ability to capture key temporal features, thus enhancing the model's performance in multi-step prediction. The specific operation flow and model architecture of FAM-LSTM are existing technologies, and this embodiment does not modify them.

[0082] Furthermore, this embodiment uses Kling-Gupta Efficiency (KGE), coefficient of determination (R²), and root mean square error (RMSE) to evaluate the model's predictive performance.

[0083] The bias measures the degree of linear correlation between model predictions and observed values. The calculation formula is as follows: ; in, These are observed values; These are model predictions; It is the average of the observed values, i.e. N is the number of data points.

[0084] and In contrast, KGE is a comprehensive evaluation metric, with values ​​ranging from -∞ to 1.0. KGE quantifies the consistency between model predictions and observed values ​​by considering the ratio of correlation, bias, and coefficient of variation, thus measuring the model's predictive accuracy, reliability, and directionality. The KGE calculation formula is as follows: ; in, It is a linear correlation between simulated and observed values. and It is the standard deviation between the simulated value and the observed value. and It is the average of the simulated value and the observed value.

[0085] RMSE (Real-Time Error Correction) provides a clear indication of the accuracy of model predictions. A smaller RMSE value indicates a smaller discrepancy between the model's predictions and the actual observed values. The formula for calculating RMSE is as follows: ; in, These are observed values; These are model predictions. It refers to the number of samples.

[0086] In the model testing phase, this embodiment uses global data from the entire year of 2020 to test the model's performance. The model's prediction accuracy for 1, 3, and 7 days in advance is comprehensively evaluated using three metrics.

[0087] Specifically, to verify the effectiveness and generalization ability of the MTMEG feature extraction module, this embodiment compares the performance of three advanced prediction models (FAM-LSTM, ED-LSTM, and LSTM-Transformer) and a simple two-layer LSTM model before and after applying MTMEG. The evaluation includes global predictions of soil moisture and evapotranspiration for 1 day, 3 days, and 7 days. Because MTMEG employs a multi-task learning approach, the baseline model without MTMEG needs to be trained separately for each task (SM and ET) for effective comparison.

[0088] The model is trained using the state-of-the-art Adam optimization algorithm with a learning rate of 0.001. This embodiment selects MSE as the loss function. The training batch size is set to 64, and the number of epochs is set to 1000. As a regularization measure, this embodiment employs early stopping, meaning training stops if there is no improvement after a certain number of epochs. This threshold, called "patience," was set to 10 after experimentation. During the experiments, to ensure rigor, this embodiment performed five tests with different model parameter initializations for all experiments, and the average of the five evaluation metrics was taken as the final result.

[0089] When calculating the multi-task loss, this embodiment uses a simple average summation method that combines the losses from both soil moisture and evapotranspiration tasks. This is because experiments have shown that using some loss weighting algorithms, such as gradient normalization, results in very small improvements in model performance while incurring significant computational resource consumption.

[0090] Furthermore, this embodiment conducted ablation experiments, specifically an ablation study of MTMEG. This study aimed to examine the effectiveness of the designed components (i.e., Share-Transformer, Task-Transformer, and gating mechanism) by removing each component from the model each time and comparing the prediction accuracy.

[0091] Furthermore, in this embodiment, ablation experiments were conducted on the MTMEG components on the LandBench dataset, and the results are shown in Table 2: Table 2

[0092] This embodiment first evaluates the role of the gating mechanism in the MTMEG module. To this end, this embodiment replaces the adaptive weight allocation of the gating mechanism with average weights to process all expert outputs. The results are shown in rows 1 and 2 of Table 2. After removing the gating mechanism, the model's R² and KGE metrics decreased by 0.021 and 0.053 respectively, while the RMSE increased by 0.0019. This indicates that the gating mechanism can improve model performance by assigning more relevant weights when fusing the final expert network outputs. Next, to verify the effectiveness of the Share-Transformer in data sharing, this embodiment replaces the Share-Transformer with a Multilayer Perceptron (MLP) architecture while keeping other modules unchanged. The MLP processes the data from both tasks through linear layers and non-linear activation functions, and then inputs the data into a self-attention mechanism layer for data interaction. The comparison results in rows 1 and 3 of Table 2 show that all three performance metrics decreased after removing the Share-Transformer. This demonstrates the advantages of Share-Transformer in information sharing. Without it to fuse input data from different input tasks and capture long-term dependencies, the model's predictive performance would be affected.

[0093] Finally, to verify the importance of the Task-Transformer, this embodiment also replaced the Task-Transformer with an MLP architecture, while keeping other modules unchanged. In Table 2, the comparison results in rows 1 and 4 show a significant difference, indicating that the Task-Transformer plays an irreplaceable role in the MTMEG module. Without its help in further filtering and highlighting task-related features from shared information, and completing the final learning and capture of time-series data, model performance would drop significantly. In summary, these experiments verify that the gating mechanism, Share-Transformer, and Task-Transformer play a crucial role in improving the overall performance of the MTMEG module.

[0094] Furthermore, this embodiment uses global land as the test object, simultaneously predicting soil moisture (SM) and evapotranspiration (ET). By comparing and analyzing the prediction performance of different deep learning models before and after using the MTMEG module, this embodiment verifies the compatibility of the MTMEG module with existing models and its effect on improving the model's generalization ability (e.g., Figure 5 and Figure 6 (As shown). For ease of description, this embodiment refers to the model without the MTMEG module as the "original model" and the model with the MTMEG module as the "processed model". Figure 5The results show a comparison of the root mean square error (RMSE) of the original and processed models in predicting global SM and ET at 1, 3, and 7 days' advance notice. It can be seen that all models achieve lower average predicted RMSEs after MTMEG processing. Although the overall trend increases with increasing lead time, the processed models still have lower RMSEs than the original models in each forecast period. Furthermore, the RMSE distribution of the processed models is more concentrated globally, indicating that MTMEG not only improves the model's prediction accuracy but also enhances its generalization ability, resulting in a comprehensive improvement in its global prediction performance.

[0095] Figure 6 Further analysis using R2 and KGE metrics revealed the performance improvements of the treated models in predicting soil moisture (SM) and evapotranspiration (ET). The results demonstrated the significant advantages of the treated models in simulating both linear and nonlinear changes in variables and in reducing the impact of outliers, particularly in improving long-term prediction performance. The R2 evaluation plot showed that the treated models generally exhibited higher linear correlation compared to the original models. Notably, in predictions made one day in advance, the R2 of the treated LSTM, FAMLSTM, EDLSTM, and LSTM-TF models for predicting SM improved by 8.3%, 9.3%, 6.6%, and 3.8%, respectively; while in predictions made seven days in advance, these models achieved R2 improvements of 20.6%, 67.2%, 19.8%, and 14.9%, respectively. This indicates that MTMEG not only improves the model's ability to linearly simulate predictor variables but also increases the magnitude of the improvement with longer prediction times. The KGE evaluation plot highlighted the model's ability to nonlinearly simulate predictor variables, while the KGE metric is susceptible to the influence of outliers. Figure 6The KGE evaluation results for SM and ET in this embodiment show that the processed model significantly outperforms the original model in terms of nonlinear simulation capabilities, especially in predicting ET, where the difference is substantial. For example, in predictions seven days in advance, the improvements of the processed model are 142.7%, 249.4%, 139.8%, and 174.1%, respectively. Similar to the R² trend, in predictions one day in advance, the processed LSTM, FAMLSTM, EDLSTM, and LSTM-TF models improve the KGE of SM predictions by 6.7%, 4.8%, 5.6%, and 6.8%, respectively; while in predictions seven days in advance, these improvements are 3.7%, 12.7%, 10.6%, and 14.1%, respectively. This means that the model processed by MTMEG can better capture the linear and nonlinear relationships between variables, thus obtaining more accurate prediction results. At the same time, the significant improvement in KGE means that the processed model can significantly reduce the impact of input outliers on the prediction results, thus having more reliable and stable prediction performance.

[0096] To further verify the practical value and better generalization performance of the model processed by MTMEG, this embodiment selected five major agricultural cities located in different climate zones for analysis. The five locations are: Kaifeng, a major wheat-producing area in China; Modesto, USA, one of the world's largest almond-producing regions; Douala, Cameroon, a major cocoa-growing region in South Africa; Bloemfontein, a major maize-growing region in South Africa; and Peru, one of the world's largest potato-producing regions. These locations represent temperate monsoon climate, Mediterranean climate, tropical monsoon climate, semi-arid climate, and desert climate, respectively. Figure 7 This example demonstrates the RMSE comparison between the processed and original models for predictions on day seven in the selected regions. This embodiment found that in all five locations, the processed models exhibited superior prediction performance, with significantly reduced errors compared to the original models. Particularly in Modesto, located in the Mediterranean region, all processed models showed significantly higher prediction accuracy for SM predictions than the original models, with RMSE reductions ranging from 0.31 to 0.18. Furthermore, in Peru and Bloemfontein, the processed models again demonstrated significant improvements in prediction performance for SM and ET predictions, respectively. In addition, this embodiment found that if one task has higher prediction accuracy (e.g., SM in Douala and ET in Peru), the other task may see significant improvement. This is attributed to MTMEG's multi-task learning, which often improves the model's prediction performance on other tasks by leveraging the higher accuracy of the task performed.

[0097] To delve deeper into the reasons for the performance improvement of the model after MTMEG processing, this embodiment conducted an analysis experiment from the perspective of feature utilization. It was found that the processed model showed significant improvements over the original model in certain locations globally. This embodiment selected North America, North Africa, and Central Asia to conduct correlation analysis between input features and model output. Figure 8 As shown, the processed model exhibits significantly different correlations with different input features in these three regions compared to the original model.

[0098] In North America, the processed model, when predicting ET and SM, reduced its reliance on the t2m feature and placed greater emphasis on the use of the stl2 feature. Furthermore, the processed model, when predicting SM, paid more attention to the impact of ssrd on the prediction results. This change is similar to the findings of Benson et al. in 2021. Benson et al., through their study of heat waves in North America, first confirmed the close relationship between atmospheric temperature and soil moisture, but further discovered a closer potential link between soil moisture and soil temperature in most parts of North America, especially under extreme weather conditions. They found that combining soil moisture and soil temperature analysis often improves the accuracy of future climate predictions. Moreover, controlling evapotranspiration is a necessary condition for soil moisture to feed back to surface climate, and in the high-latitude Northern Hemisphere, solar radiation and soil temperature are key drivers influencing the intensity of this feedback.

[0099] pass Figure 8Correlation analysis in Central Asia reveals that the processed model places greater emphasis on the influence of SSRD and Q when predicting SM. A 2018 study by Lei et al. indicated that surface latent heat flux plays a crucial role in predicting soil moisture in arid or semi-arid regions of Central Asia. Surface latent heat flux interacts with SM, jointly influencing the region's water cycle, energy cycle, and biogeochemical cycle, which are closely related to SM. Solar radiation is a significant driver of surface latent heat flux, further amplifying its impact on these cyclical processes. Specific humidity levels alter soil moisture by affecting atmospheric moisture transport processes (such as large-scale atmospheric circulation, evaporation, precipitation, and convection), especially in arid and semi-arid regions, where specific humidity levels have both direct and indirect effects on soil moisture. Furthermore, Elkouk et al. analyzed soil moisture changes in North Africa and the Sahel under global warming scenarios of 1.5°C, 2°C, and 3°C. The study showed a close relationship between changes in surface air pressure and soil moisture drought under different climate warming scenarios. In particular, higher surface air pressure typically leads to lower soil moisture because lower air pressure reduces precipitation, resulting in decreased soil moisture. These studies confirm that MTMEG, by adjusting the model's utilization of different features, can more closely reflect the interactions between different variables in the real environment, thereby improving the model's predictive performance in a site-specific manner and ultimately achieving high generalization on a global scale.

[0100] Furthermore, in this embodiment, as Figure 9 As shown, this paper presents the annual soil moisture sequence predicted 1 day, 3 days, and 7 days in advance by the LSTM model after MTMEG processing at various locations in North America. The comparison with the original model and actual observation data verifies the higher fit and stability of the MTMEG-processed model in predicting soil moisture and evapotranspiration, especially its significant advantages in predicting complex seasonal variations and long-term forecasts. Figure 9As shown in the left column, whether in the relatively stable soil moisture periods of winter and spring or the rapidly changing and complex periods of summer and autumn, the predictions of the processed model are closer to the actual soil moisture trends compared to the original model. Especially when predicting one day in advance, the processed model shows a high degree of consistency with the observed data during the most volatile soil moisture periods in summer. When predicting three days in advance, the original model's prediction curve shows a significant overestimation, while the processed model, although its prediction accuracy decreases, remains relatively stable overall. This stability is even more pronounced when predicting seven days in advance. The original model's prediction curve deviates significantly from the observed data in most of the seven-day predictions, while the processed model's prediction curve, although flatter, still generally follows the trend of the observed curve. This stability demonstrates that the MTMEG method can effectively alleviate the uncertainty faced by the model in multi-day advance predictions and improve the model's simulation of the seasonal changes of real variables, thereby increasing the reliability of the prediction results. Furthermore, Figure 9 The right column shows that, when predicting evapotranspiration (ET), the processed model also demonstrates greater stability than the original model during periods of significant change. This stability becomes increasingly apparent as the number of days in advance increases, further highlighting the advantages of the processed model.

[0101] Understandably, to more clearly demonstrate the improvement of model prediction stability and accuracy brought about by MTMEG, this embodiment compares the prediction RMSE results of the processed LSTM model and the original LSTM model under five different model parameter initializations globally. Figure 10 As shown in the figure, each solid line represents the RMSE trend of different parameter initialization models when predicting 1 day, 3 days, and 7 days in advance. (a) and (c) show the RMSE curves of the processed model for SM and ET, while (b) and (d) are the results of the original model. The RMSE distribution of the five samples in the original model is large and the differences are obvious. The model prediction performance represented by the ensemble mean is greatly affected by the RMSE distribution, so the prediction stability of the original model is lower than that of the processed model. The RMSE distribution of the processed model is more concentrated and consistent, whether predicting 1 day or 7 days in advance. This indicates that the model after MTMEG processing effectively reduces the uncertainty caused by the interaction between variables and the extended prediction time. In addition, the prediction RMSE of the processed model is lower than that of the original model at all lead times, which means that MTMEG not only improves the prediction stability of the model, but also significantly improves the prediction accuracy.

[0102] It is worth noting that the feature processing module MTMEG proposed in this embodiment aims to help neural network models with LSTM models as the backbone improve their prediction accuracy and stability, especially when forecasting several days in advance. MTMEG uses two types of Transformers, Share Transformer and Task Transformer, to help the LSTM model pay attention to longer-term dependencies in the time series, as this helps the model better handle prediction uncertainty that increases with prediction time. Through correlation analysis of global observation data and input features, this embodiment found that strd has the strongest relationship with SM, and v10 has the strongest relationship with ET. To verify whether MTMEG can improve the model's ability to simulate longer-term relationships, this embodiment performs correlation analysis on the SM and ET results predicted by the processed model and the original model on the last day of the test year 2020, with the most relevant input features of the first two weeks of 2020, and compares this with the correlation between the observed data and these features. Since the processing time step of all LSTM models in this embodiment is 365 days, the correlation analysis only applies to the first two weeks of 2020. Figure 11 As shown in (a), the correlation between SM and strd is illustrated. Although the correlation curves of the processed model and the original model both show similar trends to the observed curves, the correlation curve of the processed model fits the observed curves better and has smaller errors in most time periods. Figure 11 As shown in (b), the correlation curve between the v10 feature and ET is displayed. The processed model also shows results that better match the observed correlation for most periods, especially from day 1 to day 6, where this improvement is particularly significant. These conclusions demonstrate that MTMEG can improve the model's ability to simulate and learn the long-term dependencies between real variables and input features when performing prediction tasks, thereby achieving more accurate multi-day forecasts.

[0103] Furthermore, it's understandable that in the hydrological and atmospheric cycles, all variables exist within a holistic system, meaning they are all influenced by other related variables rather than existing independently. This phenomenon presents a significant challenge to deep learning for prediction tasks in the hydrological field. When training a deep learning model, it's impossible to use all data and features related to the predicted variables as input. This means the model needs to simulate the complex relationships between variables in real atmospheric conditions using only the limited input features, facing uncertainties arising from massive inter-variable interactions. In MTMEG, multi-task learning and three-stage information interaction provide the model with data containing more information about inter-variable interactions. The multiple tasks selected for forecasting must share some common driving factors (physical, biological, or chemical relationships, etc.). Multi-task learning enables the model to learn the potential relationships between its input data based on the relationships between the selected tasks, thus helping the model simulate more complete related hydrological processes and addressing the uncertainties caused by inter-variable interactions. While using related variables as auxiliary features can help the model better understand inter-variable relationships, introducing additional variables requires modeling them at every time step, which is clearly impractical in real-world prediction scenarios.

[0104] In summary, this embodiment proposes a feature processing module (MTMEG) based on multi-task learning and multi-expert gating mechanisms. MTMEG utilizes multi-task learning methods to learn and extract the hydrological processes between soil moisture (SM) and evapotranspiration (ET), and further emphasizes this function through two Transformers (Share-Transformer and Task-Transformer) designed within the module and a multi-expert gating mechanism. Simultaneously, due to the Transformer's direct access to the entire time series, MTMEG can better capture the potential long-term dependencies when processing input data, ultimately outputting features as the backbone of a deep learning model based on LSTM. This provides the model with feature information containing a more complete picture of the interactions between variables and more information on long-term dependencies, thereby helping the model improve its accuracy and stability in predicting SM and ET.

[0105] To verify the effectiveness of MTMEG, this embodiment evaluates the performance of several state-of-the-art hydrological models on the LandBench dataset for predicting SM and ET with lead times of 1 day, 3 days, and 7 days, before and after applying MTMEG. The experimental results first show that the model applying MTMEG exhibits superior predictive generalization across different climate regions compared to the original model, demonstrating MTMEG's ability to help models cope with the predictive challenges posed by climate differences. Second, through correlation analysis between input features and output results, this embodiment finds that the model using MTMEG, when using input features for prediction, better reflects the correlations between variables under the actual local hydrological environment. This result indirectly reveals one of the reasons why MTMEG can improve the model's predictive generalization performance. Next, by displaying the prediction results of models using and without MTMEG under different parameter initializations, it is found that the model treated with MTMEG shows more stable deterministic forecasting ability, with a more concentrated distribution of predicted RMSE at lead times of 1, 3, and 7 days. Furthermore, by analyzing the long-term dependency between model prediction results and input features, this embodiment verifies that the model applying MTMEG can better capture long-term dependency information between variables. In summary, the input data processed by MTMEG contains more complete information on hydrological processes and long-term variable dependencies. This data can help deep learning models based on LSTM reduce prediction uncertainty and learn long-term dependencies, thereby achieving higher prediction accuracy and more stable prediction performance.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting hydrological variables based on multi-task learning and a hybrid expert model, characterized in that, include: Obtain historical datasets, which include first data for multiple days from multiple spatial grid points. The first data includes surface and atmospheric variable data corresponding to multiple prediction tasks, including soil moisture prediction and evapotranspiration prediction. A pre-defined hydrological variable prediction model is trained based on the historical dataset. The hydrological variable prediction model includes a multi-task feature extraction module and a backbone prediction network. The multi-task feature extraction module is used to extract a first feature tensor from the first data based on a multi-head self-attention mechanism. The multi-task feature extraction module configures an expert network and a gating network for each prediction task. The expert network and the gating network are used to perform feature enhancement on the first feature tensor to obtain a second feature tensor. One second feature tensor corresponds to one prediction task. The backbone prediction network is built based on LSTM and is used to output a sequence of predicted values ​​for spatial grid points in future time periods based on the second feature tensor. Based on the trained hydrological variable prediction model, soil moisture and evapotranspiration at multiple spatial grid points are predicted for future time periods.

2. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 1, characterized in that... The first data includes second data for predicting soil moisture and third data for predicting evapotranspiration. The second data includes first soil temperature, first solar radiation, first thermal radiation, precipitation, first air temperature, specific humidity, surface air pressure, clay content, sand content, silt content, soil water holding capacity, and vegetation type. The third data includes first soil temperature, first solar radiation, first thermal radiation, first air temperature, specific humidity, first and second components of wind, surface air pressure, vegetation type, and soil water holding capacity.

3. The hydrological variable prediction method based on multi-task learning and hybrid expert models according to claim 1, characterized in that... After obtaining the historical dataset, the process also includes: Calculate the mean and standard deviation for each variable of the first data to obtain multiple means and multiple standard deviations; Global Z-score standardization is performed on multiple first values ​​based on the first mean and the first standard deviation to obtain multiple standardized first values. The first value is the value of the first variable, the first mean and the first standard deviation are the mean and the standard deviation corresponding to the first variable, and the first variable is the variable in the first data. The first value of each variable in each of the first data is subjected to global Z-score normalization to obtain the normalized historical dataset; The standardized historical dataset is sampled using a sliding window of preset time length to obtain a batch of tensors consisting of multiple first tensors. Each first tensor is generated from one sampling operation, and the shape of the first tensor is as follows: ,in, This indicates the number of spatial grid points in one sampling. This represents the length of the time series corresponding to a spatial grid point. This represents the number of variables corresponding to a single point in time within the time series. The first tensor is used as the input to the hydrological variable prediction model to extract the first feature tensor. The hydrological variable prediction model is trained based on the tensor batch.

4. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 3, characterized in that... The multi-task feature extraction module includes a sharing module, a multi-expert module, and a gated fusion module. The shared module is used to extract the first feature tensor from the first tensor based on a multi-head self-attention mechanism; The multi-expert module includes multiple expert networks. The expert networks are used to extract a third feature tensor from the first feature tensor based on a multi-head cross-attention mechanism. Each third feature tensor corresponds to a prediction task. The expert network includes a first expert network and a second expert network. The first expert network is used for soil moisture prediction, and the second expert network is used for evapotranspiration prediction. The gated fusion module includes multiple gated networks. The gated networks are used to fuse the fourth feature tensor and the fifth feature tensor based on preset weights to obtain the second feature tensor. The fourth feature tensor and the fifth feature tensor are the third feature vectors output by the first expert network and the second expert network, respectively. The gated network includes a first gated network and a second gated network. The first gated network and the second gated network are used to perform the feature fusion based on the weights. The first gated network and the second gated network use different weights when performing the feature fusion.

5. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 4, characterized in that... The step of extracting the first feature tensor from the first tensor based on the multi-head self-attention mechanism includes: Add sine-cosine position encoding to the first tensor to obtain the third tensor; The first feature tensor is extracted from the third tensor based on a preset multi-head self-attention layer to obtain the first feature tensor.

6. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 5, characterized in that... The extraction of the third feature tensor from the first feature tensor based on the multi-head cross-attention mechanism includes: The third tensor is linearly transformed based on a preset first linear projection layer to obtain the query tensor; The first feature tensor is linearly transformed based on the preset second and third linear projection layers respectively to obtain the key tensor and the value tensor. The query tensor, the key tensor, and the value tensor are processed based on a multi-head cross-attention mechanism to obtain the fourth feature tensor or the fifth feature tensor. If the third feature tensor is extracted in the first expert network, the fourth feature tensor is obtained; if the third feature tensor is extracted in the second expert network, the fifth feature tensor is obtained.

7. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 4, characterized in that... The feature fusion of the fourth and fifth feature tensors based on preset weights includes: In the first gating network, the first weight and the second weight are calculated based on the preset first weight matrix, the preset first bias vector, the fourth feature tensor and the fifth feature tensor to obtain the first weight and the second weight, wherein the first weight matrix is ​​a learnable weight matrix and the first bias vector is a learnable bias vector. The fourth feature tensor and the fifth feature tensor are weighted and summed based on the first weight and the second weight to obtain the second feature tensor used for soil moisture prediction. In the second gating network, the third weight and the fourth weight are calculated based on the preset second weight matrix, the preset second bias vector, the fourth feature tensor and the fifth feature tensor to obtain the third weight and the fourth weight. The second weight matrix is ​​a learnable weight matrix and the second bias vector is a learnable bias vector. The fourth feature tensor and the fifth feature tensor are weighted and summed based on the third weight and the fourth weight to obtain the second feature tensor used for evapotranspiration prediction.

8. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 7, characterized in that... The sixth feature tensor is calculated based on the first tensor, the fourth feature tensor, and the fifth feature tensor. The first gating network calculates the first weight and the second weight based on the sixth feature tensor, the first weight matrix, and the first bias vector; The second gating network calculates the third weight and the fourth weight based on the sixth feature tensor, the second weight matrix, and the second bias vector.

9. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 1, characterized in that... The process of training a pre-defined hydrological variable prediction model based on the historical dataset includes: The multi-task mean square error loss function is used as the loss function of the hydrological variable prediction model. The Adam optimizer was used as the optimizer for the hydrological variable prediction model.

10. The hydrological variable prediction method based on multi-task learning and hybrid expert model according to claim 2, characterized in that... The prediction of soil moisture and evapotranspiration at multiple spatial grid points over a future time period includes: Obtain the first data of multiple first spatial grid points in the target area over several consecutive days in real time, and construct a second tensor with the same shape as the first tensor. The second tensor is input into the trained hydrological variable prediction model, and the predicted value sequence for each of the first spatial grid points over the next few days is output. The predicted value sequence includes a soil moisture prediction value sequence and an evapotranspiration prediction value sequence.

Images