Multi-source adaptive transfer learning method for predicting daily runoff in data-scarce river basins
By employing a multi-source adaptive transfer learning method, a prior knowledge base of the source watershed and a transfer branch model of the target watershed are constructed. Combined with a dynamic attention module, the overfitting and adaptability problems of runoff prediction in watersheds with scarce data are solved, achieving high-precision and robust daily runoff prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TAIYUAN UNIVERSITY OF TECHNOLOGY
- Filing Date
- 2026-03-24
- Publication Date
- 2026-06-05
AI Technical Summary
In data-scarce watersheds, existing runoff prediction methods suffer from problems such as insufficient training samples leading to difficulties in training deep learning models and easy overfitting, single transfer strategies being prone to negative transfer, and traditional static multi-source fusion being unable to adapt to the dynamic evolution of hydrological processes.
A multi-source adaptive transfer learning method is adopted. By constructing a prior knowledge base of the source watershed and a transfer branch model of the target watershed, and combining a dynamic attention module and weight calculation, adaptive prediction of the target watershed is achieved.
It improves the accuracy and robustness of daily runoff prediction in data-scarce watersheds, solves the overfitting problem of deep learning models under small sample conditions, and achieves dynamic adaptation to hydrological conditions, thereby improving the accuracy and generalization ability of prediction.
Smart Images
Figure CN122154838A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hydrological forecasting technology, and in particular to a method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data. Background Technology
[0002] Accurate runoff forecasting is a core technological support for the scientific allocation of water resources, precise decision-making in flood control and drought relief, and routine monitoring of ecological flow. However, a common technical bottleneck remains in hydrological engineering practice: a large number of watersheds worldwide are data-scarce, severely restricting the practical application of runoff forecasting technology. These watersheds are constrained by multiple factors: monitoring stations were established relatively late; remote locations make operation and maintenance difficult; frequent equipment failures or management oversights lead to gaps in observation sequences, ultimately resulting in only short-sequence measured runoff data and limited historical samples, failing to comprehensively characterize the spatiotemporal evolution and strong nonlinear characteristics of the rainfall-runoff response process. With insufficient observational information, the difficulty of identifying watershed runoff generation and concentration mechanisms increases sharply, and the uncertainty of model parameter constraints is significantly amplified, directly leading to a substantial decrease in the accuracy and robustness of runoff forecasting. Therefore, constructing reliable runoff forecasting methods suitable for areas with no or insufficient data has become a core challenge in overcoming the barriers to the promotion of hydrological forecasting technology and meeting practical engineering needs.
[0003] Currently, daily runoff prediction methods mainly include hydrological models based on physical processes and data-driven prediction models. The former has a clear mechanism and strong interpretability, but its modeling heavily relies on relatively complete watershed underlying surface attribute information and high-precision meteorological driving data. For data-scarce watersheds, such basic data is often difficult to obtain systematically, making model parameter calibration complex and unstable, further exacerbating the uncertainty of prediction results. In recent years, with the development of artificial intelligence technology, deep learning models, represented by Transformer, have received widespread attention due to their outstanding sequence modeling capabilities. The Transformer architecture, with its multi-head self-attention mechanism at its core, can accurately capture the complex nonlinear mapping relationship between meteorological driving factors and runoff response, as well as the long- and short-term dependence characteristics of hydrological processes. Theoretically, it outperforms traditional recurrent neural network models in long-range dependence modeling. However, Transformer is a typical high-data-demand model, and its performance improvement is highly dependent on large-scale, high-quality training samples. In data-scarce domains, directly training complex Transformer models with limited small sample data can easily lead to overfitting. This manifests as good fitting results during the training phase, but extremely poor generalization ability on independent test sets and prediction failure in real-world business scenarios, failing to meet the stringent requirements of prediction accuracy and stability for engineering applications.
[0004] To overcome data bottlenecks, transfer learning has been introduced into the field of hydrological forecasting. The core idea is to leverage the hydrological knowledge of data-rich watersheds to assist in the modeling and predictive extrapolation of data-scarce watersheds. Nevertheless, existing methods for predicting daily runoff migration still suffer from significant limitations. Firstly, single-source watershed migration carries a high risk. Many studies focus on selecting the single watershed most similar in geographical location or watershed attributes as the source watershed. However, watershed runoff generation and concentration processes are highly complex and diverse, making it difficult to find a source watershed that is highly consistent with the target watershed under different hydrological conditions. When the dominant runoff generation mechanisms of the source and target watersheds differ, knowledge transfer may introduce inappropriate prior constraints and systematic biases, not only failing to improve prediction accuracy but potentially causing a decline in prediction performance. Secondly, static multi-source fusion lacks adaptability. To reduce the uncertainty of single-source watershed migration, some studies employ multi-source migration strategies, but existing multi-source fusion methods largely rely on static weighting schemes based on historical statistical errors. Such methods often overlook the significant dynamic and time-varying characteristics of hydrological processes, meaning that the runoff and runoff behavior of the target basin under different environmental conditions may be more similar to that of different source basins. Static weighting is difficult to characterize the shift in similarity caused by changes in environmental factors over time, and it is also difficult to dynamically adjust the degree of dependence on knowledge of different source basins under non-stationary hydrological conditions, thus limiting further improvements in prediction accuracy and robustness. Summary of the Invention
[0006] To overcome the shortcomings of existing runoff forecasting techniques, such as the difficulty in training deep learning models and their susceptibility to overfitting due to insufficient training samples in watersheds with scarce data, the tendency of single transfer strategies to generate negative transfer, and the difficulty of adapting traditional static multi-source fusion techniques to the dynamic evolution of hydrological processes, this invention provides a multi-source adaptive transfer learning method for predicting daily runoff in watersheds with scarce data.
[0007] This invention provides a method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data, comprising the following steps:
[0008] S1. Construct a prior knowledge base for source basins, select multiple source basins with long-sequence observation data, construct a basic daily runoff prediction model based on the Transformer architecture, and independently train the basic daily runoff prediction model using historical meteorological driving data and measured daily runoff data of each source basin, thereby obtaining multiple basic models corresponding to each source basin.
[0009] S2. Construct a target watershed migration branch model to migrate the multiple basic models to the target watershed where data is scarce; adopt a parameter freezing and differential fine-tuning strategy to freeze the parameters of the Transformer layer in the basic model to retain the general feature extraction capability, and fine-tune the model output layer using only the limited measured daily runoff samples of the target watershed to construct multiple migration branch models for the target watershed.
[0010] S3. Construct a dynamic attention module and calculate weights. Establish a dynamic attention module that receives real-time meteorological and environmental element sequences of the target watershed as input, senses the hydrological environment situation at the current moment, and calculates the dynamic confidence weights of each migration branch model at the current moment.
[0011] S4. Adaptive integration and prediction output: The meteorological data of the target watershed is synchronously input into each migration branch model to obtain preliminary prediction values. The preliminary prediction values are then adaptively weighted and integrated with the dynamic confidence weights to obtain the final daily runoff prediction results for the target watershed.
[0012] Preferably, the sub-step of step S1, which involves constructing the prior knowledge base of the source watershed, is as follows:
[0013] S11. Construct source basin hydrological and meteorological data; select five typical source basins with long-sequence observation data, and for each source basin, construct a spatiotemporal environmental element dataset, which includes the input feature sequence. and measured daily runoff value ;
[0014] S12, Temporal Feature Embedding and Location Encoding: To enable the model to understand the temporal continuity and seasonal periodicity of hydrological processes, the input feature sequence is embedded and location encoded. By performing linear mapping and positional encoding superposition, it is transformed into a latent feature vector E containing temporal information:
[0015]
[0016] In the formula, The input projection matrix is used to map meteorological physical features to a high-dimensional feature space; PE is the location encoding matrix used to mark the time and location of meteorological events such as rainfall, and its calculation formula is as follows:
[0017]
[0018]
[0019] In the formula, pos represents the time step index, which is the number of days lag from the current prediction time; i is the feature dimension index, which ensures that the model can distinguish the different impacts of recent rainfall and long-term rainfall on the current runoff;
[0020] S13. Hydrological response feature extraction based on multi-head self-attention; utilizing the multi-head self-attention mechanism in the Transformer encoder, the long-term and short-term dependencies of meteorological driving factors on daily runoff and their hysteresis response features are captured; firstly, the latent feature vector E is mapped to a query matrix Q, a key matrix K, and a value matrix V:
[0021]
[0022] Then, in the multi-head self-attention mechanism, let the common... The first point of attention, the first Each attention point is based on its corresponding query matrix. Key matrix Sum matrix The attention output is calculated using the following expression:
[0023]
[0024] In the formula, Indicates the first The output of each attention head Indicates the dimension of the key vector;
[0025] The outputs of each attention head are concatenated along the feature dimension, and a linear transformation is performed using the output mapping matrix to obtain the hydrological response feature representation aggregated by the multi-head self-attention mechanism. Its expression is:
[0026]
[0027] In the formula, To output the mapping matrix;
[0028] S14, Nonlinear Transformation and Simulation of Runoff Generation and Confluence Mechanisms; The aggregated hydrological characteristics Z... attn The input is fed into a feedforward neural network, which simulates the complex nonlinear runoff generation and merging processes in a watershed through fully connected layers and nonlinear activation functions.
[0029]
[0030]
[0031] In the formula, Z out This is the output feature representation after feedforward transformation, residual connection, and normalization. and These are the weight matrices of the linear mappings of two layers in a feedforward neural network. and These are the corresponding bias terms;
[0032] S15. Independent training and knowledge consolidation of the source-watershed model; inputting high-level abstract hydrological feature vectors into a long short-term memory network to further extract temporal dependencies and dynamic response features in the rainfall-runoff evolution process; sequentially setting the first and second long short-term memory layers to perform progressive temporal modeling of the feature sequences, obtaining a high-dimensional temporal representation for regression prediction. Based on this, a fully connected regression layer is constructed as the output layer, and the extracted feature vector Z is processed. out Mapped to daily runoff values predicted by the model :
[0033]
[0034]
[0035]
[0036] In the formula, H (1) These are intermediate temporal features extracted from the first long short-term memory layer; This is a high-level temporal representation of the output of the second long short-term memory layer; and These are the weight matrix and bias term of the output layer, respectively; The daily runoff value predicted by the model;
[0037] For each of the five source basins, a mean squared error (MSE) loss function is constructed to quantify the deviation between the daily runoff values predicted by the model and the measured daily runoff. The expression for this function is as follows:
[0038]
[0039] In the formula, For the first The loss function of the model corresponding to each source basin; For the first Number of training samples per source basin; For the first The first source basin Measured daily runoff values for each sample; For the first The first source basin The daily runoff value predicted by the model for each sample;
[0040] Finally, the backpropagation algorithm was used to train the five models independently, and the model parameters were continuously updated iteratively until their respective loss functions converged, thus obtaining five basic models that solidify the rainfall-runoff response patterns and runoff generation and confluence mechanisms of different source watersheds. These models were then used as the source watershed prior knowledge base for subsequent target watershed transfer learning and adaptive weighted fusion.
[0041] Preferably, the sub-step of step S2, which constructs the target watershed migration branch model, is as follows:
[0042] S21. Construction of a small sample dataset for the target watershed; For the target watershed with scarce data to be predicted, collect its short-sequence historical meteorological driving data X. target And measured daily runoff data Y target To simulate a data-scarce scenario and verify the model's few-shot learning ability, 80% of the data from the target watershed was selected as the fine-tuning training set, and the remaining data was used as the test set; the input historical meteorological driving data X was... target Preprocessing is performed using the same standardized parameters as the source watershed to ensure consistency in feature distribution;
[0043] S22, Source Basin Parameter Transfer and Feature Extraction Layer Freezing; Load the 5 basic models trained in step S1 respectively, and freeze their model parameters. The parameters are transferred to the target watershed as initial weights; simultaneously, a parameter freezing strategy is implemented to freeze the parameters of the Transformer encoder layer in the base model. Set to an untrainable state;
[0044] S23, Output Layer Reset and Differential Fine-tuning: The parameters of the fully connected regression output layer of the model are reset or unfrozen, and its parameter set is adjusted accordingly. Set to trainable state; utilize a limited fine-tuning training set within the target watershed. Construct the target watershed loss function Only for output layer parameters Perform gradient updates:
[0045]
[0046] In the formula, To fine-tune the learning rate, it is usually set to a value smaller than the learning rate used in the source domain training, in order to ensure the stability of parameter adjustments; The model predicts runoff under the current parameters;
[0047] S24. Construct a transfer branch model library; perform the above fine-tuning process on the five base models loaded with initial weights from different source watersheds until the loss function of each base model converges on the target watershed training set, thus obtaining five transfer branch models oriented towards the target watershed. .
[0048] Preferably, the sub-steps of step S3, which involves constructing the dynamic attention module and calculating weights, are as follows:
[0049] S31, Construct a dynamic environment perception module architecture; establish a dynamic attention module independent of the transfer branch model, the dynamic attention module aims to perceive the hydrological and meteorological status of the target watershed in real time, and evaluate the applicability of each source watershed branch model accordingly; the dynamic attention module consists of two parts: a feature extraction subnet and a weight mapping subnet.
[0050] S32. Spatiotemporal Environmental Feature Extraction Based on Transformer: A feature extraction subnetwork is used to capture environmental features such as current rainfall intensity and accumulated soil moisture in the target watershed. To maintain consistency with the main prediction model architecture and efficiently capture temporal dependencies, the feature extraction subnetwork adopts a Transformer encoder structure; Input sequence After location encoding and multi-head self-attention layer processing, the hidden state vector containing the hydrological state information at the current moment is extracted. Select The feature vector of the last time step As the environmental situation representation vector of the target watershed at the current moment, its calculation formula is:
[0051]
[0052] S33. Mapping and generation of dynamic confidence weights; transforming environmental situation representation vectors The input is fed into a weighted mapping subnet, which consists of fully connected layers and Softmax activation layers. First, the unnormalized matching score of the migration branch model for each source watershed is calculated for the current environmental situation. :
[0053]
[0054] In the formula, Indicates the first One source basin branch; and These are learnable mapping parameters;
[0055] Then, the Softmax function is used to analyze the unnormalized matching scores. Normalization is performed to generate a dynamic confidence weight vector with a sum of 1. :
[0056]
[0057] In the formula, Indicates the first The source basin branch model at time... The proportion of the predicted results in the final integration;
[0058] S34. The dynamic evolution characteristics of the weights; the dynamic confidence weights With time step The dynamic attention module changes in real time as the hydrological situation evolves. When the input meteorological data indicates a heavy rainfall event in the target watershed, the module assigns higher weights to source watershed branches originating from humid areas and with similar runoff generation mechanisms. Conversely, during the dry season, it tends to rely on source watershed branches with similar baseflow characteristics. Through this mechanism, the prediction logic adaptively switches as the hydrological situation evolves.
[0059] Preferably, the sub-steps of adaptive ensemble, predicted output, and joint optimization in step S4 are as follows:
[0060] S41. Multi-branch parallel simulation; incorporating historical meteorological driving sequences of the target watershed. Parallel input to 5 transfer branch model In this process, each transfer branch model utilizes its fixed source domain Transformer feature extractor and fine-tuned output layer to generate features specific to the current time step. Preliminary daily runoff forecast ,at this time, This represents the assumption that the target watershed completely follows the first... Runoff results derived under the premise of the runoff generation and confluence mechanism of a single source basin;
[0061] S42, Adaptive weighted ensemble; calculate the real-time confidence weights generated by the dynamic attention module in step S33. The five preliminary daily runoff forecasts generated in step S41 are linearly weighted and combined to obtain the final daily runoff forecast for the target watershed. The integrated calculation formula is as follows:
[0062]
[0063] S43. Joint optimization training for the target watershed; based on limited measured daily runoff data from the target watershed. Construct a joint loss function for the target watershed. The final integrated prediction values will be Compared with measured values The mean square error between them is used as the optimization objective:
[0064]
[0065] Gradient descent is employed to jointly train the model end-to-end on the target watershed training set. Regarding the parameter update strategy, during backpropagation, the gradient flow is simultaneously propagated to the dynamic attention module and the output layer of each branch model, synchronously updating the parameters of the attention module. and the fully connected layer parameters of each branch model Meanwhile, the parameters of the Transformer encoder in each branch model remain frozen.
[0066] Compared with existing technologies, the technical solution provided by this invention has the following technical advantages: Addressing the challenges of insufficient training samples in data-scarce watersheds and the poor adaptability of traditional transfer learning strategies, this invention proposes a Transformer-based multi-source adaptive transfer learning method for daily runoff prediction. First, by utilizing parameter freezing and differentiated fine-tuning strategies, it achieves rapid adaptation to the local characteristics of the target watershed while solidifying the general rainfall-runoff patterns of the source watersheds, effectively solving the modeling bottleneck of overfitting in deep learning models under small sample conditions. Furthermore, this invention constructs a dynamic attention mechanism based on the perception of spatiotemporal environmental elements, breaking through the limitations of traditional static weighted fusion. It can perceive the dynamic evolution of the hydrological situation in the target watershed in real time, adaptively select and fuse the optimal source watershed knowledge, thereby achieving accurate spatiotemporal alignment between multi-source prior information and the instantaneous state of the target watershed. This method effectively suppresses negative transfer noise introduced by heterogeneous source watersheds by dynamically adjusting weights, significantly improving the accuracy and robustness of daily runoff prediction in data-scarce watersheds during non-stationary hydrological processes, and significantly enhancing the accuracy and generalization ability of daily runoff prediction. Attached Figure Description
[0067] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0069] Figure 1 This is a structural block diagram of a multi-source adaptive transfer learning method for predicting daily runoff in watersheds with scarce data, as described in a certain embodiment of the present invention. Detailed Implementation
[0071] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.
[0072] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.
[0073] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0074] In one embodiment, such as Figure 1 As shown, a method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data is disclosed, and the steps are as follows:
[0075] S1. Construct a prior knowledge base for source basins, select multiple source basins with long-sequence observation data, construct a basic daily runoff prediction model based on the Transformer architecture, and independently train the basic daily runoff prediction model using historical meteorological driving data and measured daily runoff data for each source basin, thereby obtaining multiple basic models corresponding to each source basin; its sub-steps are as follows:
[0076] S11. Construct source basin hydrological and meteorological data; select five typical source basins with long-sequence observation data, and for each source basin, construct a spatiotemporal environmental element dataset, which includes the input feature sequence. and measured daily runoff value ;
[0077] S12, Temporal Feature Embedding and Location Encoding: To enable the model to understand the temporal continuity and seasonal periodicity of hydrological processes, the input feature sequence is embedded and location encoded. By performing linear mapping and positional encoding superposition, it is transformed into a latent feature vector E containing temporal information:
[0078]
[0079] In the formula, The input projection matrix is used to map meteorological physical features to a high-dimensional feature space; PE is the location encoding matrix used to mark the time and location of meteorological events such as rainfall, and its calculation formula is as follows:
[0080]
[0081]
[0082] In the formula, pos represents the time step index, which is the number of days lag from the current prediction time; i is the feature dimension index, which ensures that the model can distinguish the different impacts of recent rainfall and long-term rainfall on the current runoff;
[0083] S13. Hydrological response feature extraction based on multi-head self-attention; utilizing the multi-head self-attention mechanism in the Transformer encoder, the long-term and short-term dependencies of meteorological driving factors on daily runoff and their hysteresis response features are captured; firstly, the latent feature vector E is mapped to a query matrix Q, a key matrix K, and a value matrix V:
[0084]
[0085] Then, in the multi-head self-attention mechanism, let the common... The first point of attention, the first Each attention point is based on its corresponding query matrix. Key matrix Sum matrix The attention output is calculated using the following expression:
[0086]
[0087] In the formula, Indicates the first The output of each attention head Indicates the dimension of the key vector;
[0088] The outputs of each attention head are concatenated along the feature dimension, and a linear transformation is performed using the output mapping matrix to obtain the hydrological response feature representation aggregated by the multi-head self-attention mechanism. Its expression is:
[0089]
[0090] In the formula, To output the mapping matrix;
[0091] S14, Nonlinear Transformation and Simulation of Runoff Generation and Confluence Mechanisms; The aggregated hydrological characteristics Z... attn The input is fed into a feedforward neural network, which simulates the complex nonlinear runoff generation and merging processes in a watershed through fully connected layers and nonlinear activation functions.
[0092]
[0093]
[0094] In the formula, Z out This is the output feature representation after feedforward transformation, residual connection, and normalization. and These are the weight matrices of the linear mappings of two layers in a feedforward neural network. and These are the corresponding bias terms;
[0095] S15. Independent training and knowledge consolidation of the source-watershed model; inputting high-level abstract hydrological feature vectors into a long short-term memory network to further extract temporal dependencies and dynamic response features in the rainfall-runoff evolution process; sequentially setting the first and second long short-term memory layers to perform progressive temporal modeling of the feature sequences, obtaining a high-dimensional temporal representation for regression prediction. Based on this, a fully connected regression layer is constructed as the output layer, and the extracted feature vector Z is processed. out Mapped to daily runoff values predicted by the model :
[0096]
[0097]
[0098]
[0099] In the formula, H (1) These are intermediate temporal features extracted from the first long short-term memory layer; This is a high-level temporal representation of the output of the second long short-term memory layer; and These are the weight matrix and bias term of the output layer, respectively; The daily runoff value predicted by the model;
[0100] For each of the five source basins, a mean squared error (MSE) loss function is constructed to quantify the deviation between the daily runoff values predicted by the model and the measured daily runoff. The expression for this function is as follows:
[0101]
[0102] In the formula, For the first The loss function of the model corresponding to each source basin; For the first Number of training samples per source basin; For the first The first source basin Measured daily runoff values for each sample; For the first The first source basin The daily runoff value predicted by the model for each sample;
[0103] Finally, the backpropagation algorithm was used to train the five models independently, and the model parameters were continuously iterated and updated until their respective loss functions converged, thus obtaining five basic models that solidify the rainfall-runoff response patterns and runoff generation and confluence mechanisms of different source watersheds. These models were then used as the source watershed prior knowledge base for subsequent target watershed transfer learning and adaptive weighted fusion.
[0104] S2. Construct a target watershed migration branch model to migrate the multiple base models to the data-scarce target watershed; employ a parameter freezing and differential fine-tuning strategy, freezing the parameters of the Transformer layer in the base model to retain general feature extraction capabilities, and fine-tuning the model output layer using only limited measured daily runoff samples from the target watershed, thus constructing multiple migration branch models oriented towards the target watershed; its sub-steps are as follows:
[0105] S21. Construction of a small sample dataset for the target watershed; For the target watershed with scarce data to be predicted, collect its short-sequence historical meteorological driving data X. target And measured daily runoff data Y target To simulate a data-scarce scenario and verify the model's few-shot learning ability, 80% of the data from the target watershed was selected as the fine-tuning training set, and the remaining data was used as the test set; the input historical meteorological driving data X was... target Preprocessing is performed using the same standardized parameters as the source watershed to ensure consistency in feature distribution;
[0106] S22, Source Basin Parameter Transfer and Feature Extraction Layer Freezing; Load the 5 basic models trained in step S1 respectively, and freeze their model parameters. The parameters are transferred to the target watershed as initial weights; simultaneously, a parameter freezing strategy is implemented to freeze the parameters of the Transformer encoder layer in the base model. Set to an untrainable state;
[0107] S23, Output Layer Reset and Differential Fine-tuning: The parameters of the fully connected regression output layer of the model are reset or unfrozen, and its parameter set is adjusted accordingly. Set to trainable state; utilize a limited fine-tuning training set within the target watershed. Construct the target watershed loss function Only for output layer parameters Perform gradient updates:
[0108]
[0109] In the formula, To fine-tune the learning rate, it is usually set to a value smaller than the learning rate used in the source domain training, in order to ensure the stability of parameter adjustments; The model predicts runoff under the current parameters;
[0110] S24. Construct a transfer branch model library; perform the above fine-tuning process on the five base models loaded with initial weights from different source watersheds until the loss function of each base model converges on the target watershed training set, thus obtaining five transfer branch models oriented towards the target watershed. ;
[0111] S3. Construct a dynamic attention module and calculate weights. Establish a dynamic attention module that receives real-time meteorological and environmental element sequences from the target watershed as input, senses the current hydrological environment situation, and calculates the dynamic confidence of each migration branch model at the current moment.
[0112] Weights; its sub-steps are:
[0113] S31, Construct a dynamic environment perception module architecture; establish a dynamic attention module independent of the transfer branch model, the dynamic attention module aims to perceive the hydrological and meteorological status of the target watershed in real time, and evaluate the applicability of each source watershed branch model accordingly; the dynamic attention module consists of two parts: a feature extraction subnet and a weight mapping subnet.
[0114] S32. Spatiotemporal Environmental Feature Extraction Based on Transformer: A feature extraction subnetwork is used to capture environmental features such as current rainfall intensity and accumulated soil moisture in the target watershed. To maintain consistency with the main prediction model architecture and efficiently capture temporal dependencies, the feature extraction subnetwork adopts a Transformer encoder structure; Input sequence After location encoding and multi-head self-attention layer processing, the hidden state vector containing the hydrological state information at the current moment is extracted. Select The feature vector of the last time step As the environmental situation representation vector of the target watershed at the current moment, its calculation formula is:
[0115]
[0116] S33. Mapping and generation of dynamic confidence weights; transforming environmental situation representation vectors The input is fed into a weighted mapping subnet, which consists of fully connected layers and Softmax activation layers. First, the unnormalized matching score of the migration branch model for each source watershed is calculated for the current environmental situation. :
[0117]
[0118] In the formula, Indicates the first One source basin branch; and These are learnable mapping parameters;
[0119] Then, the Softmax function is used to analyze the unnormalized matching scores. Normalization is performed to generate a dynamic confidence weight vector with a sum of 1. :
[0120]
[0121] In the formula, Indicates the first The source basin branch model at time... The proportion of the predicted results in the final integration;
[0122] S34. The dynamic evolution characteristics of the weights; the dynamic confidence weights With time step The dynamic attention module changes in real time as the hydrological situation changes. When the input meteorological data indicates that a heavy rainfall event has occurred in the target watershed, the dynamic attention module will give higher weight to the source watershed branches that originate from the humid area and have similar runoff generation mechanisms. Conversely, during the dry season, it tends to rely on the source watershed branches with similar baseflow characteristics. Through this mechanism, the prediction logic is adaptively switched as the hydrological situation evolves.
[0123] S4. Adaptive Integration and Prediction Output: Meteorological data from the target watershed is synchronously input into each migration branch model to obtain preliminary prediction values. These preliminary prediction values are then adaptively weighted and integrated using the dynamic confidence weights to obtain the final daily runoff prediction result for the target watershed. The sub-steps are as follows:
[0124] S41. Multi-branch parallel simulation; incorporating historical meteorological driving sequences of the target watershed. Parallel input to 5 transfer branch model In this process, each transfer branch model utilizes its fixed source domain Transformer feature extractor and fine-tuned output layer to generate features specific to the current time step. Preliminary daily runoff forecast ,at this time, This represents the assumption that the target watershed completely follows the first... Runoff results derived under the premise of the runoff generation and confluence mechanism of a single source basin;
[0125] S42, Adaptive weighted ensemble; calculate the real-time confidence weights generated by the dynamic attention module in step S33. The five preliminary daily runoff forecasts generated in step S41 are linearly weighted and combined to obtain the final daily runoff forecast for the target watershed. The integrated calculation formula is as follows:
[0126]
[0127] S43. Joint optimization training for the target watershed; based on limited measured daily runoff data from the target watershed. Construct a joint loss function for the target watershed. The final integrated prediction values will be Compared with measured values The mean square error between them is used as the optimization objective:
[0128]
[0129] Gradient descent is employed to jointly train the model end-to-end on the target watershed training set. Regarding the parameter update strategy, during backpropagation, the gradient flow is simultaneously propagated to the dynamic attention module and the output layer of each branch model, synchronously updating the parameters of the attention module. and the fully connected layer parameters of each branch model Meanwhile, the parameters of the Transformer encoder in each branch model remain frozen.
[0130] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.
Claims
1. A method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data, characterized in that, The steps are as follows: S1. Construct a prior knowledge base for source basins, select multiple source basins with long-sequence observation data, construct a basic daily runoff prediction model based on the Transformer architecture, and independently train the basic daily runoff prediction model using historical meteorological driving data and measured daily runoff data of each source basin, thereby obtaining multiple basic models corresponding to each source basin. S2. Construct a target watershed migration branch model to migrate the multiple basic models to the target watershed where data is scarce; A parameter freezing and differential fine-tuning strategy was adopted. The parameters of the Transformer layer in the base model were frozen to retain the general feature extraction capability. The output layer of the model was fine-tuned using only a limited number of measured daily runoff samples from the target watershed, thus constructing multiple transfer branch models for the target watershed. S3. Construct a dynamic attention module and calculate weights. Establish a dynamic attention module that receives real-time meteorological and environmental element sequences of the target watershed as input, senses the hydrological environment situation at the current moment, and calculates the dynamic confidence weights of each migration branch model at the current moment. S4. Adaptive integration and prediction output: The meteorological data of the target watershed is synchronously input into each migration branch model to obtain preliminary prediction values. The preliminary prediction values are then adaptively weighted and integrated with the dynamic confidence weights to obtain the final daily runoff prediction results for the target watershed.
2. The method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data according to claim 1, characterized in that, The sub-steps of step S1, which involves constructing the source watershed prior knowledge base, are as follows: S11. Construct source basin hydrological and meteorological data; select five typical source basins with long-sequence observation data, and for each source basin, construct a spatiotemporal environmental element dataset, which includes the input feature sequence. and measured daily runoff value ; S12, Temporal Feature Embedding and Location Encoding: To enable the model to understand the temporal continuity and seasonal periodicity of hydrological processes, the input feature sequence is embedded and location encoded. By performing linear mapping and positional encoding superposition, it is transformed into a latent feature vector E containing temporal information: In the formula, The input projection matrix is used to map meteorological physical features to a high-dimensional feature space; PE is the location encoding matrix used to mark the time and location of meteorological events, and its calculation formula is as follows: In the formula, pos represents the time step index, which is the number of days lag from the current prediction time; i is the feature dimension index, which ensures that the model can distinguish the different impacts of recent rainfall and long-term rainfall on the current runoff; S13. Hydrological response feature extraction based on multi-head self-attention; utilizing the multi-head self-attention mechanism in the Transformer encoder, the long-term and short-term dependencies of meteorological driving factors on daily runoff and their hysteresis response features are captured; firstly, the latent feature vector E is mapped to a query matrix Q, a key matrix K, and a value matrix V: Then, in the multi-head self-attention mechanism, let the common... The first point of attention, the first Each attention point is based on its corresponding query matrix. Key matrix Sum matrix The attention output is calculated using the following expression: In the formula, Indicates the first The output of each attention head Indicates the dimension of the key vector; The outputs of each attention head are concatenated along the feature dimension, and a linear transformation is performed using the output mapping matrix to obtain the hydrological response feature representation aggregated by the multi-head self-attention mechanism. Its expression is: In the formula, To output the mapping matrix; S14, Nonlinear Transformation and Simulation of Runoff Generation and Confluence Mechanisms; The aggregated hydrological characteristics Z... attn The input is fed into a feedforward neural network, which simulates the complex nonlinear runoff generation and merging processes in a watershed through fully connected layers and nonlinear activation functions. In the formula, Z out This is the output feature representation after feedforward transformation, residual connection, and normalization. and These are the weight matrices of the linear mappings of two layers in a feedforward neural network. and These are the corresponding bias terms; S15. Independent training and knowledge consolidation of source basin models; inputting high-level abstract hydrological feature vectors into a long short-term memory network to further extract the temporal dependencies and dynamic response features in the precipitation-runoff evolution process; By sequentially setting up a first long short-term memory layer and a second long short-term memory layer, progressive temporal modeling of the feature sequence is performed to obtain a high-dimensional temporal representation for regression prediction. Based on this, a fully connected regression layer is constructed as the output layer, and the extracted feature vector Z is processed. out Mapped to daily runoff values predicted by the model : In the formula, H (1) These are intermediate temporal features extracted from the first long short-term memory layer; This is a high-level temporal representation of the output of the second long short-term memory layer; and These are the weight matrix and bias term of the output layer, respectively; The daily runoff value predicted by the model; For each of the five source basins, a mean squared error loss function is constructed to quantify the deviation between the daily runoff values predicted by the model and the measured daily runoff. The expression for this function is as follows: In the formula, For the first The loss function of the model corresponding to each source watershed; For the first Number of training samples per source basin; For the first The first source basin Measured daily runoff values for each sample; For the first The first source basin The daily runoff value predicted by the model for each sample; Finally, the backpropagation algorithm was used to train the five models independently, and the model parameters were continuously updated iteratively until their respective loss functions converged, thus obtaining five basic models that solidify the rainfall-runoff response patterns and runoff generation and confluence mechanisms of different source watersheds. These models were then used as the source watershed prior knowledge base for subsequent target watershed transfer learning and adaptive weighted fusion.
3. The method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data according to claim 2, characterized in that, The sub-steps of step S2, which constructs the target watershed migration branch model, are as follows: S21. Construction of a small sample dataset for the target watershed; For the target watershed with scarce data to be predicted, collect its short-sequence historical meteorological driving data X. target And measured daily runoff data Y target To simulate a data-scarce scenario and verify the model's few-shot learning ability, 80% of the data from the target watershed was selected as the fine-tuning training set, and the remaining data was used as the test set; the input historical meteorological driving data X was... target Preprocessing is performed using the same standardized parameters as the source watershed to ensure consistency in feature distribution; S22, Source Basin Parameter Migration and Feature Extraction Layer Freezing; Load the five base models trained in step S1 respectively, and set their model parameters. The parameters are transferred to the target watershed as initial weights; simultaneously, a parameter freezing strategy is implemented to freeze the parameters of the Transformer encoder layer in the base model. Set to an untrainable state; S23, Output Layer Reset and Differential Fine-tuning: The parameters of the fully connected regression output layer of the model are reset or unfrozen, and its parameter set is adjusted accordingly. Set to trainable state; utilize a limited fine-tuning training set within the target watershed. Construct the target watershed loss function Only for output layer parameters Perform gradient updates: In the formula, To fine-tune the learning rate, it is usually set to a value smaller than the learning rate used in the source domain training, in order to ensure the stability of parameter adjustments; The model predicts runoff under the current parameters; S24. Construct a transfer branch model library; perform the above fine-tuning process on the five base models loaded with initial weights from different source watersheds until the loss function of each base model converges on the target watershed training set, thus obtaining five transfer branch models oriented towards the target watershed. .
4. A method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data according to any one of claims 1 to 3, characterized in that, Step S3, the sub-steps for constructing the dynamic attention module and calculating weights, are as follows: S31, Construct a dynamic environment perception module architecture; Establish a dynamic attention module independent of the transfer branch model, which consists of a feature extraction subnet and a weight mapping subnet; S32. Spatiotemporal environment feature extraction based on Transformer; use The feature extraction subnetwork captures the current environmental situation features of the target watershed, and adopts a Transformer encoder structure; input sequence After location encoding and multi-head self-attention layer processing, the hidden state vector containing the hydrological state information at the current moment is extracted. Select The feature vector of the last time step As the environmental situation representation vector of the target watershed at the current moment, its calculation formula is: S33. Mapping and generation of dynamic confidence weights; transforming environmental situation representation vectors The input is fed into a weighted mapping subnet, which consists of fully connected layers and Softmax activation layers. First, the unnormalized matching score of the migration branch model for each source watershed is calculated for the current environmental situation. : In the formula, Indicates the first One source basin branch; and These are learnable mapping parameters; Then, the Softmax function is used to analyze the unnormalized matching scores. Normalization is performed to generate a dynamic confidence weight vector with a sum of 1. : In the formula, Indicates the first The source basin branch model at time... The proportion of the predicted results in the final integration; S34. The dynamic evolution characteristics of the weights; the dynamic confidence weights With time step The dynamic attention module changes in real time as the weather changes. When the input meteorological data indicates that a heavy rainfall event has occurred in the target watershed, the dynamic attention module will give higher weight to the source watershed branches that originate from the humid area and have similar runoff generation mechanisms. Conversely, during the dry season, it tends to rely on the source watershed branches with similar baseflow characteristics.
5. The method for predicting daily runoff in watersheds with scarce multi-source adaptive transfer learning data according to claim 4, characterized in that, The sub-steps of adaptive ensemble, predicted output, and joint optimization in step S4 are as follows: S41. Multi-branch parallel simulation; incorporating historical meteorological driving sequences of the target watershed. Parallel input to 5 transfer branch model In this process, each transfer branch model utilizes its fixed source domain Transformer feature extractor and fine-tuned output layer to generate features specific to the current time step. Preliminary daily runoff forecast ,at this time, This represents the assumption that the target watershed completely follows the first... Runoff results derived under the premise of the runoff generation and confluence mechanism of a single source basin; S42, Adaptive weighted ensemble; calculate the real-time confidence weights generated using the dynamic attention module in step S33. The five preliminary daily runoff forecasts generated in step S41 are linearly weighted and combined to obtain the final daily runoff forecast results for the target watershed. The integrated calculation formula is as follows: S43. Joint optimization training for the target watershed; based on limited measured daily runoff data from the target watershed. Construct a joint loss function for the target watershed. The final integrated prediction values will be Compared with measured values The mean square error between them is used as the optimization objective: Gradient descent is employed to jointly train the model end-to-end on the target watershed training set. Regarding the parameter update strategy, during backpropagation, the gradient flow is simultaneously propagated to the dynamic attention module and the output layer of each branch model, synchronously updating the parameters of the attention module. and the fully connected layer parameters of each branch model Meanwhile, the parameters of the Transformer encoder in each branch model remain frozen.