Data migration method for watershed lack of hydrological data
By adopting data transfer methods based on transfer learning and Transformer in the data scarcity basin, the problem of deep learning models dependence on large-scale data is solved, reliable simulation data is generated, and the accuracy of hydrological forecasting is improved.
Patent Information
- Application Number
- CN202411666418.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-06-13
AI Technical Summary
The dependence of existing deep learning models on large-scale data makes hydrological predictions in data-scarce watersheds difficult.
Using the data migration method of the hydrological data scarce basin based on transfer learning and Transformer, reliable simulation data is generated through the basin domain migration model and the basin timing lag migration model to provide a data basis for the hydrological forecast of the data scarce basin.
Effectively generate time series data, providing a reliable data basis for runoff prediction in data-scarce basins and improving prediction accuracy.
Smart Images
Figure CN120146151A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to deep learning, and particularly to a data migration method for basins with scarce hydrological data based on transfer learning and Transformer. Background Art
[0002] Due to the shortage of hydrological data for most small and medium-sized rivers, traditional forecasting schemes usually have difficulty making effective forecasts of runoff in these areas. The regionalization method is one of the ideas to solve this problem. Currently, there is little regionalization research on small and medium-sized basins with scarce data. In existing regionalization studies, the measured basins and target basins are mostly evenly distributed in space and have a relatively short spatial distance. In these areas, the regionalization method based on spatial distance is mostly used because of its simple operation and less required data. At the same time, the regionalization method based on hydrological attribute distance and the regression method are also used to improve the prediction performance. However, the regionalization method usually needs to consider the physical mechanism of the hydrological model and the physical meaning of the parameters. The more model parameters there are, the more difficult it is to implement. The hydrological model based on deep learning can extract useful hydrological information from a large amount of data without deeply understanding the physical meaning of the parameters.
[0003] There are many small and medium-sized rivers with difficult hydrological forecasting due to the lack of historical data, and the forecasting model depends on a large amount of data. How to establish an intelligent hydrological forecasting model with strong generality and high forecasting accuracy for basins with scarce data by means of machine learning, deep learning and other technologies has become a difficult problem that urgently needs to be solved in the field of hydrological forecasting in the era of intelligent water conservancy. Therefore, the present invention aims at basins with scarce data and generates a large amount of reliable simulated data by means of deep learning algorithms, so as to provide a data basis for predicting runoff. Summary of the Invention
[0004] Object of the Invention: The object of the present invention is to solve the problem that the dependence of existing deep learning models on large-scale data makes it difficult to predict basins with scarce data. By combining Transformer technology and transfer learning technology, a method for generating reliable simulated data is provided, so as to provide a reliable data basis for the research on hydrological forecasting of basins with scarce data.
[0005] Technical Solution:
[0006] A data migration method for basins with scarce hydrological data mainly consists of two parts: a basin value range migration model and a basin time series lag migration model. The basin value range migration is used to reduce the value range offset between basins, and the basin time series lag migration is used to capture the relationship between the water level time series and the precipitation time series in the source basin and realize the relationship migration. Its construction steps are as follows:
[0007] Step S1: Data preprocessing, i.e., normalization. The input of the model usually consists of multiple variables, and the dimensions and dimension units of different input variables are also different. The Min-Max standard is used to uniformly compress the input variables into the range of [0, 1].
[0008] Step S2: Input the data into the embedding layer to convert the data into vector representation and capture the relationships between the data.
[0009] Step S3: Use the range encoder E V to fully extract the range features of the water level time series W T of the source basin T and the range features of the rainfall time series P T of the source basin T
[0010] Step S4: Input the water level time series W S or the rainfall time series P S of the target basin S into the range decoder D V , and at the same time input the range features of the water level time series of the source basin generated in S3, i.e., P T or the range features of the rainfall time series for feature fusion, so as to realize the range migration of the water level time series W S and the rainfall time series range P S from the source basin S to the target basin T.
[0011] Step S5: Combine the water level time series and the rainfall time series after range migration in S4 into a new sequence
[0012] Step S6: The time series lag encoder E L extracts the time lag relationship L T between the water level time series and the precipitation time series in the target basin T.
[0013] Step S7: Input the time lag relationship L T into the time series lag decoder D L , and at the same time input the new sequence combination generated in S5 for feature fusion to realize the migration of the time series lag relationship of the basin from the source basin S to the target basin T, and obtain the knowledge after migration of the source basin Thus, the data migration from the source basin to the target basin is realized.
[0014] The basin range migration model is based on the Transformer architecture and consists of the following parts:
[0015] (1) Basin range encoder:
[0016] The basin value range encoder is composed of multiple encoder layers stacked together, and each encoder layer includes multiple components. First is the multi-head self-attention layer, which is used to capture the relationships between each time point in the input sequence; then is the feed-forward network, which is used to perform non-linear mapping on the features output by the attention layer; finally is the pooling layer, which is used to perform pooling operations on the outputs of the previous two layers to extract the value range features of the source domain sequence. During the downsampling process, if extracting the value range features of the water level time series, average pooling is adopted; if extracting the value range features of the precipitation time series, max pooling is adopted.
[0017] (2) Basin value range decoder:
[0018] The basin value range decoder is composed of multiple decoder layers stacked together, and each decoder layer includes multiple components. First is the multi-head self-attention layer, which is used to perform cross-domain feature fusion on the value range features of the source domain; then is the feed-forward network layer, which is used to further process the features output by the attention layer, usually including two fully-connected layers and an activation function.
[0019] The basin value range migration model is based on the Transformer architecture and consists of the following parts:
[0020] (1) Temporal lag encoder:
[0021] The basin temporal lag encoder is composed of multiple encoder layers stacked together, and each encoder layer includes multiple components. First is the multi-head self-attention layer, which is used to capture the relationships between each time point in the input sequence; then is the feed-forward network, which is used to perform non-linear mapping on the features output by the attention layer; finally is the max pooling layer, which is used to perform pooling operations on the outputs of the previous two layers to extract the time lag features of the source domain sequence.
[0022] (2) Temporal lag decoder:
[0023] The temporal lag decoder is composed of multiple decoder layers stacked together, and each decoder layer includes multiple components. First is the multi-head self-attention layer, which is used to perform cross-domain feature fusion on the time lag features of the source domain; then is the feed-forward network layer, which is used to further process the features output by the attention layer, usually including two fully-connected layers and an activation function.
[0024] Advantages of the present invention:
[0025] The present invention utilizes deep learning algorithms and adopts a data migration method based on Transformer for basins with scarce hydrological data, effectively generating time series data and providing a reliable data basis for runoff prediction in basins with scarce data. Description of the drawings
[0026] Figure 1It is the overall framework diagram of the hydrological time series transfer method based on Transformer in the present invention
[0027] Figure 2 It is the detailed diagram of the value range transfer model based on Transformer
[0028] Figure 3 It is the detailed diagram of the time delay transfer model based on Transformer, where (W T , P T ) is the concatenation of the source domain water level sequence and the source domain rainfall sequence, is the concatenation of the water level time series and the rainfall time series of the target domain after value range transfer. Specific implementation manners
[0029] The embodiment of the present invention provides a hydrological time series transfer method based on transfer learning and Transformer, including the following steps:
[0030] First, perform value range transfer of the water level time series W T and the rainfall time series value range P T from the source basin T to the target basin S:
[0031] Step S1: The dimensions of the source domain water level time series W T and the rainfall time series P T are both (batch_size, seq_len_source, 1), where batch_size is the batch size, seq_len_source is the source domain sequence length, and 1 represents a one-dimensional time series. Step S2: Input the water level time series W T and the rainfall time series P T into the input embedding layer, and the output data dimension is (batch_size, seq_len_source, d_model), where d_model is the hidden dimension of the model. This step can embed the time series into a high-dimensional space through a linear transformation.
[0032] Step S3: Input the output data in S2 into the position encoding layer to add position encoding information, and the dimension of the output data is (batch_size, seq_len_source, d_model).
[0033] Step S4: Input the output data in S3 into the basin value range encoder. The basin value range encoder is stacked by multiple encoder layers, and the dimension of the output data is still (batch_size, seq_len_source, d_model). This output data contains the context information of the source domain water level time series or rainfall time series.
[0034] Step S5: Perform a pooling operation on the output data of the range encoder. During downsampling, if extracting the range features of the water level time series, average pooling is used; if extracting the range features of the rainfall time series, max pooling is used to extract the range features of the source domain sequence. The dimension of the data is (batch_size, d_model), and this data represents the statistical features of the entire source domain water level time series or precipitation time series.
[0035] Step S6: Input the target domain water level time series W T and the rainfall time series P T into the input embedding layer and the position encoding layer. The dimension of the output data is (batch_size, seq_len_source, d_model), thereby adding position encoding information to the input data.
[0036] Step S7: Input the output data in S6 into the multi-head attention layer in the range decoder, and at the same time input the source domain range feature data obtained in S5. Through the self-attention mechanism, cross-domain feature fusion is performed, and the dimension of the output data is (batch_size, seq_len_target, d_model).
[0037] Step S8: Input the output data in S7 into the feed-forward network to further process the features output by the attention layer. The dimension of the output data is (batch_size, seq_len_target, d_model).
[0038] Step S9: Finally, input the output data in S8 into the output layer, and map the output of the decoder to the predicted value of the water level or the predicted value of the rainfall through a linear transformation layer. The dimension of the output data is (batch_size, seq_len_target, 1), and this data is the predicted value sequence of the target domain water level time series or rainfall time series.
[0039] Then perform watershed time series lag migration:
[0040] Step S1: Concatenate the source domain water level sequence and the source domain rainfall sequence on the last dimension to form a new input sequence (W T , P T ). The dimension is (batch_size, seq_len_source, 2).
[0041] Step S2: Input the data in S1 into the input embedding layer, and the output data dimension is (batch_size, seq_len_source, d_model), where d_model is the hidden dimension of the model. The time series can be embedded into a high-dimensional space through linear transformation or positional encoding.
[0042] Step S3: Input the output data in S2 into the positional encoding layer to add positional encoding information, and the dimension of the output data is (batch_size, seq_len_source, d_model).
[0043] Step S4: Input the output data in S3 into the basin time lag encoder. The basin value range encoder is stacked by multiple encoder layers, and the dimension of the output data is still (batch_size, seq_len_source, d_model). This data contains the context information of the source domain input sequence.
[0044] Step S5: Perform max pooling on the encoder output to extract the time lag relationship L between the water level time series and the rainfall time series in the source basin T T , with the dimension of (batch_size, d_model).
[0045] Step S6: Concatenate the water level time series and the rainfall amount series of the target domain after range migration to form the input of the target domain with the dimension of (batch_size, seq_len_target, 2). Input it into the input embedding layer and the positional encoding layer, and the dimension of the output data is (batch_size, seq_len_source, d_model), thereby adding positional encoding information to the input data.
[0046] Step S7: Input the data in S6 into the multi-head attention layer in the time lag decoder, and at the same time input the time lag relationship L between the water level time series and the precipitation time series in the source basin T obtained in S5 T , and perform cross-domain feature fusion through the self-attention mechanism. The dimension of the output data is (batch_size, seq_len_target, d_model).
[0047] Step S8: Input the output data in S7 into the feed-forward network to further process the features output by the attention layer. The dimension of the output data is (batch_size, seq_len_target, d_model).
[0048] Step S9: Finally, input the output data in S8 into the output layer, and map the decoder output back to the predicted values of the water level time series and rainfall time series through a linear transformation layer, with the dimension of (batch_size, seq_len_target, 2). Thus, the data migration from the source basin to the target basin is completed.
Claims
1. A data migration method for a basin with scarce hydrological data mainly consists of two parts: Basin value range migration model and basin time series lag migration model. The basin value range migration model is used to reduce the value range deviation between basins, and the basin time series lag migration model is used to capture the relationship between the water level time series and the precipitation time series in the source basin and realize the relationship migration. Its construction steps are: Step S1: Data preprocessing, i.e., normalization. The input of the model usually consists of multiple variables, and the dimensions and dimensional units of different input variables are also different. The Min-Max standard is used to compress the input variables uniformly into the range of [0, 1]. Step S2: Input the data into the embedding layer to convert the data into a vector representation to capture the relationship between the data. Step S3: Use the range encoder E V , fully extract the water level time series W of the source basin T T The range characteristics of and rainfall time series P T The range characteristics of Step S4: Convert the water level time series W of the target basin S to S Or rainfall time series P S Input to range decoder D V , and input the source basin water level time series value range feature P generated in S3 T Or the range characteristics of rainfall time series Perform feature fusion to achieve the water level time series W of the source basin S S And the rainfall time series range P S The value range is migrated to the target watershed T. Step S5: Convert the water level time series after range migration in S4 and rainfall time series Combine into a new sequence Step S6: Timing lag encoder E L Extract the time lag relationship L between the water level time series and the precipitation time series in the target basin T T . Step S7: Set the time lag relationship L T Input to timing lag decoder D L , and input the new sequence combination generated in S5 Perform feature fusion to achieve the migration of the temporal lag relationship from the source basin S to the target basin T, and obtain the knowledge after the source basin migration At this point, data migration from the source basin to the target basin has been achieved.
2. The watershed value range migration model is based on the Transformer architecture and consists of the following parts: (1) Stream value range encoder: The flow-valued range encoder is composed of multiple encoder layers stacked together, each of which includes multiple components. The first is a multi-head self-attention layer that captures the relationship between each time point in the input sequence; Then the feedforward network is used to perform nonlinear mapping on the features output by the attention layer; finally, the pooling layer is used to pool the outputs of the previous two layers to extract the range features of the source domain sequence. During the downsampling process, if the range features of the water level time series are extracted, average pooling is used; if the range features of the rainfall time series are extracted, maximum pooling is used. (2) Stream value range decoder: The flow domain value decoder is composed of multiple decoder layers, each of which includes multiple components. The first is the multi-head self-attention layer, which is used to perform cross-domain feature fusion of the source domain value features; This is followed by a feed-forward network layer, which is used to further process the features output by the attention layer and usually contains two fully connected layers and an activation function.
3. The watershed time series lag migration model is based on the Transformer architecture and consists of the following parts: (1) Timing lag encoder: The watershed temporal lag encoder is composed of multiple encoder layers stacked together, each of which includes multiple components. The first is a multi-head self-attention layer that captures the relationship between each time point in the input sequence; Then the feedforward network is used to perform nonlinear mapping on the features output by the attention layer; The last layer is the maximum pooling layer, which is used to pool the outputs of the previous two layers to extract the time-lag features of the source domain sequence. (2) Timing lag decoder: The temporal lag decoder is composed of multiple decoder layers, each of which includes multiple components. The first is a multi-head self-attention layer, which is used to perform cross-domain feature fusion of the time-lag features in the source domain; This is followed by a feed-forward network layer, which is used to further process the features output by the attention layer and usually contains two fully connected layers and an activation function.