A hydrological forecasting method based on deep learning and transfer learning

Abstract

The present invention relates to the technical field of hydrological forecasting, and in particular to a hydrological forecasting method based on deep learning and transfer learning, the method comprising: screening main contribution features; training a long-short-term memory model using historical data of the main contribution features; constructing a weight allocation model based on the difference in contribution rates of different main contribution features and the attenuation of the contribution rates of the corresponding main contribution features within a preset time interval; optimizing the loss function of the long-short-term memory model according to the weight difference between the calculated weights obtained by the weight allocation model and the training weights obtained by the trained long-short-term memory model to obtain a hydrological forecasting model; correcting the hydrological forecasting model according to the error between the forecast results and the actual results at different magnitudes, to obtain an optimized hydrological forecasting model corresponding to each magnitude. The present invention can optimize the loss function of the model by using the influence between different hydrological features, thereby improving the training effect of the network model and the learning ability of the latest data.

Description

Technical Field

[0001] The present invention relates to the technical field of hydrological forecasting, and in particular to a hydrological forecasting method based on deep learning and transfer learning. Background Art

[0002] Hydrological forecast is a qualitative or quantitative forecast of the hydrological situation in a certain period of time in the future based on previous and current hydrological, meteorological and other information. Hydrological forecast is an important basic water conservancy work and non-engineering measure for flood control, which directly serves the rational use and protection of water resources, the construction and management of water conservancy projects, and industrial and agricultural production. Hydrological data has continuity, periodicity and seasonality. Since hydrological data changes over time and is affected by factors such as periodic and seasonal changes, the changes in hydrological data generally follow certain rules.

[0003] At present, the methods for hydrological forecasting mainly include: traditional cause analysis method and deep learning prediction method. Among them, the traditional physical environment analysis method mainly builds a prediction model to achieve hydrological forecast through information such as atmospheric airflow, meteorological factors, and rainfall. The deep learning prediction method mainly trains a deep neural model through hydrological time series data and realizes hydrological forecast through a deep neural network model.

[0004] Traditional physical environment analysis methods are easily affected by noise data or partial mutation data, have low model prediction accuracy, and are prone to falling into local optimal problems; the problem with deep learning prediction methods is that they have high requirements for the hydrological time series data involved in training, and the model is prone to overfitting problems. Summary of the invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a hydrological forecasting method based on deep learning and transfer learning. The technical solutions adopted are as follows:

[0006] An embodiment of the present invention provides a hydrological forecasting method based on deep learning and transfer learning, the method comprising the following steps:

[0007] The random forest algorithm is used to extract features from historical hydrological data and screen the main contributing features;

[0008] The long-short-term memory model is trained using the historical data of the main contribution features; a weight allocation model is constructed based on the difference in contribution rates of different main contribution features and the contribution rate attenuation of the corresponding main contribution features within a preset time interval; the loss function of the long-short-term memory model is optimized according to the weight difference between the calculated weights obtained by the weight allocation model and the training weights obtained by the trained long-short-term memory model to obtain a hydrological forecast model;

[0009] The historical data of the main contribution characteristics of floods of different magnitudes are input into the hydrological forecasting model to obtain forecast results. The hydrological forecasting model is corrected according to the errors between the forecast results and the actual results at different magnitudes to obtain the optimized hydrological forecasting model corresponding to each magnitude.

[0010] Preferably, the method for obtaining the contribution rate is:

[0011] Obtain the probability value output by each main contribution feature after being activated by the activation function in the long short-term memory model, and obtain the contribution rate according to the probability value; the sum of the probability value and the contribution rate is a preset value.

[0012] Preferably, the process of obtaining the contribution rate attenuation is:

[0013] For each main contribution feature, its contribution rate at the current moment and its contribution rate at a preset time interval before the current moment are obtained, and the difference between the two contribution rates is used as the contribution rate attenuation coefficient to characterize the contribution rate attenuation.

[0014] Preferably, the method for obtaining the difference in contribution rate is:

[0015] The difference in contribution rate between each main contribution feature and each other main contribution feature at the same time is obtained, and the average value of all contribution rate differences less than the difference threshold is selected as the difference coefficient of the corresponding main contribution feature to characterize the difference in the contribution rate.

[0016] Preferably, the weight distribution model is constructed by:

[0017] The mapping function of each main contribution feature is obtained by fitting the contribution rate attenuation coefficients at multiple time intervals; and the weight distribution model is constructed based on the product of the difference coefficient and the mapping function.

[0018] Preferably, the optimization method of the loss function is:

[0019] The transfer learning strategy is used to perform rolling learning on the latest data of the main contributing features, and the training weights are used as the weight coefficients of the corresponding latest data. The differences between the training weights and weight coefficients and the calculated weights at the corresponding moments are obtained as the adjusted loss function, and the loss function of the long short-term memory model is optimized according to the adjusted loss function.

[0020] Preferably, the hydrological forecast model is corrected, including:

[0021] For each magnitude of historical data, the second-order difference of the error within the preset time interval before the current moment is obtained, and an autoregressive model and a moving average model are constructed to obtain the length of the stationary time series. The preset time interval is corrected using the stationary time series as the ideal time interval, and a new long-short-term memory model is obtained as the corresponding optimized hydrological forecasting model.

[0022] The embodiments of the present invention have at least the following beneficial effects:

[0023] 1. When historical data is used as input, the contribution rate difference coefficient between different hydrological characteristics and the contribution rate attenuation coefficient under different time interval lengths are constructed according to the contribution rate, and the contribution rate attenuation coefficient is fitted with a nonlinear function to obtain the calculation weights of different feature input data, and the calculation weights are used to optimize the training weights in the historical data learning process, and the transfer learning model is used to dynamically update the weights of the latest hydrological data. Compared with the existing technology, the beneficial effect is that the feature weights of the long-short term memory network model can be allocated according to the possible influence relationship between the features and the contribution rate changes between different features at different time lengths, thereby improving the training effect of the network model and its ability to learn the latest data.

[0024] 2. By analyzing the impact of hydrological data at different magnitudes in historical data on the prediction accuracy of the long-short-term memory network model, the optimized hydrological forecast model at different magnitudes is obtained. Compared with the existing technology, the beneficial effect is that it can improve the prediction accuracy of the long-short-term memory model prediction model and reduce the impact of hydrological data at different magnitudes on the prediction error of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flowchart of the steps of a hydrological forecasting method based on deep learning and transfer learning provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the hydrological forecasting method based on deep learning and transfer learning proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0028] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0029] The following is a detailed description of a specific scheme of a hydrological forecasting method based on deep learning and transfer learning provided by the present invention in conjunction with the accompanying drawings.

[0030] See also Figure 1 , which shows a flow chart of the steps of a hydrological forecasting method based on deep learning and transfer learning provided by an embodiment of the present invention, the method comprising the following steps:

[0031] Step S001, using the random forest algorithm to extract features from historical hydrological data and screen the main contributing features.

[0032] The specific steps include:

[0033] Hydrological characteristics generally include runoff, sediment content, flood season, ice flood, ice period, abundance of hydropower resources, flow rate and water level. The main features of historical hydrological data are extracted through random forest algorithm, the importance score of each hydrological feature is obtained, and the hydrological features with greater contribution are selected as the main contributing features.

[0034] Among them, the hydrological features with greater contributions are the hydrological features whose importance scores are normalized to be greater than the importance threshold. In the embodiment of the present invention, the importance threshold is 0.6.

[0035] Step S002, using the historical data of the main contribution features to train the long short-term memory model; constructing a weight allocation model based on the difference in contribution rates of different main contribution features and the attenuation of the contribution rates of the corresponding main contribution features within a preset time interval; optimizing the loss function of the long short-term memory model based on the weight difference between the calculated weights obtained by the weight allocation model and the training weights obtained by the trained long short-term memory model to obtain a hydrological forecasting model.

[0036] The specific steps include:

[0037] 1. Use historical data of main contributing features to train the long short-term memory model.

[0038] The model structure of the long short-term memory model (LSTM) model includes: input gate, forget gate, output gate, hidden layer. The task of the input gate is to continuously convert new data x t Input into the network model, and the output information h at the previous time point t-1 After activation by the sigmoid function, the memory cell C after forgetting through the forget gate is obtained t-1 The reserved information is passed to the tanh function, and the result obtained by the sigmoid function is multiplied by the result obtained by the tanh function to obtain the output information h t , the output is the hidden state, and then the new cell state C t and the new hidden state h t Move to the next time series.

[0039] 2. Obtain the contribution rate of each main contributing feature.

[0040] Obtain the probability value of each main contribution feature output after being activated by the activation function in the long short-term memory model, and obtain the contribution rate based on the probability value; the sum of the probability value and the contribution rate is the preset value.

[0041] The probability value p(x) output after activation by the sigmoid function is the forgetting probability, which is used to determine the probability of C t-1 How much is forgotten, p(x) = 1 means completely retained, p(x) = 0 means completely forgotten, the difference between the preset value and the probability value p(x) is taken as the contribution rate, in the embodiment of the present invention, the preset value is 1, that is, the contribution rate

[0042] 3. Obtain the contribution rate attenuation of each main contribution feature within a preset time interval.

[0043] For each main contribution feature, its contribution rate at the current moment and its contribution rate at a preset time interval before the current moment are obtained, and the difference between the two contribution rates is used as the contribution rate attenuation coefficient to characterize the contribution rate attenuation.

[0044] As the length of time changes, the contribution rate of each main contribution feature will also change accordingly, and the difference state of the contribution rate retained by the historical data of each main contribution feature at the same time length, that is, the preset time interval, is obtained.

[0045] Get the contribution rate attenuation of the u-th main contribution feature at a preset time interval:

[0046]

[0047] in, represents the contribution rate attenuation coefficient of the u-th main contribution feature in q time intervals, represents the contribution rate of the u-th main contribution feature at the t-th moment, Represents the contribution rate of the u-th main contributing feature at the tq-th moment.

[0048] In the embodiment of the present invention, the preset time interval is q.

[0049] 4. Obtain the difference in contribution rate of each main contributing feature.

[0050] The difference in contribution rate between each main contribution feature and each other main contribution feature at the same time is obtained, and the average value of all contribution rate differences less than the difference threshold is selected as the difference coefficient of the corresponding main contribution feature to characterize the difference in contribution rate.

[0051] Although the LSTM network model considers the contribution rates of different main contributing features at different time lengths, it does not take into account the possible influence relationships between different main contributing features.

[0052] Assuming that the number of main contribution features with greater contributions screened in step S001 is M, taking the u-th main contribution feature as an example, the difference in contribution rate between the u-th main contribution feature and the remaining M-1 main contribution features is calculated respectively:

[0053]

[0054] Where v≤M and v≠u, Z uv represents the difference in contribution rate between the u-th main contributing feature and the v-th main contributing feature, represents the contribution rate of the u-th main contributing feature, Represents the contribution rate of the vth main contributing feature.

[0055] For all contribution rate differences Z of the u-th main contribution feature uv , select the contribution rate differences that are less than the difference threshold and calculate the average value as the difference coefficient of the u-th main contribution feature.

[0056] In another embodiment, the difference Z of all contribution rates of the u-th main contribution feature can also be used. uv The average value is taken as the coefficient of variation.

[0057] 5. Construct a weight distribution model.

[0058] By fitting the contribution rate attenuation coefficients at multiple time intervals, the mapping function of each main contribution feature is obtained; and a weight allocation model is constructed based on the product of the difference coefficient and the mapping function.

[0059] According to the change of contribution rate attenuation coefficient of the same main contribution feature at different times, the variation coefficient is fitted to obtain the nonlinear mapping function f(x), where f(x) is obtained by fitting the change of contribution rate attenuation coefficient with the time interval. Then, the contribution rate difference coefficient and mapping function are used to construct the weight distribution model:

[0060]

[0061] Among them, ω u represents the contribution rate weight of the u-th main contribution feature, represents the coefficient of difference of the u-th main contributing feature, f(x) u They respectively represent the mapping functions of the u-th characteristic factor, and the value of x is the length of the current time interval q.

[0062] By considering the contribution rate difference coefficient between the input feature factors and the contribution rate attenuation coefficient at different time intervals, when the input feature data has a small contribution rate difference and a small contribution rate attenuation rate at the current time interval, the contribution rate weight of the corresponding feature data is larger; conversely, the contribution rate weight is smaller, which can fully consider the correlation between different features at different time intervals and improve the accuracy and speed of feature weight training.

[0063] 6. Optimize the loss function of the long short-term memory model.

[0064] The transfer learning strategy is used to perform rolling learning on the latest data of the main contributing features, and the training weights are used as the weight coefficients of the corresponding latest data. The differences between the training weights and weight coefficients and the calculated weights at the corresponding moments are obtained as the adjusted loss function, and the loss function of the long short-term memory model is optimized according to the adjusted loss function.

[0065] The transfer learning strategy is used to conduct rolling learning on the latest hydrological data. The parameters between the transfer learning and LSTM network models are shared, and the training weights ω′ of the long short-term memory model completed by historical data training are used. i As the weight coefficient w of the latest data corresponding to the main contributing feature, the weight coefficient w is used to guide the newly input latest hydrological data for feature learning.

[0066] Construct the loss function of the LSTM network model: Loss = L1 + L2.

[0067] in, represents the mean square error loss function, Y i represents the model prediction data of the i-th main contributing feature, represents the label data of the i-th main contributing feature, and N represents the total number of main contributing features.

[0068] ω ti represents the calculation weight of the i-th main contribution feature at the t-th moment in the historical moment, which is calculated by the weight distribution model; ω′ i Represents the training weight of the i-th main contributing feature when pre-training the model with historical data; ω si represents the calculation weight of the ith main contributing feature at the sth moment in the latest data, which is calculated by the weight distribution model; w i Represents the weight coefficient corresponding to the latest data of the i-th main contributing feature.

[0069] The model training is constrained and guided by the difference between the training weights of historical data and the weight coefficients of new data and the calculated weights at the corresponding moment. This means that the currently acquired calculated weights are calculated differently from the training weights of historical data training and the weight coefficients of newly added data. When the historical data has been trained, the first half of the loss function is frozen and no longer changes. The loss function is only updated by fine-tuning the newly input data.

[0070] For example, the loss function corresponding to the data of the main contributing features before December 31, 2021 converges, and its loss function is frozen as historical data. The data on January 1, 2022 is input into the model. Since hydrological data is continuous, periodic, and seasonal and changes over time, the loss function obtained at this time is likely to still converge, and the loss function remains unchanged. Assume that on July 19, 2022, the loss function no longer converges. At this time, the first half of the loss function corresponding to the historical data is frozen, and the loss function is updated by training the real-time data of the second half.

[0071] The LSTM network model is pre-trained using historical data to obtain the trained weight ω′, and the training weight ω′ is optimized using the calculation weights of each feature to ensure that the latest hydrological data can converge quickly under the source domain data using the calculation weights of historical data. The calculation weights of the real-time data are increased to guide the loss function, thereby improving the forecast accuracy without increasing computing resources.

[0072] Step S003, inputting historical data of main contribution characteristics of floods of different magnitudes into the hydrological forecasting model to obtain forecast results, and correcting the hydrological forecasting model according to the errors between the forecast results and actual results at different magnitudes to obtain the optimized hydrological forecasting model corresponding to each magnitude.

[0073] The specific steps include:

[0074] For each magnitude of historical data, the second-order difference of the error within the preset time interval before the current moment is obtained, and an autoregressive model and a moving average model are constructed to obtain the length of the stationary time series. The preset time interval is corrected using the stationary time series as the ideal time interval, and a new long-short-term memory model is obtained as the corresponding optimized hydrological forecasting model.

[0075] The historical flood data of different magnitudes are used as model input, and the trained LSTM network model is used to make hydrological predictions for floods of different magnitudes. The prediction results are compared with the actual hydrological data to obtain the error sizes δ of different magnitudes. The time series features of flood hydrological data of different magnitudes are extracted based on the differential autoregressive moving average model, and the hydrological prediction errors of different magnitudes are corrected.

[0076] In the embodiment of the present invention, the hydrological data of different magnitudes include five magnitudes: high water, medium water (larger than medium water), medium water, medium water (small than medium water), and low water.

[0077] Specifically, the first-order difference is obtained by using the data difference from the sth moment to the s-1th moment, and the second-order difference from the sth moment to the s-1th moment is obtained on the basis of the first-order difference. The mean and variance of the second-order difference are calculated, and the variance and mean of the second-order difference from the sth moment to the s-1th moment are used to construct an autoregressive model and a moving average model. The parameters obtained by the two models include the length Q of the stationary time series. The preset time interval q is adjusted with the stationary time series at different magnitudes as the ideal time interval of the corresponding magnitude, so as to realize the prediction error correction of the hydrological prediction data at different magnitudes.

[0078] By using the adjusted time interval size, the model is retrained for hydrological data at different magnitudes to correct the weight coefficients of the LSTM network model and improve the prediction accuracy of the LSTM network model.

[0079] In summary, the embodiment of the present invention uses the random forest algorithm to extract features from historical hydrological data and screen the main contribution features; uses the historical data of the main contribution features to train the long-short term memory model; constructs a weight allocation model based on the difference in contribution rates of different main contribution features and the contribution rate attenuation of the corresponding main contribution features within a preset time interval; optimizes the loss function of the long-short term memory model according to the weight difference between the calculated weights obtained by the weight allocation model and the training weights obtained by the trained long-short term memory model to obtain a hydrological forecast model; inputs the historical data of the main contribution features of floods of different magnitudes into the hydrological forecast model to obtain forecast results, and corrects the hydrological forecast model according to the errors between the forecast results and the actual results at different magnitudes to obtain an optimized hydrological forecast model corresponding to each magnitude. The embodiment of the present invention can optimize the loss function of the model by using the influence between different hydrological features, thereby improving the training effect of the network model and the learning ability of the latest data.

[0080] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0082] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A hydrological forecasting method based on deep learning and transfer learning, characterized in that: The method comprises the following steps: The random forest algorithm is used to extract features from historical hydrological data and screen the main contributing features; The long-short-term memory model is trained using the historical data of the main contribution features; a weight allocation model is constructed based on the difference in contribution rates of different main contribution features and the contribution rate attenuation of the corresponding main contribution features within a preset time interval; the loss function of the long-short-term memory model is optimized according to the weight difference between the calculated weights obtained by the weight allocation model and the training weights obtained by the trained long-short-term memory model to obtain a hydrological forecast model; Inputting historical data of main contribution characteristics of floods of different magnitudes into the hydrological forecasting model to obtain forecast results, and correcting the hydrological forecasting model according to the errors between the forecast results and actual results at different magnitudes to obtain an optimized hydrological forecasting model corresponding to each magnitude; The method for obtaining the contribution rate is: Obtaining the probability value outputted by each main contribution feature after being activated by the activation function in the long short-term memory model, and obtaining the contribution rate according to the probability value; the sum of the probability value and the contribution rate is a preset value; The process of obtaining the contribution rate attenuation is as follows: For each main contribution feature, obtain its contribution rate at the current moment and its contribution rate at a preset time interval before the current moment, and use the difference between the two contribution rates as the contribution rate attenuation coefficient to characterize the contribution rate attenuation; The method for obtaining the difference in the contribution rate is: Obtain the difference in contribution rate between each main contribution feature and each other main contribution feature at the same time, and select the average value of all contribution rate differences less than the difference threshold as the difference coefficient of the corresponding main contribution feature to characterize the difference in the contribution rate; The construction method of the weight distribution model is: The mapping function of each main contribution feature is obtained by fitting the contribution rate attenuation coefficients at multiple time intervals; and the weight distribution model is constructed based on the product of the difference coefficient and the mapping function.

2. A hydrological forecasting method based on deep learning and transfer learning according to claim 1, characterized in that: The optimization method of the loss function is: The transfer learning strategy is used to perform rolling learning on the latest data of the main contributing features, and the training weights are used as the weight coefficients of the corresponding latest data. The differences between the training weights and weight coefficients and the calculated weights at the corresponding moments are obtained as the adjusted loss function, and the loss function of the long short-term memory model is optimized according to the adjusted loss function.

3. A hydrological forecasting method based on deep learning and transfer learning according to claim 1, characterized in that: Correcting the hydrological forecast model includes: For each magnitude of historical data, the second-order difference of the error within the preset time interval before the current moment is obtained, and an autoregressive model and a moving average model are constructed to obtain the length of the stationary time series. The preset time interval is corrected using the stationary time series as the ideal time interval, and a new long-short-term memory model is obtained as the corresponding optimized hydrological forecasting model.

Images