A data-driven rolling runoff forecasting method and system

Abstract

The present invention discloses a rolling runoff forecasting method and system based on data drive, which divides historical runoff data and historical rainfall data according to sliding window size L and sliding window step size Step, constructs a sample database; obtains current runoff sequence R and current rainfall sequence F; uses current rainfall sequence F to perform similarity calculation in the sample database, obtains K samples with the highest similarity; and calculates runoff forecast value based on current runoff sequence R, current rainfall sequence F, and K samples with the highest similarity using a runoff forecasting model. The present invention can significantly reduce the amount of data involved in runoff forecasting, especially for the flood season and rainy season, improves the speed and accuracy of forecasting, and is of great significance for flood prevention and disaster reduction work.

Description

Technical Field

[0001] The present invention relates to the technical field of river basin hydrological prediction, and in particular to a data-driven rolling runoff forecasting method and system. Background Art

[0002] Runoff forecasting plays an important role in flood prevention and disaster reduction. At present, the methods of runoff forecasting mainly include hydrological model forecasting and data-driven. The hydrological model can well describe the runoff generation and convergence process, but the modeling process requires high professional knowledge, and the parameter calibration is relatively difficult. Different river basins need to be modeled separately, and the generalization ability is insufficient. With the continuous accumulation of hydrological historical data, the data-driven model uses massive historical data to achieve in-depth analysis, which can dig out the hidden laws in the historical data and gradually show good adaptability and generalization ability. Using data models such as neural networks, support vector machines, and gradient boosting decision trees to forecast runoff in various situations such as floods in the flood season, no floods in the flood season, and dry season requires more and more data and more and more model parameters, which also makes the training process more and more difficult.

[0003] After research, it was found that for a specific region and a certain geographical environment, the process of generating runoff under similar rainfall conditions is also similar. In other words, predicting runoff based on similar historical rainfall conditions not only has strong explanatory power but can also significantly reduce the amount of data involved in model training. It is also of great significance to improving forecast accuracy and extending the forecast period. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a data-driven rolling runoff forecasting method and system, which obtains the runoff conditions under similar rainfall scenarios in history according to the current rainfall characteristics, and forecasts the current runoff according to the runoff characteristics under similar rainfall scenarios. The runoff forecast is achieved using a relatively small amount of data, which is of great significance to flood prevention and disaster reduction, and improving forecast accuracy.

[0005] In a first aspect, the present invention discloses a data-driven rolling runoff forecasting method, comprising the following steps:

[0006] Step S101: dividing the historical runoff data and the historical rainfall data according to the sliding window size L and the sliding window step size Step to construct a sample database; wherein L and Step are integers, L is greater than Step, and Step is greater than 0;

[0007] Step S102: Obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1},F={f t-1 ,ft-2 ,…,f t-L+1}, t represents the current time;

[0008] Step S103: using the current rainfall sequence F to perform similarity calculation in the sample database, and obtaining K samples with the highest similarity;

[0009] Step S104: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models and weighted mean models.

[0010] Preferably, the historical runoff data and the historical rainfall data are divided according to the sliding window size L and the sliding window step size Step to construct a sample database, specifically:

[0011] Read historical runoff data and historical rainfall data, align the historical runoff data and historical rainfall data on the time axis, slide on the historical runoff data and historical rainfall data according to the set sliding window step Step, and obtain a sample for each sliding window step Step. Each sample includes L historical runoff data and historical rainfall data.

[0012] As a further improvement scheme, the sample database also includes historical prediction data; the historical prediction data includes runoff prediction values ​​and types of runoff forecast models;

[0013] Read historical prediction data; align the historical prediction data with the historical runoff data and the historical rainfall data on the time axis, slide on the historical prediction data, historical runoff data and historical rainfall data according to the set sliding window step Step, and obtain a sample for each sliding window step Step. Each sample includes L historical runoff data, L historical rainfall data and L historical prediction data.

[0014] Preferably, the current rainfall sequence F is used to perform similarity calculation in the sample database, and the K samples with the highest similarity are specifically:

[0015] Traverse each sample in the sample database, organize the L historical rainfall data of the sample into a vector, and calculate the vector and the current rainfall sequence F={f t-1 ,f t-2 ,…,f t-L+1}; sort the samples in descending order according to the similarity, obtain the K samples with the highest similarity, and the similarity set S of the K samples with the highest similarity, S={S1,S2,…,S K}.

[0016] As a further improvement, the K-Means algorithm is used to cluster the historical rainfall data in the sample database to obtain N clusters and the feature vector of each cluster; the similarity between the current rainfall sequence F and the feature vector of each cluster is calculated; the clusters are sorted in descending order according to the similarity to obtain the first a clusters; the samples in the first a clusters are obtained; the similarity between each sample in the first a clusters and the current rainfall sequence F={f t-1 ,f t-2 ,…,f t-L+1}; sort the samples in the first a clusters in descending order according to the similarity, obtain the K samples with the highest similarity, and the similarity set S of the K samples with the highest similarity, S={S1,S2,…,S K}.

[0017] Preferably, the runoff forecast value r is calculated using a runoff forecast model. t , specifically: normalize the similarity set S according to formula (1);

[0018] (1);

[0019] Where 1≤i≤K, 1≤j≤K, and the runoff forecast value r is calculated using the weighted mean model according to formula (2): t ,

[0020] (2).

[0021] As a further improvement, obtain the Lth historical runoff data of the i-th sample among the K samples with the highest similarity And the corresponding historical forecast data ; Calculate the prediction error according to formula (3) ,

[0022] (3);

[0023] The similarity set S is normalized according to formula (1);

[0024] (1);

[0025] Where, 1≤i≤K, 1≤j≤K, and the runoff forecast value r is calculated using the weighted mean model according to formula (4): t ,

[0026] (4).

[0027] Preferably, the runoff forecast value r is calculated using a runoff forecast model. t , specifically:

[0028] Use K similar samples to train the time series neural network model; use the current rainfall sequence F to input the trained time series neural network model to obtain the runoff forecast value r t .

[0029] As a further improvement, the current runoff sequence R={r t-1 ,r t-2 ,…,r t-L+1} and the corresponding predicted value sequence PR={pr t-1 ,pr t-2 ,…,pr t-L+1};

[0030] According to formula (5), the total prediction error α is calculated.

[0031] (5);

[0032] Where, t-1≤m≤t-L+1, and the runoff forecast value r is calculated using formula (6): t Correction is performed to obtain the corrected runoff forecast value ,

[0033] (6).

[0034] In a second aspect, the present invention discloses a data-driven rolling runoff forecasting system, comprising the following functional modules:

[0035] Sample database construction module: divide the historical runoff data and historical rainfall data according to the sliding window size L and the sliding window step Step to construct a sample database; where L and Step are integers, L is greater than Step, and Step is greater than 0;

[0036] Data acquisition module: obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1}, F={f t-1 ,f t-2 ,…,f t-L+1}, t represents the current time;

[0037] Similar sample acquisition module: Use the current rainfall sequence F to perform similarity calculation in the sample database and obtain the K samples with the highest similarity;

[0038] Runoff forecast module: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models and weighted mean models.

[0039] The present invention obtains K relatively similar samples by calculating the similarity between the current rainfall characteristics and the historical rainfall characteristics in the sample database, and predicts the current runoff by weighted calculation of the mean using the runoff characteristics of the similar samples. Alternatively, the present invention trains a time series neural network model using the similar samples and predicts the runoff using the time series neural network model, thereby reducing the data size involved in the runoff forecast and improving the accuracy and speed of the forecast, which is of great significance for flood prevention and disaster reduction during the flood season. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a flow chart of a data-driven rolling runoff forecasting method of the present invention;

[0042] Figure 2 is a flow chart of a method for accelerating similarity matching through clustering of the present invention;

[0043] Figure 3 It is a structural diagram of a data-driven rolling runoff forecasting system of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0045] The present invention discloses a data-driven rolling runoff forecasting method, comprising the following steps: Figure 1 As shown:

[0046] Step S101: divide the historical runoff data and historical rainfall data according to the sliding window size L and the sliding window step size Step to construct a sample database; wherein L and Step are integers, L is greater than Step, and Step is greater than 0.

[0047] With the development of data collection and storage technology, a large amount of complete historical hydrological and meteorological data has been accumulated, including measured runoff data, rainfall, evapotranspiration, etc. These data are continuous on the time axis. In order to facilitate the use of these data, the historical data is divided based on the time axis by the sliding window method. Read the historical runoff data and historical rainfall data, align the historical runoff data and historical rainfall data on the time axis, slide on the historical runoff data and historical rainfall data according to the set sliding window step Step, and obtain a sample for each sliding window step Step. Each sample includes L historical runoff data and historical rainfall data. The size of L and S can be set according to actual needs. For example, L is 6 (hours) and S is 3 (hours). For example, the historical data from 0 to 5 hours is one sample, and after sliding for 3 hours, the historical data from 3 to 8 hours is another sample.

[0048] As a further improvement, the sample database also includes historical prediction data, which includes runoff prediction values ​​and types of runoff forecast models. As the prediction model is used, the runoff prediction values ​​of the historical prediction models are also stored, and the runoff prediction values ​​and the runoff measured values ​​have a corresponding relationship. Types of runoff forecast models include time series neural network models and weighted mean models. By reading historical prediction data; aligning the historical prediction data with the historical runoff data and the historical rainfall data on the time axis, sliding on the historical prediction data, historical runoff data and historical rainfall data according to the set sliding window step Step, and obtaining a sample for each sliding window step Step, each sample includes L historical runoff data, L historical rainfall data, and L historical prediction data.

[0049] Each sample includes L historical runoff data, which can be expressed as {r1, r2, …, r L-1 |r L}, where, when training the temporal neural network model, (r1, r2, …, r L-1 ) as the input sequence, r L as the output target, and r L It is used to calculate the runoff forecast value in the weighted mean model. For L historical rainfall data, it can be expressed as {f1,f2,…,f L-1 |f L}, where when calculating the similarity with the current rainfall sequence F, (f1,f2,…,f L-1) as a vector to participate in the similarity calculation. For L historical prediction data, it can be expressed as {pr1,pr2,…,pr L-1 |p L}, where p L Participate in the adjustment or correction of the predicted value of the runoff forecast model. If there are multiple types of runoff forecast models, there are multiple groups of L historical forecast data.

[0050] Step S102: Obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1},F={f t-1 ,f t-2 ,…,f t-L+1}, t represents the current time. The current runoff data can be measured by a flow meter or a water level meter, and the current rainfall can be measured by a rain sensor, which will not be described in detail here.

[0051] Step S103: Use the current rainfall sequence F to perform similarity calculation in the sample database to obtain K samples with the highest similarity.

[0052] Preferably, the current rainfall sequence F is used to perform similarity calculation in the sample database to obtain K samples with the highest similarity, specifically:

[0053] Traverse each sample in the sample database and organize the historical rainfall data of the sample into a vector (f1, f2, …, f L-1 ), calculate the vector and the current rainfall sequence F={f t-1 ,f t-2 ,…,f t-L+1}. The method for calculating the similarity can be cosine similarity, which is not specifically limited. After each sample in the sample database has participated in the similarity calculation, the samples are sorted in descending order according to the similarity to obtain the K samples with the highest similarity and the similarity set S of the K samples with the highest similarity, S={S1,S2,…,S K}.

[0054] If there is a lot of data in the sample database, in order to further speed up the calculation of similarity, the present invention proposes a further improvement scheme, such as Figure 2As shown. Use the K-Means algorithm to cluster the historical rainfall data in the sample database to obtain N clusters and the feature vector of each cluster. The feature vector is preferably the mean of all samples in the cluster, which is not specifically limited. Calculate the similarity between the current rainfall sequence F and the feature vector of each cluster. Sort the clusters in descending order according to the similarity and obtain the first a clusters. Obtain the samples in the first a clusters. Calculate the similarity between each sample in the first a clusters and the current rainfall sequence F={f t-1 ,f t-2 ,…,f t-L+1}. Sort the samples in the first a clusters in descending order according to the similarity, obtain the K samples with the highest similarity, and the similarity set S of the K samples with the highest similarity, S={S1,S2,…,S K}.

[0055] Through the clustering algorithm, the sample database is clustered into several clusters. The current rainfall sequence F is first calculated with the feature vector of each cluster for similarity, the clusters are screened, and then the rainfall sequence F is calculated with the samples in the screened clusters for similarity calculation, which improves the speed of screening similar samples.

[0056] Step S104: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models and weighted mean models.

[0057] Preferably, the runoff forecast value r is calculated using a weighted mean model. t Specifically, the similarity set S is normalized according to formula (1). .

[0058] (1) ;

[0059] According to formula (2), the runoff forecast value r is calculated using the weighted mean model. t , S in formula (2) i After normalization by formula (1),

[0060] (2).

[0061] As a further improvement, obtain the Lth historical runoff data of the i-th sample among the K samples with the highest similarity , and obtain the corresponding historical prediction data according to the type of runoff forecast model For example, if the type of runoff forecast model is a time series neural network model, then obtain the historical forecast data of the time series neural network model If the runoff forecast model type is a weighted mean model, then obtain the historical forecast data of the weighted mean model , calculate the prediction error according to formula (3) ,

[0062] (3);

[0063] The similarity set S is normalized according to formula (1);

[0064] (1);

[0065] According to formula (4), the runoff forecast value r is calculated using the weighted mean model. t ,

[0066] (4).

[0067] The present invention calculates the runoff forecast value based on the historical runoff data and prediction errors of the K samples with the highest similarity, and the similarity between the K samples with the highest similarity and the current rainfall sequence F, thereby further improving the forecast accuracy.

[0068] Preferably, the runoff forecast value r is calculated using a runoff forecast model. t , specifically:

[0069] Use K similar samples to train the time series neural network model, use the current rainfall sequence F to input the trained time series neural network model, and obtain the runoff forecast value r t The temporal neural network model may adopt RNN or LSTM, and there is no specific limitation on this.

[0070] As a further improvement, the current runoff sequence R={r t-1 ,r t-2 ,…,r t-L+1} and the corresponding predicted value sequence PR={pr t-1 ,pr t-2 ,…,pr t-L+1}.

[0071] According to formula (5), the total prediction error α is calculated.

[0072] (5);

[0073] Where, t-1≤m≤t-L+1, and the runoff forecast value r is calculated using formula (6): t Correction is performed to obtain the corrected runoff forecast value ,

[0074] (6).

[0075] Construct a new current runoff sequence R={r t ,r t-1 ,r t-2 ,…,r t-L+2} and the current rainfall sequence F={f t ,f t-1 ,f t-2 ,…,f t-L+2}, execute steps 103-104 to realize rolling runoff forecast.

[0076] In a second aspect, the present invention discloses a data-driven rolling runoff forecasting system, comprising the following functional modules: Figure 3 As shown:

[0077] Sample database construction module: divide the historical runoff data and historical rainfall data according to the sliding window size L and the sliding window step Step to construct a sample database; where L and Step are integers, L is greater than Step, and Step is greater than 0;

[0078] Data acquisition module: obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1}, F={f t-1 ,f t-2 ,…,f t-L+1}, t represents the current time;

[0079] Similar sample acquisition module: Use the current rainfall sequence F to perform similarity calculation in the sample database and obtain the K samples with the highest similarity;

[0080] Runoff forecast module: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models and weighted mean models.

[0081] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0082] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the device described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the device described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the device shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the device shown here.

[0083] Those of ordinary skill in the art will appreciate that the units and device steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different devices to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0084] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A data-driven rolling runoff forecasting method, characterized in that: The following steps are involved: Step S101: dividing the historical runoff data and the historical rainfall data according to the sliding window size L and the sliding window step size Step to construct a sample database; wherein L and Step are integers, L is greater than Step, and Step is greater than 0; Read historical runoff data and historical rainfall data, align the historical runoff data and historical rainfall data on the time axis, slide on the historical runoff data and historical rainfall data according to the set sliding window step Step, and obtain a sample for each sliding window step Step, and each sample includes L historical runoff data and historical rainfall data; The sample database also includes historical prediction data; the historical prediction data includes runoff prediction values ​​and types of runoff forecast models; Read historical prediction data; align the historical prediction data with the historical runoff data and the historical rainfall data on the time axis, slide on the historical prediction data, historical runoff data and historical rainfall data according to the set sliding window step Step, and obtain a sample for each sliding window step Step. Each sample includes L historical runoff data, L historical rainfall data and L historical prediction data; Each sample includes L historical runoff data, which can be expressed as {r1, r2, …, r L-1 |r L }, where, when training the temporal neural network model, (r1, r2, …, r L-1 ) as the input sequence, r L As the output target; for L historical rainfall data, it can be expressed as {f1,f2,…,f L-1 |f L }, where when calculating the similarity with the current rainfall sequence F, (f1,f2,…,f L-1 ) as a vector to participate in the similarity calculation; for L historical prediction data, it can be expressed as {pr1,pr2,…,pr L-1 |p L }, where p L Participate in the adjustment or correction of the predicted values ​​of the runoff forecast model; Step S102: Obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1 }, F={f t-1 ,f t-2 ,…,f t-L+1 }, t represents the current time; Step S103: using the current rainfall sequence F to perform similarity calculation in the sample database, and obtaining K samples with the highest similarity; Use K-Means algorithm to cluster the historical rainfall data in the sample database to obtain N clusters and the feature vector of each cluster; calculate the similarity between the current rainfall sequence F and the feature vector of each cluster; sort the clusters in descending order according to the similarity to obtain the first a clusters; obtain the samples in the first a clusters, and calculate the similarity between each sample in the first a clusters and the current rainfall sequence F={f t-1 ,f t-2 ,…,f t-L+1 }; sort the samples in the first a clusters in descending order according to the similarity, obtain the K samples with the highest similarity, and the similarity set S of the K samples with the highest similarity, S={S1,S2,…,S K }; Step S104: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models; Use K similar samples to train the time series neural network model; use the current rainfall sequence F to input the trained time series neural network model to obtain the runoff forecast value r t .

2. A data-driven rolling runoff forecasting method as claimed in claim 1, characterized in that: Get the current runoff sequence R={r t-1 ,r t-2 ,…,r t-L+1 } and the corresponding predicted value sequence PR={pr t-1 ,pr t-2 ,…,pr t-L+1 }; According to formula (5), the total prediction error α is calculated. （5）； Where, t-1≤m≤t-L+1, and the runoff forecast value r is calculated using formula (6): t Correction is performed to obtain the corrected runoff forecast value , （6）。 3. A data-driven rolling runoff forecasting system, used to execute the data-driven rolling runoff forecasting method according to any one of claims 1 to 2, characterized in that: Includes the following functional modules: Sample database construction module: divide the historical runoff data and historical rainfall data according to the sliding window size L and the sliding window step Step to construct a sample database; where L and Step are integers, L is greater than Step, and Step is greater than 0; Data acquisition module: obtain the current runoff sequence R and the current rainfall sequence F; where R={r t-1 ,r t-2 ,…,r t-L+1 },F={f t-1 ,f t-2 ,…,f t-L+1 }, t represents the current time; Similar sample acquisition module: Use the current rainfall sequence F to perform similarity calculation in the sample database and obtain the K samples with the highest similarity; Runoff forecast module: Based on the current runoff sequence R, the current rainfall sequence F, and the K samples with the highest similarity, the runoff forecast value r is calculated using the runoff forecast model. t ; Among them, the types of runoff forecast models include time series neural network models.

Images