Transform-based river extreme flow measuring and calculating method and system

Abstract

The invention discloses a Transform-based river extreme flow measuring and calculating method and system, and the method comprises the steps: firstly collecting the daily rainfall data of each station in a river basin, enabling the starting and ending time and length of the daily rainfall data of each station to be consistent, obtaining the collected data, carrying out the position coding of the collected data, and obtaining the coded data; according to the method, a time window mechanism and a runoff classification technology are introduced into a Transform model to limit an attention calculation range and capture local short-term dependency features, so that the calculation complexity is reduced; according to the method, extreme traffic event simulation can be optimized, the simulation capability of extreme traffic is improved under the condition that the simulation precision of medium and small traffic is ensured, the function of balancing local and global feature relationships among a plurality of time windows in a sliding window mode is also realized, the learning capability of the model for global information is enhanced, and the learning efficiency of the model is improved. And the situation that the complete flood process is cut by a time window and information is missed is avoided, and the method is suitable for wide application and popularization.

Description

Technical Field

[0001] The present invention relates to the technical field of hydraulic engineering measurement and calculation, and in particular to a method and system for calculating extreme flow in a river channel based on Transformer. Background Art

[0002] Physical models and data-driven machine learning models are widely used in the field of hydrological simulation. As an improved form of recurrent neural network (RNN), long short-term memory (LSTM) is widely used in hydrological simulation due to its excellent performance in processing time series data.

[0003] At present, the long short-term memory network (LSTM) has certain limitations in processing ultra-long sequence data and capturing complex long-distance dependencies, such as long training time, difficulty in parallel computing, and limited ability to capture global features. In addition, extreme flow events such as floods and droughts occur less frequently in training data, and the model does not learn enough about these rare events during training, which leads to low simulation accuracy of extreme flows. The attention mechanism tends to focus on global information and may ignore local short-term dependency features, which is not conducive to accurately simulating extreme events. Therefore, it is necessary to design a Transformer-based river extreme flow measurement method and system. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art, in order to better and effectively solve the problems that the long short-term memory network LSTM has certain limitations when processing ultra-long sequence data and capturing complex long-distance dependencies, such as long training time, difficulty in parallel calculation and limited ability to capture global features; and because extreme flow events such as floods and droughts appear less frequently in training data, and the model does not learn these rare events enough during the training process, this leads to low simulation accuracy of extreme flows, and the attention mechanism tends to focus on global information and may ignore local short-term dependency features, which is also a disadvantage for accurately simulating extreme events. A method and system for calculating extreme flow in a river channel based on Transformer is provided, which realizes the function of introducing a time window mechanism in the Transformer model to limit the scope of attention calculation and capture local short-term dependency features, which not only reduces the computational complexity, but also optimizes the simulation of extreme flow events and improves the simulation ability of extreme flow while ensuring the accuracy of small and medium flow simulation, and also realizes the function of balancing the local and global feature relationships between multiple time windows by using a sliding window method, which not only enhances the model's learning ability of global information, but also avoids the situation where the time window cuts the complete flood process and information omission occurs.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A method for calculating river extreme flow based on Transformer, comprising the following steps: Step A, collecting daily rainfall data of each station in the river basin and keeping the start and end time and length of the daily rainfall data of each station consistent to obtain collected data; Step B, position encoding the collected data to obtain encoded data; Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data; Step D, inputting the window-divided data into the classification encoder and classification decoder in sequence for processing to obtain a runoff classification data set; Step E: input the classified runoff data set into the low flow encoder, high flow encoder and extremely high flow encoder respectively to obtain the river extreme flow calculation results and complete the river extreme flow calculation operation.

[0006] The above-mentioned Transformer-based river extreme flow measurement method, step A, collects daily rainfall data of each station in the river basin and keeps the start and end time and length of the daily rainfall data of each station consistent to obtain the collected data, wherein keeping the start and end time and length of the daily rainfall data of each station consistent is specifically forming The matrix of is the total number of days for recording meteorological data, is the number of sites.

[0007] In the above-mentioned Transformer-based river channel extreme flow measurement method, step B performs position encoding on the collected data to obtain encoded data, wherein the position encoding is used to map the output vector to a time series. The specific position encoding formula is shown in formula (1): , (1) in, is the position code, is the encoding position, is the position encoding dimension, is the vector dimension.

[0008] In the above-mentioned Transformer-based river channel extreme flow measurement method, step C, the encoded data is divided into sliding time windows using a sliding window mechanism to obtain window-divided data, wherein the sliding window mechanism is to limit the attention calculation in the encoded data to a length of The time window and reduce the computational complexity to , and the time series in the sliding time window partition is , and each time window contains time steps, the time window is as shown in formula (2), (2) in, is the index number of the sliding window, used to identify the kth time window.

[0009] In the above-mentioned Transformer-based river extreme flow measurement method, step D, the window-divided data is sequentially input into the classification encoder and the classification decoder for processing to obtain the runoff classification data set. The specific steps are as follows: Step D1, inputting the window-divided data into a classification encoder to obtain the classified encoded data, wherein the classification encoder includes a multi-head self-attention layer and a fully connected neural network layer, wherein the multi-head self-attention layer is used to construct the long-range dependency of each position so as to capture the relationship between the data after the window division, wherein the fully connected neural network layer is composed of an input layer, a hidden layer and an output layer, wherein the fully connected neural network layer is used to further integrate and map the relationship between the window-divided data captured by the multi-head self-attention layer, wherein the multi-head self-attention layer specifically uses a linear transformation on the input sequence to obtain a query vector Q, a key vector K and a value vector V, and then calculates the attention weight for each pair of query vector Q, key vector K and value vector V and performs a weighted sum to obtain an attention representation, and then linearly transforms the spliced ​​attention representations and outputs them; Step D2, input the classified encoded data into the classified decoder to obtain the classified decoded data, the classified decoder includes a fully masked self-attention layer, a multi-head cross-attention layer and a fully connected neural network layer, the fully masked self-attention layer is used to shield the future information that has not been predicted during the decoding process and make the classified decoder only access the determined historical position signal at each time step to prevent information leakage and ensure that the prediction process conforms to the causal relationship of the time series, the multi-head cross-attention layer is used to establish the long-range dependency between the classification encoder output and the classification decoder input, and the fully connected neural network layer is used to process the classified encoded data using residual connection and layer normalization.

[0010] In the above-mentioned Transformer-based river extreme flow measurement method, step E, the runoff classification data set is input into the low flow encoder, the high flow encoder and the extremely high flow encoder respectively to obtain the river extreme flow measurement result, and the river extreme flow measurement operation is completed, wherein each time window in the runoff classification data set The traffic forecast results for the last day of the time window will be output Then, according to the data set division order and position coding after runoff classification, the flow output results of each sliding window are reorganized and the river extreme flow calculation results are obtained as follows: .

[0011] A river extreme flow measurement system based on Transformer includes a data acquisition module, a data encoding module, a data division module, a data classification module and a data measurement module, wherein the data acquisition module is used to collect daily rainfall data of each station in a river basin and keep the start and end time and length of the daily rainfall data of each station consistent, thereby obtaining the collected data; the data encoding module is used to perform position encoding on the collected data and obtain the encoded data; the data division module is used to perform sliding time window division on the encoded data using a sliding window mechanism and obtain the window-divided data; the data classification module is used to input the window-divided data into a classification encoder and a classification decoder in sequence for processing and obtain a runoff classification data set; the data measurement module inputs the runoff classification data set into a low flow encoder, a high flow encoder and an extremely high flow encoder respectively and obtains the river extreme flow measurement result, thereby completing the river extreme flow measurement operation.

[0012] The beneficial effects of the present invention are as follows: a method and system for calculating extreme flow in a river based on Transformer of the present invention first collects daily rainfall data of each station in the river basin and keeps the start and end time and length of the daily rainfall data of each station consistent, thereby obtaining collected data, then position-encodes the collected data and obtains encoded data, then uses a sliding window mechanism to divide the encoded data into sliding time windows and obtains window-divided data, then sequentially inputs the window-divided data into a classification encoder and a classification decoder for processing and obtains a runoff classification data set, then inputs the runoff classification data set into a low-flow encoder, a high-flow encoder and an extremely high-flow encoder respectively and obtains the river extreme flow measurement result, thereby completing the river extreme flow measurement operation; effectively realizing that the method and system for calculating extreme flow in a river have The Transformer model introduces a time window mechanism to limit the scope of attention calculation and capture local short-term dependency features, and can directly model the dependency relationship between any positions in the sequence, with the advantage of parallel computing, which shows higher efficiency and performance when processing long sequence data. It not only reduces the computational complexity, but also optimizes the simulation of extreme flow events and improves the simulation ability of extreme flow while ensuring the accuracy of small and medium flow simulation. It also realizes the function of balancing the local and global feature relationships between multiple time windows using a sliding window method, which not only enhances the model's ability to learn global information, but also avoids the situation where the time window cuts the complete flood process and information is missed. At the same time, the runoff classification simulation method is introduced for training and prediction respectively, which enhances the model's ability to learn extreme events. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is an overall flow chart of a method for calculating river channel extreme flow based on Transformer of the present invention; Figure 2 It is a schematic diagram of the working principle of the multi-head self-attention layer of the present invention; Figure 3 It is a comparison chart of the simulation results of daily runoff during the test period in the real-time example of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be further described below in conjunction with the accompanying drawings.

[0015] like Figure 1 As shown, a method for calculating river extreme flow based on Transformer of the present invention comprises the following steps: Step A, collecting daily rainfall data of each station in the river basin and keeping the start and end time and length of the daily rainfall data of each station consistent, to obtain the collected data, wherein keeping the start and end time and length of the daily rainfall data of each station consistent is specifically forming The matrix of is the total number of days for recording meteorological data, is the number of sites.

[0016] Step B, position encoding the collected data to obtain encoded data, wherein the position encoding is used to map the output vector to a time series. The specific position encoding formula is shown in formula (1): , (1) in, is the position code, is the encoding position, is the position encoding dimension, is the vector dimension.

[0017] Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data, wherein the sliding window mechanism is to limit the attention calculation in the encoded data to a length of The time window and reduce the computational complexity to , and the time series in the sliding time window partition is , and each time window contains time steps, the time window is as shown in formula (2), (2) in, is the index number of the sliding window, used to identify the kth time window.

[0018] like Figure 2 As shown, in step D, the window-divided data is sequentially input into the classification encoder and classification decoder for processing to obtain the runoff classification data set. The specific steps are as follows: Step D1, inputting the window-divided data into a classification encoder to obtain the classified encoded data, wherein the classification encoder includes a multi-head self-attention layer and a fully connected neural network layer, wherein the multi-head self-attention layer is used to construct the long-range dependency of each position so as to capture the relationship between the data after the window division, wherein the fully connected neural network layer is composed of an input layer, a hidden layer and an output layer, wherein the fully connected neural network layer is used to further integrate and map the relationship between the window-divided data captured by the multi-head self-attention layer, wherein the multi-head self-attention layer specifically uses a linear transformation on the input sequence to obtain a query vector Q, a key vector K and a value vector V, and then calculates the attention weight for each pair of query vector Q, key vector K and value vector V and performs a weighted sum to obtain an attention representation, and then linearly transforms the spliced ​​attention representations and outputs them; Step D2, input the classified encoded data into the classified decoder to obtain the classified decoded data, the classified decoder includes a fully masked self-attention layer, a multi-head cross-attention layer and a fully connected neural network layer, the fully masked self-attention layer is used to shield the future information that has not been predicted during the decoding process and make the classified decoder only access the determined historical position signal at each time step to prevent information leakage and ensure that the prediction process conforms to the causal relationship of the time series, the multi-head cross-attention layer is used to establish the long-range dependency between the classification encoder output and the classification decoder input, and the fully connected neural network layer is used to process the classified encoded data using residual connection and layer normalization.

[0019] Step E: Input the classified runoff dataset into the low flow encoder, high flow encoder and extremely high flow encoder respectively to obtain the river extreme flow calculation results, and complete the river extreme flow calculation operation, wherein each time window in the classified runoff dataset The traffic forecast results for the last day of the time window will be output Then, according to the data set division order and position coding after runoff classification, the flow output results of each sliding window are reorganized and the river extreme flow calculation results are obtained as follows: .

[0020] In order to better illustrate the use effect of the present invention, a specific embodiment of the present invention is introduced below.

[0021] S1. Improve the computational efficiency of the model: The time window mechanism is introduced to reduce the complexity of attention calculation and improve the computational efficiency of the model. A numerical implementation example is constructed and a random data set containing 10,000, 50,000, and 100,000 samples is generated. Each sample contains 20 feature values ​​and 1 target value. The LSTM model, the original The Transformer model and the Transformer model proposed in the present invention were trained. To ensure that the results are statistically significant, this embodiment independently trained each model 10 times, recorded the time consumed for each training and calculated the average training time. The computer CPU configuration used in the numerical embodiment is i7-12700k and the GPU configuration is RTX3060Ti. The following table shows the training efficiency of LSTM (abbreviated as L), the original Transformer (abbreviated as T) and the Transformer (abbreviated as MT) models of the present invention under different data sets of different sizes. The results show that the training efficiency of the Transformer model proposed in the present invention is 6.5~7.5 times and 3.8~4 times higher than that of the LSTM and the original Transformer models, respectively.

[0022] Table 1. Comparison of training efficiency of various models under different scale data sets (unit: seconds)

[0023] S2. Enhance the simulation capability of extreme flow: The model's learning and prediction capabilities for extreme flow events are significantly improved through runoff classification simulation, which makes up for the shortcomings of traditional models in extreme event simulation, and can also maintain good accuracy in small and medium flow simulation. This embodiment is based on the daily precipitation data of 39 rain gauges in a basin from 1960 to 2019 and the daily runoff data of a hydrological station in the basin. The Transformer model of the present invention, the original Transformer model, the LSTM model and other machine learning models commonly used in the field of rainfall runoff prediction proposed by the present invention are used for training and verification. The model training period is 1972-2007 (12783 days of valid data records), the verification period is 2008-2012 (1827 days of valid data records), and the test period is 2014-2018 (1826 days of valid data records). Some years are excluded due to the lack of flow data at some stations. The number of days in the training period, verification period and test period account for 78%, 11% and 11% respectively. In order to ensure the comparability of the training effects of each model, a consistent early stopping strategy for training is adopted. The simulation effect of the validation period is evaluated in each training. When the evaluation coefficient of the validation period model has not been significantly improved after multiple consecutive trainings, the training is terminated, and the simulation results of the test period are used as the basis for evaluating the simulation effect of the model. The Nash-Sutcliffe efficiency (NSE), the ratio of the root mean square error to the standard deviation of the observed value (RSR), the percentage deviation (PBIAS), and the absolute percentage error of the extreme flow in the top 2% of the frequency (APE-2%) commonly used in hydrological models are used as simulation effect evaluation indicators. The simulation results of each model in the validation period and the test period are shown in the following table. The Transformer model of the present invention shows a simulation performance that is significantly better than other models in both the validation period and the test period. The NSE values ​​of the Transformer model of the present invention during the validation and test periods are 0.91 and 0.85, respectively, which are significantly higher than the original Transformer (0.79 and 0.79), LSTM (0.73 and 0.68) and other commonly used models (ANN, WLSTM, BiLSTM NSE during the validation and test periods are lower than 0.8). At the same time, the RSR of the Transformer model of the present invention during the validation and test periods are 0.29 and 0.39, respectively, which are significantly lower than other models, showing the robustness of the model in the relative standard deviation measurement of errors. In terms of the deviation index (PBIAS), the Transformer of the present invention is 3.96% during the validation period and -0.14% during the test period, which are basically controlled within a very low deviation range.For the simulation of high flow extreme values ​​(APE-2% index), the absolute percentage error of the Transformer model of the present invention is 23.80% and 21.53% in the validation period and the test period, respectively, which is significantly better than the original Transformer (about 24.46%~30.07%) and LSTM (about 27.59%~28.86%) models.

[0024] In general, the Transformer model of the present invention is significantly superior to traditional and existing deep learning models in all evaluation indicators, and has significant improvements in fitting accuracy, error control, and extreme event capture capabilities. This result shows that the Transformer structure of the present invention has better generalization and stability performance in runoff prediction based on precipitation data from multiple rainfall stations.

[0025] Table 2. Comparison of simulation results between the validation period and the test period of each model

[0026] like Figure 3 As shown, in the visual comparison of the runoff simulation results during the test period, the prediction curve of the Transformer model of the present invention is closer to the measured data at the high flow peak. In comparison, the original Transformer has more obvious deviations and underestimations in several flood season peak simulations, especially the timing and amplitude capture of large flow events is not accurate enough. The Transformer of the present invention is not only more consistent with the observed data in terms of the overall trend, but also more sensitive to the flow peak, thus showing better characterization capabilities in extreme runoff simulation. This visualization result is consistent with the improvement trend of the aforementioned quantitative indicators (such as NSE, RSR, PBIAS, APE-2%), which further verifies the superior performance of the Transformer of the present invention in actual application scenarios.

[0027] A river extreme flow measurement system based on Transformer includes a data acquisition module, a data encoding module, a data division module, a data classification module and a data measurement module, wherein the data acquisition module is used to collect daily rainfall data of each station in a river basin and keep the start and end time and length of the daily rainfall data of each station consistent, thereby obtaining the collected data; the data encoding module is used to perform position encoding on the collected data and obtain the encoded data; the data division module is used to perform sliding time window division on the encoded data using a sliding window mechanism and obtain the window-divided data; the data classification module is used to input the window-divided data into a classification encoder and a classification decoder in sequence for processing and obtain a runoff classification data set; the data measurement module inputs the runoff classification data set into a low flow encoder, a high flow encoder and an extremely high flow encoder respectively and obtains the river extreme flow measurement result, thereby completing the river extreme flow measurement operation.

[0028] In summary, the present invention provides a method and system for calculating extreme flow in a river based on Transformer. First, daily rainfall data of each station in the river basin is collected and the start and end time and length of the daily rainfall data of each station are kept consistent, thereby obtaining collected data. Then, the collected data is position-encoded and the encoded data is obtained. Then, the encoded data is divided into sliding time windows using a sliding window mechanism and the window-divided data is obtained. Then, the window-divided data is sequentially input into a classification encoder and a classification decoder for processing and a runoff classification data set is obtained. Subsequently, the runoff classification data set is respectively input into a low-flow encoder, a high-flow encoder and an extremely high-flow encoder to obtain the river extreme flow calculation result, thereby completing the river extreme flow measurement operation. The method and system for calculating extreme flow in a river are effectively realized. The time window mechanism is introduced in the ransformer model to limit the scope of attention calculation and capture the function of local short-term dependency features, and can directly model the dependency relationship between any positions in the sequence, with the advantage of parallel computing, which shows higher efficiency and performance when processing long sequence data. It not only reduces the computational complexity, but also optimizes the simulation of extreme flow events and improves the simulation ability of extreme flow while ensuring the accuracy of small and medium flow simulation. It also realizes the function of balancing the local and global feature relationships between multiple time windows using a sliding window method, which not only enhances the model's ability to learn global information, but also avoids the situation where the time window cuts the complete flood process and information is missed. At the same time, the runoff classification simulation method is introduced for training and prediction respectively, which enhances the model's learning ability for extreme events.

[0029] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for calculating extreme flow in a river based on Transformer, characterized in that: The following steps are included: Step A, collecting daily rainfall data of each station in the river basin and keeping the start and end time and length of the daily rainfall data of each station consistent to obtain collected data; Step B, position encoding the collected data to obtain encoded data; Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data; Step D, inputting the window-divided data into the classification encoder and classification decoder in sequence for processing to obtain a runoff classification data set; Step E: input the classified runoff data set into the low flow encoder, high flow encoder and extremely high flow encoder respectively to obtain the river extreme flow measurement results, thus completing the river extreme flow measurement operation.

2. According to the Transformer-based river extreme flow measurement method of claim 1, it is characterized by: Step A, collecting daily rainfall data of each station in the river basin and keeping the start and end time and length of the daily rainfall data of each station consistent, to obtain the collected data, wherein keeping the start and end time and length of the daily rainfall data of each station consistent is specifically forming The matrix of is the total number of days for recording meteorological data, is the number of sites.

3. The method for calculating river extreme flow based on Transformer according to claim 2, characterized in that: Step B, position encoding the collected data to obtain encoded data, wherein the position encoding is used to map the output vector to a time series. The specific position encoding formula is shown in formula (1): ， （1） in, is the position code, is the encoding position, is the position encoding dimension, is the vector dimension.

4. The method for calculating river extreme flow based on Transformer according to claim 3 is characterized by: Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data, wherein the sliding window mechanism is to limit the attention calculation in the encoded data to a length of The time window and reduce the computational complexity to , and the time series in the sliding time window partition is , and each time window contains time steps, the time window is as shown in formula (2), （2） in, is the index number of the sliding window, used to identify the kth time window.

5. The method for calculating river extreme flow based on Transformer according to claim 4, characterized in that: Step D: Input the window-divided data into the classification encoder and classification decoder in turn for processing to obtain the runoff classification data set. The specific steps are as follows: Step D1, inputting the window-divided data into a classification encoder to obtain the classified encoded data, wherein the classification encoder includes a multi-head self-attention layer and a fully connected neural network layer, wherein the multi-head self-attention layer is used to construct the long-range dependency of each position so as to capture the relationship between the data after the window division, wherein the fully connected neural network layer is composed of an input layer, a hidden layer and an output layer, wherein the fully connected neural network layer is used to further integrate and map the relationship between the window-divided data captured by the multi-head self-attention layer, wherein the multi-head self-attention layer specifically uses a linear transformation on the input sequence to obtain a query vector Q, a key vector K and a value vector V, and then calculates the attention weight for each pair of query vector Q, key vector K and value vector V and performs a weighted sum to obtain an attention representation, and then linearly transforms the spliced ​​attention representations and outputs them; Step D2, input the classified encoded data into the classified decoder to obtain the classified decoded data, the classified decoder includes a fully masked self-attention layer, a multi-head cross-attention layer and a fully connected neural network layer, the fully masked self-attention layer is used to shield the future information that has not been predicted during the decoding process and make the classified decoder only access the determined historical position signal at each time step to prevent information leakage and ensure that the prediction process conforms to the causal relationship of the time series, the multi-head cross-attention layer is used to establish the long-range dependency between the classification encoder output and the classification decoder input, and the fully connected neural network layer is used to process the classified encoded data using residual connection and layer normalization.

6. The method for calculating river extreme flow based on Transformer according to claim 5, characterized in that: Step E: Input the classified runoff dataset into the low flow encoder, high flow encoder and extremely high flow encoder respectively to obtain the river extreme flow measurement results, and complete the river extreme flow measurement operation, wherein each time window in the classified runoff dataset The traffic forecast results for the last day of the time window will be output Then, according to the data set division order and position coding after runoff classification, the flow output results of each sliding window are reorganized and the river extreme flow calculation results are obtained as follows: .

7. A river channel extreme flow measurement system based on Transformer, wherein the specific measurement process of the river channel extreme flow measurement system is based on the river channel extreme flow measurement method according to any one of claims 1 to 6, characterized in that: It includes a data collection module, a data encoding module, a data division module, a data classification module and a data calculation module. The data collection module is used to collect daily rainfall data of each station in the river basin and keep the start and end time and length of the daily rainfall data of each station consistent, so as to obtain the collected data; The data encoding module is used to perform position encoding on the collected data and obtain encoded data; The data partitioning module is used to partition the encoded data into sliding time windows using a sliding window mechanism and obtain the data after window partitioning; The data classification module is used to input the window-divided data into the classification encoder and the classification decoder in sequence for processing and obtain the runoff classification data set; The data calculation module inputs the runoff classification data set into the low flow encoder, the high flow encoder and the extremely high flow encoder respectively and obtains the river extreme flow calculation result, thereby completing the river extreme flow calculation operation.

Images