Space-time attention-based sequence-to-sequence multi-step flood forecasting method and device

Abstract

The application provides a sequence-to-sequence multi-step flood forecasting method and device based on space-time attention, and adds a space attention model to an input item and adds a time attention model to a deep learning framework. The space attention model can sufficiently learn the spatial distribution information of rainfall, and has a good effect in a small-flow typhoon simulation, and the time attention model can strengthen the learning of a rapid confluence process, and has a good effect in a large-flow typhoon simulation. In the application of the GRU-Seq2Seq model added with the space-time attention mechanism, it can be found that the model can effectively learn different types of floods, and effectively solve the time lag phenomenon in the mountain small watershed forecasting and the problem of low flood peak forecasting precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of flood forecasting, and particularly relates to a sequence-to-sequence multi-step flood forecasting method and device based on spatiotemporal attention. BACKGROUND

[0002] Mountain flood disaster is a global problem of flood control and watershed water resources management, and is also a difficult and weak link of China's flood control work, which seriously threatens global water security and people's life and property safety. Especially in the mountainous small watershed under the extreme typhoon, due to the lack of monitoring sites and insufficient station network density of small watershed and the uneven spatiotemporal distribution of rainstorm, the short confluence time makes it difficult to achieve rapid and accurate multi-step prediction in advance in the mountainous small watershed.

[0003] In recent years, using deep learning network to forecast flood has become a new development direction. Deep learning network can use only historical data to forecast flood, and compared with traditional methods, deep learning network can get better prediction results with lower computational cost and simpler modeling process. However, due to the lack of measured data and the decrease of prediction accuracy with the increase of prediction lead time, the application of deep learning network in flood forecasting is limited.

[0004] In order to be able to use less measured data for prediction and improve the accuracy of multi-step prediction, GRU algorithm can be combined with sequence-to-sequence (Seq2Seq) model to construct GRU-Seq2Seq flood forecasting model. The GRU-Seq2Seq flood forecasting model can improve the conversion relationship of rainfall runoff sequence and the correlation on runoff time step, improve the model's ability to mine time information, and improve the model prediction effect, especially in the prediction of peak occurrence time. The time lag phenomenon of flood process is effectively alleviated. However, in the practical application of engineering, it is found that the performance of GRU-Seq2Seq model will be limited by the intermediate vector of Seq2Seq model, and part of the information of the long input sequence will be covered, which may cause the GRU-Seq2Seq model to be unable to fully mine the three-dimensional spatiotemporal information of hydrological data. This leads to the phenomenon of flood peak time lag in flood forecasting under some extreme typhoon weather. SUMMARY

[0005] In order to solve the time lag phenomenon in mountain small watershed prediction and the low precision of flood peak prediction of the GRU-Seq2Seq model, a multi-step flood prediction method based on space-time attention is provided on the basis of the GRU-Seq2Seq model, and a space attention model is added to the input item and a time attention model is added to the deep learning framework. The space attention model can fully learn the spatial distribution information of rainfall, and the effect is good in small flow typhoon simulation, and the time attention model can strengthen the learning of the rapid confluence process, and the effect is good in large flow typhoon simulation. In the application of the GRU-Seq2Seq model added with the space-time attention mechanism, it can be found that the model can effectively learn different types of floods, and effectively solve the time lag phenomenon in mountain small watershed prediction and the low precision of flood peak prediction.

[0006] According to one aspect of the present application, a sequence-to-sequence multi-step flood prediction method based on space-time attention includes the following steps:

[0007] Obtain the flow and extreme typhoon climate rainfall data of the research basin, represent the typhoon rainfall of each time step with sequence according to the rainfall characteristic number, represent the flow of each time step with sequence according to the flow characteristic number, and finally splice the rainfall sequence and flow sequence of k time steps into an input matrix X;

[0008] Divide the input matrix X into k m+n-dimensional feature vectors;

[0009] Calculate the space attention weight of each feature vector respectively;

[0010] Dynamically allocate the weight of each input feature, add a space attention model to the input matrix X, and obtain an input matrix X' containing space attention weights;

[0011] Build a GRU encoder and input the input matrix X' containing the space attention weights, output the encoder hidden layer sequence H and the context encoding vector c;

[0012] Build a GRU decoder and input the output sequence of the last time and the context encoding vector c, and output the decoder hidden layer sequence H';

[0013] Calculate the time attention weight β T ;

[0014] Perform matrix multiplication on the time attention weight and the encoder hidden layer state, and distribute the time attention weight to each time step of the input to obtain the final attention hidden state H at ;

[0015] ​​The context encoding vector c obtained by the GRU encoder is taken as the initial hidden state of the GRU, and the output value of the GRU at the previous moment is input into the GRU decoder, and the multi-step prediction result is obtained by combining the attention hidden state and the decoder hidden state and activating.

[0016] Further, the expression of the input matrix X is as follows:

[0017]

[0018] The expressions of the k m+n-dimensional feature vectors are as follows:

[0019]

[0020]

[0021]

[0022] In the formula, y T 1 ~ y T n is the rainfall and flow data in a single time step, T is the time, m and n are the feature numbers of the rainfall and flow sequence in a single time step, and k is the input time step.

[0023] Further, the step of calculating the spatial attention weight of each feature vector respectively comprises:

[0024] The input k feature vectors x1-x k are sequentially activated by the function Sigmoid and mapped to the interval (0, 1):

[0025]

[0026] In the formula, x is the rainfall and flow data in a single time step feature vector, that is, y T 1 ~ y T n ;

[0027] The feature vector ratio obtained by normalizing by using the function Softmax is the spatial attention weight:

[0028]

[0029] In the formula, α i is the spatial attention weight, and x i is the rainfall and flow data in the feature vector after Sigmoid activation.

[0030] Further, the step of dynamically assigning the weight of each input feature, adding a spatial attention model to the input matrix X to obtain an input matrix X' containing spatial attention weights, comprises:

[0031] Obtaining spatial attention weights of each time step feature vector

[0032] Then, the model input feature vector x' after adding the spatial attention term is calculated according to the following formula: k And the model input matrix X' is:

[0033]

[0034]

[0035] In the formula, · is the Hadamard product, that is, the element product operation.

[0036] Further, the step of constructing a GRU encoder and inputting the input matrix X' containing spatial attention weights, outputting an encoder hidden layer sequence H and a context encoding vector c, comprises:

[0037] The encoder is composed of a GRU, which reads the information of an input sequence of any length in order, and when it reads each data, the hidden state of the GRU changes according to the following formula:

[0038]

[0039] Wherein:

[0040]

[0041]

[0042]

[0043] The context encoding vector c is:

[0044] c = h k

[0045] The encoder hidden layer sequence H is:

[0046] H = [h1, h2, …, h k ] k×u

[0047] In the formula, h t represents the hidden state of the current time of the encoder, h t-1 the hidden state of the previous time of the encoder, x t the model input feature after adding the spatial attention term, h kFor the k-th eigenvector x′ k The hidden information of the input, k is the time step of the input, u is the dimension of the hidden layer state, W, W r W z Both are weight matrices. For the candidate hidden state, r t To reset the door, z t For the update gate, σ represents the sigmoid function, which can transform data into a value in the range of 0-1, and tanh represents the tanh function, which can transform data into a value in the range of [-1,1].

[0048] Further, the step of constructing the GRU decoder and inputting the output value and context encoding vector c from the previous time step to output the decoder hidden layer sequence H′ includes:

[0049] The decoder is another GRU, which is trained to utilize a given hidden state h′ t To predict the output value y at the next time step t The hidden state of the decoder at time t is calculated as follows:

[0050]

[0051] in:

[0052]

[0053]

[0054]

[0055] The decoder hidden layer sequence H′ is:

[0056] H' = [h'1, h'2, ..., h'] n ] n×u

[0057] In the formula, h′ t Let h′ be the hidden state of the decoder at time t. t-1 y represents the hidden state of the decoder in the previous time step; T+t-1 Let n be the output value of the GRU decoder at the previous time step, n be the prediction time step, u be the dimension of the hidden layer state, and W and W' be the output value of the GRU decoder at the previous time step. r W z Both are weight matrices. Let r′ be the candidate hidden state. t To reset the door, z′ t For the update gate, σ is the sigmoid function and tanh is the tanh function.

[0058] Further, the step of calculating the time attention weight β at each time step hidden layer state of the GRU encoder and the GRU decoder comprises: T

[0059] The encoder hidden layer state H is multiplied by the decoder hidden state H' to obtain the score value of the input and output sequence, and the time attention weight β is obtained by normalizing the function Softmax T , and the calculation formula is as follows:

[0060]

[0061] In the formula, is a matrix multiplication operation.

[0062] Further, in the step of performing matrix multiplication on the time attention weight and the encoder hidden layer state to distribute the time attention weight to each time step of the input to obtain the final attention hidden state H at , the specific calculation formula is as follows:

[0063]

[0064] In the formula, β T is the time attention weight, H is the encoder hidden layer sequence, n is the prediction time step, and u is the dimension of the hidden layer state.

[0065] Further, in the step of taking the context encoding vector c obtained by the GRU encoder as the initial hidden state of the GRU, and inputting the output value of the previous time step GRU into the GRU decoder, and combining the attention hidden state and the decoder hidden state to obtain the multi-step prediction result, the specific calculation formula is as follows:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] h′0=c

[0073] In the formula, n is the prediction time step, W s , W h , W, W r , W z ​is a weight matrix, and is a sigmoid function, is an attention hidden state, and y T+t-1 is an output value of the GRU decoder at the previous time, is a joint state of the attention hidden state and the decoder hidden state activated by a tanh function, is a candidate hidden state, and r t is a reset gate, and z′ t is an update gate, is a sigmoid function, and tanh is a tanh function.

[0074] According to another aspect of the present application, the present application provides a sequence-to-sequence multi-step flood forecasting device based on spatial-temporal attention, comprising the following modules:

[0075] An input matrix construction module is configured to obtain flow data and extreme typhoon climate rainfall data of a study basin, represent the typhoon rainfall at each time step by a sequence according to a rainfall characteristic number, represent the flow at each time step by a sequence according to a flow characteristic number, and finally splice the k time-step rainfall sequence and the flow sequence into an input matrix X;

[0076] An input matrix division module is configured to divide the input matrix X into k m+n-dimensional feature vectors;

[0077] A spatial attention weight calculation module is configured to calculate the spatial attention weight of each feature vector respectively;

[0078] A spatial attention adding module is configured to dynamically allocate the weight of each input feature, add a spatial attention model to the input matrix X, and obtain an input matrix X′ containing spatial attention weights;

[0079] An encoder construction and encoding module is configured to construct a GRU encoder and input the input matrix X′ containing the spatial attention weights, and output an encoder hidden layer sequence H and a context encoding vector c;

[0080] A decoder construction and decoding module is configured to construct a GRU decoder and input the output sequence at the previous time and the context encoding vector c, and output a decoder hidden layer sequence H′;

[0081] A temporal attention weight calculation module is configured to calculate the temporal attention weight β T by using the hidden layer state of the GRU encoder and the GRU decoder at each time step;

[0082] An attention hidden state calculation module is configured to perform matrix multiplication on the temporal attention weight and the encoder hidden layer state, allocate the temporal attention weight to each time step of the input, and obtain the final attention hidden state H at;

[0083] A multi-step prediction module is configured to take the context encoding vector c obtained by the GRU encoder as the initial hidden state of the GRU, and input the output value of the GRU at the previous time into the GRU decoder, and combine the attention hidden state and the decoder hidden state to obtain a multi-step prediction result.

[0084] The technical scheme provided by the application has the following beneficial effects:

[0085] (1) By constructing a spatial attention model for input features, the spatial weight in a single time step is calculated, and the weight of the input features is dynamically allocated, thereby improving the stability of the multi-step flood forecasting model and the accuracy of peak value prediction.

[0086] (2) A time attention model is constructed by using the hidden layer state of the GRU encoder and decoder at each time step, and the time attention weight is allocated to each hidden step state of the encoder. The time attention model can accurately predict the peak occurrence time and reduce the time lag phenomenon in flood forecasting. BRIEF DESCRIPTION OF DRAWINGS

[0087] The specific effects of the application will be further described below in combination with the drawings and examples, and the drawings are as follows:

[0088] Figure 1 is a flowchart of the sequence-to-sequence multi-step flood forecasting method based on spatial-temporal attention in the embodiment of the application;

[0089] Figure 2 is a rainfall runoff model structure diagram based on spatial-temporal attention in the embodiment of the application;

[0090] Figure 3 is a scatter plot of the four models at T+1 in the embodiment of the application;

[0091] Figure 4 is a scatter plot of the four models at T+2 in the embodiment of the application;

[0092] Figure 5 is a scatter plot of the four models at T+3 in the embodiment of the application;

[0093] Figure 6 is a Soudelor typhoon flood hydrograph in the embodiment of the application;

[0094] Figure 7 is a Matmo typhoon flood hydrograph in the embodiment of the application;

[0095] Figure 8 is a structure diagram of the sequence-to-sequence multi-step flood forecasting device based on spatial-temporal attention in the embodiment of the application. Detailed Implementation

[0096] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0097] Example 1:

[0098] Please refer to Figure 1 and Figure 2 This invention provides a sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention, comprising the following steps:

[0099] S1: Collect watershed flow and extreme typhoon rainfall data for the research basin to provide training, validation, and testing data for the deep learning model. Then, based on rainfall feature numbers, sequence the typhoon rainfall at each time step. This indicates that the flow at each time step is then sequenced based on the flow characteristic number. This means that the rainfall and flow sequences at k time steps are finally concatenated into the input matrix X.

[0100]

[0101] In the formula, y T 1 ~y T n The data represents rainfall and flow rate within a single time step, where T is time, m and n are the number of rainfall and flow rate sequence features within a single time step, and k is the input time step size.

[0102] S2: Divide the input matrix X into k m+n dimensional eigenvectors, as shown in the following formula:

[0103]

[0104] In the formula, y T 1 ~y T n The data represents rainfall and flow rate within a single time step, where T is time, and m and n are the number of features in the rainfall and flow rate sequence within a single time step.

[0105] S3: In order to fully learn the spatial distribution information of rainfall in the study area, the spatial attention weight of each feature vector is calculated.

[0106] The spatial attention weight calculation process in this embodiment can be summarized as follows:

[0107] Because rainfall and flow characteristics differ significantly, the input k feature vectors, i.e., x1 to x2, are used. k, sequentially through the function Sigmoid activation. After activation, the rainfall and flow data in the feature vector are mapped to the interval (0, 1) to improve the stability of model training.

[0108]

[0109] In the formula, x is the rainfall and flow data in the single time step feature vector, that is, y T 1 ~ y T n .

[0110] Then, the function Softmax is used for normalization, and the proportion of the calculated feature vector is the spatial attention weight:

[0111]

[0112] In the formula, α i is the spatial attention weight, and x i is the rainfall and flow data in the feature vector after Sigmoid activation.

[0113] S4: Dynamically allocate the weight of each input feature, add a spatial attention model to the input matrix X, and finally obtain an input matrix containing spatial attention weights.

[0114] The dynamic allocation of spatial weights in this embodiment can be summarized as follows:

[0115] According to formula (4), the spatial attention weight of each time step feature vector is obtained Then, the input feature vector x' of the model after adding the spatial attention term is calculated according to the following formula: k and the model input matrix X'.

[0116]

[0117]

[0118] In the formula, · is the Hadamard product, that is, the element product operation.

[0119] S5: Build a GRU encoder and input the input matrix X' after adding the spatial attention term, and finally output the encoder hidden layer sequence H and the context encoding vector c.

[0120] In this embodiment, the specific process of GRU encoder input and output is as follows:

[0121] The encoder is composed of a GRU, which reads the information of an input sequence of arbitrary length in order. When it reads each data, the hidden state of the GRU changes according to formula (7).

[0122]

[0123]

[0124]

[0125]

[0126] c = h k (8)

[0127] H = [h1, h2, ..., h k ] k×u (9)

[0128] In the formula, h t h represents the hidden state of the encoder at the current moment. t-1 The hidden state of the encoder at the previous time step, x t h is the input feature of the model after adding a spatial attention term. k For the k-th eigenvector x′ k The hidden information of the input, k is the time step of the input, u is the dimension of the hidden layer state, W, W r W z Both are weight matrices. For the candidate hidden state, r t To reset the door, z t For the update gate, σ represents the sigmoid function, which can transform data into a value in the range of 0-1, and tanh represents the tanh function, which can transform data into a value in the range of [-1,1].

[0129] In this embodiment, the specific input and output process of the GRU decoder is as follows:

[0130] The decoder is another GRU. It is trained to utilize a given hidden state h′ (t) To predict the output sequence y at the next time step t The decoder's hidden state at time t is calculated as follows:

[0131]

[0132]

[0133]

[0134]

[0135] H' = [h'1, h'2, ..., h'] n ]n×u (11)

[0136] where h′ t is the hidden state of the decoder at time t, h′ t-1 is the hidden state of the decoder at the previous time step; y T+t-1 is the output value of the GRU decoder at the previous time step, n is the prediction time step, u is the dimension of the hidden layer state, W, W r , W z are weight matrices, is the candidate hidden state, r′ t is the reset gate, z′ t is the update gate, and σ is the sigmoid function, and tanh is the tanh function.

[0137] S7: Calculate the time attention weight β T using the hidden layer state of each time step of the GRU encoder and decoder.

[0138] In this embodiment, the calculation process of the time attention weight is as follows:

[0139] Get the hidden layer sequence H, H′ output in S5 and S6:

[0140] H = [h1, h2, …, hn] k k×u (9)

[0141] where h is the hidden layer state of each time step calculated by formula (7).

[0142] H′ = [h′1, h′2, …, h′n] n n×u (11)

[0143] where h′ is the hidden layer state of each time step calculated by formula (10)

[0144] Multiply the encoder hidden layer state and the decoder hidden state to get the score value of the input and output sequence, and then normalize it by the function Softmax to get the time attention weight β T , which is calculated as follows:

[0145]

[0146] where, is the matrix multiplication operation.

[0147] S8: Perform matrix multiplication on the time attention weight and the encoder hidden layer state, and distribute the time attention weight to each time step of the input to get the final attention hidden state H at :​​

[0148]

[0149] S9: In the decoding stage, the GRU unit generates the output value of the predicted traffic in sequence, the GRU decoder takes the context vector c obtained by the GRU encoder as the initial hidden state of the GRU, and the output value of the previous time GRU is input into the GRU decoding structure, and the subsequent units no longer accept the input of c.

[0150] At the time of T+1 prediction, since there is no output prediction value of the previous time GRU, the runoff at the current time is input into the GRU-Seq2Seq model, and the attention hidden state and the decoding decoder hidden state h' t are combined to obtain the final multi-step-ahead prediction result, and the specific calculation formula is as follows:

[0151]

[0152]

[0153]

[0154]

[0155]

[0156]

[0157] h'0=c (16)

[0158] In the formula, n is the prediction time step, W s , W h , W, W r , W z is a weight matrix, sigma is a sigmoid function, is an attention hidden state, y T+t-1 is the output value of the previous time GRU decoder, is the joint state of the attention hidden state and the decoder hidden state activated by the tanh function, is a candidate hidden state, r t ' is a reset gate, z' t is an update gate, sigma is a sigmoid function, and tanh is a tanh function. In the formula, n is the prediction time step.

[0159] The implementation mode of the present application is described in detail in combination with specific embodiments.

[0160] The embodiment uses 7 extreme typhoon rainfall recorded by 9 control rainfall stations in the study basin range and reservoir inflow records with a time interval of 1 hour. The characteristics of the 7 extreme typhoon rainfalls are shown in Table 1. The typhoon rainfall in the basin concentrates in July-September, and the rainfall is relatively concentrated, more than 80% of the rainfall concentrates within 24 hours, resulting in short flood concentration time and large flood peak in the basin.

[0161] Table 1 Typhoon basic characteristics table

[0162]

[0163] According to the training rule of the neural network, the 7 typhoon data sets are divided in the embodiment. Under the premise of maintaining similar statistical characteristics, the entire data set is generally divided into a training set, a validation set and a test set according to 6:2:2, and according to the cumulative rainfall size and duration, the data of 5 typhoons of Saola, Dujuan, Megi, Maria and Nepartak are finally used as the training set and the validation set; the data of two typhoons of Soudelor and Matmo are used as the test set, and Table 2 shows the characteristic values of the data set.

[0164] Table 2 Data set characteristics table

[0165]

[0166] The STA-Seq2Seq model (Seq2Seq model with added space-time attention) of the embodiment will be compared with GRU-Seq2Seq (Seq2Seq model based on GRU algorithm), SA-Seq2Seq (Seq2Seq model with added space attention) and TA-Seq2Seq (Seq2Seq model with added time attention) below. The advantages of the present application compared with other models are illustrated by analyzing the prediction results of the validation set and the test set of the four trained models.

[0167] (1) Improve peak prediction accuracy

[0168] Figures 3-5 The scatter plots of the four models for Soudelor and Matmo two typhoons in the 1-3 hour prediction period are shown respectively. Overall, the STA-Seq2Seq model is better than the GRU-Seq2Seq model, and can well realize 1-3 hour advance prediction. In 3 hours of advance prediction, the prediction result of the STA-Seq2Seq model is more concentrated, and is also more accurate in peak prediction.

[0169] (2) Reduce flood peak time lag phenomenon

[0170] Using GRU-Seq2Seq, SA-Seq2Seq, TA-Seq2Seq, STA-Seq2Seq, the four models for Soudelor and Matmo multi-step runoff prediction, see Figures 6-7 wherein Figure 6 (a) is the rainfall of Soudelor typhoon, Figure 6 (b) is the flood hydrograph predicted by the four models 1 hour in advance, Figure 6 (c) is the flood hydrograph predicted 2 hours in advance, Figure 6 (d) is the flood hydrograph predicted 3 hours in advance, Figure 7 (a) is the rainfall of Matmo typhoon, Figure 7 (b) is the flood hydrograph predicted by the four models 1 hour in advance, Figure 7 (c) is the flood hydrograph predicted 2 hours in advance, Figure 7 (d) is the flood hydrograph predicted 3 hours in advance according to Figure 6 and Figure 7 It can be seen that the overall prediction results of the STA-Seq2Seq model are satisfactory, and the simulation of the peak value and the fitting results of the hydrograph are relatively excellent. In the 1-3 hour prediction period, the STA-Seq2Seq model shows good prediction ability, especially in the peak time, which can accurately predict the flood peak time and further reduce the time lag phenomenon. In addition, the model has stronger learning ability for the early rainfall, and the initial rainfall will not immediately cause the runoff to rise, and the simulation effect of the runoff before the peak is more excellent.

[0171] Through the analysis of two typhoon events, it can be found that the TA-Seq2Seq model has better simulation effect for large flow typhoon, and the time attention model can strengthen the learning of the rapid confluence process, which leads to overestimation of small flow typhoon events; while the SA-Seq2Seq model is just the opposite, the prediction level for large flow becomes mediocre, but the learning ability for small flow typhoon events is greatly enhanced. The STA-Seq2Seq model can well adapt to different typhoon situations.

[0172] (3) Improve prediction performance

[0173] As shown in Table 3, the STA-Seq2Seq model can effectively improve the prediction effect of the GRU-Seq2Seq model, and the spatio-temporal attention model can fully exploit the inherent three-dimensional spatio-temporal information of hydrological data, and the maximum improvement of the RMSE and NSE of the GRU-Seq2Seq model is 47% and 15%, respectively. Therefore, the STA-Seq2Seq model can improve the prediction ability of the GRU-Seq2Seq model, and make full use of the limited data to learn the rainfall runoff process, and realize accurate 3-hour advance prediction in the study area.

[0174] Table 3 STA-Seq2Seq model performance improvement in 3-hour-ahead prediction

[0175]

[0176] Please refer to Figure 8 The embodiment of the application provides a sequence-to-sequence multi-step flood forecasting device based on space-time attention, comprising the following modules:

[0177] The input matrix construction module 01 is used for acquiring flow and extreme typhoon climate rainfall data of a research basin, representing typhoon rainfall at each time step by a sequence according to rainfall characteristic numbers, representing flow at each time step by a sequence according to flow characteristic numbers, and finally splicing the k time step rainfall sequence and the flow sequence into an input matrix X.

[0178] The input matrix division module 02 is used for dividing the input matrix X into k m+n-dimensional feature vectors.

[0179] The space attention weight calculation module 03 is used for calculating the space attention weight of each feature vector.

[0180] The space attention adding module 04 is used for dynamically allocating the weight of each input feature, adding a space attention model to the input matrix X, and obtaining an input matrix X' containing space attention weights.

[0181] The encoder construction and encoding module 05 is used for constructing a GRU encoder and inputting the input matrix X' containing the space attention weights, and outputting an encoder hidden layer sequence H and a context encoding vector c.

[0182] The decoder construction and decoding module 06 is used for constructing a GRU decoder and inputting the output sequence at the last time and the context encoding vector c, and outputting a decoder hidden layer sequence H'.

[0183] The time attention weight calculation module 07 is used for calculating the time attention weight β T ;

[0184] The attention hidden state calculation module 08 is used for performing matrix multiplication on the time attention weight and the encoder hidden layer state, allocating the time attention weight to each time step of the input, and obtaining the final attention hidden state H at ;

[0185] ​​The multi-step prediction module 09 is configured to take the context encoding vector c obtained by the GRU encoder as the initial hidden state of the GRU, and input the output value of the GRU at the previous time into the GRU decoder, and activate the attention hidden state and the decoder hidden state to obtain a multi-step prediction result.

[0186] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or system. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or system that includes the element.

[0187] The above-mentioned embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments. In the unit claims in which several devices are listed, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third does not represent any order, and these words can be interpreted as identifiers.

[0188] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the application, is also included in the patent protection scope of the application.

Claims

1. A sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention, characterized in that, Includes the following steps: Acquire flow and extreme typhoon rainfall data for the study basin, and sequence the typhoon rainfall at each time step based on rainfall characteristic numbers. This means that the flow at each time step is represented by a sequence based on the flow characteristic number. It indicates that, in the end, k The time-step rainfall and flow sequences are concatenated into an input matrix. X ; Input matrix X Divided into k m+n dimensional feature vectors; Calculate the spatial attention weights for each feature vector; Dynamically assign weights to each input feature on the input matrix. X Add a spatial attention model to obtain an input matrix that includes spatial attention weights. ; Construct a GRU encoder and input an input matrix containing spatial attention weights. Output encoder hidden layer sequence H and context encoding vector c ; Construct a GRU decoder and input the output sequence and context encoding vector from the previous time step. c Output decoder hidden layer sequence H′ ; Temporal attention weights are computed using the hidden layer states at each time step of the GRU encoder and GRU decoder. ; Matrix multiplication is performed on the temporal attention weights and the encoder hidden state to distribute the temporal attention weights to each time step of the input, resulting in the final attention hidden state. ; The context encoding vector obtained by the GRU encoder The initial hidden state of the GRU is used as the input of the output value of the GRU from the previous time step into the GRU decoder. The attention hidden state and the decoder hidden state are combined to activate and obtain the multi-step prediction result. The expression for the input matrix X is as follows: The expressions for the k m+n dimensional feature vectors are as follows: In the formula, , The data represents rainfall and flow rate within a single time step, where T is time, m and n are the number of rainfall and flow rate sequence features within a single time step, and k is the input time step size.

2. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The step of calculating the spatial attention weights for each feature vector includes: Input k eigenvectors The sigmoid function is activated sequentially, mapping the values ​​to the (0,1) interval: In the formula, For rainfall and flow data in a single time step feature vector, i.e. , ; The spatial attention weights are calculated by normalizing the feature vectors using the Softmax function. In the formula, Spatial attention weights, The features are rainfall and flow data activated by the Sigmoid function.

3. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 2, characterized in that, The dynamic allocation of weights for each input feature is applied to the input matrix. X Add a spatial attention model to obtain an input matrix that includes spatial attention weights. The steps include: Obtain the spatial attention weights of the feature vector at each time step. ; Then, calculate the model input feature vector after adding the spatial attention term according to the following formula. and model input matrix : In the formula, This is the Hadamard product, i.e., element-wise multiplication.

4. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The process involves constructing a GRU encoder and inputting an input matrix containing spatial attention weights. Output encoder hidden layer sequence H and context encoding vector c The steps include: The encoder consists of a GRU that reads information from an input sequence of arbitrary length sequentially. As it reads each piece of data, the hidden state of the GRU changes according to the following formula: In the formula, This indicates the current hidden state of the encoder. The encoder's hidden state at the previous moment. Input features to the model after adding spatial attention terms. For the k-th eigenvector Hidden information in the input k The input time step, u The dimension of the hidden layer state. , W r , W z Both are weight matrices. In the candidate hidden state, To reset the door, To update the door, The sigmoid function is used to transform data into a value in the range of 0-1, and the tanh function is used to transform data into a value in the range of [-1, 1].

5. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The GRU decoder is constructed by inputting the output value and context encoding vector from the previous time step. c Output decoder hidden layer sequence H′ The steps include: The decoder is another GRU, which is trained to utilize the given hidden state. To predict the output value at the next time step ; decoder in time t The hidden state is calculated as follows: In the formula, For the decoder in time t The hidden state, This represents the hidden state of the decoder in the previous time step; This is the output value of the GRU decoder at the previous time step. n To predict the time step, u The dimension of the hidden layer state. , W r , W z Both are weight matrices. In the candidate hidden state, To reset the door, To update the door, Here, is the sigmoid function, and tanh is the tanh function.

6. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The temporal attention weights are calculated using the hidden layer states at each time step of the GRU encoder and GRU decoder. The steps include: encoder hidden layer state With decoder hidden state Matrix multiplication is performed to obtain the score values ​​of the input and output sequences, and then the temporal attention weights are obtained by normalization using the Softmax function. The calculation formula is as follows: In the formula, This refers to matrix multiplication.

7. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The matrix multiplication of the temporal attention weights and the encoder hidden state distributes the temporal attention weights to each time step of the input, resulting in the final attention hidden state. The specific calculation formula for the steps is as follows: In the formula, For time attention weights, H For encoder hidden layer sequence, n To predict the time step, u is the dimension of the hidden layer state.

8. The sequence-to-sequence multi-step flood forecasting method based on spatiotemporal attention according to claim 1, characterized in that, The context encoding vector obtained by the GRU encoder The specific calculation formulas for the steps of using the initial hidden state of the GRU, inputting the output value of the GRU from the previous time step into the GRU decoder, and combining the attention hidden state and the decoder hidden state for activation to obtain the multi-step prediction result are as follows: In the formula, n To predict the time step, , , , W r , W z This is the weight matrix. For the sigmoid function, This is a state of attention hiding. This is the output value of the GRU decoder at the previous time step. This is the joint state of the attention hidden state and the decoder hidden state activated by the tanh function. In the candidate hidden state, To reset the door, To update the door, Here, is the sigmoid function, and tanh is the tanh function.

9. A sequence-to-sequence multi-step flood forecasting device based on spatiotemporal attention, characterized in that, Includes the following modules: The input matrix construction module is used to acquire the flow and extreme typhoon precipitation data of the study basin, and to sequence the typhoon precipitation at each time step based on the precipitation feature number. This means that the flow at each time step is represented by a sequence based on the flow characteristic number. It indicates that, in the end, k The time-step rainfall and flow sequences are concatenated into an input matrix. ; The input matrix partitioning module is used to partition the input matrix. X Divided into k m+n dimensional feature vectors; The spatial attention weight calculation module is used to calculate the spatial attention weight for each feature vector. The spatial attention addition module is used to dynamically assign weights to each input feature on the input matrix. X Add a spatial attention model to obtain an input matrix that includes spatial attention weights. ; The encoder construction and encoding module is used to construct the GRU encoder and input an input matrix containing spatial attention weights. Output encoder hidden layer sequence H and context encoding vector c ; The decoder construction and decoding module is used to build the GRU decoder and take the output sequence and context encoding vector from the previous time step as input. c Output decoder hidden layer sequence H′ ; The temporal attention weight calculation module is used to calculate temporal attention weights using the hidden layer states at each time step of the GRU encoder and GRU decoder. ; The attention hidden state calculation module performs matrix multiplication on the temporal attention weights and the encoder hidden layer state, distributing the temporal attention weights to each time step of the input to obtain the final attention hidden state. ; The multi-step prediction module is used to convert the context encoding vector obtained by the GRU encoder. The initial hidden state of the GRU, along with the output value of the GRU from the previous time step, is input into the GRU decoder. The attention hidden state and the decoder hidden state are then combined for activation to obtain the multi-step prediction result. The expression for the input matrix X is as follows: The expressions for the k m+n dimensional feature vectors are as follows: In the formula, , The data represents rainfall and flow rate within a single time step, where T is time, m and n are the number of rainfall and flow rate sequence features within a single time step, and k is the input time step size.

Images