A flood flow prediction method based on pre-training enhancement
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HOHAI UNIV
- Filing Date
- 2024-04-03
- Publication Date
- 2026-08-07
AI Technical Summary
考虑到时间序列通常是有噪声的，模型可能很难区分不同上下文中的短期时间序列，难以根据有限的历史数据来准确预测其不同的未来趋势
Smart Images
Figure CN118520908B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of flood forecasting and involves deep learning technology, specifically a flood flow prediction method based on pre-training enhancement. Background Technology
[0002] Flood forecasting is a non-engineering measure for preventing flood disasters. Establishing effective and high-precision flood flow prediction models can effectively avoid floods, reduce casualties and economic losses, and has strong social and production significance and application value. Currently, flood forecasting generally adopts two approaches: hydrological models based on runoff processes and data-driven intelligent models. These two approaches complement each other in actual forecasting. Runoff process models are based on hydrophysical processes and consider the interaction between rivers and rainfall, but many parameters need to be calibrated and estimated, and the modeling is difficult. Data-driven modeling, on the other hand, basically does not consider the physical mechanisms of hydrological processes. It is a black-box method aimed at establishing the optimal mathematical relationship between input and output data and requires sufficient historical data for training.
[0003] Most data-driven flood flow forecasting models typically use historical data within a small window for prediction, such as using the past twelve time steps (one hour) to predict the next twelve time steps. This makes the model blind to contextual information outside the window. Given that time series are often noisy, the model may struggle to distinguish short-term time series in different contexts and accurately predict their varying future trends based on limited historical data. Due to the limitation of the model's input window size, enhancing the model's ability to learn about future flood trends is also crucial, as it helps improve the model's robustness and generalization ability. Summary of the Invention
[0004] Purpose of the invention: In order to overcome the shortcomings of the existing technology, a flood flow prediction method based on pre-training enhancement is provided, which effectively mines the contextual information of short-term windows from historical data, generates a segmented representation that correctly reflects the complex patterns in the time series, and improves the accuracy of flood forecasting.
[0005] Technical solution: The flood discharge prediction method based on pre-training enhancement described in this invention includes the following steps:
[0006] (1) Process historical hydrological station data, construct hydrological station dataset, generate hydrological station watershed topology representing the spatial relationship between hydrological stations, and divide the time series data of all hydrological stations into K time series segments of length L.
[0007] (2) In the pre-training stage, segments are randomly masked. A pre-trained encoder and a pre-trained decoder are constructed based on the attention mechanism for time-series data. The feature vectors of the unmasked segments are learned by the pre-trained encoder, and the feature vectors are parsed and the masked segments are predicted by the pre-trained decoder.
[0008] (3) In the prediction stage, the time series segments of the hydrological station and the watershed topology of the hydrological station are used as inputs. The encoder in the pre-training stage encodes all segments and generates feature vectors. Based on the feature vectors and watershed topology, the vector representation is parsed using time attention blocks, graph attention networks and gating fusion mechanisms to realize flood flow prediction.
[0009] Furthermore, the hydrological station data mentioned in step (1) includes long-term historical monitoring data on rainfall and flow.
[0010] Furthermore, the implementation process of step (1) is as follows:
[0011] A sliding window process is applied to rainfall and flow data to transform unsupervised data into supervised data. The first 70% of the supervised data is used as the training set, 10% as the validation set, and 20% as the test set to construct a hydrological station dataset. Pearson correlation coefficients are calculated based on the rainfall and flow data from the hydrological stations to determine the correlation between them. Combined with geospatial information, a watershed topology for each hydrological station is generated. This topology is expressed as an adjacency matrix, where elements represent the correlation between two hydrological stations, with values {0, 1}, where 0 indicates no obvious spatial correlation and 1 indicates a correlation. The time-series data of the hydrological stations is represented as X = (x0, x1, ..., x...). s Let the length of the time series data be s, and the segment length be b. Then the data is divided into K segments, satisfying s = K * b. The data after division is represented as P = (p0, p1, ..., p...). K ).
[0012] Furthermore, the implementation process of step (2) is as follows:
[0013] (21) Randomly mask time segments and divide the time segments into masked segments and unmasked segments;
[0014] (22) Construct a pre-trained encoder to process the unmasked segments and encode the unmasked segments into feature vectors;
[0015] (23) Construct a pre-trained decoder, initialize the masked segment as a feature vector, combine the feature vectors of the masked segment and the unmasked segment, parse all feature vectors, and predict the masked segment;
[0016] (24) Construct a loss function based on the predicted and actual shielded segments, and train a pre-trained model; the loss function is expressed as:
[0017]
[0018] Where, N i N represents the set of hydrological stations. j Let L represent the set of masked segments, and y represent the segment length. l This represents the actual value of the l-th time series data in the masked segment. This represents the true value of the l-th time series data in the masked segment; based on the loss function, the parameters in the pre-trained encoder and pre-trained decoder are updated to prepare for the prediction stage.
[0019] Furthermore, the implementation process of step (22) is as follows:
[0020] (221) Based on multi-channel convolution, unmasked temporal segments are converted into hidden representations;
[0021] (222) Add positional encoding information to the hidden representation to help the pre-trained model better understand the order and positional information in the input time segment;
[0022] (223) Construct a temporal attention block, which adaptively simulates the nonlinear association between different segments based on the attention mechanism to capture the interrelationships between segments; specifically, for segment i, it is first converted into a query vector, key vector, and value vector through multiple multilayer perceptrons; the conversion process is expressed as:
[0023] q i =MLP(h i ),k i =MLP(h i ),v i =MLP(h i )
[0024] This represents a group of segments preceding segment i, including segment i itself. In the element fragment 'a', the attention vector from 'a' to 'i' is represented as:
[0025]
[0026] Here, Softmax calculates the weights from segment a to segment i by weighting all messages from segment a to segment i. Assuming that multiple attention heads are concatenated using the σ function and transformed to the same dimension as the hidden representation, the update method of the hidden representation of segment i can be obtained based on the multi-head attention mechanism:
[0027]
[0028] Furthermore, the implementation process of step (23) is as follows:
[0029] (231) Construct random feature vectors for the masked segments, combine the masked segments and unmasked segments according to their order in the original time series data, and add position codes to the masked segments;
[0030] (232) Based on the temporal attention block, update the feature vectors of all segments, and then use a one-layer feedforward neural network to perform a linear transformation on the feature vectors to generate the prediction results of the masked segments.
[0031] Furthermore, the implementation process of step (3) is as follows:
[0032] (31) Encode all time segments using a pre-trained encoder to generate feature vectors representing the time segments;
[0033] (32) Based on the time attention block, the feature vector is encoded to generate a time-related time feature vector h. t Subsequently, using the feature vector and the watershed topology of the hydrological station as input, a spatially relevant spatial feature vector h is generated based on a graph attention network. s ;
[0034] (33) Fusion of time feature vector h based on gating mechanism t and spatial eigenvector h s Construct a fusion gate g to capture spatiotemporal relationships and generate spatiotemporal feature vector h. s,r A feedforward neural network is used to perform a linear transformation on the spatiotemporal feature vectors to generate flood flow prediction results.
[0035] (34) Construct a loss function based on the predicted and actual flow rates, and train the system for the entire prediction phase; specifically, the loss function is expressed as:
[0036]
[0037] Where N represents the total number of predicted flow values, y n This represents the actual value of the traffic data. This represents the predicted value of the traffic data.
[0038] Furthermore, the implementation process of step (32) is as follows:
[0039] For a hydrological station node i, assume that node i's neighbors j∈N in the hydrological station's watershed topology. i First, based on the shared learnable vector a and parameters W, the attention coefficient α of node i and each of its neighbors j is calculated. ij :
[0040]
[0041] Among them, hi and h j Let i and j represent the feature vectors corresponding to nodes i and j, respectively. || represents the vector concatenation operation, LeakyReLu represents the normalization function, and exp represents the exponential function. Assuming there are K attention heads, || represents the attention head connection operation, and σ represents the nonlinear activation function, based on the normalized attention coefficients, the feature vectors of neighboring nodes in the watershed topology are linearly combined, and nodes are updated through multi-head attention.
[0042]
[0043] Furthermore, the process of constructing the fusion gate g in step (33) is expressed as follows:
[0044] g = tanh(h) t w t +h s w s )
[0045] Among them, w t and w s The learnable parameters are represented, and tanh is used as the activation function; the spatiotemporal feature vector h is obtained based on the fusion gate g and the Hadamard product ⊙. s,t :
[0046] h s,t =g⊙h t +(1-g)⊙h s .
[0047] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are: The present invention improves the model's ability to learn future flood change trends based on the pre-training strategy, and can capture temporal and spatial feature information more efficiently and improve the accuracy of flood forecasting based on the attention mechanism and graph attention network for joint learning of temporal and spatial feature information. Attached Figure Description
[0048] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0049] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0050] This invention provides a flood forecasting method based on a distributed adaptive graph attention network, such as... Figure 1 As shown, it includes the following steps:
[0051] Step 1: Process hydrological station data, including rain gauge data and flow rate data, and generate a hydrological station watershed topology representing the spatial relationship between hydrological stations. Divide all rainfall and flow rate data into K time series segments of length L.
[0052] The collected flow data are generally historical flow data from the outlet section of the watershed, denoted as X. flow =(x flow,1 ,x flow,2 ,…,x flow,n The collected rainfall data mainly includes historical rainfall data from multiple rain gauge stations, with an interval of 1 hour, expressed in millimeters. For rain gauge i, it can be represented as... This embodiment takes a river basin in southern China as an example, and collects data from 11 rain gauge stations and 1 flow station from June 1981 to July 2003.
[0053] Data cleaning includes handling missing values and outliers; data normalization is then performed using the following formula:
[0054]
[0055] Where X is the standardized value, X i The original value, X min X is the minimum value in the original sequence. max This represents the maximum value in the original sequence.
[0056] A sliding window process was applied to rainfall and flow data to transform unsupervised data into supervised data. The supervised data was then partitioned, with the top 70% used as the model training set, 10% as the model validation set, and 20% as the model test set, thus constructing a hydrological station dataset. Pearson correlation coefficients were calculated based on the rainfall and flow data from the hydrological stations to determine the correlation between them. Combined with geospatial information, a watershed topology for each hydrological station was generated. This topology was expressed as an adjacency matrix, where elements represent the correlation between two hydrological stations, with values ranging from {0, 1}, where 0 indicates no significant spatial correlation and 1 indicates a correlation. This watershed topology was used as input for flood flow prediction during the forecasting phase.
[0057] The time-series data of all hydrological stations are divided to obtain K unique time-series segments for each station. Specifically, assume that the time-series data of the hydrological stations are represented as X = (x0, x1, ..., x...). s The time-series data has a length of s. In this embodiment, the segment length is 12, so the data can be divided into K segments, satisfying s = K * 12. The divided data can be represented as P = (p0, p1, ..., p...). K ).
[0058] Step 2: In the pre-training stage, segments are randomly masked. A pre-trained encoder and decoder are constructed based on an attention mechanism for time-series data. The pre-trained encoder learns the feature vectors of the unmasked segments, and the pre-trained decoder parses the feature vectors and predicts the masked segments. Long-term historical information is combined to calculate the correlation between time-series data, generating features that reflect complex patterns and trends in the time series. The specific implementation process is as follows:
[0059] Randomly mask time segments to divide them into masked and unmasked segments; construct a pre-trained encoder to process the unmasked segments and encode them into feature vectors.
[0060] The process of building a pre-trained encoder is as follows:
[0061] First, based on multi-channel convolution, the unmasked temporal segments are converted into hidden representations. More specifically, in this embodiment, 256 one-dimensional convolutional kernels are set, with both the kernel length and stride being 12. The i-th convolutional kernel is denoted as... Through convolution kernel C i The operation of convolving the j-th unmasked segment can be represented as:
[0062]
[0063] in, Let represent the hidden representation generated after convolution. Therefore, the features generated by the j-th unmasked segment after passing through 256 convolutional kernels can be represented as:
[0064] Subsequently, positional encoding information is added to the hidden representation to help the model better understand the order and positional information in the input temporal segments. Specifically, the positional encoding of the i-th segment in the j-th dimension can be represented as:
[0065]
[0066] in, The frequency of the function decreases continuously along the dimension of the hidden representation. Therefore, the positional encoding of the i-th segment can be represented as:
[0067]
[0068] By summing the hidden representation of a fragment with its hidden encoding, we obtain the hidden representation h of the i-th fragment containing positional information. i =z i +PE i .
[0069] Finally, a temporal attention block is constructed to adaptively simulate the nonlinear relationships between different segments based on an attention mechanism, capturing the interrelationships between segments. Specifically, for segment i, it is first converted into a query vector, key vector, and value vector through multiple multilayer perceptrons. The conversion process can be represented as:
[0070] q i =MLP(h i ),k i =MLP(h i ),v i =MLP(h i )
[0071] Assumption This represents a group of segments preceding segment i (including i itself), for... The attention vector from element segment a to i in the given information can be represented as:
[0072]
[0073] Softmax calculates the weights from segment a to segment i by weighting all messages from segment a to segment i. Based on the attention vector, multi-head attention allows us to obtain the update method for the hidden representation of segment i:
[0074]
[0075] The σ function is used to connect multiple attention heads and transform them into the same dimension as the hidden representation.
[0076] The pre-trained decoder is constructed by initializing the masked segment as a feature vector, combining the feature vectors of the masked and unmasked segments, parsing all feature vectors, and predicting the masked segment. The process of constructing the pre-trained encoder is as follows:
[0077] First, a random feature vector of dimension 256 is constructed for the masked segment. The masked and unmasked segments are then combined according to their order in the original time series data, and positional encoding is added to the masked segment. Subsequently, based on the temporal attention block, the feature vectors of all segments are updated. Finally, a feedforward neural network layer is used to perform a linear transformation on the feature vectors to generate the predicted results of the segments.
[0078] A loss function is constructed based on the predicted and actual masked segments, and the entire pre-trained model is trained. The loss function can be expressed as:
[0079]
[0080] Where, N i N represents the set of hydrological stations. jLet L represent the set of masked segments, and y represent the segment length. l This represents the actual value of the l-th time series data in the masked segment. This represents the true value of the l-th time-series data in the masked segment. Based on the loss function, the parameters in the pre-trained encoder and pre-trained decoder can be updated to prepare for the prediction stage.
[0081] Step 3: In the prediction phase, using the time-series segments and watershed topology of the hydrological station as input, the encoder from the pre-training phase encodes all segments and generates feature vectors. Based on the feature vectors and watershed topology, a parsed vector representation is generated using temporal attention blocks, graph attention networks, and a gating fusion mechanism to achieve flood discharge prediction. The specific implementation process is as follows:
[0082] Using feature vectors and the watershed topology of hydrological stations as inputs, a flood discharge prediction module is constructed to predict flood discharge. First, the feature vectors are encoded using time attention blocks to generate time-related time feature vectors h. t Subsequently, using the feature vector and the watershed topology of the hydrological station as input, a spatially relevant spatial feature vector h is generated based on a graph attention network. s Specifically, for hydrological station node i, assume that node i's neighbors j∈N in the hydrological station's watershed topology. i First, based on the shared learnable vector a and parameters W, the attention coefficient α of node i and each of its neighbors j is calculated. ij :
[0083]
[0084] Among them, h i and h j Let represent the feature vectors corresponding to nodes i and j, respectively. || denotes the vector concatenation operation, LeakyReLU represents the normalization function, and exp represents the exponential function. Assuming there are K attention heads, || represents the attention head connection operation, and σ represents the nonlinear activation function, based on the normalized attention coefficients, the feature vectors of neighboring nodes in the watershed topology are linearly combined to update nodes through multi-head attention:
[0085]
[0086] Finally, based on the time feature vector h t and spatial eigenvector h s A fusion gate is constructed, and then the temporal and spatial feature vectors are fused based on the fusion gate to capture the spatiotemporal relationship and generate a spatiotemporal feature vector h. s,t The process of generating the fusion gate g is represented as follows:
[0087] g = tanh(h) t wt +h s w s )
[0088] Among them, w t and w s Let represent the learnable parameters, and tanh be used as the activation function. Based on the fusion gate g and the Hadamard product ⊙, the spatiotemporal feature vector h can be obtained. s,t :
[0089] h s,t =g⊙h t +(1-g)⊙h s .
[0090] A feedforward neural network is used to perform a linear transformation on the spatiotemporal feature vectors to generate flood discharge prediction results.
[0091] A loss function is constructed based on the predicted and actual traffic volumes, and the entire prediction model is trained accordingly. Specifically, the loss function can be expressed as:
[0092]
[0093] Where N represents the total number of predicted flow values, y n This represents the actual value of the traffic data. This represents the predicted value of the flow data. This embodiment uses the rainfall and flow time series data of the past 12 hours to predict the flood flow of the next 12 hours, that is, the forecast period is 12 hours. Since flood flow is more concerned with the change of flow at each different prediction time point, this embodiment uses T as the base time and introduces different time points T+1, T+3, T+6, T+9, and T+12, and dynamically measures the change of flood flow at 3-hour intervals.
[0094] This invention addresses the difficulty of data-driven flood flow prediction models in distinguishing short-term window forecasts across different contexts. It proposes a pre-training enhancement method to capture contextual information outside the window, generating feature vectors containing trend information through a strategy of randomly masking segments. To address the encoding and fusion of temporal and spatial features, multi-head attention and graph attention networks are used to capture temporal and spatial features, learning latent temporal and spatial dependencies, and fusing the temporal and spatial relationships based on a gating mechanism. This invention utilizes a pre-training mechanism to capture trend information in time-series data, encodes time-series information based on convolution, and learns and fuses spatiotemporal feature vectors through attention mechanisms and gating fusion methods, effectively improving the performance of flood flow prediction.
[0095] The technical solution of the present invention has been described in conjunction with the specific experimental procedures shown in the accompanying drawings. However, the scope of protection of the present invention is not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A flood discharge prediction method based on pre-training enhancement, characterized in that, Includes the following steps: (1) Process historical hydrological station data, construct hydrological station dataset, generate hydrological station watershed topology representing the spatial relationship between hydrological stations, and divide the time series data of all hydrological stations into K time series segments of length L. (2) In the pre-training stage, segments are randomly masked, and a pre-trained encoder and a pre-trained decoder are constructed based on the attention mechanism for time-series data; The feature vectors of the unmasked segment are learned through a pre-trained encoder. The specific implementation process is as follows: Based on multi-channel convolution, unmasked temporal segments are converted into hidden representations; positional encoding information is added to the hidden representations to help the pre-trained model better understand the order and positional information in the input temporal segments; temporal attention blocks are constructed to adaptively simulate the nonlinear relationships between different segments based on the attention mechanism, capturing the interrelationships between segments; specifically, for segments... First, the vectors are transformed into query vectors, key vectors, and value vectors using multiple multilayer perceptrons; the transformation process is represented as follows: Representing fragments A previous set of clips, including clips For oneself, element fragments , arrive The attention vector is represented as: in, It is calculating all segments to segments After receiving the message, a weighted fragment was obtained. To the fragment The weights; assuming the use of The function concatenates multiple attention heads and transforms them to the same dimension as the hidden representation, obtaining a fragment based on the multi-head attention mechanism. Hidden representation update method: ； The process of parsing feature vectors and predicting masked segments using a pre-trained decoder is as follows: A random feature vector is constructed for the masked segment, and the masked and unmasked segments are combined according to their order in the original time series data. Position encoding is added to the masked segment. Based on the temporal attention block, the feature vectors of all segments are updated, and then a one-layer feedforward neural network is used to perform a linear transformation on the feature vectors to generate the prediction results of the masked segments; (3) In the prediction stage, the time series segments of the hydrological station and the watershed topology of the hydrological station are used as inputs. The encoder in the pre-training stage is used to encode all segments and generate feature vectors. Based on feature vectors and watershed topology, this paper utilizes temporal attention blocks, graph attention networks, and gating fusion mechanisms to analyze vector representations and achieve flood flow prediction.
2. The flood discharge prediction method based on pre-training enhancement according to claim 1, characterized in that, The hydrological station data mentioned in step (1) includes long-term historical monitoring data of rainfall and flow.
3. The flood discharge prediction method based on pre-training enhancement according to claim 1, characterized in that, The implementation process of step (1) is as follows: A sliding window process is applied to rainfall and flow data to transform unsupervised data into supervised data. Pearson correlation coefficients are calculated based on rainfall and flow data from hydrological stations to determine the correlation between stations. Combined with geospatial information, a watershed topology is generated for each hydrological station. This topology is expressed as an adjacency matrix, where each element represents the correlation between two hydrological stations, with values ranging from 1 to 2. Where 0 indicates that the two hydrological stations are not spatially related, and 1 indicates that they are related; the time series data of the hydrological stations are represented as The length of the time series data is The length of the segment is denoted as The data will then be divided into A segment, satisfying ; The partitioned data is represented as .
4. The flood discharge prediction method based on pre-training enhancement according to claim 1, characterized in that, The implementation process of step (2) is as follows: (21) Randomly mask time segments and divide the time segments into masked segments and unmasked segments; (22) Construct a pre-trained encoder, process the unmasked segments, and encode the unmasked segments into feature vectors; (23) Construct a pre-trained decoder, initialize the masked segment as a feature vector, combine the feature vectors of the masked segment and the unmasked segment, parse all feature vectors, and predict the masked segment; (24) Construct a loss function based on the predicted and actual shielded segments, and train a pre-trained model; the loss function is expressed as: in, This refers to a set of hydrological stations. This represents a set of masked segments. Indicates the length of the segment. Indicates the first segment in the masked segment The true value of a time series data point Indicates the first segment in the masked segment The true values of each time series data point are used; based on the loss function, the parameters in the pre-trained encoder and pre-trained decoder are updated to prepare for the prediction stage.
5. The flood discharge prediction method based on pre-training enhancement according to claim 1, characterized in that, The implementation process of step (3) is as follows: (31) Encode all time segments using a pre-trained encoder to generate feature vectors representing the time segments; (32) Based on the time attention block, the feature vector is encoded to generate time-related time feature vectors. Subsequently, using the feature vectors and the watershed topology of the hydrological station as input, spatially relevant spatial feature vectors are generated based on a graph attention network. ; (33) Fusion of temporal feature vectors based on gating mechanism and spatial eigenvectors Constructing a fusion gate Capture spatiotemporal relationships and generate spatiotemporal feature vectors. A feedforward neural network is used to perform a linear transformation on the spatiotemporal feature vectors to generate flood flow prediction results. (34) Construct a loss function based on the predicted and actual flow rates, and train the system for the entire prediction phase; specifically, the loss function is expressed as: in, This represents the total number of all predicted flow values. This represents the actual value of the traffic data. This represents the predicted value of the traffic data.
6. The flood discharge prediction method based on pre-training enhancement according to claim 5, characterized in that, The implementation process of step (32) is as follows: For hydrological monitoring station nodes Assuming node Neighbors in the watershed topology of a hydrological station First, based on shared learnable vectors and parameters compute nodes and each of its neighbors Attention coefficient : in, and Representing nodes respectively and nodes The corresponding feature vector, This represents the vector concatenation operation. Represents the normalization function. This represents an exponential function; assuming the number of attention heads is K, This indicates the connection operation of the attention head. This represents a non-linear activation function, based on normalized attention coefficients, linearly combining the feature vectors of neighboring nodes in the watershed topology, and updating nodes through multi-head attention: 。 7. The flood discharge prediction method based on pre-training enhancement according to claim 5, characterized in that, The construction of the fusion gate described in step (33) The process is represented as: in, and Indicates learnable parameters, and uses As an activation function; based on the fusion gate and Hadamaji To obtain the spatiotemporal feature vector : 。 8. The flood discharge prediction method based on pre-training enhancement according to claim 3, characterized in that, The supervised data is used as the training set (70%), the validation set (10%), and the test set (20%) to construct the hydrological station dataset.