An LSTM-CAM-Transformer Flood Forecasting Model for Long- and Short-Term Dependence Feature Fusion

Abstract

The present invention discloses an LSTM-CAM-Transformer flood forecasting model for long- and short-term dependent feature fusion. The LSTM-CAM-Transformer flood forecasting model of the present invention is trained based on the calibration period data by using the root mean square propagation method. This model couples LSTM and Transformer through a cross-attention mechanism, and fuses the short-term feature extraction ability of LSTM and the long-term dependence modeling ability of Transformer to construct. And the accuracy of the LSTM-CAM-Transformer flood forecasting model is verified based on the validation period data by using the coefficient of determination, and the trained LSTM-CAM-Transformer flood forecasting model is used for flood prediction. The present invention realizes the joint recognition of long- and short-term flood features, and improves the accuracy and reliability of flood forecasting.

Description

Technical Field

[0001] The present invention relates to the technical field of flood forecasting, and specifically provides an LSTM-CAM-Transformer flood forecasting model for the fusion of long-term and short-term dependence features. Background Art

[0002] Floods are one of the major natural disasters that seriously threaten the safety of human life and property and the development of social economy. Accurate flood forecasting is crucial for flood prevention and mitigation. During the formation process of floods, there are both long-term characteristics affected by the basin's regulation effect and short-term change characteristics affected by sudden changes in rainfall intensity within a short period. Traditional flood forecasting models, such as empirical models, conceptual models, and distributed models, have problems such as low forecasting accuracy and difficulty in adapting to changing environments when facing complex hydrological and meteorological conditions.

[0003] With the development of deep learning technology, data-driven flood forecasting models have made some progress, but existing models still have deficiencies in capturing long-term and short-term dependence features during the flood process. A single model structure is difficult to comprehensively capture various time-scale information involved in the formation and development of floods. For example, the long short-term memory network (LSTM) can handle short-term dependence relationships well, but it has limited ability in capturing long-term climate trends and basin hydrological characteristics; the Transformer architecture is good at capturing long-range dependence information, but it is not sensitive enough to short-term local feature changes. In addition, when traditional models fuse features of different time scales, they often use simple weighting or splicing methods, which cannot fully explore the internal connections between features, resulting in limited forecasting accuracy.

[0004] Existing flood forecasting models often cannot make full use of data information of different time scales and multi-modalities, resulting in insufficient accuracy and timeliness of forecasting and being difficult to meet the actual needs of flood prevention and mitigation. Therefore, it is of great practical significance to develop a flood forecasting model that can effectively fuse long-term and short-term dependence features. Summary of the Invention

[0005] The purpose of the present invention is to provide an LSTM-CAM-Transformer flood forecasting model for the fusion of long-term and short-term dependence features, which couples LSTM and Transformer through a cross-attention mechanism (CAM), fuses the short-term feature extraction ability of LSTM and the long-term dependence modeling ability of Transformer, realizes the joint recognition of long-term and short-term flood features, and improves the accuracy and reliability of flood forecasting.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] An LSTM-CAM-Transformer flood forecasting model for long-short dependence feature fusion, the steps of which include:

[0008] Obtain flood runoff data and flood prediction factor data, and use the mutual information method to screen flood prediction factors;

[0009] Divide the screened flood prediction factors and flood data after normalization processing into calibration period data and verification period data;

[0010] Construct an LSTM-CAM-Transformer flood forecasting model, the structure of the LSTM-CAM-Transformer flood forecasting model includes a long-short dependence feature extraction module and a flood prediction module;

[0011] Among them, the long-short dependence feature extraction module is used to extract the long-short dependence fusion feature vector in the hydrological dynamic features of flood prediction factors. The long-short dependence feature extraction module includes: three independent LSTM subnets, a multi-scale feature fusion module, a first Transformer encoder, a feature alignment module, and a cross-attention mechanism feature fusion module;

[0012] The flood prediction module is used to perform flood forecasting based on the long-short dependence fusion feature vector and flood data. The flood prediction module includes: a second Transformer decoder and a multi-layer perceptron (MLP);

[0013] Train the LSTM-CAM-Transformer flood forecasting model based on the calibration period data using the Root Mean Square Propagation (RMSprop) method; the Root Mean Square Propagation (RMSprop) method can adaptively adjust the learning rate for different parameters by performing an exponentially weighted moving average on the squared gradients, accelerate convergence, and reduce the interference of noise on the model convergence and forecasting accuracy;

[0014] Verify the accuracy of the LSTM-CAM-Transformer flood forecasting model based on the verification period data using the coefficient of determination, and use the trained LSTM-CAM-Transformer flood forecasting model for flood prediction.

[0015] Among them, for a given time series of flood data, the Hurst exponent (H) is calculated using the Rescaled Range Analysis (R / S analysis) method, and its functional expression is:

[0016] ;

[0017] In the formula: t = 1, 2,...; R(t) is the range of the flood time series, S(t) is the standard deviation S of the time series, and R / S is the rescaled range.

[0018] By calculating the rescaled range R / S for different time intervals t and plotting the slope of log(R / S) against log(t), the Hurst exponent (H) is obtained. The H values of many runoff sequences are significantly greater than 0.5, indicating that flood sequences have long memory. Therefore, long-term and short-term characteristics need to be combined when constructing the LSTM-CAM-Transformer flood forecasting model.

[0019] According to the above technical solution, the three independent LSTM subnets are the hourly LSTM subnet, the daily LSTM subnet, and the weekly LSTM subnet respectively;

[0020] The hourly LSTM subnet is used to capture the hydrological dynamic characteristics at the hourly time scale;

[0021] The daily LSTM subnet is used to capture the hydrological dynamic characteristics at the daily time scale;

[0022] The weekly LSTM subnet is used to capture the hydrological dynamic characteristics at the weekly time scale.

[0023] Among them, each LSTM subnet can effectively capture the hydrological dynamic changes at different time scales through the coordinated action of its unique input gate, forget gate, and output gate. The hourly LSTM subnet focuses on capturing short-term characteristics such as the rapid change of rainfall intensity and the rapid response of runoff in a short time; the daily LSTM subnet is used to analyze the impact of the overall changes in rainfall and runoff within a day on the basin's water storage capacity; the weekly LSTM subnet focuses on the comprehensive hydrological dynamic changes in the basin within a week and integrates cumulative information such as rainfall over a longer period.

[0024] According to the above technical solution, the steps executed by the LSTM-CAM-Transformer flood forecasting model include:

[0025] Input the selected flood prediction factors into the three independent LSTM subnets to obtain the hydrological dynamic characteristics h at the hourly time scale （1） 、the hydrological dynamic characteristics h at the daily time scale （2） and the hydrological dynamic characteristics h at the weekly time scale （3） ;

[0026] Use the multi-scale feature fusion module to dynamically weight and fuse the hydrological dynamic characteristics h at the hourly time scale （1） 、the hydrological dynamic characteristics h at the daily time scale （2） and the hydrological dynamic characteristics h at the weekly time scale （3） to obtain the comprehensive short-term feature vector ;

[0027] The downsampled flood prediction factors are mapped to a low-dimensional vector space through the embedding layer of the first Transformer encoder and the comprehensive short-term feature vector After fusion, the long-term feature vector is obtained through the self-attention mechanism, normalization layer, feed-forward network layer, and normalization layer in sequence ;

[0028] The comprehensive short-term feature vector is passed through the feature alignment module and the long-term feature vector are mapped to the same dimension;

[0029] The comprehensive short-term feature vector with the same dimension and the long-term feature vector are fused using the cross-attention mechanism feature fusion module and the long-term feature vector to obtain the long and short dependence fusion feature vector ;

[0030] The flood feature vector obtained by processing the previous moment's prediction output through the masked self-attention mechanism of the second Transformer decoder and the long and short dependence fusion feature vector are fused through the cross-attention mechanism and then passed through the normalization layer, feed-forward network layer, and multi-layer perceptron of the second Transformer decoder in sequence to obtain the flood forecast result.

[0031] According to the above technical solution, the influence degree of features at different time scales on flood changes varies dynamically with time and specific conditions. Therefore, the feature vectors output by each LSTM subnet are dynamically weighted and fused. According to the importance of features at different time scales, the scale feature fusion module uses the softmax function to assign different weights to the outputs of each LSTM subnet for weighted summation to obtain the comprehensive short-term feature vector h s ;

[0032] The comprehensive short-term feature vector Calculation formula:

[0033] ;

[0034] ;

[0035] ;

[0036] In the formula, is the input feature of the k-th scale at time t, represents the hidden state output by the k-th scale at time t-1, represents the hidden state of the k-th scale at time t, represents the feature vector of the kth scale, represents the hidden state output by the kth scale at time 1, represents the hidden state output by the kth scale at time 2, represents the hidden state output by the kth scale at time T, represents the weight matrix of the kth scale in the softmax function, represents the weight matrix of the j-th scale in the softmax function, represents the feature vector of the jth scale, represents the bias vector of the jth scale in the softmax function, Represents the bias vector of the kth scale in the softmax function.

[0037] Among them, k and j can be 1, 2, and 3, respectively representing the hydrological dynamic characteristics of the hourly time scale, the hydrological dynamic characteristics of the daily time scale, and the hydrological dynamic characteristics of the weekly time scale.

[0038] According to the above technical solution, the downsampled flood prediction factor is mapped to the low-dimensional vector space and the comprehensive short-term feature vector through the embedding layer of the first Transformer encoder. After merging, we get the fused vector sequence , and fused vector sequence Perform position encoding;

[0039] The downsampled flood prediction factor is mapped to a low-dimensional vector space and the comprehensive short-term feature vector through the embedding layer of the first Transformer encoder Fusion enables Transformer to combine these short-term features while capturing long-term dependencies, better understand the changing patterns of floods at different time scales, and further explore the potential complex patterns and features in the data. At the same time, in order to enable Transformer to better process time series data, the fusion vector sequence Perform positional encoding to enable the model to distinguish data from different time points.

[0040] The position encoding formula includes:

[0041] ;

[0042] ;

[0043] Where: Fusion vector sequence The position index of the element in, i represents the dimension index, The model dimension is set to 128 dimensions. Represents the fused vector sequence The position encoding value of the elements in the even dimensions in Represents the fused vector sequence The position encoding value of the elements in the odd dimensions in

[0044] According to the above technical solution, the LSTM-CAM-Transformer flood forecasting model is characterized in that the long-term feature vector Calculation formula:

[0045] ;

[0046] ;

[0047] ;

[0048] In the formula, x is the fused vector sequence , Represents the weight matrix of the query, Represents the weight matrix of the key, Represents the weight matrix of the value, Represents the query vector, Represents the key vector, Represents the value vector, Represents the dimension of the key vector, T represents the transpose operation on the key vector, Represents the activation function, Represents the multi-head attention concatenation function, Represents the single-head attention calculation function.

[0049] According to the above technical solution, since the comprehensive short-term feature vector and the long-term feature vector may have different time steps or dimensions, the feature alignment module uses the line interpolation method to refine the comprehensive short-term feature vector and the long-term feature vector to the same time step, and uses a fully connected layer to map the comprehensive short-term feature vector and the long-term feature vector to the same dimension.

[0050] According to the above technical solution, the cross-attention mechanism feature fusion module uses the cross-attention mechanism CAM to perform long and short dependence feature fusion on the comprehensive short-term feature vector and the long-term feature vector with the same dimension, and obtains the long and short dependence fusion feature vector .

[0051] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention couples LSTM and Transformer through the Cross Attention Mechanism (CAM), constructs a hierarchical long-term and short-term feature collaborative learning framework, integrates the short-term feature extraction ability of LSTM and the long-term dependence modeling ability of Transformer, realizes the joint recognition of long-term and short-term flood features, effectively solves the problem of feature fragmentation between local mutation response and global trend evolution in hydrological prediction, avoids the limitations of a single model in terms of computational efficiency and feature coverage, and provides a new solution with both scientificity and engineering practicability for accurate hydrological forecasting. This coupled architecture demonstrates more reliable multi-scale feature decoupling analysis and dynamic decision support, can efficiently identify the coupling effects of heterogeneous hydrological elements, enhance the robustness of the system to real-world constraints such as data missing and basin heterogeneity, adaptively adjust the interaction weights of long-term and short-term features, simultaneously capture the subtle signs of sudden flood peaks and the macroscopic evolution laws of slow-onset floods in complex hydrological scenarios, and significantly improve the flood prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the drawings:

[0053] Figure 1 is the overall architecture diagram of an LSTM-CAM-Transformer flood forecasting model for long-term and short-term dependence feature fusion according to the present invention;

[0054] Figure 2 is the schematic diagram of the cross-attention mechanism. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0056] The present invention provides the following technical solutions:

[0057] An LSTM-CAM-Transformer flood forecasting model for long-term and short-term dependence feature fusion, and its construction steps are as follows:

[0058] S1. Obtain flood runoff data and flood prediction factor data, and use the mutual information method to screen flood prediction factors. Taking the variable to be calculated as the flood discharge Taking [example], the mutual information method is used to screen flood prediction factors, and the specific steps are as follows:

[0059] Collect a large amount of data related to flood prediction factors within the basin, such as historical rainfall data R from meteorological stations, previous water level data Wpre and previous flood flow data Qpre from hydrological monitoring stations, etc. The time resolution of the data needs to cover scales such as hours, days, and weeks to meet the needs of multi-scale modeling.

[0060] And use the mutual information method to calculate the mutual information values between various flood prediction factors and the variable to be sought, flood flow After that, it is found that the rainfall data R, the previous water level Wpre, the previous flood flow Qpre, and the underlying surface indexes such as the soil texture, vegetation coverage, elevation, slope, and aspect of the basin area are all highly correlated with the flood flow and can be used as flood prediction factors.

[0061] Among them, the mutual information value formula is as follows:

[0062] ;

[0063] In the formula: is the flood prediction factor, is the variable to be sought, flood flow, N is the total number of samples of the flood prediction factor, is the flood prediction factor in the k-th gear and the flood flow in the m-th gear, represents the number of samples of the flood prediction factor in the k-th gear; represents the number of samples of the flood flow in the m-th gear.

[0064] S2. Normalize the screened flood prediction factors and flood runoff data; specifically, combine the actual situation of the local basin, set a reasonable data range, and remove the outliers in the screened flood prediction factor data and flood runoff data that exceed the range or do not conform to physical laws. And use linear interpolation to fill in the missing values to ensure data integrity and obtain an equal-time interval time series. Normalize different types of data and map the values to the interval [0,1] or [-1,1]. The normalization can be carried out through the following formula:

[0065] ;

[0066] In the formula: is the normalized value, is the original value, and are the maximum and minimum values of the screened flood prediction factor data respectively.

[0067] S3. Build an LSTM-CAM-Transformer flood forecasting model.

[0068] The LSTM-CAM-Transformer flood forecasting model structure includes:

[0069] The long-short dependency feature extraction module is used to extract the long-short dependency fusion feature vector in the hydrological dynamic characteristics of flood prediction factors, including: three independent LSTM subnetworks, a multi-scale feature fusion module, the first Transformer encoder, a feature alignment module, and a cross-attention mechanism feature fusion module;

[0070] The flood prediction module is used to perform flood forecasting based on the long-short dependency fusion feature vector and flood data, including: a second Transformer decoder and a multi-layer perceptron (MLP).

[0071] Among them, the three independent LSTM subnetworks are hourly LSTM subnetwork, daily LSTM subnetwork and weekly LSTM subnetwork; among them, the hourly LSTM subnetwork is used to capture the hydrological dynamic characteristics of the hourly time scale; the daily LSTM subnetwork is used to capture the hydrological dynamic characteristics of the daily time scale; the weekly LSTM subnetwork is used to capture the hydrological dynamic characteristics of the weekly time scale.

[0072] Among them, the execution steps of the LSTM-CAM-Transformer flood forecasting model include:

[0073] S301, input the selected flood prediction factors into three independent LSTM subnetworks to obtain the hydrological dynamic characteristics h at the hourly time scale. （1） , hydrological dynamic characteristics on daily time scale （2） and hydrological dynamic characteristics at weekly time scales （3） .

[0074] Taking the hour-level LSTM subnetwork as an example, the prediction factor data x with a time resolution of 1 hour t （1） Input into the hour-level LSTM subnet, refer to the hidden state h obtained by the model at the previous moment t-1 （1） , which information to keep and discard is determined by the input gate and the forget gate. Combining the screening results of the forget gate and the input gate with the candidate cell state, the hour-level LSTM subnet updates the cell state at the current moment. The output gate determines how much information in the current cell state can be used as the hidden state h at the current moment. t （1） Output. The hidden state h output at each time step t （1）Finally, it constitutes the hourly hydrological dynamic feature h at the hourly time scale of the output of the hourly LSTM subnet （1） ; Similarly, the hidden state output by the daily LSTM subnet constitutes the hydrological dynamic feature h at the daily time scale （2） ; The hidden state output by the weekly LSTM subnet constitutes the hydrological dynamic feature h at the weekly time scale （3） .

[0075] S302. Use the multi-scale feature fusion module to use the softmax function to perform dynamic weighted fusion on the hydrological dynamic feature h at the hourly time scale （1） , the hydrological dynamic feature h at the daily time scale （2） and the hydrological dynamic feature h at the weekly time scale （3） to obtain the comprehensive short-term feature vector .

[0076] The calculation formula of the comprehensive short-term feature vector :

[0077] ;

[0078] ;

[0079] ;

[0080] In the formula, is the input feature at the k-th scale at time t, represents the hidden state output at the k-th scale at time t-1, represents the hidden state at the k-th scale at time t, represents the feature vector of the k-th scale, represents the weight matrix of the k-th scale in the softmax function, represents the weight matrix of the j-th scale in the softmax function, represents the feature vector of the j-th scale, represents the bias vector of the j-th scale in the softmax function, represents the bias vector of the k-th scale in the softmax function, represents the hidden state output at the k-th scale at time 1, represents the hidden state output at the k-th scale at time 2, represents the hidden state output at the k-th scale at time T

[0081] Among them, k and j can take 1, 2, and 3 respectively, representing the hydrological dynamic feature at the hourly time scale, the hydrological dynamic feature at the daily time scale, and the hydrological dynamic feature at the weekly time scale

[0082] S303. Map the downsampled flood prediction factors to a low-dimensional vector space through the embedding layer of the first Transformer encoder and after fusing with the comprehensive short-term feature vector, a fused vector sequence is obtained , and perform positional encoding on the fused vector sequence , and sequentially pass the positionally encoded fused vectors through the self-attention mechanism, normalization layer, feed-forward network layer, and normalization layer to obtain the long-term feature vector .

[0083] Among them, the positional encoding formula includes:

[0084] ;

[0085] ;

[0086] In the formula: represents the position index of the element in the fused vector sequence , i represents the dimension index, is the model dimension set to 128 dimensions, represents the positional encoding value of the element in the fused vector sequence on the even dimensions, represents the positional encoding value of the element in the fused vector sequence on the odd dimensions.

[0087] Among them, the calculation formula of the long-term feature vector is:

[0088] ;

[0089] ;

[0090] ;

[0091] In the formula, x is the fused vector sequence , represents the weight matrix of the query, represents the weight matrix of the key, represents the weight matrix of the value, represents the query vector, represents the key vector, represents the value vector, represents the dimension of the key vector, T represents the transpose operation on the key vector, represents the activation function, represents the multi-head attention concatenation function, represents the single-head attention calculation function.

[0092] S304. Align the comprehensive short-term feature vector and the long-term feature vector to the same dimension. Specifically, the feature alignment module uses linear interpolation to refine the comprehensive short-term feature vector and the long-term feature vector to the same time step, and uses a fully connected layer to map the comprehensive short-term feature vector and the long-term feature vector to the same dimension.

[0093] Among them, let have a dimension of , have a dimension of , be the target dimension:

[0094] ;

[0095] ;

[0096] In the formula: , are fully connected layers, , are bias vectors, has a dimension of , has a dimension of . is the comprehensive short-term feature vector, is the comprehensive short-term feature vector for achieving dimension alignment, is the long-term feature vector for achieving dimension alignment, is the long-term feature vector.

[0097] S305. Use the cross-attention mechanism feature fusion module to fuse the comprehensive short-term feature vector and the long-term feature vector with the same dimension by using the cross-attention mechanism CAM to obtain the long and short dependence fusion feature vector ; Specifically, the aligned is linearly transformed to obtain the query matrix Q, the aligned is linearly transformed to obtain the key matrix K and the value matrix V, and the cross-attention mechanism is used to calculate the long and short dependence fusion feature vector ; The principle of the cross-attention mechanism (CAM) is shown in Figure 2 .

[0098] S306. Process the predicted output of the previous moment through the masked self-attention mechanism in the second Transformer decoder to obtain a feature vector and a long-short dependence fusion feature vector After performing cross-attention mechanism fusion, successively pass through the normalization layer, feed-forward network layer, and multi-layer perceptron of the second Transformer decoder to obtain the flood forecast result.

[0099] S4. Divide the flood prediction factors and flood data after normalization processing into calibration period data by two-thirds, and the remaining one-third into verification period data.

[0100] S5. Train the LSTM-CAM-Transformer flood forecasting model based on the calibration period data using the Root Mean Square Propagation (RMSprop) method; specifically, use the selected RMSprop optimization method to update the parameters of the LSTM-CAM-Transformer flood forecasting model according to the gradient information of the loss function, and continuously iterate and train about 200 rounds to determine the structural parameters such as the number of layers and the number of neurons in each layer of the LSTM-CAM-Transformer flood forecasting model until the LSTM-CAM-Transformer flood forecasting model converges or the decrease in the loss function is very small.

[0101] And input the calibration period data into the LSTM-CAM-Transformer flood forecasting model, calculate the mean square error loss function MSE according to the difference between the prediction result of the LSTM-CAM-Transformer flood forecasting model and the actual value: Among them, the prediction result of the LSTM-CAM-Transformer flood forecasting model can be flood water level data and / or flood flow data.

[0102] ;

[0103] In the formula: n is the number of samples, is the predicted value of the i-th sample, is the true value of the i-th sample.

[0104] S6. Verify the accuracy of the LSTM-CAM-Transformer flood forecasting model based on the verification period data using the coefficient of determination, and use the trained LSTM-CAM-Transformer flood forecasting model for flood prediction.

[0105] Specifically, input the flood prediction factors of the verification period into the trained LSTM-CAM-Transformer flood forecasting model to obtain the predicted flood forecasting result (flood water level data and / or flood flow data) to be sought, and compare it with the actual flood data. And by calculating the coefficient of determination R2 To predict the accuracy and reliability of the LSTM-CAM-Transformer flood forecasting model:

[0106] ;

[0107] Where: n is the number of samples, is the predicted value of the i-th sample, is the true value of the i-th sample, is the average value of the true values.

[0108] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0109] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An LSTM-CAM-Transformer flood forecasting model for long and short dependence feature fusion, characterized in that, The steps include: Obtain flood runoff data and flood prediction factor data, and use the mutual information method to screen flood prediction factors; Divide the screened flood prediction factors and flood data after normalization into calibration period data and verification period data; Construct an LSTM-CAM-Transformer flood forecasting model, and the structure of the LSTM-CAM-Transformer flood forecasting model includes a long-short term dependence feature extraction module and a flood prediction module; Among them, the long- and short-term dependence feature extraction module is used to extract the long- and short-term dependence fusion feature vector in the hydrological dynamic features of flood prediction factors. The long- and short-term dependence feature extraction module includes: three independent LSTM subnets, a multi-scale feature fusion module, a first Transformer encoder, a feature alignment module, and a cross-attention mechanism feature fusion module; the multi-scale feature fusion module uses the softmax function to assign different weights to the outputs of each LSTM subnet and perform weighted summation to obtain a comprehensive short-term feature vector ; The comprehensive short-term feature vector Calculation formula: ； ； ； wherein, represents the hidden state of the k-th scale at time t, is the input feature of the k-th scale at time t, represents the hidden state output by the k-th scale at time t-1, represents the feature vector of the k-th scale, represents the hidden state output by the k-th scale at time 1, represents the hidden state output by the k-th scale at time 2, represents the hidden state output by the k-th scale at time T; represents the weight matrix of the k-th scale in the softmax function, represents the weight matrix of the j-th scale in the softmax function, represents the feature vector of the j-th scale, represents the bias vector of the j-th scale in the softmax function, represents the bias vector of the k-th scale in the softmax function; Among them, k and j can take 1, 2, and 3 respectively, representing the hydrological dynamic characteristics at the hourly time scale, the hydrological dynamic characteristics at the daily time scale, and the hydrological dynamic characteristics at the weekly time scale; the flood prediction module is used to perform flood forecasting according to the long-short term dependence fusion feature vector and flood data, and the flood prediction module includes: a second Transformer decoder and a multi-layer perceptron; The execution steps of the LSTM-CAM-Transformer flood forecasting model include: Input the screened flood prediction factors into three independent LSTM subnets to obtain the hydrological dynamic characteristics at the hourly time scale, the hydrological dynamic characteristics at the daily time scale, and the hydrological dynamic characteristics at the weekly time scale respectively; The multi-scale feature fusion module is used to dynamically and weightedly fuse the hydrological dynamic features at the hourly time scale, the hydrological dynamic features at the daily time scale, and the hydrological dynamic features at the weekly time scale to obtain a comprehensive short-term feature vector ; The downsampled flood prediction factors are mapped to a low-dimensional vector space through the embedding layer of the first Transformer encoder and the comprehensive short-term feature vector After fusion, the long-term feature vector is obtained through the self-attention mechanism, normalization layer, feed-forward network layer, and normalization layer of the first Transformer encoder in sequence ; Align the comprehensive short-term feature vector through the feature alignment module and the long-term feature vector to the same dimension; Using the cross-attention mechanism feature fusion module to fuse the comprehensive short-term feature vectors with the same dimension and the long-term feature vectors to obtain a long-short dependence fusion feature vector ; The predicted output of the previous moment will be processed by the masked self-attention mechanism of the second Transformer decoder to obtain a flood feature vector and a long-short dependence fusion feature vector After cross-attention mechanism fusion, it will pass through the normalization layer, feed-forward network layer and multi-layer perceptron of the second Transformer decoder in sequence to obtain the flood forecast result; Train the LSTM-CAM-Transformer flood forecasting model based on the calibration period data using the root mean square propagation method; Verify the accuracy of the LSTM-CAM-Transformer flood forecasting model based on the verification period data using the coefficient of determination, and use the trained LSTM-CAM-Transformer flood forecasting model for flood prediction.

2. The LSTM-CAM-Transformer flood forecasting model for long- and short-term dependence feature fusion according to claim 1, wherein The three independent LSTM subnets are an hourly LSTM subnet, a daily LSTM subnet, and a weekly LSTM subnet respectively; The hourly LSTM subnet is used to capture the hydrological dynamic characteristics at the hourly time scale; The daily LSTM subnet is used to capture the hydrological dynamic characteristics at the daily time scale; The weekly LSTM subnet is used to capture the hydrological dynamic characteristics at the weekly time scale.

3. The LSTM-CAM-Transformer flood forecasting model for long and short dependence feature fusion according to claim 1, wherein, The downsampled flood prediction factors are mapped to a low-dimensional vector space through the embedding layer of the first Transformer encoder and after being fused with the comprehensive short-term feature vectors, a fused vector sequence is obtained , and position encoding is performed on the fused vector sequence . The position encoding formula includes: ； ； Wherein: represents the position index of the elements in the fusion vector sequence where i represents the dimension index, the model dimension is set to 128 dimensions, represents the position encoding value of the elements in the fusion vector sequence represents the position encoding value of the elements in the fusion vector sequence on the odd dimensions.

4. A LSTM-CAM-Transformer flood forecasting model for long and short dependence feature fusion according to claim 1, wherein The long-term feature vector Calculation formula: ； ； ； where \(x\) is the sequence of fusion vectors , denotes the weight matrix of the query, denotes the weight matrix of the key, denotes the weight matrix of the value, denotes the query vector, denotes the key vector, denotes the value vector, denotes the dimension of the key vector, \(T\) denotes the transpose operation on the key vector, denotes the activation function, denotes the multi - head attention concatenation function, denotes the single - head attention calculation function.

5. A LSTM-CAM-Transformer flood forecasting model for long and short dependence feature fusion according to claim 1, characterized in that The feature alignment module uses the line interpolation method to refine the comprehensive short-term feature vector and the long-term feature vector to the same time step, and uses a fully connected layer to map the comprehensive short-term feature vector and the long-term feature vector to the same dimension.

6. The LSTM-CAM-Transformer flood forecasting model for long and short dependence feature fusion according to claim 1, wherein, The cross-attention mechanism feature fusion module uses the cross-attention mechanism to perform long-short dependence feature fusion on the comprehensive short-term feature vectors and the long-term feature vectors to obtain long-short dependence fusion feature vectors .

Images