Runoff prediction method based on local attention enhanced model

The runoff prediction method using local attention enhancement models solves the problems of parallel training difficulties and large memory consumption in existing models in runoff prediction, achieving more accurate runoff prediction and improving the robustness and prediction accuracy of the model.

CN116050652BActive Publication Date: 2026-05-05CHONGQING UNIV OF POSTS & TELECOMM +1
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CHONGQING UNIV OF POSTS & TELECOMM
Filing Date
2023-02-22
Publication Date
2026-05-05

AI Technical Summary

Technical Problem

Existing time series prediction models, such as recurrent neural networks and convolutional neural networks, suffer from problems such as difficulty in parallel training, vanishing gradients, and large memory consumption in runoff prediction, making it difficult to accurately capture the changing trends of long-term series.

Method used

A runoff prediction method based on a local attention enhancement model is adopted. By combining a variable selection module, a local information enhancement module, and an attention module with a temporal convolutional network and a self-attention mechanism, the data processing and model architecture are optimized to improve prediction accuracy.

Benefits of technology

More accurate runoff predictions were achieved under the same memory conditions, improving the model's robustness and prediction accuracy, especially showing greater advantages in long-term series prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116050652B_ABST
    Figure CN116050652B_ABST
Patent Text Reader

Abstract

This invention relates to a runoff prediction method based on a local attention enhancement model, belonging to the field of time series prediction. The method includes: acquiring and preprocessing data, inputting the processed data into a trained prediction model to obtain the predicted runoff sequence; wherein the runoff prediction model includes a variable selection module, a local information enhancement module, an attention module, and a model extraction module. This invention's prediction model considers the long runoff cycle, irregular trends, and varying degrees of influence of different covariates on the results. It utilizes a variable selection module to weight covariates, a local information enhancement module to capture local information of the runoff sequence, enabling data at individual time points to obtain short-term trend characteristics, and a self-attention module to obtain similarity and attention information between trends. Furthermore, by appropriately setting the depth of the encoder and decoder and the model extraction module, more accurate predictions are achieved with the same available memory.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of time series forecasting and relates to a runoff forecasting method based on a local attention enhancement model. Background Technology

[0002] Time series simulation and forecasting play a crucial supporting role in resource optimization and disaster prevention and mitigation. Currently, they are mainly divided into physical-driven models that describe the sequence generation process and data-driven models based on data. Physical-driven models, based on data-related physical concepts, focus on depicting the complex process of time series formation. Specifically in the field of hydrological forecasting, these models typically need to consider multiple parameters such as meteorological, topographical, and soil factors. They can achieve good predictive results in runoff forecasting in areas with complete data, but due to significant differences in data completeness across regions and the uncertainty of model structure and parameters leading to cumulative errors, these models are greatly limited by parameters and future meteorological input conditions. In contrast, data-driven models do not require explicit knowledge of the physical generation process of the time series; by establishing a functional relationship between driving factors and predictive factors, they can also produce good predictive results. Therefore, relying on data-driven models to predict time series processes is considered a simple and effective forecasting method even when data is incomplete.

[0003] Currently, improved versions of Recurrent Neural Networks (RNNs) such as Long Short-Term Memory (LSTM) networks have demonstrated good performance in multi-step prediction. However, due to limitations in model structure, RNNs still suffer from the problem of capturing sequence length caused by the inability to train in parallel and the vanishing gradient problem. Furthermore, Temporal Convolutional Networks (TCNs) based on Convolutional Neural Networks (CNNs) have solved the problem of parallel training, but relying on stacking hidden layers to obtain a larger receptive field results in a greater memory footprint. Therefore, TCNs are ill-suited for capturing long sequences. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a runoff prediction method based on a local attention enhancement model, thereby improving the prediction accuracy of the prediction model.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A runoff prediction method based on a local attention enhancement model is proposed. This method acquires and preprocesses runoff data, then inputs the processed data into a trained prediction model to obtain predicted runoff sequences. The runoff prediction model includes a variable selection module, a local information enhancement module, an attention module, and a model extraction module.

[0007] The runoff prediction model designed in this invention takes into account the characteristics of long runoff cycles and irregular trends, and considers the different degrees of influence of different covariates on the results. It uses a variable selection module to weight the covariates, a local information enhancement module to capture local information of the runoff sequence, so that the data at a single time point can obtain short-term trend characteristics, and a self-attention module to obtain similarity and attention information between trends. By reasonably setting the depth of the encoder and decoder and the model extraction module, more accurate predictions are achieved with the same available memory.

[0008] The method specifically includes the following steps:

[0009] S1: Prepare and preprocess the dataset;

[0010] Historical runoff data and related covariates are acquired, aligned according to the time dimension, and normalized to eliminate the influence of the data's numerical value on the model; then, the data is divided into training, validation, and test sets.

[0011] S2: Input the training set into the variable selection module; the variable selection module uses modules such as gated residual networks to weight the input variables, filter out the relevant covariates that contribute more to the results, optimize the impact of data on the model, and thus obtain better prediction accuracy;

[0012] S3: Location encoding of time series: The location encoding layer injects location information into the input data;

[0013] S4: Input the position-encoded sequence into the encoder-decoder structure; wherein, the encoder-decoder structure includes a local information enhancement module, an attention module, and a model extraction module, etc. The output of the encoder is used to participate in the cross-attention calculation of the decoder, and the output of the decoder is the predicted sequence;

[0014] S5: The loss function is calculated using the mean absolute error (MAE) and the Nash correlation coefficient (NSE) for the prediction results;

[0015] S6: Set an initial learning rate for the training model and optimize the model's decay learning rate using the Adam algorithm;

[0016] S7: Use the validation set data to verify the training effect. Stop training early when the validation set results continue to decline to prevent overfitting.

[0017] Furthermore, in step S1, daily runoff data, atmospheric circulation index, sea surface temperature index, and other indices can be used as the dataset for this method.

[0018] Obtaining relevant covariates from runoff data specifically includes: preliminary screening of variables using the Pearson correlation coefficient method, followed by standardization; the formula for calculating the Pearson correlation coefficient between the prediction object Y and factor X is as follows:

[0019]

[0020] Where n is the number of data samples, X i Let X be the i-th sample value, and Y be the i-th sample value. i Let Y be the i-th sample value. Let X be the sample mean. Let Y be the sample mean.

[0021] Standardization processing, transformation function is Here, μ is the mean of all samples, and σ is the standard deviation of the samples. This ensures that each data point has a variance of 1 and a mean of 0. This avoids an imbalance in variable selection weights due to excessively large feature values ​​for some variables at the variable selection layer. It also avoids a decrease in prediction accuracy caused by outliers and extreme values.

[0022] Furthermore, in step S2, the variable selection module includes a gated residual block (GRB) and a softmax function, enabling each variable to obtain the weight of the flattened input under the softmax operation; the formula is as follows:

[0023]

[0024]

[0025]

[0026] in, Ξ represents the transformed input of the j-th variable at time t. t Represents all flattened vectors at time t. Choose weights for variables. Let be the output of the j-th variable at time t after being transformed by the GRB module; we note that each variable has its own GRB module, and we share the weights across all time steps t. It is the output of all variables at time t after passing through the variable selection module, m x Indicates the number of variables.

[0027] Furthermore, in step S3, the location encoding layer injects location information into the input data, and the corresponding calculation formula is as follows:

[0028]

[0029]

[0030] Where PE represents Position Embedding, pos represents the position of the data in the sequence, and d model Let 2i represent the dimension of PE, where 2i represents an even-numbered dimension and 2i+1 represents an odd-numbered dimension. This method makes it easier to calculate relative positions. For a fixed-length spacing k, the relative positional relationship between PE(pos+k) and PE(pos) can be calculated using the sum-difference formula, as follows:

[0031] sin(pos+k)=sin(pos)cos(k)+cos(pos)sin(k)

[0032] cos(pos+k)=cos(pos)cos(k)-sin(pos)sin(k)

[0033] Furthermore, in step S4, the local information enhancement module uses a temporal convolutional network to capture the local context of the time series; the attention module performs global attention calculation on the time series; and the model extraction module acts on the encoder part to perform one-dimensional pooling operation on the intermediate model of the encoder to reduce the size of the stacked encoding layer model.

[0034] Furthermore, in step S4, the model architecture includes: a stackable encoder, a decoder, model distillation layers between the encoding layers, and an output layer. The encoder consists of temporal convolutional layers, multi-head self-attention layers, residual connections, and layer normalization; multiple encoder layers can be stacked in series. The decoder consists of temporal convolutional layers, multi-head self-attention layers, multi-head cross-attention layers, residual connections, and layer normalization; multiple decoder layers can be stacked in series. The decoder needs to pad the predicted sequence positions with zeros.

[0035] The temporal convolutional layer uses a convolutional neural network and includes causal convolution, dilated convolution, and residual modules to learn short-term local features. The causal convolution expands the receptive field by stacking convolutional layers, learning local sequence information. The dilated convolution injects holes into standard convolutions to increase the receptive field. Unlike traditional convolution, dilated convolution allows for interval sampling of the input during convolution. Increasing the number of convolutional layers can lead to gradient vanishing and network degradation. The residual module addresses these issues by creating identity mappings in redundant layers of the network. The residual connection calculation formula for the residual module is as follows:

[0036] O = x + subLayer(x)

[0037] Where x represents the input of the sublayer, subLayer(x) represents the input of the sublayer, and O represents the final input;

[0038] The attention prediction sequence obtained after the multi-head self-attention layer and the multi-head cross-attention layer is as follows:

[0039]

[0040] Q = W Q x t

[0041] K = W K x t

[0042] V = W V x t

[0043]

[0044] in, This represents a time series sample after positive standardization, where Linear represents a linear layer, and x... t The initial time series components are represented by Q, K, and V, which represent the query component, key-value component, and value component, respectively. Q W K W V F represents the weight matrices corresponding to the query component, key component, and value component, respectively. A Let K represent the attention prediction sequence, Softmax represent the normalized exponential function, and K represent the normalized exponential function. T d represents the transpose of the key-value component. k Indicates the model scale;

[0045] Residual connections are used to prevent gradient vanishing and to preserve local features after passing through a temporal convolutional network.

[0046] Layer normalization maps the values ​​of intermediate layers or outputs during training to a range of 0 to 1. This eliminates the influence of differences between data on the model.

[0047] The output layer transforms the output dimension of the decoder through a fully connected layer to make the output result meet expectations.

[0048] Furthermore, in step S5, the formulas for calculating the Mean Absolute Error (MAE) and the Nash Correlation Coefficient (NSE) are as follows:

[0049]

[0050]

[0051] Where N represents the length of the runoff forecast period, This represents the predicted runoff volume at time i. This represents the observed runoff volume at time i. This represents the average value of the observed runoff, and i is the prediction time.

[0052] The beneficial effects of this invention are as follows: It adds a variable weight selection function for multivariate prediction, which is beneficial for filtering more relevant time-series variables. Simultaneously, it uses a temporal convolutional network, that is, it utilizes one-dimensional temporal convolution combined with causal convolution and dilated convolution operations to increase the receptive field, enabling each single time node to obtain local contextual information. Furthermore, it uses one-dimensional max-pooling layers between the encoding layers to reduce the model size, improve training speed, and enhance the model's robustness.

[0053] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0054] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0055] Figure 1 This is a schematic diagram of the runoff prediction method model of the present invention;

[0056] Figure 2 This is a schematic diagram of the variable selection module in this invention;

[0057] Figure 3 This is a schematic diagram of the gated residual block structure in this invention;

[0058] Figure 4 This is a comparison image of local attention enhancement in this invention. Detailed Implementation

[0059] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0060] Please see Figures 1-4This invention provides a time series prediction method based on a local attention enhancement model, which adds variable weight selection functionality under multivariate prediction, helping us to filter more relevant time series variables. Simultaneously, a temporal convolutional network is used, employing one-dimensional temporal convolution combined with causal convolution and dilated convolution operations to increase the receptive field, enabling each individual time node to obtain local contextual information. Furthermore, one-dimensional max-pooling layers are used between the encoding layers to reduce the model size, improve training speed, and enhance the model's robustness.

[0061] like Figure 1 As shown, the specific implementation steps of this time series forecasting method are as follows:

[0062] Step 1: Prepare the dataset for model input;

[0063] Daily runoff data, atmospheric circulation index, sea surface temperature index, and other indices were used as the dataset for this method. Suitable training data was constructed, with runoff data from Pingshan station as the target value for prediction. To avoid excessive memory consumption due to irrelevant indices, preliminary variable screening was performed on historical runoff data from Pingshan station and 130 atmospheric circulation index factors. The Pearson correlation coefficient method was used. The correlation coefficient between the prediction target Y and factor X is:

[0064]

[0065] In the formula: n is the number of data samples; X i Let X be the i-th sample value; Y be the i-th sample value. i Let Y be the i-th sample value; Let X be the sample mean; R is the sample mean of Y. XY The larger the absolute value, the higher the correlation between the predicted object Y and the factor X.

[0066] The highest temperature, average temperature, sunshine duration, East Asian trough intensity index, and Eurasian zonal circulation index with high correlation were obtained through the above correlation coefficient screening operation and used for training.

[0067] Step 2: Data Preprocessing. The historical runoff data from the Pingshan station and the five preliminarily selected relevant indices are standardized. The transformation function is as follows: Here, μ is the mean of all samples, and σ is the standard deviation of the samples. This ensures that each data point has a variance of 1 and a mean of 0. This avoids an imbalance in variable selection weights due to excessively large feature values ​​for some variables at the variable selection layer. It also avoids a decrease in prediction accuracy caused by outliers and extreme values.

[0068] Step 3: Dataset Partitioning. The preprocessed dataset is divided into three mutually exclusive subsets: a training set, a validation set, and a test set. The training set is used to fit the model, the validation set is used to prevent overfitting, and the test set is used to check the model's training performance.

[0069] Step 4: Variable Selection and Weighting. The variable selection layer weights each variable to optimize the data's impact on the model, thereby achieving better prediction accuracy. Its variable selection module consists of a Gated Residual Block (GRB) and a Softmax function. This ensures that each variable receives the weights of the flattened input under the Softmax operation. The formula is as follows:

[0070]

[0071]

[0072]

[0073] in, Ξ represents the transformed input of the j-th variable at time t. t Represents all flattened vectors at time t. Choose weights for variables. Let be the output of the j-th variable at time t after being transformed by the GRB module. We note that each variable has its own GRB module, and we share weights across all time steps t. It is the output of all variables at time t after passing through the variable selection module.

[0074] Step 5: Training data enters the positional encoding layer. The positional encoding layer injects positional information into the input data, solving the problem that the transformer encoder and decoder cannot obtain the sequence position order.

[0075] calculate:

[0076]

[0077]

[0078] Where PE stands for Position Embedding, pos represents the position of the data in the sequence, d represents the dimension of PE, 2i represents an even dimension, and 2i+1 represents an odd dimension. This method makes it easier to calculate relative positions. For a fixed interval k, the relative positional relationship between PE(pos+k) and PE(pos) can be calculated using the sum-difference formula, as follows:

[0079] sin(pos+k)=sin(pos)cos(k)+cos(pos)sin(k)

[0080] cos(pos+k)=cos(pos)cos(k)-sin(pos)sin(k)

[0081] Step 6: Construct an encoder-decoder architecture based on local information enhancement and attention mechanisms. The model architecture includes: stackable encoders, decoders, model distillation layers between encoding layers, and an output layer. Specifically, it includes:

[0082] (1) The encoder consists of a temporal convolutional layer, a multi-head self-attention layer, residual links and layer normalization. The encoder can be stacked in series.

[0083] (2) The decoder consists of temporal convolutional layers, multi-head self-attention layers, multi-head cross-attention layers, residual connections, and layer normalization. Multiple decoders can be stacked in series. The decoder needs to pad the positions of the predicted sequence with zeros.

[0084] The temporal convolutional layer uses a convolutional neural network and includes causal convolution, dilated convolution, and residual modules to learn short-term local features. Specifically, it includes the following:

[0085] Causal convolution expands the receptive field by stacking convolutional layers, learning local sequence information;

[0086] Dilated convolution injects holes into standard convolution to increase the receptive field. Unlike traditional convolution, dilated convolution allows for interval sampling of the input during convolution.

[0087] Increasing the number of convolutional layers can lead to gradient vanishing and network degradation. Residual modules address these issues by creating identity mappings between redundant layers. The formula for calculating residual connections is as follows:

[0088] O = x + subLayer(x)

[0089] Where x represents the input of the sublayer, subLayer(x) represents the input of the sublayer, and O represents the final input;

[0090] The attention prediction sequence obtained after the multi-head self-attention layer and the multi-head cross-attention layer is as follows:

[0091]

[0092] Q = W Q x t

[0093] K = W K x t

[0094] V = W V x t

[0095]

[0096] in, This represents a time series sample after positive standardization, where Linear represents a linear layer, and x... t The initial time series components are represented by Q, K, and V, which represent the query component, key-value component, and value component, respectively. Q W K W V F represents the weight matrices corresponding to the query component, key component, and value component, respectively. A Let K represent the attention prediction sequence, Softmax represent the normalized exponential function, and K represent the normalized exponential function. T d represents the transpose of the key-value component. k Indicates the model scale.

[0097] Residual connections are used to prevent gradient vanishing and to preserve local features after passing through the temporal convolutional network;

[0098] Layer normalization maps the values ​​of intermediate layers or outputs during training to a range of 0 to 1. This eliminates the influence of differences between data on the model.

[0099] (3) The output layer transforms the output dimension of the decoder through a fully connected layer to make the output result meet expectations.

[0100] Step 7: The model extraction layer reduces the size of the intermediate layers of the encoder stack by using one-dimensional max pooling operation, thereby reducing the model's memory footprint and enabling the model to capture longer sequence relationships with the same memory footprint.

[0101] Step 8: Feed the preprocessed training and validation sets into the model, and train the model using the MSE loss function and the Adam optimizer. Set the training to 100 epochs, and stop training early when the loss function decreases consecutively for 5 times.

[0102] Step 9: To verify the effectiveness of the method proposed in this invention, the method of this invention is compared with the classic time series prediction methods Long Short-Term Memory (LSTM) and Transformer for predicting runoff at the Pingshan hydrological station along the Yangtze River. Mean Absolute Error (MAE) and Nash-Sutcliffe Efficiency (NSE) are used as evaluation indicators.

[0103] MAE is the average of the absolute values ​​of the differences between all individual predicted and observed values. It can effectively reflect the magnitude of the prediction error. Its calculation expression is:

[0104]

[0105] in,

[0106] N represents the length of the runoff forecasting period;

[0107] This represents the predicted runoff volume at time i.

[0108] This represents the observed runoff volume at time i.

[0109] NSE is a commonly used index in hydrology to evaluate the prediction results of hydrological models, and its value ranges from negative infinity to 1. An NSE value close to 1 indicates high prediction reliability; a value close to 0 indicates that the prediction result is close to the average level of the observed values, that is, the overall result is reliable, but there is a certain error in individual prediction values; a value much less than 0 indicates that the prediction result is unreliable.

[0110]

[0111] in,

[0112] N represents the length of the runoff forecasting period;

[0113] This represents the predicted runoff volume at time i.

[0114] This represents the observed runoff volume at time i;

[0115] This represents the average value of the observed runoff.

[0116] Table 1 shows the MAE (Momentum Efforts) of the Pingshan Hydrological Station under different prediction periods (three days, one week, half a month, one month, one quarter, and half a year). The unit of data in the table is cubic meters per second. Observation of the data in the table shows that the MAE value of the method described in this invention is lower than that of the comparative method in all test periods, indicating that the overall prediction error is the smallest, and it shows a greater advantage in long-term prediction.

[0117] Table 1 Comparison of mean absolute error under different prediction periods.

[0118]

[0119] Table 2 shows the NSE (Number of Sequences) at the Pingshan Hydrological Station under different prediction periods (three days, one week, half a month, one month, one quarter, and half a year). The data in the table shows that, compared with the comparative methods, the NSE value of the method described in this invention is closer to 1 in all test periods, indicating the highest reliability of the runoff prediction results, and it also shows a greater advantage in long-term predictions.

[0120] Table 2 Comparison of Nash correlation coefficients under different prediction periods.

[0121]

[0122] like Figure 4 As shown, the output attention layer attention point matrix shows that after TCN local attention enhancement, the attention in this invention obtains more local contextual information and can capture similar historical sequences more accurately.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A runoff prediction method based on a local attention enhancement model, characterized in that, The method involves acquiring and preprocessing runoff data, then inputting the processed data into a trained prediction model to obtain the predicted runoff sequence. The runoff prediction model includes a variable selection module, a local information enhancement module, an attention module, and a model extraction module. The specific steps of this method are as follows: S1: Prepare and preprocess the dataset; Historical runoff data and related covariates were acquired, aligned according to the time dimension, and normalized; then, the data were divided into training, validation, and test sets. S2: Input the training set into the variable selection module; the variable selection module uses a gated residual network to weight the input variables and filter out the relevant covariates that contribute more to the results; the variable selection module includes a gated residual block (GRB) and... Function; which allows each variable to obtain The weights of the flattened inputs under the operation; the formula is as follows: in, Indicates the first Variables in The input after the time change, express All flattened vectors at time 1, Choose weights for variables. for Time of the first The output of each variable after being transformed by the GRB module; yes At any given moment, all variables pass through the output of the variable selection module. Indicates the number of variables; S3: Location encoding of time series: The location encoding layer injects location information into the input data; S4: Input the position-encoded sequence into the encoder-decoder structure; wherein, the encoder-decoder structure includes a local information enhancement module, an attention module and a model extraction module, the encoder output is used to participate in the cross-attention calculation of the decoder, and the decoder output is the predicted sequence; S5: The loss function is calculated using the mean absolute error (MAE) and the Nash correlation coefficient (NSE) for the prediction results; S6: Set an initial learning rate for the training model and optimize the model's decay learning rate using the Adam algorithm; S7: Use the validation set data to verify the training effect. Stop training early when the validation set results continue to decline to prevent overfitting.

2. The runoff prediction method according to claim 1, characterized in that, In step S1, relevant covariates of runoff data are obtained, specifically including: preliminary screening of variables using the Pearson correlation coefficient method, and standardization processing; among which, the prediction object and factors The formula for calculating the Pearson correlation coefficient between them is: in, The number of data samples. for The Each sample value for The Each sample value for The sample mean, for The sample mean.

3. The runoff prediction method according to claim 1, characterized in that, In step S3, the location encoding layer injects location information into the input data, and the corresponding calculation formula is as follows: in, Indicates positional embedding, Indicates the position of the data in the sequence. express Dimensions The dimension representing an even number. Represents odd-numbered dimensions; for fixed-length spacing... , and The relative positional relationship can be calculated using the sum and difference formula of two angles, as follows: 。 4. The runoff prediction method according to claim 1, characterized in that, In step S4, the local information enhancement module uses a temporal convolutional network to capture the local context of the time series; the attention module performs global attention calculation on the time series; and the model extraction module acts on the encoder part to perform one-dimensional pooling operation on the intermediate model of the encoder to reduce the size of the stacked encoding layer model.

5. The runoff prediction method according to claim 1 or 4, characterized in that, In step S4, the encoder consists of a temporal convolutional layer, a multi-head self-attention layer, a residual connection, and a layer normalization layer. The encoder can be stacked in series with multiple layers. The decoder consists of a temporal convolutional layer, a multi-head self-attention layer, a multi-head cross-attention layer, a residual connection, and a layer normalization layer. The decoder can be stacked in series with multiple layers. The decoder needs to pad the positions of the predicted sequence with zeros; where, The temporal convolutional layer uses a convolutional neural network and includes causal convolution, dilated convolution, and residual modules for learning short-term local features. The causal convolution expands the receptive field by stacking convolutional layers, learning local sequence information. The dilated convolution injects holes into a standard convolution to increase the receptive field; dilated convolution allows for interval sampling of the input during convolution. The residual connection calculation formula for the residual module is as follows: in, Indicates the input of the sublayer. Indicates the input of the sublayer. Indicates the final input; The attention prediction sequence obtained after the multi-head self-attention layer and the multi-head cross-attention layer is as follows: in, This represents a time series sample after positive standardization; Linear indicates a linear layer. Indicates the initial time series components. , K , V These represent the query component, key-value component, and numeric component, respectively. , These represent the weight matrices corresponding to the query component, key-value component, and numeric component, respectively. Represents the attention prediction sequence, Represents the normalized exponential function, This represents the transpose of the key-value component. Indicates the model scale.

6. The runoff prediction method according to claim 1, characterized in that, In step S5, the formulas for calculating the mean absolute error (MAE) and the Nash correlation coefficient (NSE) are as follows: in, Indicates the length of the runoff forecast period. express Predicted runoff volume at any time express Runoff observation values ​​at specific times This represents the average value of the observed runoff. For predicting time.

Citation Information

Patent Citations

  • Universal image target detection method and device based on self-attention mechanism

    CN113902926A

  • Runoff prediction method based on Transform sequence-to-sequence model

    CN115146700A