A runoff prediction method based on improved Transformer model

Abstract

The present invention discloses a runoff prediction method based on an improved Transformer model. By introducing a pre-trained encoder CoST, the bottleneck of the traditional Transformer encoder in prediction is eliminated, and the trend and seasonal representation of the runoff sequence can be effectively extracted. The present invention introduces a past attention decoder and adopts the M0 method to pre-process the input, trend and seasonal representation, remove the long-term distribution trend of the data, and improve the model's ability to capture short-term and long-term dependency patterns. The present invention proposes a data sliding extraction window, which reduces the complexity of the attention mechanism by compressing the input data length, so as to achieve the effect of less time consumption and high precision. The present invention solves the problem of large local runoff prediction errors of the traditional Transformer model by coupling the pre-trained encoder CoST, the past attention decoder and the data sliding extraction window, and improves the prediction accuracy and efficiency.

Description

Technical Field

[0001] The invention relates to the technical field of basin runoff prediction, and in particular to a runoff prediction method based on an improved Transformer model. Background Art

[0002] Accurate and reliable runoff prediction plays an important role in water resource management and flood prevention and disaster reduction, such as water supply, hydropower generation and ecological restoration. Existing runoff prediction methods can be roughly divided into physical process models and data-driven models. The physical process model is based on the understanding of the natural water cycle process. The model has many parameters and complex calculations, and cannot accurately capture the runoff change law under a changing environment.

[0003] In recent years, with the improvement of computing power and the increase in the amount of hydrological data, data-driven models represented by machine learning have gradually been applied to flood forecasting, runoff prediction, etc. and have achieved good results. Among them, the long short-term memory neural network (LSTM) overcomes the modeling difficulty of long-term dependencies of RNN by introducing a gating mechanism, and is widely used in runoff prediction. Different from the LSTM model, the Transformer model fundamentally solves the problem of long-term memory loss by introducing an attention mechanism, and shows great potential in processing time data series.

[0004] However, the Transformer model still has the following problems when predicting runoff. First, the complexity of the attention mechanism is quadratically related to the input length, which significantly increases the model's training and prediction time for long sequence data; second, the dynamic decoder of the Transformer model shows good performance in short sequence prediction, but the error will be amplified in long sequence prediction; finally, the Transformer model was originally designed to solve natural language processing tasks, and it still has insufficient ability to capture the changing laws of runoff sequences. Summary of the invention

[0005] The purpose of the present invention is to provide a runoff prediction method based on an improved Transformer model, which solves the problems of low efficiency and accuracy and poor robustness of the Transformer model in runoff prediction.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A runoff prediction method based on an improved Transformer model, wherein the runoff prediction method comprises the following steps:

[0008] Select and collect hydrological data of the target basin, and preprocess the hydrological data to obtain training sets, validation sets, and test sets;

[0009] Build an improved Transformer model;

[0010] Use the training set data and optimization algorithm to train and optimize the improved Transformer model, use the validation set data to verify the trained improved Transformer model, and optimize and adjust the improved Transformer model based on the verification results;

[0011] Based on the test set, the optimized improved Transformer model is used to predict runoff, and the prediction results are denormalized to obtain the final runoff prediction results.

[0012] According to the above technical solution, the hydrological data preprocessing step includes:

[0013] Correct and interpolate missing and abnormal hydrological data; the hydrological data include but are not limited to precipitation data, evapotranspiration data, and runoff data;

[0014] The corrected hydrological data are sorted according to the time scale to obtain a new data series, and then the new data series are standardized;

[0015] The hydrological data obtained by standardization processing were divided into training set, validation set and test set in a ratio of 8:1:1;

[0016] Wherein, the standardization formula is:

[0017] ;

[0018] in, The data are obtained after standardization; is the new data sequence; and are the mean and standard deviation of the new data series respectively.

[0019] According to the above technical solution, the step of building the improved Transformer model includes:

[0020] An input of the traditional Transformer model is divided into long sequence input and short sequence input, including: determining the time step of predicted runoff , according to the time period type and step size of the predicted runoff, set the long sequence input length and short sequence input length The value range of long sequence input length The value range is , short sequence input length The value range is ,in The lower limit is greater than The data in the long sequence input and the short sequence input are precipitation data and evapotranspiration data, etc.;

[0021] The pre-trained encoder CoST is introduced, and seven categories of timestamp information are set as the time representation of the runoff sequence. The pre-trained encoder CoST is used to pre-train the runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input to obtain the long sequence error-free runoff and short sequence error-free runoff ; The timestamp information includes: minute, hour, day of the week, day of the month, day of the year, month, week of the year;

[0022] Introduce a data sliding extraction window, which is used to set the length of the long sequence extraction window and short sequence window length , using the data sliding extraction window to extract the long sequence error-free runoff Step length Shortened to , the short sequence error-free runoff Step length Shortened to ;

[0023] Introducing a past attention decoder that uses the M0 method to perform error-free runoff on long sequences After preprocessing, the input without long-term distribution trend representation is obtained , using the past long-term multi-head attention mechanism and the past long-term and short-term cross multi-head attention mechanism , , The corresponding long sequence input and The corresponding short sequence input is processed to capture the long-term dependency pattern and short-term dependency pattern of the data, and then the captured long-term dependency pattern features and short-term dependency pattern features are output through the normalization layer, convolution feedforward layer group, flattening layer and linear layer to output the final runoff prediction result; the long-term dependency pattern refers to the seasonality or periodicity of the runoff, and the short-term dependency pattern refers to the short-term variation law of the runoff in a specific period of time.

[0024] According to the above technical solution, the specific steps of the pre-training encoder CoST include:

[0025] For each time step, it is assumed that the runoff data consists of three components: trend, seasonality, and error, namely ,in represents the trend component of runoff data, represents the seasonal component of runoff data, Indicates the error, is the corresponding step length of the given timestamp information, represents the sum of component dimensions, represents the latent space;

[0026] The runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input are transformed into has the sum of component dimensions The sum of input channels and trend component dimensions The output channel uses a hybrid autoregressive expert with one-dimensional causal convolution to extract trend representations. After the extracted trend representations pass through the average pooling layer, the time domain contrast loss function is used. Learning, obtaining the final trend representation ;in, , is the corresponding step length of the given timestamp information;

[0027] The runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input are mapped to the frequency domain using discrete Fourier transform, and then the frequency domain seasonal trend representation is obtained through a learnable Fourier layer. The frequency domain seasonal trend representation is learned using the frequency domain contrast loss function, and the frequency domain seasonal trend representation is converted back to the time domain using the inverse operation. ;

[0028] The total loss function formula is used to connect the trend representation and the seasonal trend representation to obtain error-free runoff. ,in The long sequence input corresponds to the long sequence error-free runoff. , the short sequence input corresponds to the short sequence error-free runoff .

[0029] Among them, the sum of the component dimensions represents the sum of the number of latent space dimensions of the trend component and the number of latent space dimensions of the seasonal component; the sum of the trend component dimensions Represents the total number of dimensions of the latent space of the trend component.

[0030] Among them, the core of the frequency domain contrast loss function is to distinguish between different periodic modes of a given frequency. However, in order to overcome the problem of constructing the loss function using complex number representation, each frequency is uniquely represented by amplitude and phase.

[0031] According to the above technical solution, the time domain contrast loss function :

[0032] ;

[0033] in, is the temperature parameter, The momentum encoder is used to obtain the encoding representation of the trend representation, Indicates a positive trend. Indicates a negative trend. is the total number of negative trend samples; Indicates the number of samples in a batch.

[0034] The core is a MoCo variant, whose momentum encoder is used to obtain the encoded representation of the trend representation ,The dynamic dictionary with queues is used to obtain the positive and negative trend representations,respectively.

[0035] According to the above technical solution, the frequency domain contrast loss function is composed of the frequency domain amplitude contrast loss function And the frequency domain phase contrast loss function composition;

[0036] The frequency domain amplitude contrast loss function :

[0037] ;

[0038] The frequency domain phase contrast loss function :

[0039] ;

[0040] The inverse operation:

[0041] ;

[0042] in, represents the jth sample in the batch of samples, express The corresponding extended sample, represents the amplitude of the frequency corresponding to the jth sample, represents the phase of the frequency corresponding to the jth sample, express The phase of the frequency corresponding to the jth sample in the corresponding extended sample, express The amplitude of the corresponding extended sample frequency, represents the sum of component dimensions, represents the frequency domain, and represents the linear layer parameters, represents the kth sample in the batch of samples, Indicates the phase of the frequency corresponding to the kth sample in the batch of samples, represents the operation function of mapping the frequency domain representation to the time domain (the inverse operation of mapping the time domain representation to the frequency domain), N represents the number of batch samples, represents the batch number, PN represents the total number of samples, represents the time domain output matrix, i.e., seasonal representation.

[0043] According to the above technical solution, the total loss function formula is as follows:

[0044] ;

[0045] in, represents the time domain contrast loss function, represents the frequency domain amplitude contrast loss function, represents the frequency domain phase contrast loss function, A parameter that balances the trade-off between trend and seasonal factors.

[0046] According to the above technical solution, the specific execution steps of the M0 method include:

[0047] The long sequence of error-free runoff Subtract the corresponding mean To input without long-term distribution trend ;

[0048] Using the past long-term multi-head attention mechanism and the past long-term and short-term cross multi-head attention mechanism , The corresponding long sequence input and The corresponding short sequence input is processed to obtain the preliminary prediction value;

[0049] Characterize the trend of the initial forecast value and the long series of error-free runoff The corresponding mean The final runoff prediction value is obtained by adding them together.

[0050] According to the above technical solution, the past attention decoder consists of a past long-term multi-head attention mechanism, a normalization layer, and a convolutional feedforward layer;

[0051] The past long-term multi-head attention mechanism:

[0052] ;

[0053] The normalization layer:

[0054] ;

[0055] The convolutional feed-forward layer:

[0056] ;

[0057] in, Indicates long-term long-term attention, represents the feature fusion method, represents the attention head, represents the parameter matrix, represents batch normalization, represents the standardization method, represents the residual connection layer, represents the convolutional feed-forward layer, represents the parameters of the feedforward sublayer, Indicates input, Represents the sum of component dimensions, and the activation function is GELU. Indicates input with no long-term distribution trend , short sequence error-free runoff , The corresponding long sequence input and The corresponding short sequence input, Indicates transpose.

[0058] According to the above technical solution, the improved Transformer model optimization adjustment takes the mean absolute error MAE as the objective function, uses the Bayesian algorithm to optimize the model hyperparameters, and uses the Nash coefficient NSE, Kling-Gupta efficiency coefficient KGE and root mean square error, RMSE as model evaluation indicators. When the prediction effect of the validation set is poor, different optimization algorithms (such as the sparrow algorithm, the whale algorithm, etc.) and objective functions (NSE, KGE, etc.) can be used to adjust the model.

[0059] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0060] 1. By introducing the pre-trained encoder CoST, the present invention eliminates the bottleneck that the traditional Transformer encoder can only start running after the encoder completes the training process, and can effectively extract the trend and seasonal representation of the runoff series;

[0061] 2. The present invention uses a data sliding extraction window to extract the length of The runoff sequence is compressed into , while sacrificing some input information, the complexity of the attention mechanism is reduced, so that the model can still have high prediction accuracy while reducing the calculation time;

[0062] 3. The present invention introduces a past attention decoder and uses the M0 method to preprocess the input data, thereby removing the long-term distribution trend of the input data, effectively improving the ability of the past attention mechanism to capture short-term and long-term dependency patterns, and enhancing the model's runoff prediction ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0064] Figure 1 It is a flow chart of the runoff prediction method based on the improved Transformer model of the present invention;

[0065] Figure 2 It is a comparison chart of runoff prediction between the improved Transformer model and the Transformer model in Example 2. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Example 1: In the Ganjiang River Basin, historical hydrological data in the basin are collected and studied, and a runoff prediction method based on an improved Transformer model of the present invention is used to perform runoff prediction, and the specific steps include:

[0068] S1. Select and collect hydrological data such as precipitation, evapotranspiration and runoff of the target basin, and preprocess the hydrological data to obtain training set, validation set and test set. The specific steps of hydrological data preprocessing include:

[0069] S101. Correct and interpolate missing and abnormal hydrological data;

[0070] S102, sorting the corrected hydrological data according to the time scale to obtain a new data sequence, and performing standardization processing on the new data sequence;

[0071] S103, dividing the hydrological data obtained by the standardized processing into a training set, a validation set and a test set in a ratio of 8:1:1;

[0072] Wherein, the standardization formula is:

[0073] ;

[0074] In the formula, The data are obtained after standardization; is the new data sequence; and are the mean and standard deviation of the new data series respectively.

[0075] S2. Build an improved Transformer model. The specific steps include:

[0076] S201, divide an input of the traditional Transformer model into long and short sequence inputs, specifically including: determining the time step M for predicting runoff, setting the length of the long sequence input according to the time period type and step length of the predicted runoff and short sequence input length The value range of long sequence input length The value range is , short sequence input length The value range is ,in The lower limit is greater than The upper limit of

[0077] S202, introduce the pre-trained encoder CoST, set seven categories of timestamp information (minute, hour, day of the week, day of the month, day of the year, month, week of the year) as the time representation of the runoff sequence, and use the pre-trained encoder CoST to pre-train the runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input to obtain the long sequence error-free runoff and short sequence error-free runoff The specific execution steps include:

[0078] S202.1. For each time step, assume that the runoff data consists of three components: trend, seasonality, and error, namely ,in represents the trend component of runoff data, represents the seasonal component of runoff data, Indicates the error, is the corresponding step length of the given timestamp information, represents the sum of component dimensions, represents the latent space;

[0079] S202.2. The runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input are processed by has the sum of component dimensions The sum of input channels and trend component dimensions The output channel uses a hybrid autoregressive expert with one-dimensional causal convolution to extract trend representations. After the extracted trend representations pass through the average pooling layer, the time domain contrast loss function is used. Learning, obtaining the final trend representation ;

[0080] Temporal Contrast Loss Function :

[0081] ;

[0082] in, is the temperature parameter, The momentum encoder is used to obtain the encoding representation of the trend representation, Indicates a positive trend. Indicates a negative trend. is the total number of negative trend samples; Indicates the number of samples in a batch.

[0083] S202.3. Use discrete Fourier transform to map the time domain representation of the runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input to the frequency domain, then use a learnable Fourier layer to obtain the frequency domain seasonal trend representation, use the frequency domain contrast loss function to learn to obtain the frequency domain seasonal trend representation, and use the inverse operation to convert the frequency domain seasonal trend representation back to the time domain to obtain the seasonal trend representation ;

[0084] The frequency domain contrast loss function is derived from the frequency domain amplitude contrast loss function And the frequency domain phase contrast loss function composition;

[0085] Frequency domain amplitude contrast loss function :

[0086] ;

[0087] Frequency domain phase contrast loss function :

[0088] ;

[0089] The inverse operation:

[0090] ;

[0091] in, represents the jth sample in the batch of samples, express The corresponding extended sample, represents the amplitude of the frequency corresponding to the jth sample, represents the phase of the frequency corresponding to the jth sample, express The phase of the frequency corresponding to the jth sample in the corresponding extended sample, express The amplitude of the corresponding extended sample frequency, represents the sum of component dimensions, represents the frequency domain, and represents the linear layer parameters, represents the kth sample in the batch of samples, Indicates the phase of the frequency corresponding to the kth sample in the batch of samples, represents the operation function that maps the frequency domain representation to the time domain, N represents the number of batch samples, represents the batch number, PN represents the total number of samples, represents the time domain output matrix, i.e., seasonal representation;

[0092] S202.4. Use the total loss function formula to connect the trend representation of the sequence and the seasonal trend representation extraction results to obtain the final output representation; the total loss function formula is as follows:

[0093] ;

[0094] in, represents the time domain contrast loss function, represents the frequency domain amplitude contrast loss function, represents the frequency domain phase contrast loss function, Expressed as a parameter that balances the trade-off between trend and seasonal factors.

[0095] S203: Introduce a data sliding extraction window, wherein the data sliding extraction window is used to set the length of the long sequence extraction window. and short sequence window length , using the data sliding extraction window to extract the long sequence error-free runoff Step length Shortened to , the short sequence error-free runoff Step length Shortened to ;

[0096] S204, introducing a past attention decoder, wherein the past attention decoder is used to correct the error-free runoff of the long sequence through the M0 method. After preprocessing, the input without long-term distribution trend representation is obtained , using the past long-term multi-head attention mechanism and the past long-term and short-term cross multi-head attention mechanism , , The corresponding long sequence input and The corresponding short sequence input is processed to capture the long-term dependency pattern and short-term dependency pattern of the data, and then the captured long-term dependency pattern features and short-term dependency pattern features are output through the normalization layer, convolutional feedforward layer group, flattening layer and linear layer to output the final runoff prediction result.

[0097] The past attention decoder consists of a past long-term multi-head attention mechanism, a normalization layer, and a convolutional feed-forward layer;

[0098] Past long-term multi-head attention mechanism:

[0099] ;

[0100] Normalization layer:

[0101] ;

[0102] Convolutional feed-forward layer:

[0103] ;

[0104] in, Indicates long-term long-term attention, represents the feature fusion method, represents the attention head, represents the parameter matrix, represents batch normalization, represents the standardization method, represents the residual connection layer, represents the convolutional feed-forward layer, represents the parameters of the feed-forward sublayer, Indicates input with no long-term distribution trend , short sequence error-free runoff , The corresponding long sequence input and The corresponding short sequence input, Represents the sum of component dimensions, and the activation function is GELU. Indicates transpose.

[0105] S3. Use the training set data and optimization algorithm to train and optimize the improved Transformer model, use the verification set data to verify the trained improved Transformer model, and optimize and adjust the improved Transformer model according to the verification results;

[0106] Specifically, the improved Transformer model optimization adjustment takes the mean absolute error MAE as the objective function, uses the Bayesian algorithm to optimize the model hyperparameters, and uses the Nash coefficient NSE, Kling-Gupta efficiency coefficient KGE and root mean square error, RMSE as model evaluation indicators. When the prediction effect of the validation set is poor, different optimization algorithms (such as sparrow algorithm, whale algorithm, etc.) and objective functions (NSE, KGE, etc.) can be used to adjust the model.

[0107] S4. Based on the test set, the optimized improved Transformer model is used to predict runoff, and the prediction results are denormalized to obtain the final runoff prediction results.

[0108] Example 2: Comparison of daily runoff prediction using the improved Transformer model and the Transformer model based on the same training set, validation set, and test set. The results are as follows: Figure 2 As shown, through calculation, it can be seen that the MAE, RMSE and NSE of the Transformer model are 907m 3 / s、1380m 3 / s and 0.78; the MAE, RMSE and NSE of the improved Transformer model are 572m 3 / s、933m 3 / s and 0.9.

[0109] It can be seen that the prediction effect of the improved Transformer model is better than that of the Transformer model, especially the runoff prediction ability of the Transformer model is significantly improved.

[0110] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0111] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A runoff prediction method based on an improved Transformer model, characterized in that: The runoff prediction method includes the following steps: Select and collect hydrological data of the target basin, and preprocess the hydrological data to obtain training sets, validation sets, and test sets; Constructing an improved Transformer model; the steps of constructing the improved Transformer model include: An input of the traditional Transformer model is divided into long sequence input and short sequence input, including: determining the time step M of the predicted runoff, setting the length of the long sequence input according to the time period type and step length of the predicted runoff and short sequence input length The value range of The pre-trained encoder CoST is introduced to set the timestamp information of seven categories as the time representation of the runoff sequence. The pre-trained encoder CoST is used to pre-train the runoff sequence with time representation corresponding to the long sequence input to obtain the long sequence error-free runoff , the pre-trained encoder CoST is used to pre-train the runoff sequence with time representation corresponding to the short sequence input to obtain the short sequence error-free runoff ; Introduce a data sliding extraction window, which is used to set the length of the long sequence extraction window and short sequence window length , using the data sliding extraction window to extract the long sequence error-free runoff Step length shorten , the short sequence error-free runoff Step length Shortened to ; Introducing a past attention decoder that uses the M0 method to perform error-free runoff on long sequences After preprocessing, the input without long-term distribution trend representation is obtained , using the past long-term multi-head attention mechanism and the past long-term and short-term cross multi-head attention mechanism and its corresponding long sequence input and and its corresponding short sequence input to capture the long-term dependency pattern and short-term dependency pattern of the data, and then pass the captured long-term dependency pattern features and short-term dependency pattern features through the normalization layer, convolutional feedforward layer group, flattening layer and linear layer to output the final runoff prediction result; Use the training set data to train and optimize the improved Transformer model, use the validation set data to verify the trained improved Transformer model, and optimize and adjust the improved Transformer model based on the validation results; Based on the test set, the optimized improved Transformer model is used to predict runoff, and the prediction results are denormalized to obtain the final runoff prediction results.

2. A runoff prediction method based on an improved Transformer model according to claim 1, characterized in that: The hydrological data preprocessing step comprises: Correct and interpolate missing and abnormal hydrological data; the hydrological data include but are not limited to precipitation data, evapotranspiration data, and runoff data; The corrected hydrological data are sorted according to the time scale to obtain a new data series, and then the new data series are standardized; The hydrological data obtained by standardization processing were divided into training set, validation set and test set in a ratio of 8:1:1; Wherein, the standardization formula is: ； in, The data are obtained after standardization; is the new data sequence; and are the mean and standard deviation of the new data series respectively.

3. The runoff prediction method based on the improved Transformer model according to claim 1 is characterized in that: The specific process of the pre-trained encoder CoST is as follows: For each time step, it is assumed that the runoff data consists of three components: trend, seasonality, and error, namely ,in represents the trend component of runoff data, represents the seasonal component of runoff data, Indicates the error, is the corresponding step length of the given timestamp information, represents the sum of component dimensions, represents the latent space; The runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input are transformed into has the sum of component dimensions The sum of input channels and trend component dimensions The output channel uses a hybrid autoregressive expert with one-dimensional causal convolution to extract trend representations. After the extracted trend representations pass through the average pooling layer, the time domain contrast loss function is used. Learning, obtaining the final trend representation ; The runoff sequence with time representation corresponding to the long sequence input and the runoff sequence with time representation corresponding to the short sequence input are mapped to the frequency domain using discrete Fourier transform, and then the frequency domain seasonal trend representation is obtained through a learnable Fourier layer. The frequency domain seasonal trend representation is learned using the frequency domain contrast loss function, and the frequency domain seasonal trend representation is converted back to the time domain using the inverse operation. ; The total loss function formula is used to connect the trend representation and the seasonal trend representation to obtain error-free runoff. ,in ; Among them, the sum of the component dimensions represents the sum of the number of latent space dimensions of the trend component and the number of latent space dimensions of the seasonal component; the sum of the trend component dimensions Represents the total number of dimensions of the latent space of the trend component.

4. A runoff prediction method based on an improved Transformer model according to claim 3, characterized in that: The time domain contrast loss function : ； in, is the temperature parameter, The momentum encoder is used to obtain the encoding representation of the trend representation, Indicates a positive trend. Indicates a negative trend. is the total number of negative trend samples; Indicates the number of samples in a batch.

5. The runoff prediction method based on the improved Transformer model according to claim 3 is characterized in that: The frequency domain contrast loss function is derived from the frequency domain amplitude contrast loss function And the frequency domain phase contrast loss function composition; The frequency domain amplitude contrast loss function : ； The frequency domain phase contrast loss function : ； The inverse operation: ； in, represents the jth sample in the batch of samples, express The corresponding extended sample, represents the amplitude of the frequency corresponding to the jth sample, represents the phase of the frequency corresponding to the jth sample, express The phase of the frequency corresponding to the jth sample in the corresponding extended sample, express The amplitude of the corresponding extended sample frequency, represents the sum of component dimensions, represents the frequency domain, and represents the linear layer parameters, represents the kth sample in the batch of samples, Indicates the phase of the frequency corresponding to the kth sample in the batch of samples, represents the operation function that maps the frequency domain representation to the time domain, N represents the number of batch samples, represents the batch number, PN represents the total number of samples, represents the time domain output matrix, i.e., seasonal representation.

6. The runoff prediction method based on the improved Transformer model according to claim 3 is characterized in that: The total loss function formula is as follows: ； in, represents the time domain contrast loss function, represents the frequency domain amplitude contrast loss function, represents the frequency domain phase contrast loss function, A parameter that balances the trade-off between trend and seasonal factors.

7. The runoff prediction method based on the improved Transformer model according to claim 1 is characterized in that: The specific execution steps of the M0 method include: The long sequence of error-free runoff Subtract the corresponding mean To input without long-term distribution trend ; Utilize the past long-term multi-head attention mechanism and the past long-term and short-term cross multi-head attention mechanism , The corresponding long sequence input and The corresponding short sequence input is processed to obtain the preliminary prediction value; Characterize the trend of the initial forecast value and the long series of error-free runoff The corresponding mean The final runoff prediction value is obtained by adding them together.

8. The runoff prediction method based on the improved Transformer model according to claim 1 is characterized in that: The past attention decoder consists of a past long-term multi-head attention mechanism, a normalization layer, and a convolutional feed-forward layer; The past long-term multi-head attention mechanism: ； The normalization layer: ； The convolutional feed-forward layer: ； in, Indicates long-term long-term attention, represents the feature fusion method, represents the first attention head, represents the hth attention head, represents the parameter matrix, represents batch normalization, represents the standardization method, represents the residual connection layer, represents the convolutional feed-forward layer, , , and are the parameters of the feedforward sublayer, Indicates input with no long-term distribution trend , short sequence error-free runoff , The corresponding long sequence input and The corresponding short sequence input, Represents the sum of component dimensions, and the activation function is GELU. Indicates transpose.

9. The runoff prediction method based on the improved Transformer model according to claim 1, characterized in that: The optimization adjustment of the improved Transformer model takes the mean absolute error MAE as the objective function, uses the Bayesian algorithm to optimize the model hyperparameters, and uses the Nash coefficient NSE, Kling-Gupta efficiency coefficient KGE and root mean square error, RMSE as model evaluation indicators. When the prediction effect of the validation set is poor, different optimization algorithms and objective functions are used to adjust the model.

Images