Hydropower station reservoir runoff intelligent forecasting method based on large language model

Abstract

The invention discloses a hydropower station reservoir runoff intelligent forecasting method based on a large language model, and belongs to the technical field of hydrological forecasting, and the method comprises the following steps: S1, data collection and preprocessing; s2, constructing a BERT-Hydrology deep learning model for the runoff forecast of the reservoir of the hydropower station; s3, designing a weighted mixed loss function, and introducing an optimizer, learning rate scheduling and an early stop mechanism to improve the convergence speed and numerical stability of model training; and S4, establishing an evaluation system, performing rolling forecasting, and outputting an evaluation index and a significance test result. According to the intelligent hydropower station reservoir runoff forecasting method based on the large language model, the accuracy and the stability of runoff forecasting are remarkably improved by constructing the runoff forecasting model, excavating the hidden rule in the hydrological sequence and establishing the multi-step forecasting mapping, and the forecasting accuracy and the forecasting stability are improved. And technical support is provided for intelligent scheduling of a hydropower station reservoir and efficient management of water resources.

Description

Technical Field

[0001] This invention belongs to the field of hydrological forecasting technology, specifically relating to an intelligent forecasting method for hydropower station reservoir runoff based on a large language model. Background Technology

[0002] Hydropower station reservoir runoff forecasting is a core component of water conservancy project operation and water resource management, playing a vital supporting role in flood control safety, power generation scheduling, water supply security, and ecological regulation. Driven by a combination of factors including meteorological conditions, underlying surface characteristics, historical runoff, and human intervention, hydrological processes exhibit significant nonlinearity, non-stationarity, and spatiotemporal variability, increasing the difficulty of forecasting.

[0003] However, existing technologies have the following shortcomings: Represented by the Xin'anjiang model, they rely on assumptions such as time invariance and linear response, making it difficult to capture the dynamic time-varying mechanism and long-range dependence of runoff under climate change, and the forecast accuracy drops significantly under extreme hydrological scenarios; Recurrent neural networks such as LSTM and GRU suffer from the gradient vanishing / exploding problem, making it difficult to model long-distance dependence of ultra-long sequences; CNNs have limited receptive fields and are insufficient in capturing global time patterns and cross-variable interactions; Hydrological data have limited sample size, high noise, and many missing values, which easily leads to overfitting of complex deep learning models, resulting in weak generalization performance and difficulty in adapting to actual business scenarios with a wide range and multiple forecast periods.

[0004] Therefore, a new method is urgently needed. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent forecasting method for hydropower station reservoir runoff based on a large language model. This method processes data in parallel to capture long-term dependencies and nonlinear interactions in hydrological sequences, breaking through the bottleneck of long-distance dependencies in traditional models; it takes into account both the peak capture of extreme hydrological events and the consistency of overall distribution, making up for the inadequacy of sample imbalance handling; it improves training stability, adapts to the characteristics of hydrological data, and alleviates the performance degradation of traditional models over multiple forecast periods and the problem of poor multivariate adaptation.

[0006] To achieve the above objectives, this invention provides an intelligent forecasting method for hydropower station reservoir runoff based on a large language model, comprising the following steps: S1. Acquire and examine hydrological data, process missing factor data sequences, and then analyze the correlation between external factors and runoff sequences to form different factor sets. Further transform the runoff data, perform time series division, sliding window reconstruction, and standardization on different factor set data to obtain standardized sample data. The standardized sample data includes training set, validation set, and test set. S2. Based on the standardized sample data of S1, the model structure is obtained by inputting the projection layer, Transformer encoder, attention pooling layer and regression output layer; S3. Based on the model structure of S2, the model structure is trained using the training set of S1. The trained model is obtained through the designed weighted mixed loss function, AdamW optimizer, one-cycle learning rate scheduler and early stopping mechanism. S4. Using the training model from S3 and the test set data from S1, output multi-foreseeability runoff forecast results through a rolling forecast strategy, and calculate RMSE, MAE, NSE and R indexes to complete the accuracy evaluation.

[0007] Preferably, S1 includes the following steps: S101. For the obtained historical hydrological data such as runoff sequences and external factors, first check the completeness of the data. If there are missing values ​​in the runoff, fill them with the average value of the runoff in the same period of history. If there are missing values ​​in the external factors and the number of missing values ​​is no more than 10% of the sequence length, fill them with the average value of the series in the same period of history. If the missing value is greater than 10% of the sequence length, remove the data of that factor. S102. The correlation between external factors and runoff sequences is analyzed using the maximum mutual information coefficient and Pearson correlation coefficient, forming different factor sets; S103. Perform a logarithmic transformation on the runoff values ​​in the factor set to alleviate their skewed distribution characteristics; S104. Divide the factor set data into training set, validation set and test set according to time sequence; S105. Reconstruct continuous time series into structured samples using a sliding window. Each sample consists of a fixed-length historical observation window and a future runoff sequence for a specified forecast period. S106. Perform Z-score standardization on the data based on the mean and standard deviation calculated from the training set to eliminate dimensional differences and improve training stability.

[0008] Preferably, S2 includes the following steps: S201. Through learnable linear transformation, multivariate hydrological data is mapped to the hidden space of the BERT model, transforming it into a high-dimensional feature representation, as follows: ; in, The original input matrix; This is the projection weight matrix; For activation functions; It is the bias vector; S202, a Transformer encoder block composed of a multi-head self-attention mechanism and a feedforward neural network, with each sub-layer equipped with residual connections and layer normalization operations, in order to capture the long-term dependencies and nonlinear interactions of hydrological sequences; The self-attention mechanism dynamically calculates the global dependency weight between any two elements in the sequence. The calculation process is as follows: ; ; ; ; in, These represent the query, key, and value matrices, respectively. , , These represent different weight matrices; To calculate the dot product similarity between the query and the key; This is the scaling factor; The function normalizes the similarity. Multi-head attention mechanisms perform computation by executing multiple self-attention mechanisms in parallel. The computation process is as follows: ; ; in, The concatenation of the outputs of all attention heads along the feature dimension; For the first i The result vector of each attention head; The output projection matrix of the linear layer; ; ; in, For position In the dimensional The encoded value; This refers to the time step position; For dimension indexing; For the hidden layer dimension; Residual connections and layer normalization are key mechanisms in the Transformer encoder block that ensure training stability and effective information transfer. Their calculation process is as follows: ; ; in, and These are the mean and standard deviation calculated along the feature dimension, respectively. For learnable scaling parameters; These are learnable offset parameters; It is a very small constant; The feedforward neural network is an important component in the Transformer encoder block that provides nonlinear transformation capabilities. It consists of two linear transformation layers and the GELU nonlinear activation function between them, and is expressed as: ; ; in, The input vector; , These are the weight matrices for the two linear layers; , These are the corresponding bias vectors; The cumulative distribution function of the standard normal distribution; The Gaussian error function; S203 replaces the traditional CLS label, dynamically aggregates hydrological time-series information, and generates a vector that reflects the global context. The specific calculation process is as follows: ; ; ; in, This is the context parameter vector in the attention mechanism; and For the learnable weight matrix and bias vector of the linear transformation; For the first t The hidden state at each time step; For the first t Attention weights for each time step; This is the final generated context vector; A two-layer feedforward network, incorporating the GELU activation function and Dropout, is used to map the context vector to the final runoff forecast value. The calculation process can be represented as follows: ; in, This is the forecast value; , , For different weight matrices; It is the bias vector; This is a regularization method.

[0009] Preferably, S3 includes the following steps: S301. Design a joint loss function that combines the weighted mean square error and the Nash efficiency coefficient; the weighted mean square error loss is expressed as: ; ; ; in, Batch size; For the first i Individual sample weights; This represents the actual runoff value; Forecast flow rate; and These are the mean and standard deviation of the true values ​​in the training set, respectively. Hyperparameters for controlling weight strength; It is the sigmoid function, which maps input values ​​to (0,1); The function limits the weights to a range; The Nash efficiency coefficient loss is expressed as: ; in, The total number of samples; The final loss function is the weighted sum of the above two terms, expressed as: ; in, For hyperparameters; S302. The AdamW optimizer is selected, and a decoupled weight decay mechanism is introduced to suppress model overfitting, expressed as: ; ; ; ; ; in, , These are first-order and second-order momentum estimates, respectively; , The momentum decay coefficient; The current gradient; , These are the first-order and second-order momentum estimates after bias correction, respectively; In time step t Updated parameters; The learning rate; This is the weight decay coefficient; It is the numerical stability constant; Simultaneously, a cosine annealing one-cycle learning rate scheduling strategy with linear warm-up is adopted to balance the exploratory ability of model parameters with convergence stability, expressed as: ; in, This refers to the number of steps in the upward phase. The maximum learning rate; Minimum learning rate; Enable the early stopping mechanism; if the performance does not improve within a set number of iterations, stop training early.

[0010] Preferably, S4 includes the following steps: S401. For the forecast period 1, the forecast value at time T+1 is output using the full observation data window; for the period T+2 and later, the forecast value at the previous time is introduced to complete the data window, and the data window is updated by sliding as the forecast progresses. S402. Calculate the root mean square error, mean absolute error, Nash coefficient, and Pearson correlation coefficient to verify the forecast accuracy and output the evaluation indicators and significance test results.

[0011] This invention also provides an intelligent forecasting system for hydropower station reservoir runoff based on a large language model, comprising: The data processing module is used to acquire and inspect hydrological data, process missing factor data sequences, and then analyze the correlation between external factors and runoff sequences to form different factor sets. It further transforms the runoff data, performs time series partitioning, sliding window reconstruction, and standardization on different factor set data to obtain standardized sample data. The standardized sample data includes training set, validation set, and test set. The model building module, connected to the data processing module, obtains the model structure based on standardized sample data through an input projection layer, a Transformer encoder, an attention pooling layer, and a regression output layer. The model training module is connected to the model building module. Based on the model structure, it trains the model structure using the training set. Through the designed weighted mixed loss function, AdamW optimizer, One-Cycle learning rate scheduler and early stopping mechanism, the trained model is obtained. The forecast evaluation module is connected to the model training module. It uses the trained model and test set data to output multi-foreseeability runoff forecast results through a rolling forecast strategy, and calculates RMSE, MAE, NSE and R index to complete the accuracy evaluation.

[0012] Therefore, the present invention employs the above-mentioned intelligent forecasting method for hydropower station reservoir runoff based on a large language model. Compared with the prior art, the technical solution of the present invention has the following beneficial effects: (1) This invention uses the Transformer encoder and self-attention mechanism of BERT architecture to process time window data in parallel, effectively capturing the long-term dependency relationship and nonlinear interaction of hydrological sequences, and solving the bottleneck of long-distance dependency modeling in traditional models. (2) The present invention designs a weighted MSE+NSE hybrid loss function, which gives greater weight to high flow samples, takes into account both the peak capture of extreme hydrological events and the overall distribution consistency, and makes up for the shortcomings of traditional model sample imbalance processing. (3) This invention uses AdamW optimizer, One-Cycle learning rate scheduling, early stopping mechanism and Dropout regularization to suppress overfitting, improve training convergence speed and numerical stability, and adapt to the noise and missing characteristics of hydrological data. (4) This invention supports multi-factor input combination and achieves continuous rolling forecasting for multiple time periods through rolling forecasting strategy. It has a small error accumulation effect and solves the problems of fast performance decay and poor multivariate adaptability of traditional models with multiple forecast periods.

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0014] Figure 1 This is a flowchart of an embodiment of the intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to the present invention; Figure 2 This is a factor correlation heatmap of an embodiment of the intelligent forecasting method for hydropower station reservoir runoff based on a large language model of the present invention. Figure 2 In the diagram, (a) represents the heatmap of the maximum mutual information coefficient (MIC); Figure 2 (b) in the figure represents the Pearson correlation coefficient (R) heatmap; Figure 3 This is a data construction diagram of the rolling forecast strategy in an embodiment of the intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to the present invention. Figure 4 The above are a forecast flow process diagram and scatter plot of an embodiment of the intelligent forecasting method for hydropower station reservoir runoff based on a large language model of the present invention; Figure 4 (a), (c), (e), (g), and (i) in the diagram represent a comparison of runoff processes; Figure 4 In the diagram, (b), (d), (f), (h), and (j) represent scatter plots. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used in the present invention should have the ordinary meaning understood by those skilled in the art.

[0016] Example 1 like Figure 1 As shown, the intelligent forecasting method for hydropower station reservoir runoff based on a large language model of the present invention includes the following steps: S1. Acquire and examine hydrological data, process missing factor data sequences, and then analyze the correlation between external factors and runoff sequences to form different factor sets. Further transform the runoff data, perform time-series partitioning, sliding window reconstruction, and standardization on the different factor set data to obtain standardized sample data. The standardized sample data includes training set, validation set, and test set. This includes the following steps: S101. For the daily runoff observation data of the Chitan Hydropower Station from January 1, 2004 to December 31, 2023 (20 years), and the daily rainfall observation data of 20 rain gauge stations including Taining and Jianning, check the completeness of the data. If there are missing values ​​in the runoff, use the average runoff of the same historical period to fill them in. If there are missing values ​​in the external factors, and the number of missing values ​​is less than or equal to 10% of the series length, use the average of the same historical period to fill them in. If the number of missing values ​​is greater than 10% of the series length, remove the data for that factor. After checking, the data is complete, and there is no need to fill in missing data or remove factor data. S102. The correlation between external factors and runoff sequences was analyzed using the maximum mutual information coefficient (MIC) and Pearson correlation coefficient (R) to form different factor sets. The specific calculation results of MIC and R are shown in Tables 1, 2, 3, and 4, and the correlation heatmap is shown below. Figure 2 As shown; Based on the combined performance of the two indicators across different leading time periods, the rainfall at the Chitan surface, the rainfall at the Chitan dam, the rainfall in Taining, and the rainfall in Dabu were ultimately selected as the rainfall factors for the model input. The selection criteria are as follows: (1) Early correlation: In the preceding 1-3 periods, the MIC and R of rainfall at Chitan surface, Chitan dam, Taining, Jianning, Junkou, Meikou and Dabu were all at a high level. (2) Stability of Change: As the preceding time period increases, the MIC of rainfall at Junkou and Meikou decreases significantly (down to a minimum of 0.111), while the other factors remain relatively stable. The R of rainfall at Jianning is higher than that at Taining at time period 1, but they tend to be consistent by time period 2, after which Taining rainfall maintains a higher correlation. The R trends of rainfall at Chitan Dam and Dabu are basically consistent in the preceding time periods 3-12. Rainfall at Chitan surface generally shows the highest R in all time periods.

[0017] Based on the above analysis, the inflow rate into the ponds and reservoirs, combined with the four rainfall factors mentioned above, forms the following four factor sets, which are used to evaluate the model forecasting performance under different input configurations: Factor set 1: Inflow into the ponds and reservoirs, and surface rainfall in the ponds and reservoirs (2 factors); Factor set 2: Inflow into the pond, rainfall over the pond surface, and rainfall over the pond dam (3 factors); Factor set 3: Inflow into Chitan Reservoir, surface rainfall in Chitan, rainfall above Chitan Dam, rainfall in Taining (4 factors). Factor set 4: Inflow into Chitan Reservoir, surface rainfall in Chitan, rainfall above Chitan Dam, rainfall in Taining, and rainfall in Dabu (5 factors).

[0018] Table 1. Calculation Results of Maximum Mutual Information Coefficient

[0019] Table 2 Calculation Results of Maximum Mutual Information Coefficient

[0020] Table 3. Results of Pearson Correlation Coefficient Calculation

[0021] Table 4. Results of Pearson Correlation Coefficient Calculation

[0022] S103. Perform a logarithmic transformation on the inflow values ​​of the pools and ponds in the factor set to alleviate their skewed distribution characteristics. S104. Strictly divide the factor set data into training set, validation set and test set according to time sequence, with a ratio of 6:2:2, to avoid future information leakage; S105. Reconstruct continuous time series into structured samples using a sliding window. Each sample consists of a fixed-length historical observation window and a future runoff sequence for a specified forecast period. S106. Perform Z-score standardization on the data based on the mean and standard deviation calculated from the training set to eliminate dimensional differences and improve training stability. S2. Construct a runoff forecasting model based on a large language model. The model adopts the BERT architecture, including an input projection layer, a Transformer encoder, an attention pooling layer, and a regression output layer, to learn the long-term dependencies of runoff sequences and output runoff forecast values ​​for different lead times; including the following steps: The main function of the input projection layer (S201) is to map multivariate hydrological data such as flow and rainfall within a historical time window to the hidden space of the BERT model. This addresses the mismatch between the original feature dimensions and the model's hidden dimensions, and provides preliminary nonlinear abstraction capabilities for subsequent feature extraction. This layer maps multivariate hydrological data to the BERT hidden space, transforming it into a high-dimensional representation through a learnable linear transformation, thereby more effectively capturing the complex nonlinear relationships in hydrological variables. Its calculation process can be represented as follows: ; in, The original input matrix has dimensions of batch size × time window length × input feature dimension; This is the projection weight matrix, used to map features from the input dimension to the hidden dimension; As the activation function, nonlinear transformations are introduced to enhance the model's representational power; It is the bias vector; S202. Each Transformer encoder block contains two core sub-layers: a multi-head self-attention mechanism and a feedforward neural network. Each sub-layer is supplemented with residual connections and layer normalization, thereby enhancing training stability, mitigating the gradient vanishing problem, and improving the model's expressive power. This characteristic makes it suitable for hydrological time series forecasting tasks. The model can process data in parallel throughout the entire time window, effectively capturing long-term dependencies and understanding complex nonlinear interactions between different time steps.

[0023] As a core component of the Transformer architecture, the self-attention mechanism achieves efficient modeling of long-term temporal relationships by dynamically calculating the global dependency weights between any two elements in a sequence. This effectively overcomes the bottleneck problem of traditional temporal models in capturing long-term dependencies. Specifically, this mechanism first processes the input matrix... The values ​​are mapped to three different representation spaces through linear transformations: query space, key space, and value space, where each mapping is implemented using a trainable parameter matrix. The calculation process is as follows: ; ; ; ; in, These represent the query, key, and value matrices, respectively, derived from the input matrix. Obtained through linear mapping; , , These represent different weight matrices; To calculate the dot product similarity between the query and the key, which is used to characterize the association strength between different time steps; This is a scaling factor used to control the order of magnitude of the dot product result, preventing gradient vanishing or instability when the dimension is high; The function normalizes the similarity and transforms it into attention weights in the form of a probability distribution; Multi-head attention significantly enhances the model's ability to model diverse dependencies by executing multiple self-attention computations in parallel. Specifically, multi-head attention replicates the original self-attention computation process ℎ times, each time using a different projection matrix to map the input to the query, key, and value spaces, allowing each attention head to learn a different representation pattern. Then, the outputs of all attention heads are concatenated along the feature dimension and fused using a shared projection matrix to form the final context-aware representation. The computation process is as follows: ; ; in, The concatenation of the outputs of all attention heads along the feature dimension; This is the vector of calculation results for the i-th attention head; The output projection matrix of the linear layer; Since self-attention mechanisms themselves lack the ability to perceive sequence order, positional encoding is needed to provide the model with temporal sequence information, thereby ensuring its correct understanding of causal and temporal constraints in hydrological events (e.g., rainfall events occur before runoff responses). This mechanism, by explicitly encoding positional information, ensures the causality of the time series, meaning that past states influence the future, while future states do not influence the past. , The position encoding matrix is ​​represented as follows: ; ; in, For position In the dimensional The encoded value; This refers to the time step position; For dimension indexing; For the hidden layer dimension; Residual connections and layer normalization are key mechanisms in the Transformer encoder block that ensure training stability and efficient information transfer. Residual connections, through cross-layer identity mappings, ensure that gradients can propagate directly backward, effectively alleviating the gradient vanishing and representation bottleneck problems in deep networks, thus supporting the construction of deeper network structures. Normalization, by standardizing the output of each layer, further improves training stability and convergence speed. The calculation process is as follows: ; ; in, and These are the mean and standard deviation calculated along the feature dimension, respectively. For learnable scaling parameters; These are learnable offset parameters used to enhance the model's activity; It is a very small constant to prevent the denominator from being zero; Feedforward neural networks (FNNs) are a crucial component in the Transformer model, providing nonlinear transformation capabilities. They consist of two linear transformation layers and a nonlinear activation function between them, used to learn complex interactions between features and enhance the model's representational power. This paper employs GELU as the activation function, which, compared to ReLU which allows some negative information to pass through, often preserves more distribution information in practice. Its mathematical expression can be approximated as: ; ; in, The input vector; , These are the weight matrices for the two linear layers; , These are the corresponding bias vectors; The cumulative distribution function of the standard normal distribution; The Gaussian error function is used to calculate the correlation between the input and the cumulative probability density of the normal distribution. S203, Attention Pooling Layer: This layer replaces the [CLS] label in the traditional BERT model with an attention pooling layer to dynamically aggregate hydrological time series information. This layer adaptively focuses on important moments within historical time windows and generates vector representations containing global contextual information, thus more effectively capturing historical hydrological events that significantly impact current forecasts. The specific calculation process is as follows: ; ; ; in, This is the context parameter vector in the attention mechanism; and For the learnable weight matrix and bias vector of the linear transformation; For the first t The hidden state at each time step; For the first t The attention weights at each time step are obtained by normalizing using the softmax function; The final generated context vector is obtained by weighted summation, representing the aggregated information of the entire time window; The regression output layer maps the context vector generated by the attention pooling layer to the final runoff forecast value. This layer transforms the time-series patterns learned by the model into the actual runoff output through nonlinear transformation, supporting forecasting tasks for different lead times. Its structure consists of a two-layer feedforward network, using the GELU activation function to provide continuous and smooth nonlinear transformation capabilities, and introducing Dropout to suppress overfitting and improve the model's generalization ability. The computation process of this layer can be represented as follows: ; in, This is the forecast value; , , For different weight matrices; It is the bias vector; For regularization methods; S3. Train the model using the training set, and improve training performance by employing a weighted mixture loss function, AdamW optimizer, one-cycle learning rate scheduler, and early stopping mechanism; including the following steps: S301. To address the sample imbalance problem in hydrological forecasting and improve the accuracy of forecasting extreme hydrological events, a joint loss function combining weighted mean square error and Nash efficiency coefficient is designed. This function ensures accurate capture of peak flows by assigning greater weight to high-flow samples, while introducing evaluation indicators to optimize the overall distribution consistency of forecast results; the weighted mean square error loss is expressed as: ; ; ; in, Batch size; The weight of the i-th sample; This represents the actual runoff value; Forecast flow rate; and These are the mean and standard deviation of the true values ​​in the training set, respectively. Hyperparameters for controlling weight strength; It is the sigmoid function, which maps input values ​​to (0,1); The function limits the weights to a range to prevent extreme weights from dominating the training process; The Nash efficiency coefficient loss is expressed as: ; in, The total number of samples; The final loss function is a weighted sum of the above two terms, aiming to balance the ability to predict extreme events with the consistency of the overall distribution, and is expressed as: ; in, These are hyperparameters used to adjust the weighting ratio of the two losses. This design allows the model to focus on peak forecast accuracy while also considering the overall simulation capability of the hydrological process. S302. To improve the convergence speed and numerical stability of model training, the AdamW optimizer is adopted. This optimizer introduces decoupled weight decay on top of Adam, effectively solving the incompatibility problem between L2 regularization and gradient normalization in traditional Adam by separating weight decay from the adaptive learning rate mechanism. This improves generalization ability while better suppressing overfitting. The calculation process can be expressed as: ; ; ; ; ; in, , These are first-order and second-order momentum estimates, respectively; , The momentum decay coefficient; The current gradient; , These are the first-order and second-order momentum estimates after bias correction, respectively; In time step t Updated parameters; The learning rate; This is the weight decay coefficient; It is the numerical stability constant; Meanwhile, to better balance parameter space exploration and convergence stability, a cosine annealing learning rate scheduling strategy with linear warm-up is adopted. This strategy rapidly heats up in the initial stage to broadly explore the parameter space, and then slowly cools down to achieve fine-tuning, effectively avoiding getting trapped in local optima and accelerating convergence to the global optimum. The learning rate adjustment method is as follows: ; in, This represents the number of steps in the upward phase (accounting for 30% of the total steps). The maximum learning rate; Minimum learning rate; Early stopping is a technique to prevent overfitting. It determines whether to stop training early by monitoring the model's performance on the validation set. If the model's performance on the validation set does not improve significantly within a certain number of iterations, the model is considered to have converged, and training can be stopped early. S4. Use the trained model to perform multi-step rolling forecasts on the test set, and verify the forecast accuracy using RMSE, MAE, NSE, and R metrics. This includes the following steps: The core idea of ​​S401 rolling forecasts is to continuously predict future runoff events based on feedback updates between real-time observation data and forecast results. In this strategy, the model input is typically a fixed-length data window containing historical and recent meteorological and runoff data. When the lead time is one period, since the actual observation values ​​for all input data times have been obtained, the data window consists entirely of observation values, allowing direct output of the forecast value for time period T+1. From time period T+2 onwards, because observation data for some future times is not yet available, the data window gradually incorporates forecast results from the previous time period as input to replace missing observation values. After each forecast for the current time period is completed, the data window slides forward one time period, discarding data from the earliest time period and incorporating the latest observation or forecast data, thus forming the input sequence for a new round of forecasts. The data construction for the rolling forecast strategy is as follows: Figure 3 As shown; S402. Calculate the deterministic evaluation indicators, including root mean square error (RMSE), mean absolute error (MAE), Nash coefficient (NSE), and Pearson correlation coefficient (R). like Figure 4As shown, to verify the effectiveness of the model constructed in step S2, under the condition of consistent parameter settings, four different factor sets were used as inputs to compare and analyze the runoff forecasting model based on the large language model (BERT model) with four benchmark models: LSTM, ANN, GBR, and ELM. The performance of each model in runoff forecasting was comprehensively evaluated using four evaluation indicators: RMSE, MAE, NSE, and R. The parameter settings are shown in Table 5, and the results are shown in Table 6. The BERT model outperformed the comparison models in multiple indicators, demonstrating good accuracy and stability, as shown in Table 7.

[0024] Table 5 Parameter Setting Table

[0025] Table 6 Statistical Indicators of Forecast Results from Different Models

[0026] Table 7. Statistical table of peak forecast results for different models

[0027] The multi-step rolling forecasts mentioned in S4 were tested with lead times ranging from 1 to 5 years for different combinations of forecast factors. The LSTM model was used as a comparison model, and the forecast accuracy and stability were compared and analyzed. The results are shown in Table 8. Table 8. Rolling forecast results of the model under different factors

[0028] In summary, the BERT model exhibits the following characteristics in rolling runoff forecasts: extremely high accuracy and stability at a lead time of 1, making it directly applicable to real-time operational forecasting; a slight decrease in performance at lead times 2-3, but all indicators remain at excellent levels, meeting operational accuracy requirements; and while some error accumulation exists at lead times 4-5, the forecast results decay slowly and remain highly stable, retaining significant practical reference value. Clearly, the BERT model demonstrates robust and reliable forecasting capabilities across different lead times, validating its effectiveness and applicability in multi-step runoff forecasting within practical hydrological forecasting operations.

[0029] Therefore, this invention adopts the above-mentioned intelligent forecasting method for hydropower station reservoir runoff based on a large language model. This method processes data in parallel to capture the long-term dependence and nonlinear interaction of hydrological sequences, breaking through the bottleneck of long-distance dependence in traditional models; it takes into account both the peak capture of extreme hydrological events and the consistency of overall distribution, making up for the insufficient handling of sample imbalance; it improves training stability, adapts to the characteristics of hydrological data, and alleviates the problems of performance decay and poor multivariate adaptation of traditional models over multiple forecast periods.

[0030] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for intelligent forecasting of hydropower station reservoir runoff based on a large language model, characterized in that, Includes the following steps: S1. Acquire and examine hydrological data, process missing factor data sequences, and then analyze the correlation between external factors and runoff sequences to form different factor sets. Further transform the runoff data, perform time series division, sliding window reconstruction, and standardization on different factor set data to obtain standardized sample data. The standardized sample data includes training set, validation set, and test set. S2. Based on the standardized sample data of S1, the model structure is obtained by inputting the projection layer, Transformer encoder, attention pooling layer and regression output layer; S3. Based on the model structure of S2, the model structure is trained using the training set of S1. The trained model is obtained through the designed weighted mixed loss function, AdamW optimizer, one-cycle learning rate scheduler and early stopping mechanism. S4. Using the training model from S3 and the test set data from S1, output multi-foreseeability runoff forecast results through a rolling forecast strategy, and calculate RMSE, MAE, NSE and R indexes to complete the accuracy evaluation.

2. The intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to claim 1, characterized in that, S1 includes the following steps: S101. For the obtained historical hydrological data such as runoff sequences and external factors, first check the completeness of the data. If there are missing values ​​in the runoff, fill them with the average value of the runoff in the same period of history. If there are missing values ​​in the external factors and the number of missing values ​​is no more than 10% of the sequence length, fill them with the average value of the series in the same period of history. If the missing value is greater than 10% of the sequence length, remove the data of that factor. S102. The correlation between external factors and runoff sequences is analyzed using the maximum mutual information coefficient and Pearson correlation coefficient, forming different factor sets; S103. Perform a logarithmic transformation on the runoff values ​​in the factor set to alleviate their skewed distribution characteristics; S104. Divide the factor set data into training set, validation set and test set according to time sequence; S105. Reconstruct continuous time series into structured samples using a sliding window. Each sample consists of a fixed-length historical observation window and a future runoff sequence for a specified forecast period. S106. Perform Z-score standardization on the data based on the mean and standard deviation calculated from the training set to eliminate dimensional differences and improve training stability.

3. The intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to claim 1, characterized in that, S2 includes the following steps: S201. Through learnable linear transformation, multivariate hydrological data is mapped to the hidden space of the BERT model, transforming it into a high-dimensional feature representation, as follows: ； in, The original input matrix; This is the projection weight matrix; For activation functions; It is the bias vector; S202, a Transformer encoder block composed of a multi-head self-attention mechanism and a feedforward neural network, with each sub-layer equipped with residual connections and layer normalization operations, in order to capture the long-term dependencies and nonlinear interactions of hydrological sequences; The self-attention mechanism dynamically calculates the global dependency weight between any two elements in the sequence. The calculation process is as follows: ； ； ； ； in, These represent the query, key, and value matrices, respectively. , , These represent different weight matrices; To calculate the dot product similarity between the query and the key; This is the scaling factor; The function normalizes the similarity. Multi-head attention mechanisms perform computation by executing multiple self-attention mechanisms in parallel. The computation process is as follows: ； ； in, The concatenation of the outputs of all attention heads along the feature dimension; For the first i The result vector of each attention head; The output projection matrix of the linear layer; ； ； in, For position In the dimensional The encoded value; This refers to the time step position; For dimension indexing; For the hidden layer dimension; Residual connections and layer normalization are key mechanisms in the Transformer encoder block that ensure training stability and effective information transfer. Their calculation process is as follows: ； ； in, and These are the mean and standard deviation calculated along the feature dimension, respectively. For learnable scaling parameters; These are learnable offset parameters; It is a very small constant; The feedforward neural network is an important component in the Transformer encoder block that provides nonlinear transformation capabilities. It consists of two linear transformation layers and the GELU nonlinear activation function between them, and is expressed as: ； ； in, The input vector; , These are the weight matrices for the two linear layers; , These are the corresponding bias vectors; The cumulative distribution function of the standard normal distribution; The Gaussian error function; S203 replaces the traditional CLS label, dynamically aggregates hydrological time-series information, and generates a vector that reflects the global context. The specific calculation process is as follows: ； ； ； in, This is the context parameter vector in the attention mechanism; and For the learnable weight matrix and bias vector of the linear transformation; For the first t The hidden state at each time step; For the first t Attention weights for each time step; This is the final generated context vector; A two-layer feedforward network, incorporating the GELU activation function and Dropout, is used to map the context vector to the final runoff forecast value. The calculation process can be represented as follows: ； in, This is the forecast value; , , For different weight matrices; It is the bias vector; This is a regularization method.

4. The intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to claim 1, characterized in that, S3 includes the following steps: S301. Design a joint loss function that combines the weighted mean square error and the Nash efficiency coefficient; the weighted mean square error loss is expressed as: ； ； ； in, Batch size; For the first i Individual sample weights; This represents the actual runoff value; Forecast flow rate; and These are the mean and standard deviation of the true values ​​in the training set, respectively. Hyperparameters for controlling weight strength; It is the sigmoid function, which maps input values ​​to (0,1); The function limits the weights to a range; The Nash efficiency coefficient loss is expressed as: ； in, The total number of samples; The final loss function is the weighted sum of the above two terms, expressed as: ； in, For hyperparameters; S302. The AdamW optimizer is selected, and a decoupled weight decay mechanism is introduced to suppress model overfitting, expressed as: ； ； ； ； ； in, , These are first-order and second-order momentum estimates, respectively; , The momentum decay coefficient; The current gradient; , These are the first-order and second-order momentum estimates after bias correction, respectively; In time step t Updated parameters; The learning rate; This is the weight decay coefficient; It is the numerical stability constant; Simultaneously, a cosine annealing one-cycle learning rate scheduling strategy with linear warm-up is adopted to balance the exploratory ability of model parameters with convergence stability, expressed as: ； in, This refers to the number of steps in the upward phase. The maximum learning rate; Minimum learning rate; Enable the early stopping mechanism; if the performance does not improve within a set number of iterations, stop training early.

5. The intelligent forecasting method for hydropower station reservoir runoff based on a large language model according to claim 1, characterized in that, S4 includes the following steps: S401. For the forecast period 1, the forecast value at time T+1 is output using the full observation data window; for the period T+2 and later, the forecast value at the previous time is introduced to complete the data window, and the data window is updated by sliding as the forecast progresses. S402. Calculate the root mean square error, mean absolute error, Nash coefficient, and Pearson correlation coefficient to verify the forecast accuracy and output the evaluation indicators and significance test results.

6. A smart forecasting system for hydropower station reservoir runoff based on a large language model, applied to the smart forecasting method for hydropower station reservoir runoff based on a large language model as described in any one of claims 1-5, characterized in that, include: The data processing module is used to acquire and inspect hydrological data, process missing factor data sequences, and then analyze the correlation between external factors and runoff sequences to form different factor sets. It further transforms the runoff data, performs time series partitioning, sliding window reconstruction, and standardization on different factor set data to obtain standardized sample data. The standardized sample data includes training set, validation set, and test set. The model building module, connected to the data processing module, obtains the model structure based on standardized sample data through an input projection layer, a Transformer encoder, an attention pooling layer, and a regression output layer. The model training module is connected to the model building module. Based on the model structure, it trains the model structure using the training set. Through the designed weighted mixed loss function, AdamW optimizer, One-Cycle learning rate scheduler and early stopping mechanism, the trained model is obtained. The forecast evaluation module is connected to the model training module. It uses the trained model and test set data to output multi-foreseeability runoff forecast results through a rolling forecast strategy, and calculates RMSE, MAE, NSE and R index to complete the accuracy evaluation.

7. A computer device, characterized in that, include: A processor configured to be coupled to memory, read and execute instructions and / or program code in the memory to perform the method as described in any one of claims 1-5.

8. A computer-readable medium, characterized in that, The computer-readable medium stores computer program code that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1-5.

Images