Water-wind complementary optimal scheduling methods, equipment and media

Abstract

This invention provides a method, device, and medium for hydro-wind complementary optimal scheduling, relating to the field of hydro-wind optimal scheduling technology in power systems. The method includes: predicting wind power using a TPA-BiLSTM model; obtaining data related to the uncertainty of wind power prediction based on predicted and historical data; converting the uncertainty of wind power prediction into deterministic positive and negative reserve constraints for hydropower using the probability density function of the established two-dimensional interval general distribution model, and establishing a hydro-wind stochastic optimal scheduling model; converting the hydro-wind stochastic optimal scheduling model into an environment suitable for reinforcement learning interaction, setting relevant states, actions, and rewards, and solving the hydro-wind stochastic optimal scheduling model using the xLSTM-TD3 algorithm to obtain a hydro-wind complementary optimal scheduling scheme. Compared to traditional optimal scheduling methods and classical reinforcement learning methods, this method can find a globally optimal solution while balancing solution efficiency and accuracy, effectively improving processing efficiency and computational accuracy.

Description

Technical Field

[0001] This invention relates to the field of power system water-wind complementary optimal dispatching technology, and more specifically, to a water-wind complementary optimal dispatching method, equipment, and medium. Background Technology

[0002] Developing clean energy sources such as wind and hydropower and constructing a new power system is imperative. New energy output is characterized by uncertainty and volatility, posing a series of challenges to power system dispatch. With the continuous increase in wind power installed capacity, the uncertainty, volatility, and randomness of wind power have brought great difficulties to grid dispatch and control. To avoid grid dispatch difficulties caused by the output characteristics of wind power, a large-capacity energy source with rapid adjustment is needed as compensation. Hydropower's high installed capacity and fast ramp-up rate perfectly meet these characteristics, making it suitable for dispatching alongside wind power to balance the uncertainty brought by wind power. By rationally planning the capacity of hydropower and wind power, utilizing wind power to avoid power shortages during the dry season, and achieving complementary dispatch between hydropower and wind power, it is possible to effectively utilize hydropower and wind energy resources, replacing fossil fuels while improving economic efficiency.

[0003] Research on hydro-wind complementary optimal dispatch mainly includes two aspects: establishing and solving the hydro-wind complementary optimal dispatch model. Characterizing the uncertainty of wind power prediction and comprehensively considering the benefits of power generation and dispatch are key to establishing a reasonable hydro-wind complementary optimal dispatch model. Hydro-wind complementary optimal dispatch models are usually quite complex, with numerous constraints and strong nonlinearity, requiring appropriate algorithms to solve them. Many scholars have adopted heuristic learning methods to solve this model. However, traditional intelligent algorithms are inefficient when dealing with complex high-dimensional problems and are prone to getting trapped in local optima.

[0004] Traditional physical models and statistical learning methods struggle to accurately predict wind power variation trends, while deep learning, with its powerful learning capabilities, can effectively predict wind power under sufficient sample conditions. Furthermore, as the complexity of scheduling models continues to increase, finding a globally optimal solution while balancing search efficiency and solution accuracy within a large space of independent variables remains a challenge. Reinforcement learning, by its very nature, excels at handling temporal problems, especially high-dimensional action spaces, demonstrating stronger global search capabilities and less susceptibility to local optima compared to traditional optimization algorithms. Therefore, employing Deep Reinforcement Learning (DRL) algorithms can effectively improve the processing efficiency of the hydro-wind hybrid optimal scheduling problem. Summary of the Invention

[0005] The present invention aims to at least solve one of the technical problems of existing water-wind complementary optimization scheduling methods, namely poor calculation speed and solution accuracy, low processing efficiency, and easy getting trapped in local optima.

[0006] Therefore, the first aspect of the present invention provides a water-wind complementary optimal scheduling method.

[0007] A second aspect of the present invention provides a computer device.

[0008] A third aspect of the present invention provides a computer-readable storage medium.

[0009] This invention provides a water-wind complementary optimal scheduling method, comprising:

[0010] Wind power is predicted using the TPA-BiLSTM neural network model, which combines a time-pattern attention mechanism and a bidirectional long short-term memory network. Based on the predicted data and historical data, data related to the uncertainty of wind power prediction are obtained. Among them, since the uncertainty of wind power has significant characteristics in both the power and time dimensions, probability density functions of wind power prediction error are established for different power-time dimension intervals. In particular, a general distribution model is used to establish the probability density functions of wind power uncertainty data in different intervals.

[0011] Based on the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power prediction is transformed into deterministic positive and negative reserve constraints of hydropower, and a hydro-wind stochastic optimization scheduling model is established. Among them, the economic efficiency of hydro-wind power generation and line power fluctuation are used as the optimization objectives of the hydro-wind stochastic optimization scheduling model, and constraints are set.

[0012] The water-wind stochastic optimization scheduling model is transformed into an environment suitable for reinforcement learning interaction. Relevant states, actions, and rewards are set, and the xLSTM-TD3 method, which combines long short-term memory network and dual-delay deep deterministic policy gradient algorithm, is used to solve the water-wind stochastic optimization scheduling model and obtain a water-wind complementary optimization scheduling scheme.

[0013] The water-wind complementary optimal scheduling method according to the above technical solution of the present invention may also have the following additional technical features:

[0014] In the above technical solution, based on the significant characteristics of wind power uncertainty in both power and time dimensions, the probability density function of wind power prediction error is established for different power-time dimension intervals, including:

[0015] A two-dimensional power-time interval is divided, where the power interval is divided and the time interval is divided separately;

[0016] A general distribution model is used to establish the probability density function of wind power uncertainty data for each interval; the general distribution model has a closed cumulative distribution function form and can fit wind power uncertainty distributions of arbitrary shapes; the expressions for the probability density function and cumulative distribution function of the general distribution model are as follows:

[0017]

[0018] F(xa,b,c)=(1+e -a(x-c) ) -b

[0019] Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; and a, b, and c all represent shape parameters that can be adjusted.

[0020] The prediction error confidence interval is obtained using the inverse function of the cumulative distribution function; the expression for the inverse function of the cumulative distribution function is:

[0021]

[0022] Where α represents the confidence level.

[0023] In the above technical solution, the power range division includes:

[0024] Determine the number of power intervals to make the number of samples in each power interval approximately the same;

[0025] The time interval division includes:

[0026] The time intervals are initially determined and divided into several time intervals. Based on the similarity of data distribution between different time intervals, the time intervals are merged. The similarity of data distribution is represented by relative entropy.

[0027]

[0028] Where KL(P||Q) represents the relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different values ​​of the random variable.

[0029] In the above technical solution, the power range division includes:

[0030] Determine the number of power intervals to make the number of samples in each power interval approximately the same;

[0031] The time interval division includes:

[0032] The time intervals were initially determined, dividing the time into several intervals. Based on the similarity of data distribution between different time intervals, the time intervals were merged. The similarity of data distribution was represented by JS divergence.

[0033]

[0034] Where JS(P||Q) represents the JS divergence; KL(*) represents the relative entropy; and P and Q represent two different data distributions.

[0035] In the above technical solution, the objective function of the water-wind stochastic optimization scheduling model includes:

[0036]

[0037] Where, N t This represents the local electricity price for time period t; L represents the external electricity price for time period t. t P represents the local load during time period t; t,w P represents the wind power generated at time t; t,h The hydroelectric power generated at time t represents the dispatching period; T represents the dispatching period. N represents the average power output of hydro-wind power generation over the entire scheduling cycle; k represents the weighting parameter; N t L t This indicates the benefits generated from power generation supplying local load. This represents the benefits generated from the transmission of excess electricity; the sum of these two items represents the total revenue from electricity sales. This indicates the level of power fluctuations in wind and hydropower on the transmission lines to the system, i.e., the expected hydropower output to smooth out the fluctuations in wind power output.

[0038] In the above technical solution, the constraints of the water-wind stochastic optimization scheduling model include:

[0039] Upper and lower limits of hydropower generation capacity constraints:

[0040]

[0041] in, This indicates the lower limit of the hydropower unit's output. Indicates the upper limit of the hydropower unit's output;

[0042] Average power generation water flow upper and lower limits constraints:

[0043]

[0044] Among them, Q t,h This represents the average water flow rate for power generation during time period t. This indicates the lower limit of average water consumption for power generation. This indicates the upper limit of average water consumption for power generation;

[0045] Water level upper and lower limits constraints:

[0046]

[0047] Among them, Z t,h This represents the water level during time period t. Indicates the lower limit of the water level. Indicates the upper limit of the water level;

[0048] Storage capacity upper and lower limits constraints:

[0049]

[0050] Among them, V t,h This represents the storage capacity during time period t. Indicates the lower limit of storage capacity. Indicates the upper limit of reservoir capacity; transmission line capacity constraints:

[0051] P t,h +P t,w ≤P TL

[0052] Among them, P TL This indicates the maximum transmission power of the power transmission line;

[0053] Load constraints:

[0054] P t,h +P t,w ≥L t ;

[0055] Water level constraints at the end of the period:

[0056] |Z T,h -Z 1,h |≤Z b

[0057] Among them, Z T,h Z represents the storage capacity at the end of the period. 1,h Z represents the initial storage capacity during the time period. b Indicates the water level error threshold;

[0058] Uncertainty constraints of wind power:

[0059] Hydropower reserves positive and negative backups for wind power capacity, including:

[0060]

[0061] Where α represents the confidence level. This represents the minimum wind power output at confidence level α. P represents the maximum wind power output at confidence level α. r Represents probability;

[0062] The relationship between hydropower generation capacity and average power generation flow, reservoir capacity, and water level includes:

[0063] P t,h =AQ t,h H t

[0064] Q t,e Δt+V t,h =V t+1,h

[0065] Z t,h =f(V t,h )

[0066] Among them, P t,h H represents the output of the hydropower unit during time period t; A represents the output coefficient of the hydropower unit; H t Q represents the net head of the hydropower unit during time period t; t,e V represents the average water flow rate of the reservoir during time period t; a negative value indicates that water is being drawn from the reservoir for power generation. Δt represents the length of the time period. t+1,h represents the reservoir capacity during the (t+1)th time period; f(*) represents the reservoir capacity-water level function relationship.

[0067] In the above technical solution, the step of converting the water-wind stochastic optimization scheduling model into an environment suitable for reinforcement learning interaction, and setting relevant states, actions, and rewards, includes:

[0068] Define a state, where state S t It is a sequence of length T+1, including:

[0069] S t =[N t V 1,h V 2,h ,...,V t,h ,...,V T,h ]

[0070] Among them, V t,h This represents the storage capacity at time t, after which the storage capacity is initialized to 0.

[0071] Define an action, where action A t The average hydropower generation water flow rate in the t-th time period includes:

[0072] A t =Q t,h

[0073] Among them, Qt,h This represents the average water flow rate for power generation during time period t;

[0074] Define rewards, which include power generation rewards, power fluctuation penalties, and end-point water level penalties. The expression for the rewards includes:

[0075]

[0076] Where R1 represents the power generation reward, R2 represents the power fluctuation penalty, R3 represents the end-point water level penalty, R represents the total reward, k1 represents the power generation reward parameter, k2 represents the power fluctuation penalty parameter, and k3 represents the end-point water level penalty parameter;

[0077] By adjusting the values ​​of k1, k2, and k3, we ensure that different rewards are on the same order of magnitude, enabling the Long Short-Term Memory (LSM) network xLSTM to combine the rewards from each part and make the most appropriate action.

[0078] In the above technical solution, the dual-delay deep deterministic policy gradient algorithm includes a policy network and an evaluation network. The policy network is used to make the next decision based on the current environmental state, and the evaluation network is used to judge the quality of the decision made by the policy network based on the current state. The evaluation network includes a first online evaluation network and a second online evaluation network.

[0079] The method xLSTM-TD3, which combines a long short-term memory network and a dual-delay deep deterministic policy gradient algorithm, is used to solve the water-wind stochastic optimization scheduling model and obtain a water-wind complementary optimization scheduling scheme, including:

[0080] The Long Short-Term Memory (xLSTM) network is used to continuously give actions based on the current state of the environment, and obtain new states and rewards after interacting with the environment.

[0081] The online policy network weights are updated to form the target policy network. Update methods include:

[0082]

[0083] Where Q1 represents the first online evaluation network, w1 represents the network parameters of the first online evaluation network, m represents the sample size, and J(θ) represents the loss function of the online policy network. π represents the gradient of the online policy network weights; π represents the online action network; θ represents the network parameters of the online action network.

[0084] The network parameters of the online evaluation network are updated according to the temporal difference algorithm, including:

[0085]

[0086] Among them, J(w j) represents the loss function of the online evaluation network; y t This represents the target Q value obtained using the time-difference algorithm; Q j Indicates an online evaluation network; w j This represents the network parameters of the online evaluation network; Q′ j The network represents the target evaluation network, where j represents the evaluation network number; π′ represents the target action network; θ' represents the network parameters of the target action network; w' j R represents the network parameters of the target evaluation network. t Represents the total reward at time t; γ represents the decay factor;

[0087] The parameters of the target network are not used in training, but are updated periodically from the online network based on a soft update strategy to ensure the stability of the online network updates, i.e.:

[0088]

[0089] Where τ represents the update factor;

[0090] The long short-term memory network xLSTM is trained according to the preset training algebra, and the water-wind complementary optimal scheduling scheme is obtained based on the trained long short-term memory network xLSTM.

[0091] The present invention provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is loaded and executed by the processor, it implements the water-wind complementary optimal scheduling method as described in any of the above technical solutions.

[0092] The present invention provides a computer-readable storage medium storing a program that, when loaded by a processor, implements the water-wind complementary optimal scheduling method as described in any of the above technical solutions.

[0093] In summary, due to the adoption of the above-mentioned technical features, the beneficial effects of the present invention are:

[0094] This invention employs a two-dimensional power-time interval partitioning and a universal distribution to characterize the uncertainty of wind power prediction, and establishes a hydro-wind complementary optimal scheduling model, which provides a more refined characterization of the model and more constraints. Finally, the xLSTM-TD3 algorithm based on reinforcement learning is used to solve the hydro-wind stochastic optimal scheduling model. The results show that, compared with traditional heuristic learning methods and the DDPG algorithm (a deep reinforcement learning algorithm for solving continuous action problems), this invention can find a better solution with higher efficiency.

[0095] Specifically, this invention establishes a water-wind stochastic optimization scheduling model based on stochastic optimization methods and solves it using a dual-delay deep deterministic policy gradient algorithm. First, the established scheduling model is converted into a reinforcement learning environment, constructing corresponding actions, rewards, and states. The xLSTM-TD3 algorithm is then used to interact with the environment, training the policy network and evaluation network to provide the final optimized scheduling scheme. Compared to traditional optimization scheduling methods and classical reinforcement learning methods, the xLSTM-TD3 algorithm can find a globally optimal solution while maintaining both efficiency and accuracy, effectively improving the processing efficiency and computational accuracy of the water-wind complementary optimization scheduling problem.

[0096] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0097] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0098] Figure 1 This is a schematic diagram of the network structure of the Actor network and the Critic network in a water-wind complementary optimization scheduling method according to an embodiment of the present invention;

[0099] Figure 2 This is a flowchart of the water-wind random optimization scheduling based on xLSTM-TD3 in a water-wind complementary optimization scheduling method according to an embodiment of the present invention;

[0100] Figure 3 This is a schematic diagram of a power generation scheme obtained by using the xLSTM-TD3 algorithm in a water-wind complementary optimal scheduling method according to an embodiment of the present invention;

[0101] Figure 4 This is a schematic diagram of water level and electricity price curves in a water-wind complementary optimal scheduling method according to an embodiment of the present invention. Detailed Implementation

[0102] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0103] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0104] The following reference Figures 1 to 4 This describes a water-wind complementary optimized scheduling method provided according to some embodiments of the present invention.

[0105] Some embodiments of this application provide a water-wind complementary optimal scheduling method.

[0106] The first embodiment of the present invention proposes a water-wind complementary optimal scheduling method, which mainly includes three parts: S1-S3.

[0107] S1. Wind power is predicted using the TPA-BiLSTM neural network model (hereinafter referred to as the TPA-BiLSTM model), which combines a time-pattern attention mechanism and a bidirectional long short-term memory network. Based on the predicted data and historical data, data related to the uncertainty of wind power prediction are obtained. Among them, based on the significant characteristics of wind power uncertainty in both power and time dimensions, probability density functions of wind power prediction error are established for different power-time dimension intervals. In particular, a general distribution model is used to establish the probability density function of wind power uncertainty data in different intervals.

[0108] It should be noted that the TPA-BiLSTM model is one of the commonly used models for predicting wind power, and its specific usage is well known to those skilled in the art and will not be elaborated here. It is understood that those skilled in the art can also use other suitable prediction models to make preliminary predictions of wind power to obtain prediction data, such as the Autoregressive Moving Average (ARMA) model, the two-dimensional Jensen model, and using Artificial Neural Networks (ANNs) to dynamically model multivariate time series data to achieve wind power prediction.

[0109] To address the uncertainty in wind power prediction within a hydro-wind power optimization scheduling model using stochastic optimization methods, it's necessary to establish a probability density function (PDF) of the wind power prediction error based on historical data to determine the prediction error confidence interval. However, directly establishing a single PDF for the entire wind power prediction error dataset results in excessively wide confidence intervals, significantly reducing scheduling efficiency. Research has found that wind power uncertainty typically exhibits distinct characteristics in both the power and time dimensions. Therefore, establishing multiple PDFs for different power-time dimension intervals can improve the economic efficiency of the scheduling process.

[0110] In some embodiments, the step of establishing probability density functions for wind power prediction errors for different power-time dimension intervals, based on the significant characteristics of wind power uncertainty in both power and time dimensions, includes:

[0111] A two-dimensional power-time interval partitioning is performed, specifically dividing the interval into power intervals and time intervals. The number of power-time intervals should be reasonable. Too many intervals may result in insufficient sample sizes in some intervals, making it impossible to fit the true prediction error PDF; too few intervals may lead to poor scheduling economy.

[0112] Specifically, for power interval division, first determine the number of power intervals to be divided, and then try to make the number of samples in each power interval the same.

[0113] For time interval division, the interval is first initially determined, and then the time interval is divided into several time intervals. Based on the similarity of data distribution between different time intervals, the time intervals are merged.

[0114] In one specific embodiment, the similarity of data distribution during the time interval division process can be represented by relative entropy (Kullback-Leibler divergence, KLD), specifically:

[0115]

[0116] Where KL(P||Q) represents the relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different values ​​of the random variable.

[0117] However, KLD has asymmetry, which is not conducive to determining the distance between two distributions. Therefore, the JS divergence, which has symmetry, can be used to measure the similarity between two distributions. In another specific embodiment, the similarity of data distributions during the time interval division is represented by JS divergence:

[0118]

[0119] Where JS(P||Q) represents the JS divergence; KL(*) represents the relative entropy; and P and Q represent two different data distributions.

[0120] That is, when dividing the time interval, it is necessary to first determine the probability distribution of each prediction error among several equally divided subdivisions, and then determine the JS divergence of the prediction error distribution in different time periods.

[0121] To establish a probability density function (PDF) for wind power uncertainty data under different distributions, a distribution model capable of describing different shapes is needed. In this embodiment, a general distribution model is used to establish the probability density function of wind power uncertainty data for each interval; the general distribution model has a closed cumulative distribution function form and can fit wind power uncertainty distributions of arbitrary shapes; the expressions for the probability density function and cumulative distribution function of the general distribution model are as follows:

[0122]

[0123] F(xa,b,c)=(1+e -a(x-c) ) -b

[0124] Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; and a, b, and c all represent shape parameters that can be adjusted.

[0125] The prediction error confidence interval is obtained using the inverse function of the cumulative distribution function; the expression for the inverse function of the cumulative distribution function is:

[0126]

[0127] Here, α represents the confidence level. Since the general probability distribution has an explicit inverse CDF (cumulative distribution function), the confidence interval can be easily determined. When fitting the uncertainty of wind power, the least squares method or gradient descent method can be used to obtain the three shape parameters of the general distribution.

[0128] After obtaining the wind power prediction error PDF for each interval, the uncertainty of wind power can be quantified based on this, and then a stochastic optimization scheduling model for water and wind can be established.

[0129] S2. The hydropower and wind power stochastic optimization scheduling model treats hydropower and wind power generation as a whole for scheduling. It not only needs to fully meet local load but also ensures that surplus power can be transmitted to other regions via transmission lines. Based on the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power prediction is transformed into deterministic positive and negative reserve constraints for hydropower, thus establishing the hydropower and wind power stochastic optimization scheduling model. The economic efficiency of hydropower and wind power generation and the fluctuation of transmission line power are used as the optimization objectives of the model, and constraints are set accordingly.

[0130] In some embodiments, the objective function of the water-wind stochastic optimization scheduling model includes:

[0131]

[0132] Where, N tThis represents the local electricity price for time period t; L represents the external electricity price for time period t. t P represents the local load during time period t; t,w P represents the wind power generated at time t; t,h The value represents the hydropower generated at time t; T represents the scheduling cycle, which is once per hour for day-ahead scheduling, i.e., T = 24. N represents the average power output of hydro-wind power generation over the entire scheduling cycle; k represents the weighting parameter; N t L t This indicates the benefits generated from power generation supplying local load. This represents the benefits generated from the transmission of excess electricity; the sum of these two items represents the total revenue from electricity sales. This indicates the level of power fluctuations in wind and hydropower on the transmission lines to the system, i.e., the expected hydropower output to smooth out the fluctuations in wind power output.

[0133] In some embodiments, the constraints of the hydropower stochastic optimization scheduling model include upper and lower limits of hydropower generation capacity, upper and lower limits of average power generation water flow, upper and lower limits of water level, upper and lower limits of reservoir capacity, transmission line capacity, load, water level at the end of the period, and wind power uncertainty constraints.

[0134] Upper and lower limits of hydropower generation capacity constraints:

[0135]

[0136] in, This indicates the lower limit of the hydropower unit's output. This indicates the upper limit of the hydropower unit's output.

[0137] Average power generation water flow upper and lower limits constraints:

[0138]

[0139] Among them, Q t,h This represents the average water flow rate for power generation during time period t. This indicates the lower limit of average water consumption for power generation. This indicates the upper limit of average water flow for power generation.

[0140] Water level upper and lower limits constraints:

[0141]

[0142] Among them, Z t,h This represents the water level during time period t. Indicates the lower limit of the water level. This indicates the upper limit of the water level.

[0143] Storage capacity upper and lower limits constraints:

[0144]

[0145] Among them, V t,h This represents the storage capacity during time period t. Indicates the lower limit of storage capacity. This indicates the maximum storage capacity.

[0146] Transmission line capacity constraints:

[0147] P t,h +P t,w ≤P TL

[0148] Among them, P TL This indicates the maximum transmission power of the power transmission line.

[0149] Load constraints:

[0150] P t,h +P t,w ≥L t

[0151] Under normal circumstances, hydropower and wind power should at least meet the local load.

[0152] Water level constraints at the end of the period:

[0153] |Z T,h -Z 1,h |≤Z b

[0154] Among them, Z T,h Z represents the storage capacity at the end of the period. 1,h Z represents the initial storage capacity during the time period. b This indicates the water level error threshold. The water level at the end of the period should not differ significantly from the water level at the beginning of the period to facilitate scheduling in the next cycle.

[0155] Uncertainty constraints of wind power:

[0156] Given the uncertainty of wind power output and the difficulty in achieving completely accurate forecasts, hydropower needs to both supplement insufficient wind power when forecasts are too high and reduce its output when forecasts are too low to prevent wind curtailment. Therefore, hydropower needs to reserve positive and negative reserves for wind power, including:

[0157]

[0158] Where α represents the confidence level. This represents the minimum wind power output at confidence level α. P represents the maximum wind power output at confidence level α. r Represents probability;

[0159] The relationship between hydropower generation capacity and average power generation flow, reservoir capacity, and water level includes:

[0160] P t,h =AQ t,h H t

[0161] Q t,e Δt+V t,h =V t+1,h

[0162] Z t,h =f(V t,h )

[0163] Among them, P t,h H represents the output of the hydropower unit during time period t; A represents the output coefficient of the hydropower unit; H t Q represents the net head of the hydropower unit during time period t; t,e V represents the average water flow rate of the reservoir during time period t; a negative value indicates that water is being drawn from the reservoir for power generation. Δt represents the length of the time period. t+1,h represents the reservoir capacity during the (t+1)th time period; f(*) represents the reservoir capacity-water level function relationship.

[0164] The water-wind stochastic optimization scheduling model established in this disclosure is quite complex. Traditional optimization methods may easily get trapped in local optima and have low solution efficiency. Common reinforcement learning methods such as Q-learning and DQN (Deep Q-Network, an existing deep learning-based reinforcement learning algorithm) require discretization of continuous actions, making it difficult to balance solution accuracy and computational speed. Therefore, this disclosure adopts a method combining Long Short-Term Memory Network (xLSTM) and the dual-delay deep deterministic policy gradient algorithm (TD3) (hereinafter referred to as the xLSTM-TD3 algorithm), which can directly solve the continuous action problem and effectively balance computational accuracy and efficiency.

[0165] S3. The water-wind stochastic optimization scheduling model is transformed into an environment suitable for reinforcement learning interaction. Relevant states, actions and rewards are set, and the water-wind stochastic optimization scheduling model is solved using the xLSTM-TD3 algorithm to obtain a water-wind complementary optimization scheduling scheme.

[0166] Among them, the Long Short-Term Memory (xLSTM) network is constructed based on LSTM, and its construction steps are as follows:

[0167] Replacing the sigmoid activation function of the LSTM forget gate with xigmoid allows the LSTM forget gate to continue training, thus achieving long-term memory.

[0168] Turn the input gate i of the LSTM t Replace with (1-f) t On the one hand, it can save the number of learnable parameters of LSTM and speed up model training. On the other hand, adding more prior knowledge to the model can reduce the size of the search space of model parameters, reduce the difficulty of model learning, avoid the model getting stuck in a large range of non-optimal solutions during training, and reduce the amount of training data and training rounds required for convergence.

[0169] The Dual-Delay Deep Deterministic Policy Gradient Algorithm (TD3) is an improved deep reinforcement learning algorithm based on DDPG, which to some extent solves the overestimation problem of DDPG. TD3 mainly consists of two networks: a policy network (Actor network) that makes the next decision based on the current environment state; and a criterion network that judges the quality of the Actor network's decision based on the current state. The online Actor network selects its actions based on the current state and its network parameters.

[0170] A t =π(S) t |θ)

[0171] Among them, S t As the current state, A t π represents the decision-making behavior made by the online Actor network based on the current state, θ represents the online action network, and θ represents the network parameters.

[0172] The combination of xLSTM and TD3 can improve the performance of deep reinforcement learning agents, especially in tasks where the state space exhibits temporal dependencies or sequential structures. By leveraging xLSTM's ability to capture long-term dependencies, the agent may be better able to learn complex behaviors and make more informed decisions in dynamic environments.

[0173] To solve the water-wind stochastic optimal scheduling model using the xLSTM-TD3 algorithm, it is necessary to define the state, action, and reward so that the xLSTM network can effectively learn from the environment and obtain an optimal scheduling scheme.

[0174] In some embodiments, the step of converting the water-wind stochastic optimization scheduling model into an environment suitable for reinforcement learning interaction, and setting relevant states, actions, and rewards, includes:

[0175] Define a state, where state S t It is a sequence of length T+1, including:

[0176] S t =[N t V 1,h V 2,h ,...,V t,h ,...,V T,h ]

[0177] Among them, V t,h Let represent the normalized storage capacity at time t, after which the storage capacity is initialized to 0; define actions, where action A... t The average hydropower generation water flow rate in the t-th time period includes:

[0178] A t =Q t,h

[0179] Among them, Q t,h A represents the average water flow rate for power generation during time period t; t As a continuous variable, it needs to satisfy the upper and lower limits of the average power generation water flow in the constraints.

[0180] Define rewards, including power generation rewards, power fluctuation penalties, and end-of-pipe water level penalties. The power generation reward is returned after each action, while the power fluctuation penalties and end-of-pipe water level penalties are only returned after the final action. The expression for the rewards includes:

[0181]

[0182] Where R1 represents the power generation reward, R2 represents the power fluctuation penalty, R3 represents the end-point water level penalty, R represents the total reward, k1 represents the power generation reward parameter, k2 represents the power fluctuation penalty parameter, and k3 represents the end-point water level penalty parameter;

[0183] By adjusting the values ​​of k1, k2, and k3, the different rewards are kept on the same order of magnitude, enabling the Long Short-Term Memory (xLSTM) network to synthesize the various rewards and make the most appropriate action. During the interaction between the xLSTM network and the environment, it checks whether the constraints are met. If not, it returns a large negative reward and ends the interaction.

[0184] In some embodiments, the dual-delay deep deterministic policy gradient algorithm includes a policy network (Actor network) and an evaluation network (Critic network), wherein the policy network is used to make the next decision based on the current environmental state, and the evaluation network is used to judge the quality of the decision made by the policy network based on the current state. The evaluation network includes a first online evaluation network and a second online evaluation network.

[0185] Policy network and evaluation network structures are as follows Figure 1 As shown, both the Actor network and the Critic network use fully connected (FC) layers as their hidden layers. To enable the Actor network to determine the current hydropower generation, the current electricity price and the hydropower generation for each time period are fed to the Actor network as state variables. The Actor network processes these two variables separately using Dense layers (fully connected layers) to extract information, and then combines them to provide the hydropower generation value for the current scheduling period using a Dense layer. The Critic network, based on the current state and the Actor network's action value, uses multiple fully connected layers combined with the electricity price and scheduling schemes for each time period to evaluate the current Actor's value. The Critic network trains its neural network parameters based on the reward for the current action and the time-series difference algorithm as improvement directions, while the Actor network trains its neural network to maximize the score of the output action value.

[0186] In some embodiments, such as Figure 2 As shown, the method xLSTM-TD3, which combines a long short-term memory network and a dual-delay deep deterministic policy gradient algorithm, is used to solve the water-wind stochastic optimization scheduling model and obtain a water-wind complementary optimization scheduling scheme. This includes the following steps S31-S35.

[0187] Among them, the upper and lower reserves of hydropower should be reserved according to the wind power forecast and the following formula, and the environment should be initialized:

[0188]

[0189] Where α represents the confidence level. This represents the minimum wind power output at confidence level α. This represents the maximum wind power output at confidence level α. It is important to note that... and It needs to be calculated separately based on the PDF for different power-time two-dimensional intervals.

[0190] S31. Utilize a Long Short-Term Memory (xLSTM) network to continuously provide action A based on the current state of the environment. t After interacting with the environment, one gains new status and rewards;

[0191] S32. Update the weights of the online policy network to form the target policy network. The update methods include:

[0192] The online Critic network evaluates the actions selected by the action network through a neural network, thereby guiding the weight adjustment of the online action network. The online Actor network calculates the loss function and the gradients of the network weights based on the gradient boosting method.

[0193]

[0194] Q1 represents the first online evaluation network, i.e. Figure 2 The online Critic network 1 is shown; w1 represents the network parameters of the first online evaluation network, m represents the sample size; J(θ) represents the loss function of the online policy network; π represents the gradient of the online policy network weights; π represents the online action network; θ represents the network parameters of the online action network.

[0195] S33. Update the network parameters of the online evaluation network according to the temporal difference algorithm, including:

[0196]

[0197] Among them, J(w j ) represents the loss function of the online evaluation network; y t This represents the target Q value obtained using the time-difference algorithm; Q j This refers to online review networks, including Figure 2 The online Critic network 1 and online Critic network 2 are shown; w j Q represents the network parameters of the online evaluation network; j ′ represents the target evaluation network, including Figure 2 The target Critic network 1 and target Critic network 2 are shown; j represents the evaluation network number; π′ represents the target action network; θ′ represents the network parameters of the target action network; w′ j R represents the network parameters of the target evaluation network. t γ represents the total reward at time t; γ represents the decay factor, and the larger the value, the more emphasis is placed on the potential rewards in the future.

[0198] The TD3 algorithm constructs the target value of the online Critic network using the relatively smaller value from the two target Critic networks to address its overestimation problem. Furthermore, the Critic network is updated more frequently than the Actor network because inaccurate evaluations of states and actions by the Critic network cannot effectively guide the Actor network's updates.

[0199] S34. The parameters of the target network are not used in training, but are updated periodically from the online network based on a soft update strategy to ensure the stability of the online network updates, that is:

[0200]

[0201] Where τ represents the update factor, which reflects the update speed of the target network. Usually, a small value, such as 0.1, is taken to ensure the training stability of the online network.

[0202] S35. Train the Long Short-Term Memory (xLSTM) network according to the preset training number of generations, and obtain the water-wind complementary optimal scheduling scheme based on the trained xLSTM network. Figure 2 The method shown determines whether the training generation Epoch is greater than the preset training generation E. If the preset training generation is not reached, the environment is re-initialized and training continues; otherwise, training is stopped. A water-wind random optimization scheduling scheme is obtained based on the trained xLSTM network.

[0203] In one specific embodiment, the scheduling results based on the xLSTM-TD3 algorithm are analyzed. The relevant parameters of the scheduling model and the xLSTM-TD3 algorithm are shown in Table 1; the predicted load and wind power for each scheduling period are shown in Table 2; the electricity price for each scheduling period is shown in Table 3; and the relevant parameters and network structure of the xLSTM-TD3 algorithm are shown in Table 4. A comparison of the results of the xLSTM-TD3 algorithm with those of the DDPG and GA (Genetic Algorithm) algorithms is listed in Table 5.

[0204] Table 1. Parameters of the stochastic optimization scheduling model considering the uncertainty of wind power forecasting

[0205]

[0206] Table 2. Load and wind power forecasts for each scheduling period.

[0207]

[0208] Table 3 Electricity Prices for Each Scheduling Period

[0209]

[0210] Table 4 xLSTM-TD3 Algorithm Parameters and Network Structure

[0211]

[0212] Table 5. Scheduling results of each algorithm

[0213]

[0214]

[0215] As can be seen from the table, the xLSTM-TD3 algorithm ultimately achieves the highest total reward. Compared to the DDPG algorithm, although the power fluctuation is slightly larger, it achieves a better increase in power generation revenue, resulting in a higher total reward. Compared to the GA algorithm, it significantly reduces power fluctuation with a slight loss in power generation revenue. The xLSTM-TD3 algorithm is better able to avoid getting trapped in local optima and find better solutions. This is because the xLSTM-TD3 algorithm uses two Critic networks and delays the update of the Actor network, avoiding the overestimation problem in traditional DDPG, and also making the Actor training more stable, which is more conducive to finding the global optimum.

[0216] The solution found by the xLSTM-TD3 algorithm is as follows: Figure 3 As shown. From Figure 3 As can be seen, the total power generation of hydropower and wind power is always higher than the load, and the excess power is transmitted through external transmission channels. The total power generation remains at a relatively stable level, which helps to reduce power fluctuations. In addition, hydropower always has sufficient positive and negative reserves, which ensures that the load is met while minimizing wind curtailment.

[0217] Water level change curve and electricity price curve as shown Figure 4 As shown. From Figure 4 As can be seen, the water level remains consistently between the highest water level (1330m) and the dead water level (1328m), and the water level at the end of the scheduling period is almost equal to the water level at the beginning of the scheduling period, satisfying the water level-related constraints of the model. Furthermore, it can be seen that during periods of low electricity prices, such as 10 PM to 7 AM, the agent stores as much water as possible to maximize electricity sales revenue. Conversely, during periods of high electricity prices, such as 2 PM to 5 PM and 8 PM to 9 PM, the agent sells electricity as much as possible, utilizing the water stored during periods of low electricity prices to generate electricity and increase sales revenue. The graph shows that water level and electricity price are approximately inversely proportional, but not simply linearly related. This indicates that the agent is trying to find a nonlinear optimal solution while satisfying all model constraints.

[0218] In summary, the model was solved using the xLSTM-TD3 algorithm. The results show that the xLSTM-TD3 algorithm, with its dual Critic network and Actor delayed update strategy, can effectively avoid overestimation and improve algorithm stability. It can also effectively escape local optima and further search for better global solutions.

[0219] Other embodiments of the present invention provide a computer device including a processor and a memory, wherein the memory stores a computer program that, when loaded and executed by the processor, implements the water-wind complementary optimal scheduling method as described in any of the above embodiments.

[0220] Some embodiments of the present invention provide a computer-readable storage medium storing a program that, when loaded by a processor, implements the water-wind complementary optimal scheduling method as described in any of the above embodiments.

[0221] In this specification, the illustrative expressions of the terms used do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0222] Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention shall be included within the scope of protection of this invention.

Claims

1. A water-wind complementary optimal scheduling method, characterized in that, include: Wind power is predicted using the TPA-BiLSTM neural network model, which combines a time-pattern attention mechanism and a bidirectional long short-term memory network. Based on the predicted data and historical data, data related to the uncertainty of wind power prediction are obtained. Among them, since the uncertainty of wind power has significant characteristics in both the power and time dimensions, probability density functions of wind power prediction error are established for different power-time dimension intervals. In particular, a general distribution model is used to establish the probability density functions of wind power uncertainty data in different intervals. Based on the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power prediction is transformed into deterministic positive and negative reserve constraints of hydropower, and a hydro-wind stochastic optimization scheduling model is established. Among them, the economic efficiency of hydro-wind power generation and line power fluctuation are used as the optimization objectives of the hydro-wind stochastic optimization scheduling model, and constraints are set. The water-wind stochastic optimization scheduling model is transformed into an environment suitable for reinforcement learning interaction. Relevant states, actions, and rewards are set, and the xLSTM-TD3 method, which combines long short-term memory network and dual-delay deep deterministic policy gradient algorithm, is used to solve the water-wind stochastic optimization scheduling model and obtain a water-wind complementary optimization scheduling scheme.

2. The water-wind complementary optimal scheduling method according to claim 1, characterized in that, Based on the significant characteristics of wind power uncertainty in both the power and time dimensions, probability density functions for wind power prediction errors are established for different power-time dimension intervals, including: A two-dimensional power-time interval is divided, where the power interval is divided and the time interval is divided separately; A general distribution model is used to establish the probability density function of wind power uncertainty data for each interval; the general distribution model has a closed cumulative distribution function form and can fit wind power uncertainty distributions of arbitrary shapes; the expressions for the probability density function and cumulative distribution function of the general distribution model are as follows: F(x|a,b,c)=(1+e -a(x-c) ) -b Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; and a, b, and c all represent shape parameters that can be adjusted. The prediction error confidence interval is obtained using the inverse function of the cumulative distribution function; the expression for the inverse function of the cumulative distribution function is: Where α represents the confidence level.

3. The water-wind complementary optimal scheduling method according to claim 2, characterized in that, The power range division includes: Determine the number of power intervals to make the number of samples in each power interval approximately the same; The time interval division includes: The time intervals are initially determined and divided into several time intervals. Based on the similarity of data distribution between different time intervals, the time intervals are merged. The similarity of data distribution is represented by relative entropy. Where KL(P||Q) represents the relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different values ​​of the random variable.

4. The water-wind complementary optimal scheduling method according to claim 2, characterized in that, The power range division includes: Determine the number of power intervals to make the number of samples in each power interval approximately the same; The time interval division includes: The time intervals were initially determined, dividing the time into several intervals. Based on the similarity of data distribution between different time intervals, the time intervals were merged. The similarity of data distribution was represented by JS divergence. Where JS(P||Q) represents the JS divergence; KL(*) represents the relative entropy; and P and Q represent two different data distributions.

5. The water-wind complementary optimal scheduling method according to claim 1, characterized in that, The objective function of the water-wind stochastic optimization scheduling model includes: Where, N t This represents the local electricity price for time period t; L represents the external electricity price for time period t. t P represents the local load during time period t; t,w P represents the wind power generated at time t; t,h The hydroelectric power generated at time t represents the dispatching period; T represents the dispatching period. N represents the average power output of hydro-wind power generation over the entire scheduling cycle; k represents the weighting parameter; N t L t This indicates the benefits generated from power generation supplying local load. This represents the benefits generated from the transmission of excess electricity; the sum of these two items represents the total revenue from electricity sales. This indicates the level of power fluctuations in wind and hydropower on the transmission lines to the system, i.e., the expected hydropower output to smooth out the fluctuations in wind power output.

6. The water-wind complementary optimal scheduling method according to claim 5, characterized in that, The constraints of the water-wind stochastic optimization scheduling model include: Upper and lower limits of hydropower generation capacity constraints: in, This indicates the lower limit of the hydropower unit's output. Indicates the upper limit of the hydropower unit's output; Average power generation water flow upper and lower limits constraints: Among them, Q t,h This represents the average water flow rate for power generation during time period t. This indicates the lower limit of average water consumption for power generation. This indicates the upper limit of average water consumption for power generation; Water level upper and lower limits constraints: Among them, Z t,h This represents the water level during time period t. Indicates the lower limit of the water level. Indicates the upper limit of the water level; Storage capacity upper and lower limits constraints: Among them, V t,h This represents the storage capacity during time period t. Indicates the lower limit of storage capacity. Indicates the maximum storage capacity; Transmission line capacity constraints: P t,h +P t,w ≤P TL Among them, P TL This indicates the maximum transmission power of the power transmission line; Load constraints: P t,h +P t,w ≥L t ; Water level constraints at the end of the period: |Z T,h -WITH 1,h |≤Z b Among them, Z T,h Z represents the storage capacity at the end of the period. 1,h Z represents the initial storage capacity during the time period. b Indicates the water level error threshold; Uncertainty constraints of wind power: Hydropower reserves positive and negative backups for wind power capacity, including: Where α represents the confidence level. This represents the minimum wind power output at confidence level α. P represents the maximum wind power output at confidence level α. r Represents probability; The relationship between hydropower generation capacity and average power generation flow, reservoir capacity, and water level includes: P t,h =AQ t,h H t Q t,e Δt+V t,h =V t+1,h With t,h =f(V t,h ) Among them, P t,h H represents the output of the hydropower unit during time period t; A represents the output coefficient of the hydropower unit; H t Q represents the net head of the hydropower unit during time period t; t,e V represents the average water flow rate of the reservoir during time period t; a negative value indicates that water is being drawn from the reservoir for power generation. Δt represents the length of the time period. t+1,h represents the reservoir capacity during the (t+1)th time period; f(*) represents the reservoir capacity-water level function relationship.

7. The water-wind complementary optimal scheduling method according to claim 6, characterized in that, The process of converting the water-wind stochastic optimization scheduling model into an environment suitable for reinforcement learning interaction, and setting relevant states, actions, and rewards, includes: Define a state, where state S t It is a sequence of length T+1, including: S t =[N t ,V 1,h ,V 2,h ,...,V t,h ,...,V T,h ] Among them, V t,h This represents the storage capacity at time t, after which the storage capacity is initialized to 0. Define an action, where action A t The average hydropower generation water flow rate in the t-th time period includes: A t =Q t,h Among them, Q t,h This represents the average water flow rate for power generation during time period t; Define rewards, which include power generation rewards, power fluctuation penalties, and end-point water level penalties. The expression for the rewards includes: Where R1 represents the power generation reward, R2 represents the power fluctuation penalty, R3 represents the end-point water level penalty, R represents the total reward, k1 represents the power generation reward parameter, k2 represents the power fluctuation penalty parameter, and k3 represents the end-point water level penalty parameter; By adjusting the values ​​of k1, k2, and k3, we ensure that different rewards are on the same order of magnitude, enabling the Long Short-Term Memory (LSM) network xLSTM to combine the rewards from each part and make the most appropriate action.

8. The water-wind complementary optimal scheduling method according to claim 7, characterized in that, The dual-delay deep deterministic policy gradient algorithm includes a policy network and an evaluation network. The policy network is used to make the next decision based on the current environment state, and the evaluation network is used to judge the quality of the decision made by the policy network based on the current state. The evaluation network includes a first online evaluation network and a second online evaluation network. The method xLSTM-TD3, which combines a long short-term memory network and a dual-delay deep deterministic policy gradient algorithm, is used to solve the water-wind stochastic optimization scheduling model and obtain a water-wind complementary optimization scheduling scheme, including: The Long Short-Term Memory (xLSTM) network is used to continuously give actions based on the current state of the environment, and obtain new states and rewards after interacting with the environment. The online policy network weights are updated to form the target policy network. Update methods include: Where Q1 represents the first online evaluation network, W1 represents the network parameters of the first online evaluation network, m represents the sample size, and J(θ) represents the loss function of the online policy network. π represents the gradient of the online policy network weights; π represents the online action network; θ represents the network parameters of the online action network. The network parameters of the online evaluation network are updated according to the temporal difference algorithm, including: Among them, J(w j ) represents the loss function of the online evaluation network; y t This represents the target Q value obtained using the time-difference algorithm; Q j Indicates an online evaluation network; w j This represents the network parameters of the online evaluation network; Q′ j The network represents the target evaluation network, where j represents the evaluation network number; π′ represents the target action network; θ′ represents the network parameters of the target action network; w′ j R represents the network parameters of the target evaluation network. t Represents the total reward at time t; γ represents the decay factor; The parameters of the target network are not used in training, but are updated periodically from the online network based on a soft update strategy to ensure the stability of the online network updates, i.e.: Where τ represents the update factor; The long short-term memory network xLSTM is trained according to the preset training algebra, and the water-wind complementary optimal scheduling scheme is obtained based on the trained long short-term memory network xLSTM.

9. A computer device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when loaded and executed by the processor, implements the water-wind complementary optimal scheduling method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The system stores a program that, when loaded by a processor, implements the water-wind complementary optimal scheduling method as described in any one of claims 1 to 8.

Images