Water-wind complementary optimization scheduling method, equipment and medium

Abstract

The invention provides a water-wind complementary optimal scheduling method, equipment and a medium, and relates to the technical field of water-wind optimal scheduling of a power system. The method comprises the following steps: predicting wind power according to a TPA-BiLSTM model, and obtaining wind power prediction uncertainty related data according to prediction data and historical data; according to a probability density function of the established universal distribution model of the two-dimensional interval, wind power prediction uncertainty is converted into deterministic hydroelectric power positive and negative standby constraints, and a water-wind random optimization scheduling model is established; and converting the water-wind random optimization scheduling model into an environment suitable for reinforcement learning interaction, setting related states, actions and rewards, solving the water-wind random optimization scheduling model by using an xLSTM-TD3 algorithm, and obtaining a water-wind complementary optimization scheduling scheme. Compared with a traditional optimization scheduling method and a classical reinforcement learning method, the method can give consideration to the solving efficiency and precision, finds a global optimal solution, and effectively improves the processing efficiency and the calculation precision.

Description

Technical Field

[0001] The present invention relates to the technical field of water-wind optimization scheduling of power systems, and in particular to a water-wind complementary optimization scheduling method, equipment and medium. Background Art

[0002] It is imperative to vigorously develop clean energy such as wind power and hydropower and build a new power system. The output of new energy has the characteristics of uncertainty and volatility, and faces a series of challenges in the dispatching of power systems. With the continuous increase in wind power installed capacity, the uncertainty, volatility and randomness of wind power have brought great difficulties to the dispatching and control of power grids. In order to avoid the difficulty of power grid dispatching caused by the output characteristics of wind power, a large-capacity energy that can be adjusted quickly is needed as compensation. The high installed capacity and fast ramp rate of hydropower just meet these characteristics, and it is suitable for dispatching with wind power to balance the uncertainty brought by wind power. By rationally planning the capacity of hydropower and wind power, using wind power to avoid the power supply shortage of hydropower in the dry season, and realizing complementary dispatching between water and wind, it is possible to effectively utilize hydropower and wind energy resources, replace fossil fuels and improve economic benefits.

[0003] The research on water-wind complementary optimization scheduling mainly includes two aspects: the establishment of water-wind complementary optimization scheduling model and the solution of the model. How to characterize the uncertainty of wind power prediction and comprehensively consider the interests of power generation and scheduling are the key to establishing a reasonable water-wind complementary optimization scheduling model. The water-wind complementary optimization scheduling model is usually more complex, with more constraints and strong nonlinearity, and a suitable algorithm is needed to solve the model. Many scholars use heuristic learning methods to solve the model. However, traditional intelligent algorithms are inefficient in dealing with complex high-dimensional problems and are easily trapped in local optimality.

[0004] Traditional physical models and statistical learning methods are difficult to accurately predict the changing trend of wind power, while deep learning, with its powerful learning ability, can effectively predict wind power under the condition of sufficient samples. In addition, as the complexity of the scheduling model continues to increase, how to balance the search efficiency and solution accuracy in a larger independent variable space to find the global optimal solution is also a difficult problem. Reinforcement learning is naturally good at dealing with timing problems, especially when dealing with high-dimensional action space problems. Compared with traditional optimization algorithms, it has stronger global search capabilities and is not easy to fall into local optimality. It can be seen that the use of deep reinforcement learning (DRL) algorithm can effectively improve the processing efficiency of water-wind complementary optimization scheduling problems. Summary of the invention

[0005] The present invention aims to at least solve one of the technical problems in the prior art that the existing water-wind complementary optimization scheduling method has poor calculation speed and solution accuracy, low processing efficiency, and is easy to fall into local optimality.

[0006] To this end, the first aspect of the present invention provides a water-wind complementarity optimization scheduling method.

[0007] A second aspect of the present invention provides a computer device.

[0008] A third aspect of the present invention provides a computer-readable storage medium.

[0009] The present invention provides a water-wind complementary optimization scheduling method, comprising:

[0010] Wind power is predicted based on the neural network model TPA-BiLSTM that combines the temporal pattern attention mechanism and the bidirectional long short-term memory network, and the wind power prediction uncertainty-related data is obtained based on the predicted data and historical data; wherein, since the wind power uncertainty has significant characteristics in the two dimensions of power and time, the probability density function of the wind power prediction error is established for different power-time dimension intervals, wherein the general distribution model is used to establish the probability density function of the wind power uncertainty data in different intervals;

[0011] According to the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power forecast is converted into deterministic positive and negative reserve constraints of hydropower power, and a hydro-wind random optimization scheduling model is established; wherein, the hydro-wind power economy and line power fluctuation are used as the optimization objectives of the hydro-wind random optimization scheduling model, and constraints are set;

[0012] The water-wind random optimization scheduling model is converted into an environment suitable for reinforcement learning interaction, and the relevant states, actions and rewards are set. The water-wind random optimization scheduling model is solved using the xLSTM-TD3 method that combines the long short-term memory network and the double-delay deep deterministic policy gradient algorithm to obtain the water-wind complementary optimization scheduling plan.

[0013] The water-wind complementary optimization scheduling method according to the above technical solution of the present invention may also have the following additional technical features:

[0014] In the above technical solution, according to the significant characteristics of wind power uncertainty in the two dimensions of power and time, the probability density function of wind power prediction error is established for different power-time dimension intervals, including:

[0015] Performing power-time two-dimensional interval division, wherein power interval division and time interval division are performed separately;

[0016] A general distribution model is used to establish the probability density function of wind power uncertainty data in each interval; the general distribution model has a closed cumulative distribution function form and can fit wind power uncertainty distribution of any shape; the expressions of the probability density function and cumulative distribution function of the general distribution model are:

[0017]

[0018] F(xa,b,c)=(1+e -a(x-c) ) -b

[0019] Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; a, b, c all represent adjustable shape parameters;

[0020] The prediction error confidence interval is obtained by using the inverse function of the cumulative distribution function; the expression of the inverse function of the cumulative distribution function is:

[0021]

[0022] Among them, α represents the confidence level.

[0023] In the above technical solution, the power interval division includes:

[0024] Determine the number of divided power intervals so that the number of samples in each power interval tends to be the same;

[0025] The time interval division includes:

[0026] Preliminarily determine the time interval division interval, divide it into several time intervals, and merge the time intervals according to the similarity of data distribution between different time intervals; the similarity of data distribution is expressed by relative entropy:

[0027]

[0028] Among them, KL(P||Q) represents relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different random variable values.

[0029] In the above technical solution, the power interval division includes:

[0030] Determine the number of divided power intervals so that the number of samples in each power interval tends to be the same;

[0031] The time interval division includes:

[0032] Preliminarily determine the time interval division interval, divide it into several time intervals, and merge the time intervals according to the similarity of data distribution between different time intervals; the similarity of data distribution is expressed by JS divergence:

[0033]

[0034] Among them, JS(P||Q) represents JS divergence; KL(*) represents relative entropy; P and Q represent two different data distributions respectively.

[0035] In the above technical solution, the objective function of the water-wind stochastic optimization scheduling model includes:

[0036]

[0037] Among them, N t represents the local electricity price in time period t; represents the electricity price delivered during time period t, L t represents the local load in period t; P t,w represents the wind power generated at time t; P t,h represents the hydropower generated at time t; T represents the scheduling period; represents the average value of hydropower and wind power generation in the entire dispatching period; k represents the weight parameter; N t L t Represents the benefits of power generation to supply local loads, It represents the benefit generated by the transmission of excess power generation. The sum of these two items represents the total revenue from electricity sales. It represents the power fluctuation level of wind power and hydropower on the lines sent to the system, that is, hydropower power is expected to smooth the volatility of wind power.

[0038] In the above technical solution, the constraints of the water-wind stochastic optimization scheduling model include:

[0039] The upper and lower limits of the hydropower unit power generation capacity:

[0040]

[0041] in, Indicates the lower limit of the hydropower unit output. Indicates the upper limit of the output of the hydropower unit;

[0042] The upper and lower limits of the average water flow for power generation are:

[0043]

[0044] Among them, Q t,h represents the average water flow for power generation in period t, Indicates the lower limit of the average water flow for power generation, It indicates the upper limit of the average water flow for power generation;

[0045] Water level upper and lower limit constraints:

[0046]

[0047] Among them, Z t,h represents the water level at time period t, Indicates the lower limit of water level. Indicates the upper limit of the water level;

[0048] Storage capacity upper and lower limit constraints:

[0049]

[0050] Among them, V t,h represents the storage capacity at time period t, Indicates the lower limit of storage capacity, Represents the upper limit of storage capacity; transmission line capacity constraint:

[0051] P t,h +P t,w ≤P TL

[0052] Among them, P TL Indicates the maximum transmission power of the transmission line;

[0053] Load constraints:

[0054] P t,h +P t,w ≥L t ;

[0055] Water level constraints at the end of the period:

[0056] |Z T,h -Z 1,h |≤Z b

[0057] Among them, Z T,h represents the storage capacity at the end of the period, Z 1,h represents the storage capacity in the initial period, Z b Indicates the water level error threshold;

[0058] Wind power uncertainty constraints:

[0059] Hydropower reserves positive and negative reserves for wind power, including:

[0060]

[0061] Among them, α represents the confidence level, represents the minimum value of wind power under the confidence level α, represents the maximum value of wind power under the condition of confidence level α, P r represents probability;

[0062] Among them, the relationship between hydropower generation power and average power generation flow, reservoir capacity and water level includes:

[0063] P t,h =AQ t,h H t

[0064] Q t,e Δt+V t,h =V t+1,h

[0065] Z t,h =f(V t,h )

[0066] Among them, P t,h represents the output of the hydropower unit in time period t; A represents the output coefficient of the hydropower unit; H t represents the net water head of the hydropower unit in period t; Q t,e It represents the average water flow of the reservoir in time period t. When it is negative, it means that the water storage is used to generate electricity from the reservoir. Δt represents the length of the time period. V t+1,h represents the storage capacity in the t+1th period; f(*) represents the functional relationship between the reservoir capacity and water level.

[0067] In the above technical solution, the water-wind random optimization scheduling model is converted into an environment suitable for reinforcement learning interaction, and the relevant states, actions and rewards are set, including:

[0068] Define the state, where state S t is a sequence of length T+1, consisting of:

[0069] S t =[N t ,V 1,h ,V 2,h ,...,V t,h ,...,V T,h ]

[0070] Among them, V t,h Represents the storage capacity at the tth period, and the storage capacity after that is initialized to 0;

[0071] Define actions, where action A t is the average water flow rate for hydropower generation in the tth period, including:

[0072] A t =Q t,h

[0073] Among them, Qt,h represents the average water flow for power generation in period t;

[0074] A reward is defined, which includes a power generation reward, a power fluctuation penalty, and a terminal water level penalty. The expression of the reward includes:

[0075]

[0076] Among them, R 1 represents the power generation reward, R 2 represents the power fluctuation penalty, R 3 represents the terminal water level penalty, R represents the total reward, k 1 represents the power generation reward parameter, k 2 represents the power fluctuation penalty parameter, k 3 Represents the terminal water level penalty parameter;

[0077] By adjusting k 1 , k 2 and k 3 The size of ensures that different rewards are at the same order of magnitude, allowing the long short-term memory network xLSTM to comprehensively consider the rewards of each part and make the most appropriate action.

[0078] In the above technical solution, the dual-delay deep deterministic policy gradient algorithm includes a policy network and an evaluation network, wherein the policy network is used to make the next decision according to the current environment state, and the evaluation network is used to judge the quality of the decision made by the policy network according to the current state, and the evaluation network includes a first online evaluation network and a second online evaluation network;

[0079] The method xLSTM-TD3 combining the long short-term memory network and the double-delayed deep deterministic policy gradient algorithm is used to solve the water-wind random optimization scheduling model and obtain the water-wind complementary optimization scheduling plan, including:

[0080] Use the long short-term memory network xLSTM to continuously give actions based on the current state of the environment, and obtain new states and rewards after interacting with the environment;

[0081] Update the online policy network weights to form the target policy network. The update method includes:

[0082]

[0083] Among them, Q 1 represents the first online evaluation network, w 1 represents the network parameters of the first online evaluation network, m represents the sample size; J(θ) represents the loss function of the online strategy network; represents the gradient of the online policy network weight; π represents the online action network; θ represents the network parameters of the online action network;

[0084] Update the network parameters of the online evaluation network according to the temporal difference algorithm, including:

[0085]

[0086] Among them, J(w j ) represents the loss function of the online evaluation network; y t represents the target Q value obtained by the temporal difference algorithm; Q j represents the online evaluation network; w j represents the network parameters of the online evaluation network; Q′ j represents the target evaluation network, j represents the evaluation network number; π′ represents the target action network; θ' represents the network parameters of the target action network; w' j represents the network parameters of the target evaluation network; R t represents the total reward at time t; γ represents the decay factor;

[0087] The parameters of the target network are not involved in the training, but are regularly updated from the online network based on the soft update strategy to ensure the stability of the online network update, namely:

[0088]

[0089] Among them, τ represents the update factor;

[0090] The long short-term memory network xLSTM is trained according to the preset training algebra, and the water-wind complementary optimization scheduling plan is obtained according to the trained long short-term memory network xLSTM.

[0091] The present invention provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is loaded and executed by the processor, a water-wind complementary optimization scheduling method as described in any one of the above technical solutions is implemented.

[0092] The present invention provides a computer-readable storage medium storing a program, which, when loaded by a processor, implements the water-wind complementary optimization scheduling method as described in any one of the above technical solutions.

[0093] In summary, due to the adoption of the above technical features, the beneficial effects of the present invention are:

[0094] The present invention adopts power-time two-dimensional interval division and general distribution to characterize the uncertainty of wind power prediction, and establishes a water-wind complementary optimization scheduling model, which has a more refined characterization of the model and more constraints. Finally, the xLSTM-TD3 algorithm based on reinforcement learning is used to solve the water-wind random optimization scheduling model. The results show that compared with traditional heuristic learning methods and DDPG algorithm (deep reinforcement learning algorithm for solving continuous action problems), the present invention can find a better solution with higher efficiency.

[0095] Specifically, the present invention establishes a water-wind random optimization scheduling model based on a random optimization method, and solves it using a double-delayed deep deterministic policy gradient algorithm. First, the established scheduling model is converted into a reinforcement learning environment, and the corresponding actions, rewards, and states are constructed. The xLSTM-TD3 algorithm is used to interact with the environment, train the policy network and the evaluation network, and give the final optimization scheduling solution. Compared with traditional optimization scheduling methods and classical reinforcement learning methods, the xLSTM-TD3 algorithm can take into account both solution efficiency and accuracy to find a global optimal solution, effectively improving the processing efficiency and calculation accuracy of the water-wind complementary optimization scheduling problem.

[0096] Additional aspects and advantages of the present invention will become apparent from the following description or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0098] Figure 1 It is a schematic diagram of the network structure of the Actor network and the Critic network in the water-wind complementary optimization scheduling method according to an embodiment of the present invention;

[0099] Figure 2 This is a flow chart of random optimization scheduling of water and wind based on xLSTM-TD3 in a water-wind complementary optimization scheduling method according to an embodiment of the present invention;

[0100] Figure 3 It is a schematic diagram of a power generation scheme obtained by using the xLSTM-TD3 algorithm in a water-wind complementary optimization scheduling method according to an embodiment of the present invention;

[0101] Figure 4 It is a schematic diagram of water level and electricity price curves in a water-wind complementary optimization scheduling method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0102] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0103] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0104] Refer to the following Figures 1 to 4 To describe the water-wind complementary optimization scheduling method provided according to some embodiments of the present invention.

[0105] Some embodiments of the present application provide a method for optimizing the scheduling of water-wind complementarity.

[0106] The first embodiment of the present invention proposes a water-wind complementary optimization scheduling method, which mainly includes three parts S1-S3.

[0107] S1. Predict wind power based on the neural network model TPA-BiLSTM (hereinafter referred to as the TPA-BiLSTM model) that combines the temporal pattern attention mechanism and the bidirectional long short-term memory network, and obtain wind power prediction uncertainty related data based on the predicted data and historical data; wherein, according to the significant characteristics of wind power uncertainty in the two dimensions of power and time, probability density functions of wind power prediction errors are established for different power-time dimension intervals, wherein a general distribution model is used to establish probability density functions of wind power uncertainty data in different intervals.

[0108] It should be noted that the TPA-BiLSTM model is one of the commonly used models for predicting wind power. The specific method of use is well known to those skilled in the art and will not be repeated here. It is understandable that those skilled in the art can also use other suitable prediction models to make preliminary predictions on wind power to obtain prediction data, such as the AutoRegressive Moving Average (ARMA) model, the two-dimensional Jensen model, and the use of artificial neural networks (ANN) to dynamically model multivariate time series to achieve wind power prediction.

[0109] In order to use the stochastic optimization method to solve the uncertainty problem of wind power prediction in the water-wind optimization scheduling model, it is necessary to establish the PDF (probability density function) of wind power prediction error based on historical data to obtain the prediction error confidence interval. However, directly establishing the same PDF for the entire wind power prediction error data set will result in a confidence interval that is too wide, greatly reducing the economic efficiency of scheduling. Studies have found that wind power uncertainty usually has obvious characteristics in the two dimensions of power and time. Therefore, multiple PDFs can be established for different power-time dimension intervals to improve the economy of the scheduling process.

[0110] In some embodiments, according to the significant characteristics of wind power uncertainty in the two dimensions of power and time, the probability density functions of wind power prediction errors are established for different power-time dimension intervals, including:

[0111] Perform power-time two-dimensional interval division, in which power interval division and time interval division are performed separately. The number of power-time two-dimensional interval divisions should be reasonable. If the number of intervals is too large, the number of samples in some intervals may be too small, and the true prediction error PDF cannot be fitted; if the number of intervals is too small, the dispatch economy may deteriorate.

[0112] Specifically, for power interval division, the number of divided power intervals is first determined, and then the number of samples in each power interval is made the same as much as possible.

[0113] For time interval division, first preliminarily determine the time interval division interval, then divide it into several time intervals, and merge the time intervals according to the similarity of data distribution between different time intervals.

[0114] In a specific embodiment, the similarity of data distribution during the time interval division process can be expressed by relative entropy (Kullback-Leibler divergence, KLD), specifically:

[0115]

[0116] Among them, KL(P||Q) represents relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different random variable values.

[0117] However, KLD is asymmetric, which is not conducive to determining the distance between two distributions. Therefore, the symmetric JS divergence can be used to measure the similarity between the two distributions. In another specific embodiment, the similarity of data distribution in the time interval division process is expressed by JS divergence:

[0118]

[0119] Among them, JS(P||Q) represents JS divergence; KL(*) represents relative entropy; P and Q represent two different data distributions respectively.

[0120] That is, when dividing the time interval, it is necessary to first determine the probability distribution of each prediction error among several equally divided small areas and then determine the JS divergence of the prediction error distribution in different time periods.

[0121] In order to establish PDF for wind power uncertainty data under different distributions, a distribution model that can describe different shapes is required. In the embodiment of the present disclosure, a general distribution model is used to establish the probability density function of wind power uncertainty data in each interval; the general distribution model has a closed cumulative distribution function form and can fit any shape of wind power uncertainty distribution; the probability density function and cumulative distribution function of the general distribution model are expressed as:

[0122]

[0123] F(xa,b,c)=(1+e -a(x-c) ) -b

[0124] Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; a, b, c all represent adjustable shape parameters;

[0125] The prediction error confidence interval is obtained by using the inverse function of the cumulative distribution function; the expression of the inverse function of the cumulative distribution function is:

[0126]

[0127] Among them, α represents the confidence level. Since the general probability distribution has an explicit inverse CDF (cumulative distribution function), the confidence interval can be easily obtained. When fitting the uncertainty of wind power, the least squares method or gradient descent method can be used to obtain the three shape parameters of the general distribution.

[0128] After obtaining the wind power prediction error PDF in each interval, the uncertainty of wind power can be quantified based on it, and then a water-wind stochastic optimization scheduling model can be established.

[0129] S2. The hydro-wind random optimization dispatch model dispatches hydropower and wind power as a whole, which not only needs to fully meet the local load, but also the surplus power can be sent to other areas through transmission lines. According to the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power prediction is converted into deterministic hydropower positive and negative reserve constraints, and a hydro-wind random optimization dispatch model is established; among them, the hydro-wind power economy and line power fluctuation are used as the optimization objectives of the hydro-wind random optimization dispatch model, and constraints are set.

[0130] In some embodiments, the objective function of the water-wind stochastic optimization scheduling model includes:

[0131]

[0132] Among them, N t represents the local electricity price in time period t; represents the electricity price delivered during time period t, L t represents the local load in period t; P t,w represents the wind power generated at time t; P t,h represents the hydropower generated at time t; T represents the dispatching period, which is dispatched once every hour for day-ahead dispatching, i.e., T = 24; represents the average value of hydropower and wind power generation in the entire dispatching period; k represents the weight parameter; N t L t Represents the benefits of power generation to supply local loads, It represents the benefit generated by the transmission of excess power generation. The sum of these two items represents the total revenue from electricity sales. It represents the power fluctuation level of wind power and hydropower on the lines sent to the system, that is, hydropower power is expected to smooth the volatility of wind power.

[0133] In some embodiments, the constraints of the water-wind stochastic optimization scheduling model include upper and lower limit constraints on the power generation of hydropower units, upper and lower limit constraints on the average water flow for power generation, upper and lower limit constraints on water levels, upper and lower limit constraints on reservoir capacity, transmission line capacity constraints, load constraints, water level constraints at the end of the period, and wind power uncertainty constraints.

[0134] The upper and lower limits of the hydropower unit power generation capacity:

[0135]

[0136] in, Indicates the lower limit of the hydropower unit output. Indicates the upper limit of the hydropower unit output.

[0137] The upper and lower limits of the average water flow for power generation are:

[0138]

[0139] Among them, Q t,h represents the average water flow for power generation in period t, Indicates the lower limit of the average water flow for power generation, Indicates the upper limit of the average water flow rate for power generation.

[0140] Water level upper and lower limit constraints:

[0141]

[0142] Among them, Z t,h represents the water level at time period t, Indicates the lower limit of water level. Indicates the upper limit of the water level.

[0143] Storage capacity upper and lower limit constraints:

[0144]

[0145] Among them, V t,h represents the storage capacity at time period t, Indicates the lower limit of storage capacity, Indicates the upper limit of storage capacity.

[0146] Transmission line capacity constraints:

[0147] P t,h +P t,w ≤P TL

[0148] Among them, P TL Indicates the maximum transmission power of the transmission line.

[0149] Load constraints:

[0150] P t,h +P t,w ≥L t

[0151] Under normal circumstances, hydropower and wind power must at least meet local loads.

[0152] Water level constraints at the end of the period:

[0153] |Z T,h -Z 1,h |≤Z b

[0154] Among them, Z T,h represents the storage capacity at the end of the period, Z 1,h represents the storage capacity in the initial period, Z b Indicates the water level error threshold. The water level at the end of the reservoir period should not differ much from the water level at the beginning of the period to facilitate the scheduling of the next cycle.

[0155] Wind power uncertainty constraints:

[0156] Considering the uncertainty of wind power, its prediction is difficult to be completely accurate. Hydropower generation must be able to make up for the shortfall when the wind power forecast is too high, and also reduce its power generation when the wind power forecast is too low to prevent wind abandonment. Therefore, hydropower needs to reserve positive and negative reserves for wind power, including:

[0157]

[0158] Among them, α represents the confidence level, represents the minimum value of wind power under the confidence level α, represents the maximum value of wind power under the condition of confidence level α, P r represents probability;

[0159] Among them, the relationship between hydropower generation power and average power generation flow, reservoir capacity and water level includes:

[0160] P t,h =AQ t,h H t

[0161] Q t,e Δt+V t,h =V t+1,h

[0162] Z t,h =f(V t,h )

[0163] Among them, P t,h represents the output of the hydropower unit in time period t; A represents the output coefficient of the hydropower unit; H t represents the net water head of the hydropower unit in period t; Q t,e It represents the average water flow of the reservoir in time period t. When it is negative, it means that the water storage is used to generate electricity from the reservoir. Δt represents the length of the time period. V t+1,h represents the storage capacity in the t+1th period; f(*) represents the functional relationship between the reservoir capacity and water level.

[0164] The water-wind random optimization scheduling model established in the present disclosure is relatively complex. Traditional optimization methods may easily fall into local optimal solutions and have low solution efficiency. Common reinforcement learning methods such as Q-learning (Q learning) and DQN (Deep Q-Network, an existing reinforcement learning algorithm based on deep learning) require continuous actions to be discretized for solution, which makes it difficult to balance solution accuracy and operation speed. Therefore, the present disclosure adopts a method that combines the long short-term memory network (xLSTM) and the double-delay deep deterministic policy gradient algorithm (TD3) (hereinafter referred to as the xLSTM-TD3 algorithm), which can directly solve the problem of continuous actions and effectively balance calculation accuracy and efficiency.

[0165] S3. Convert the water-wind random optimization scheduling model into an environment suitable for reinforcement learning interaction, set relevant states, actions and rewards, and use the xLSTM-TD3 algorithm to solve the water-wind random optimization scheduling model to obtain the water-wind complementary optimization scheduling plan.

[0166] Among them, the long short-term memory network xLSTM (Xigmoid-based Long Short-Term Memory) is constructed on the basis of LSTM, and its construction steps are as follows:

[0167] Replace the activation function sigmoid of the forget gate of LSTM with xigmoid. Replacing the sigmoid of the forget gate with xigmoid allows the forget gate of LSTM to continue training, thereby achieving long-term memory.

[0168] The input gate i of LSTM t Replace with (1-f t ), on the one hand, it can save the amount of learnable parameters of LSTM and speed up the training of the model; on the other hand, adding more prior knowledge to the model can reduce the size of the search space of model parameters, reduce the difficulty of model learning, avoid falling into non-optimal solutions in a larger space during model training, and reduce the amount of training data and the number of training rounds required for convergence.

[0169] The double-delayed deep deterministic policy gradient algorithm TD3 is an improved deep reinforcement learning algorithm based on DDPG, which solves the over-estimation problem of DDPG to a certain extent. TD3 mainly consists of two networks: one is the policy network (Actor network), which makes the next decision based on the current environment state; the other is the evaluation network (Critic network), which judges the quality of the Actor network's decision based on the current state. Among them, the online Actor network selects actions based on the current state and its network parameters:

[0170] A t =π(S t |θ)

[0171] Among them, S t is the current state, A t is the decision behavior made by the online Actor network based on the current state, π is the online action network, and θ is the network parameter.

[0172] The combination of xLSTM-TD3 can improve the performance of deep reinforcement learning agents, especially in tasks where the state space exhibits temporal dependencies or sequential structure. By leveraging the ability of xLSTM to capture long-term dependencies, agents may be better able to learn complex behaviors and make smarter decisions in dynamic environments.

[0173] In order to use the xLSTM-TD3 algorithm to solve the water-wind stochastic optimization scheduling model, it is necessary to define the state, action and reward so that the xLSTM network can effectively learn from the environment to obtain the optimal scheduling solution.

[0174] In some embodiments, the water-wind random optimization scheduling model is converted into an environment suitable for reinforcement learning interaction, and related states, actions, and rewards are set, including:

[0175] Define the state, where state S t is a sequence of length T+1, consisting of:

[0176] S t =[N t ,V 1,h ,V 2,h ,...,V t,h ,...,V T,h ]

[0177] Among them, V t,h represents the normalized storage capacity at the tth period, and the storage capacity after that is initialized to 0; define actions, where action A t is the average water flow rate for hydropower generation in the tth period, including:

[0178] A t =Q t,h

[0179] Among them, Q t,h A represents the average water flow for power generation in period t; t It is a continuous variable and needs to satisfy the upper and lower limits of the average water flow for power generation in the constraints.

[0180] Define rewards, which include power generation rewards, power fluctuation penalties, and terminal water level penalties. The power generation rewards will be returned after each step, while the power fluctuation penalties and terminal water level penalties will only be returned after the last step. The expressions of rewards include:

[0181]

[0182] Among them, R 1 represents the power generation reward, R 2 represents the power fluctuation penalty, R 3 represents the terminal water level penalty, R represents the total reward, k 1 represents the power generation reward parameter, k 2 represents the power fluctuation penalty parameter, k 3 Represents the terminal water level penalty parameter;

[0183] By adjusting k 1 , k 2 and k 3 The size of ensures that different rewards are in the same order of magnitude, so that the long short-term memory network xLSTM can combine the rewards of each part to make the most appropriate action. During the interaction between the xLSTM network and the environment, it will check whether the constraints are met. If they are not met, it will return a large negative reward and end the interaction.

[0184] In some embodiments, the dual-delay deep deterministic policy gradient algorithm includes a policy network (Actor network) and an evaluation network (Critic network), wherein the policy network is used to make the next decision based on the current environment state, and the evaluation network is used to judge the quality of the decision made by the policy network based on the current state, and the evaluation network includes a first online evaluation network and a second online evaluation network.

[0185] The structure of the policy network and evaluation network is as follows Figure 1 As shown in the figure, the hidden layers of the Actor network and the Critic network are both fully connected (FC). In order to enable the Actor network to judge the current hydropower generation, the current electricity price and the hydropower generation in each period are sent to the Actor network as state quantities. The Actor network uses the Dense layer (fully connected layer) to process these two quantities and extract information, and then uses the Dense layer to give the hydropower generation value of the current scheduling period after merging. The Critic network uses multiple fully connected layers combined with electricity prices and scheduling plans for each period to evaluate the value of the current Actor based on the current state and the action value of the Actor network. The Critic network trains the neural network parameters based on the reward of the current action and the time series difference algorithm as the improvement direction, and the Actor network trains the neural network to make the score of the output action value as high as possible.

[0186] In some embodiments, Figure 2As shown, the method xLSTM-TD3 that combines the long short-term memory network and the double-delay deep deterministic policy gradient algorithm is used to solve the water-wind random optimization scheduling model and obtain the water-wind complementary optimization scheduling plan, including the following steps S31-S35.

[0187] Among them, the upper and lower reserves of hydropower that should be reserved are determined according to the predicted wind power and the following formula, and the environment is initialized:

[0188]

[0189] Among them, α represents the confidence level, represents the minimum value of wind power under the confidence level α, represents the maximum value of wind power under the condition of confidence α. It should be noted that and It is necessary to calculate separately according to the PDF of different power-time two-dimensional intervals.

[0190] S31. Use the long short-term memory network xLSTM to continuously give action A according to the state of the current environment. t , obtain new states and rewards after interacting with the environment;

[0191] S32, updating the online policy network weights to form a target policy network, the updating method comprising:

[0192] The online critic network evaluates the actions selected by the action network through a neural network, thereby guiding the weight adjustment of the online action network. The online actor network calculates the gradient of the loss function and network weights based on the gradient boosting method:

[0193]

[0194] Among them, Q 1 represents the first online evaluation network, i.e. Figure 2 The online critic network shown in 1; w 1 represents the network parameters of the first online evaluation network, m represents the sample size; J(θ) represents the loss function of the online strategy network; represents the gradient of the online policy network weight; π represents the online action network; θ represents the network parameters of the online action network;

[0195] S33, updating the network parameters of the online evaluation network according to the temporal difference algorithm, including:

[0196]

[0197] Among them, J(w j ) represents the loss function of the online evaluation network; y trepresents the target Q value obtained by the temporal difference algorithm; Q j represents the online evaluation network, including Figure 2 The online critic network 1 and the online critic network 2 shown in the figure are as follows: j represents the network parameters of the online evaluation network; Q j ′ represents the target evaluation network, including Figure 2 The target critic network 1 and target critic network 2 are shown; j represents the evaluation network number; π′ represents the target action network; θ′ represents the network parameters of the target action network; w′ j represents the network parameters of the target evaluation network; R t represents the total reward at time t; γ represents the decay factor, the larger the value, the more importance is attached to the rewards that may be obtained in the future.

[0198] The TD3 algorithm uses the relatively smaller value of the two target critic networks to construct the target value of the online critic network to solve its overestimation problem. In addition, the critic network is updated more frequently than the actor network. This is because if the critic network's evaluation of the state and action is inaccurate, it cannot effectively guide the actor network to update.

[0199] S34. The parameters of the target network are not trained, but are updated regularly from the online network based on the soft update strategy to ensure the stability of the online network update, that is:

[0200]

[0201] Among them, τ represents the update factor, which reflects the update speed of the target network. Usually, a smaller value such as 0.1 is taken for the training stability of the online network.

[0202] S35, training the long short-term memory network xLSTM according to the preset training algebra, and obtaining the water-wind complementary optimization scheduling plan according to the trained long short-term memory network xLSTM. Figure 2 As shown in , it is determined whether the training generation Epoch is greater than the preset training generation E. If the preset training generation is not reached, the environment is reinitialized to continue training. Otherwise, the training is stopped. The water-wind random optimization scheduling scheme is obtained according to the trained xLSTM network.

[0203] In a specific embodiment, the dispatching results based on the xLSTM-TD3 algorithm are analyzed. The relevant parameters of the dispatching model and the xLSTM-TD3 algorithm are shown in Table 1, the load and wind power forecast values ​​of each dispatching period are shown in Table 2, the electricity price of each dispatching period is shown in Table 3, and the relevant parameters and network structure of the xLSTM-TD3 algorithm are shown in Table 4. The results of the xLSTM-TD3 algorithm are compared with those of the DDPG and GA (genetic algorithm) algorithms and are listed in Table 5.

[0204] Table 1 Parameters of the stochastic optimization dispatch model considering the uncertainty of wind power forecast

[0205]

[0206] Table 2 Load and wind power forecast values ​​for each dispatch period

[0207]

[0208] Table 3 Electricity prices for each dispatch period

[0209]

[0210] Table 4 xLSTM-TD3 algorithm parameters and network structure

[0211]

[0212] Table 5 Scheduling results of each algorithm

[0213]

[0214]

[0215] As can be seen from the table, the xLSTM-TD3 algorithm finally obtained the highest total reward. Compared with the results of the DDPG algorithm, although the power fluctuation is slightly larger, it is exchanged for a better increase in power generation revenue, so the total reward is higher. Compared with the GA algorithm, the power fluctuation is greatly reduced at the expense of a slight loss of power generation revenue. The xLSTM-TD3 algorithm can better avoid falling into the local optimal point and find a better solution. This is because the xLSTM-TD3 algorithm uses two Critic networks and delays the update of the Actor network, avoiding the over-estimation problem in traditional DDPG. At the same time, it also makes the Actor training more stable and more conducive to finding the global optimal point.

[0216] The solution found by the xLSTM-TD3 algorithm is as follows Figure 3 As shown. Figure 3It can be seen that the total power generation of hydropower and wind power is always higher than the load, and the excess power is sent out through the transmission channel. The total power generation is always maintained at a relatively stable level, which is conducive to reducing power fluctuations. In addition, hydropower always reserves sufficient positive and negative reserves, ensuring that the load can be met while also avoiding wind power as much as possible.

[0217] The water level change curve and electricity price curve are as follows: Figure 4 As shown. Figure 4 It can be seen that the water level is always maintained between the highest water level (1330m) and the dead water level (1328m), and the water level at the end of the scheduling period is almost equal to the water level at the beginning of the scheduling period, satisfying the water level-related constraints of the model. In addition, it can be seen that during periods with low electricity prices, such as 10 pm to 7 am, the agent stores water as much as possible during these periods, thereby maximizing the revenue from electricity sales. During periods with higher electricity prices, such as 2 pm to 5 pm and 8 pm to 9 pm, the agent sells electricity as much as possible, and uses the water stored during the previous low electricity price for power generation to increase the revenue from electricity sales. It can be seen from the figure that the water level and electricity price are approximately inversely proportional, but not simply linearly, which shows that the agent is trying to find a nonlinear optimal solution as much as possible under the premise of satisfying the constraints of the model.

[0218] In summary, the established model is solved based on the xLSTM-TD3 algorithm. The results show that the xLSTM-TD3 algorithm can effectively avoid over-estimation and improve the stability of the algorithm with its dual Critic network and Actor delayed update strategy. It can effectively jump out of the local optimal solution and further search for a better global solution.

[0219] Some other embodiments of the present invention provide a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is loaded and executed by the processor, the water-wind complementary optimization scheduling method as described in any of the above embodiments is implemented.

[0220] Some further embodiments of the present invention provide a computer-readable storage medium storing a program, which, when loaded by a processor, implements the water-wind complementary optimization scheduling method as described in any of the above embodiments.

[0221] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0222] Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A water-wind complementary optimization scheduling method, characterized in that: include: Wind power is predicted based on the neural network model TPA-BiLSTM that combines the temporal pattern attention mechanism and the bidirectional long short-term memory network, and the wind power prediction uncertainty-related data is obtained based on the predicted data and historical data; wherein, since the wind power uncertainty has significant characteristics in the two dimensions of power and time, the probability density function of the wind power prediction error is established for different power-time dimension intervals, wherein the general distribution model is used to establish the probability density function of the wind power uncertainty data in different intervals; According to the probability density function of the established two-dimensional interval general distribution model, the uncertainty of wind power forecast is converted into deterministic positive and negative reserve constraints of hydropower power, and a hydro-wind random optimization scheduling model is established; wherein, the hydro-wind power economy and line power fluctuation are used as the optimization objectives of the hydro-wind random optimization scheduling model, and constraints are set; The water-wind random optimization scheduling model is converted into an environment suitable for reinforcement learning interaction, and the relevant states, actions and rewards are set. The water-wind random optimization scheduling model is solved using the xLSTM-TD3 method that combines the long short-term memory network and the double-delay deep deterministic policy gradient algorithm to obtain the water-wind complementary optimization scheduling plan.

2. The water-wind complementary optimization scheduling method according to claim 1 is characterized in that: According to the significant characteristics of wind power uncertainty in the two dimensions of power and time, the probability density functions of wind power prediction errors are established for different power-time dimension intervals, including: Performing power-time two-dimensional interval division, wherein power interval division and time interval division are performed separately; A general distribution model is used to establish the probability density function of wind power uncertainty data in each interval; the general distribution model has a closed cumulative distribution function form and can fit wind power uncertainty distribution of any shape; the expressions of the probability density function and cumulative distribution function of the general distribution model are: F(x|a,b,c)=(1+e -a(x-c) ) -b Where f(x|a,b,c) represents the probability density function; F(x|a,b,c) represents the cumulative distribution function; x represents a random variable; a, b, c all represent adjustable shape parameters; The prediction error confidence interval is obtained by using the inverse function of the cumulative distribution function; the expression of the inverse function of the cumulative distribution function is: Among them, α represents the confidence level.

3. The water-wind complementary optimization scheduling method according to claim 2 is characterized in that: The power interval division includes: Determine the number of divided power intervals so that the number of samples in each power interval tends to be the same; The time interval division includes: Preliminarily determine the time interval division interval, divide it into several time intervals, and merge the time intervals according to the similarity of data distribution between different time intervals; the similarity of data distribution is expressed by relative entropy: Among them, KL(P||Q) represents relative entropy; P and Q represent two different data distributions; p(x) and q(x) represent the probabilities corresponding to different random variable values.

4. The water-wind complementary optimization scheduling method according to claim 2 is characterized in that: The power interval division includes: Determine the number of divided power intervals so that the number of samples in each power interval tends to be the same; The time interval division includes: Preliminarily determine the time interval division interval, divide it into several time intervals, and merge the time intervals according to the similarity of data distribution between different time intervals; the similarity of data distribution is expressed by JS divergence: Among them, JS(P||Q) represents JS divergence; KL(*) represents relative entropy; P and Q represent two different data distributions respectively.

5. The water-wind complementary optimization scheduling method according to claim 1 is characterized in that: The objective function of the water-wind stochastic optimization scheduling model includes: Among them, N t represents the local electricity price in time period t; represents the electricity price for delivery during time period t, L t represents the local load in time period t; P t,w represents the wind power generated at time t; P t,h represents the hydropower generated at time t; T represents the dispatching period; represents the average value of hydropower and wind power generation in the entire dispatching period; k represents the weight parameter; N t L t Represents the benefits of power generation to supply local loads, It represents the benefit generated by the transmission of excess power generation. The sum of these two items represents the total revenue from electricity sales. It represents the power fluctuation level of wind power and hydropower on the lines sent to the system, that is, hydropower power is expected to smooth the volatility of wind power.

6. The water-wind complementary optimization scheduling method according to claim 5 is characterized in that: The constraints of the water-wind stochastic optimization scheduling model include: The upper and lower limits of the hydropower unit power generation capacity: in, Indicates the lower limit of the hydropower unit output. Indicates the upper limit of the output of the hydropower unit; The upper and lower limits of the average water flow for power generation are: Among them, Q t,h represents the average water flow for power generation in period t, Indicates the lower limit of the average water flow for power generation, It indicates the upper limit of average water flow for power generation; Water level upper and lower limit constraints: Among them, Z t,h represents the water level at time period t, Indicates the lower limit of water level. Indicates the upper limit of the water level; Storage capacity upper and lower limit constraints: Among them, V t,h represents the storage capacity at time period t, Indicates the lower limit of storage capacity, Indicates the upper limit of storage capacity; Transmission line capacity constraints: P t,h +P t,w ≤P TL Among them, P TL Indicates the maximum transmission power of the transmission line; Load constraints: P t,h +P t,w ≥L t ; Water level constraint at the end of the period: |From T,h -WITH 1,h |≤Z b Among them, Z T,h represents the storage capacity at the end of the period, Z 1,h represents the storage capacity in the initial period, Z b Indicates the water level error threshold; Wind power uncertainty constraints: Hydropower reserves positive and negative reserves for wind power, including: Among them, α represents the confidence level, represents the minimum value of wind power under the confidence level α, represents the maximum value of wind power under the condition of confidence level α, P r represents probability; Among them, the relationship between hydropower generation power and average power generation flow, reservoir capacity and water level includes: P t,h =AQ t,h H t Q t,e Δt+V t,h =V t+1,h With t,h =f(V t,h ) Among them, P t,h represents the output of the hydropower unit in time period t; A represents the output coefficient of the hydropower unit; H t represents the net water head of the hydropower unit in period t; Q t,e It represents the average water flow of the reservoir in time period t. When it is negative, it means that the water storage is used to generate electricity from the reservoir. Δt represents the length of the time period. V t+1,h represents the storage capacity in the t+1th period; f(*) represents the functional relationship between the reservoir capacity and water level.

7. The water-wind complementary optimization scheduling method according to claim 6 is characterized in that: The water-wind random optimization scheduling model is converted into an environment suitable for reinforcement learning interaction, and the relevant states, actions and rewards are set, including: Define the state, where state S t is a sequence of length T+1, consisting of: S t =[N t ,V 1,h ,V 2,h ,...,V t,h ,...,V T,h ] Among them, V t,h Represents the storage capacity at the tth period, and the storage capacity after that is initialized to 0; Define actions, where action A t is the average water flow rate for hydropower generation in the tth period, including: A t =Q t,h Among them, Q t,h represents the average water flow for power generation in period t; A reward is defined, which includes a power generation reward, a power fluctuation penalty, and a terminal water level penalty. The expression of the reward includes: Among them, R1 represents the power generation reward, R2 represents the power fluctuation penalty, R3 represents the terminal water level penalty, R represents the total reward, k1 represents the power generation reward parameter, k2 represents the power fluctuation penalty parameter, and k3 represents the terminal water level penalty parameter; Among them, by adjusting the sizes of k1, k2 and k3 to ensure that different rewards are at the same order of magnitude, the long short-term memory network xLSTM can comprehensively consider the rewards of each part and make the most appropriate action.

8. The water-wind complementary optimization scheduling method according to claim 7 is characterized in that: The dual-delay deep deterministic policy gradient algorithm includes a policy network and an evaluation network, wherein the policy network is used to make the next decision according to the current environment state, and the evaluation network is used to judge the quality of the decision made by the policy network according to the current state, and the evaluation network includes a first online evaluation network and a second online evaluation network; The method xLSTM-TD3 combining the long short-term memory network and the double-delayed deep deterministic policy gradient algorithm is used to solve the water-wind random optimization scheduling model and obtain the water-wind complementary optimization scheduling plan, including: Use the long short-term memory network xLSTM to continuously give actions based on the current state of the environment, and obtain new states and rewards after interacting with the environment; Update the online policy network weights to form the target policy network. The update method includes: Where Q1 represents the first online evaluation network, W1 represents the network parameters of the first online evaluation network, m represents the sample size; J(θ) represents the loss function of the online strategy network; represents the gradient of the online policy network weight; π represents the online action network; θ represents the network parameters of the online action network; Update the network parameters of the online evaluation network according to the temporal difference algorithm, including: Among them, J(w j ) represents the loss function of the online evaluation network; y t represents the target Q value obtained by the temporal difference algorithm; Q j represents the online evaluation network; w j represents the network parameters of the online evaluation network; Q′ j represents the target evaluation network, j represents the evaluation network number; π′ represents the target action network; θ′ represents the network parameters of the target action network; w′ j represents the network parameters of the target evaluation network; R t represents the total reward at time t; γ represents the decay factor; The parameters of the target network are not involved in the training, but are regularly updated from the online network based on the soft update strategy to ensure the stability of the online network update, namely: Among them, τ represents the update factor; The long short-term memory network xLSTM is trained according to the preset training algebra, and the water-wind complementary optimization scheduling plan is obtained according to the trained long short-term memory network xLSTM.

9. A computer device, characterized in that: It comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded and executed by the processor, the water-wind complementary optimization scheduling method as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: A program is stored, and when the program is loaded by a processor, the water-wind complementary optimization scheduling method as described in any one of claims 1 to 8 is implemented.

Images