Power system dynamic economic dispatch decision method and system for reliability improvement

Abstract

This invention discloses a dynamic economic dispatch decision-making method and system for power systems aimed at improving reliability. It constructs a Markov decision process for dynamic economic dispatch; builds a reinforcement learning environment; updates the distributed reinforcement learning neural network; obtains feasible power output under constraints; acquires the total generation cost through an average consensus algorithm; introduces a greedy strategy ε-greedy algorithm, utilizing the agent to maximize the currently obtainable reward, ensuring the agent learns the optimal strategy in complex environments; defines the historical optimal strategy, updates the local operation strategy, making the optimal dispatch strategy monotonically decrease until it reaches its optimum. This invention exhibits strong robustness under fluctuating power demand conditions.

Description

Technical Field

[0001] This invention belongs to the field of electrical engineering technology, specifically relating to a dynamic economic dispatch decision-making method and system for power systems aimed at improving reliability. Background Technology

[0002] In power systems, the Economic Dispatch Problem (EDP) is of great significance. Its main task is to find the optimal allocation of power generation under various constraints to minimize the total generation cost. The main methods for solving EDPs include classical optimization algorithms, intelligent optimization algorithms, and reinforcement learning. Many classical optimization algorithms are only applicable to convex optimization problems, while EDPs are difficult to model mathematically to meet the convexity requirement. Although intelligent optimization algorithms can handle more complex scenarios, their convergence speed is slow, they are prone to getting trapped in local optima, and they usually only find static optimal solutions with low efficiency and weak robustness. In contrast, reinforcement learning focuses on developing flexible strategies in unknown environments, adapting to dynamic changes through continuous exploration and learning, and providing more effective solutions. Depending on whether a single time period is considered, EDPs can be divided into Dynamic EDPs (DEDPs) and Static EDPs (SEDPs). SEDPs only consider economic dispatch within a fixed time period, belonging to the optimization problem of studying a specific time period on a single time plane, ignoring the dynamic characteristics of the power system and the relationship between different time periods, and are difficult to cope with the volatility brought by renewable energy generation. DEDP makes full use of the relationships between various characteristic quantities of the power system at different time points to better cope with load-side volatility and source-side randomness brought about by the access of new energy sources.

[0003] Although existing research uses distributed reinforcement learning algorithms to solve dynamic economic scheduling problems, it does not consider the problem of quickly deciding the output of the most economical generator unit in scenarios with a high proportion of uncertain new energy sources, so as to achieve a balance between active power and output. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a dynamic economic dispatch decision-making method and system for power systems with reliability improvement in view of the shortcomings of the prior art, so as to solve the technical problem of rapid dispatch of power systems under strong uncertainty load fluctuations.

[0005] The present invention adopts the following technical solution:

[0006] A dynamic economic dispatch decision-making method for power systems aimed at improving reliability includes the following steps:

[0007] S1. Construct a Markov decision process for dynamic economic scheduling, and define the state transition probability through a Markov chain to calculate the policy and value function for reinforcement learning.

[0008] S2. Construct a reinforcement learning environment to provide a simulation scenario for the intelligent agent of the power system generator set. The intelligent agent of the power system generator set makes scheduling decisions based on the system state and schedules the generator sets to minimize the total power generation cost.

[0009] S3. Update the distributed reinforcement learning neural network, and the power system generator agent adjusts its strategy based on environmental feedback;

[0010] S4. Obtain feasible power output under constraints;

[0011] S5. Obtain the total power generation cost through the average consensus algorithm, and return the total power generation cost as a reward for reinforcement learning to the power system generator agent;

[0012] S6. Introduce the ε-greedy algorithm, which utilizes the power system generator agent to maximize the current reward, ensuring that the power system generator agent learns the optimal strategy in complex environments.

[0013] S7, Definition The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0014] Preferably, in step S1, the multi-agent Markov decision process for dynamic economic scheduling of a high-proportion renewable energy system is defined as follows:

[0015]

[0016] in, It is a global state space shared by all intelligent agents, set to the maximum output value of the new energy source. With total load D t , m represents the number of new energy sources; It is the action space of agent i, designed as the active power output p of the generator set. i,t ; Let be the reward value for agent i's action; γ∈[0,1] is the discount factor. In multi-agent decision-making, agents obtain information from the global state space. Predicting the action policy of local agent i Simultaneously receive local rewards

[0017] Preferably, in step S2, the reinforcement learning environment includes a state space s t Generator output constraints;

[0018] Each unit agent estimates the total power demand using an average consensus algorithm, which is expressed as follows:

[0019]

[0020] Where L is the graph The Laplace matrix of the communication topology between central nodes. Where, max i l ii The maximum value of the diagonal elements of the Laplace matrix L. The average power is obtained in the k-th iteration;

[0021] Derived from the consensus protocol:

[0022]

[0023] Among them, 1 N It is an N-dimensional column vector with matrix elements of 1, D t Let N be the total load of the power grid at time t, and N be the total number of nodes in the power grid.

[0024] definition To represent the estimated communication topology level of central node i, Each central node obtains the same ∈ through a distributed algorithm;

[0025] After N iterations, there exists Equal to the degree of communication topology.

[0026] Preferably, step S3 specifically includes:

[0027] S301. Design a dual neural network comprising an action network and an evaluation network. The action network is responsible for the output action to be taken under a given power grid condition; the evaluation network assesses the expected cost of the system after taking a certain action. Define... As the state-action value function under policy action π at time slot t, ω is used to establish the reinforcement learning value network estimate. t These are the network parameters, and ω is updated using the mean squared error of the target network as the loss function. t The parameters are used to obtain the minimum value of the state-action value function, thus obtaining an approximate solution for economic scheduling; a distributed method is used to obtain the minimum value of the state-action value function. Solve the problem concerning approximate states and constructing value functions;

[0028] S302, Each generator set's intelligent agent has an action network. Use parameters Approximate optimal policy; select a fully connected multilayer perceptron with two hidden layers as the approximate optimal policy; reinforcement learning state. With agent actions These are the input and output of the action network, respectively. along Compared to Update using the negative gradient.

[0029] Preferably, the loss function is defined as:

[0030]

[0031] in, This refers to calculating the expected value, where γ is the discount factor. This is an estimate of the action value function for the current state;

[0032] Update ω t :

[0033]

[0034] in, C is the learning rate; t Indicates that in state s t Find the minimum total cost from time t to t+1; yes The derivative with respect to t;

[0035] The action network of generator unit i is represented as follows:

[0036]

[0037] in, It is a weight matrix. h1 and h2 are the bias vectors, and h1 and h2 are the number of neurons in the two hidden layers. It is a non-linear activation function.

[0038] Preferably, step S4 specifically includes:

[0039] S401, Each agent makes decisions regarding the power output of each generator. Perform sampling;

[0040] S402. Add a projection controller to the power output distribution; if the constraints are not met, obtain the joint action that satisfies the constraints through projection, transform the problem into an optimization problem, and solve it using a projection optimization algorithm. If the communication topology is undirected and connected, then the equilibrium point of this equation is the optimal solution to the optimization problem.

[0041] Preferably, in step S5, the definition is... and For a local estimate of the average power generation cost at time t, c is the value of each agent i. i,t =F(p i,t );

[0042] Applying the average consistency algorithm, the time costs satisfy:

[0043]

[0044] get The local estimate of the power generation cost based on the average converges to the actual average power generation cost at time slot t, and the total power generation cost is Nc. t .

[0045] Preferably, step S6 specifically includes:

[0046] A greedy ε-greedy algorithm is introduced, where the probability of the optimal action is 1-ε, and the probability of a random exploration action is ε; an exploration strategy is designed. For the current k-th training iteration; with exploration count k ≤ K0, the agent's decision power is determined by:

[0047]

[0048] After the number of explorations k > K0, the agent's decision power is determined by:

[0049]

[0050] in, Indicates in s t The probability of choosing action 'a';

[0051] Initially, the agent's decision-making action is given by the evaluation network, which has already added some noise inside the algorithm before outputting the action, and uses a greedy strategy to ensure exploration performance;

[0052] After training, the action network makes a deterministic decision.

[0053] Preferably, in step S7, p is set as follows: i,t Actions performed in a real-world environment; definition:

[0054]

[0055] Update local operation policy:

[0056]

[0057] in, The optimal action is generated by the evaluation network; p i,t =p i ' ,t It is an exploration of the interaction between action and a real power grid environment. It is the optimal scheduling record during the training process; or p i,t When the decision-making output combination is more economical, the optimal scheduling strategy is... It is updated to reach its optimal state.

[0058] Secondly, embodiments of the present invention provide a dynamic economic dispatch decision-making system for power systems aimed at improving reliability, comprising:

[0059] The module constructs a Markov decision process for dynamic economic scheduling, defines state transition probabilities through Markov chains, and uses them to calculate the policy and value function for reinforcement learning. It also constructs a reinforcement learning environment to provide a simulation scenario for the intelligent agent of the power system generator set. The intelligent agent of the power system generator set makes scheduling decisions based on the system state, and schedules the generator sets to minimize the total power generation cost.

[0060] The update module updates the distributed reinforcement learning neural network, and the power system generator agent adjusts its strategy based on environmental feedback to obtain feasible power output under constraints.

[0061] The computation module obtains the total power generation cost through an average consensus algorithm and returns the total power generation cost as a reward for reinforcement learning to the power system generator agent.

[0062] The strategy module introduces the ε-greedy algorithm, which maximizes the current reward that the power system generator agent can obtain, ensuring that the power system generator agent learns the optimal strategy in complex environments.

[0063] Output module, definition The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0064] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned power system dynamic economic dispatch decision-making method for reliability improvement.

[0065] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described power system dynamic economic dispatch decision-making method for reliability improvement.

[0066] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned power system dynamic economic dispatch decision-making method for reliability improvement.

[0067] In a sixth aspect, embodiments of the present invention provide an electronic device including a computer program, wherein when the computer program is executed by the electronic device, it implements the steps of the above-described dynamic economic dispatch decision-making method for power systems aimed at improving reliability.

[0068] Compared with the prior art, the present invention has at least the following beneficial effects:

[0069] A dynamic economic dispatch decision-making method for power systems, geared towards reliability improvement, can better and more quickly address the volatility and randomness brought about by high-proportion renewable energy generation, minimizing generation costs while ensuring load supply. During the design process, the advantages of reinforcement learning and optimization algorithms are maximized to coordinate the output of each generating unit. Specifically, reinforcement learning is used to gradually obtain the optimal feasible power output. After the training process, when the total power demand fluctuates or changes, the algorithm can provide a feasible power output distribution for the generating units through neural network (NN) prediction and projection optimization algorithms, thereby effectively absorbing renewable energy. Furthermore, the strategy update depends not only on the optimal solution of the approximate state action value function but also on the previously explored optimal strategy. This approach ensures that the decision-making cost monotonically decreases.

[0070] Furthermore, a Markov decision process for dynamic economic dispatch is designed to describe how the agent interacts with the environment and learns the optimal strategy. This step clarifies that the agent's objective is to minimize power generation costs, the decision content is generator output, and the environmental factors are the maximum output of new energy sources and the total cost required by the grid.

[0071] Furthermore, the methods for obtaining the various elements required for reinforcement learning are described in detail. In a distributed power system, the total load is obtained by an average consensus algorithm, thus forming the state space for reinforcement learning.

[0072] Furthermore, designing a dual neural network is of significant importance for dynamic economic dispatch problems. The action network is responsible for generating output decisions under given grid conditions. This design enables the system to respond quickly to real-time changes in grid conditions, such as load fluctuations and new energy output fluctuations, ensuring the stability and economy of grid operation. The evaluation network is used to assess the expected costs or returns of the grid system after a decision, thereby effectively evaluating the long-term effects of each action.

[0073] Furthermore, by employing the projection constraint algorithm, the decision output of reinforcement learning can be effectively mapped to a feasible range of safety constraints. The method of this invention ensures that the agent not only pursues the optimization objective when making control decisions, but also complies with the physical and safety constraints of the system.

[0074] Furthermore, a consensus method is used to obtain the total cost of the generator set, ensuring that each agent aims to minimize the total cost.

[0075] Furthermore, a greedy strategy is employed, incorporating random fluctuations after each reinforcement learning decision to prevent the generator set from converging to a local optimum.

[0076] Furthermore, the local operating strategy is updated based on the historical best strategy, ensuring that the total cost monotonically decreases. This technique stores the best strategy in the experience set, which indirectly guides the learning process, thereby improving the training efficiency of reinforcement learning and accelerating convergence.

[0077] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0078] In summary, this invention can quickly predict the optimal output power when facing the output of new energy units and the power demand on the load side under highly uncertain conditions, thereby improving the decision-making efficiency of generators.

[0079] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0080] Figure 1 Topology diagram for information exchange between 4 generator nodes;

[0081] Figure 2 Topology diagram for information exchange between 14 load nodes;

[0082] Figure 3 This is a diagram illustrating the evolution of the total cost of agent decision-making during the training process.

[0083] Figure 4 Figure 1-6 shows the evolution of the crew strategy at each moment during training.

[0084] Figure 5 Figure 7-12 shows the evolution of the crew strategy at each moment during training.

[0085] Figure 6 A graph showing the consumption of new energy sources (12h).

[0086] Figure 7 A graph showing the consumption of new energy sources (48 hours).

[0087] Figure 8 A graph showing the evolution of strategy cost with training under different exploration rates;

[0088] Figure 9 This is a robustness analysis diagram;

[0089] Figure 10 A schematic diagram of a computer device provided in an embodiment of the present invention;

[0090] Figure 11 This is a block diagram of a chip according to an embodiment of the present invention;

[0091] Figure 12 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0092] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0094] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0095] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0096] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0097] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0098] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0099] The key to solving the DEDP (Digital Emissions Per Power) problem in high-proportion renewable energy power systems is finding the optimal power allocation among generating units to effectively absorb the volatility of renewable energy while minimizing total generation costs. Dynamic economic dispatch can be described as follows:

[0100]

[0101] Among them, F i (·) is the power generation cost function of unit i; γ t p is the discount factor at time t and γ∈(0,1); i,t The power output of generator unit i at time t; D t It is the total power demand over time; The ramp-up rate limit for thermal power unit i; p i and These are the minimum and maximum power outputs of the generator set, respectively. Introducing a discount factor γ∈(0,1) ensures that the economic scheduling problem exists over an infinite time period and can be uniformly expressed; it also corresponds to the discount factor used in reinforcement learning to measure future rewards, facilitating the application of reinforcement learning theory.

[0102] make If the allowable power output of thermal power unit i is set, the constraint condition simplifies to:

[0103]

[0104] The cost function for electricity generation can be expressed in various forms, including linear and logarithmic functions. The most common cost function is a quadratic function, as shown in the following equation:

[0105]

[0106] Among them, a i ,b i and c i These are some coefficients of agent i. For new energy units, without considering operation and maintenance costs, the power generation cost can be regarded as 0.

[0107] In dynamic economic dispatch problems, the operating cost of a power system is typically determined by multiple factors, including fuel costs, maintenance costs, and environmental costs. Traditional optimization methods usually rely on known cost function forms and assume these functions are convex. However, in practical applications, the actual changes in power generation costs can be complex and nonlinear, making them difficult to accurately describe using simple convex functions. Therefore, it is necessary to consider more complex cost function forms, such as combinations of quadratic and trigonometric functions, to more accurately reflect the changes in actual power generation costs. This invention considers a more complex cost function in order to better fit the power generation cost, i.e., the formula:

[0108]

[0109] Among them, e i with f i It is related to the capacity of the thermal power generator unit. This function is designed as the objective of this model solution, guiding the actions of the agent.

[0110] To ensure effective load supply and better solve the economic dispatch model, we assume the load satisfies the following equation:

[0111]

[0112] By designing complex cost functions, the actual changes in power generation costs can be better fitted, improving the accuracy and practicality of dynamic economic dispatch models. Regarding the problem DEDP that this invention aims to solve, the following assumptions are proposed:

[0113] Feasibility and optimality assumptions:

[0114] At every moment of the considered time period, there exists at least one feasible power output combination (FPOC) that satisfies the constraints: (p 1,1 ,...,p N,1 ,...,p 1,T ,...,p N,1 ) T satisfy:

[0115]

[0116] Topological connectivity assumption:

[0117] Each generating unit can be viewed as an intelligent agent. The topology of the information exchange diagram between these agents and the main grid line information exchange diagram is undirected and interconnected, ensuring that information can be exchanged between the agents to minimize overall cost. At each time t, each main grid line can obtain its own power demand and the total power demand D. t Each agent i can access the previously obtained total power demand D of the power grid. t Adjust local power output p i,t Obtain local power generation cost F i (p i,t Through information exchange, the total cost of power generation can be obtained.

[0118] Please see Figure 12 This invention provides a dynamic economic dispatch decision-making method for power systems aimed at improving reliability, comprising the following steps:

[0119] S1. Construct a Markov decision process for dynamic economic scheduling;

[0120] Dynamic economic dispatch can be viewed as a Markov Decision Process (MDP). The goal of economic dispatch is to minimize the cost of the combined output over a given period; therefore, the output of each unit needs to be considered as a decision variable, and each unit is treated as an agent. The Dynamic Economic Dispatch Problem (DEDP) can be described as a system decision MDP. This invention will design a Distributed Multiagent Markov Decision Process (MAMDP) for dynamic economic dispatch of a high-proportion renewable energy system.

[0121] Define the MAMDP of DEDP as a quintuple:

[0122]

[0123] in, It is a global state space shared by all intelligent agents, set to the maximum output value of the new energy source. With total load D t To ensure the full utilization of new energy sources, that is m represents the number of new energy sources; It is the action space of agent i, designed as the active power output p of the generator set. i,t ; Let γ be the reward value for agent i's action, which is considered to be the total cost of power generation; γ∈[0,1] is the discount factor. In this invention, it is considered that the cost of power generation in the future period is as important as the current cost of power generation, so let γ=1.

[0124] In multi-agent decision-making, the overall action space is the superposition of the action spaces of each agent. At the same time, rewards In state space The agent i performs the following actions. The agent receives a local reward, and therefore gains information about the global state space. Predicting the action policy of local agent i Simultaneously receive local rewards

[0125] S2. Construct a reinforcement learning environment;

[0126] The environment for reinforcement learning in power systems is the basic requirements and various types of information inherent in the power grid, including the state space s. t (Maximum output P of new energy generator set) j,t With total load D t , that is, ie,s t = <P j,t D t >, j = 1, 2, ..., m), generator output constraints. The maximum output of each new energy generator unit is directly transmitted, while the total load demand is obtained using an average consensus algorithm.

[0127] Because the communication topology is undirected and interconnected, each unit agent estimates the total power demand using an average consensus algorithm. Definition in, The i-th term represents the average local power demand of all units estimated by each mainline in the power grid topology.

[0128] The consensus algorithm for total demand is represented as follows:

[0129]

[0130] Where L is the graph The Laplace matrix of the communication topology between central nodes.

[0131] From the consensus protocol, we obtain:

[0132]

[0133] Among them, 1 N It is an N-dimensional column vector with matrix elements of 1.

[0134] Therefore, the local estimate of average power demand converges to the actual average power demand.

[0135] definition To represent the estimated communication topology level of central node i, Each central node obtains the same ∈ through a distributed algorithm.

[0136]

[0137] Among them, a i,j These are entries in the adjacency matrix of the communication topology.

[0138] After N iterations, there exists in, Equal to the degree of communication topology.

[0139] S3. Update the distributed reinforcement learning neural network;

[0140] The design incorporates a dual neural network consisting of an action network and an evaluation network. The action network generates decisions, determining the appropriate power output action to take given grid conditions, while the evaluation network assesses the expected cost of the system after taking a particular action. This separate architecture allows the system to make rapid decisions during execution and effectively evaluate the long-term effects of those decisions during the training phase.

[0141] S301, Update Evaluation Network

[0142] The evaluation network is used to update the state-action value function. Definition As the state-action value function under policy action π at time slot t, a reinforcement learning value network estimate is established, where ω t These are the network parameters. The mean squared error between the target network and the target network is used as the loss function to update ω. t parameter.

[0143] The loss function is defined as:

[0144]

[0145] Among them, P t i Let P be the set of active power output of all thermal power units except for agent i. t = <P1,...,P n >. To minimize the loss function, update ω. t :

[0146]

[0147] in, C is the learning rate; t Indicates that in state s t Find the minimum total cost from time t to t+1; It is Q t (s t ,P t i ;w tThe derivative of t.

[0148] To improve the stability of reinforcement learning and ensure the parameter ω t Slow updates:

[0149] ω t '←τω t +(1-τ)ω t '

[0150] in, These are feature vectors. Accordingly, we design and evaluate the parameters of the neural network. definition:

[0151]

[0152] Where i = 1, 2, ..., N, and N is the total number of agents, the state-action value function can be obtained as follows:

[0153]

[0154] The goal is to find the minimum value of the state-action value function to obtain an approximate solution for economic scheduling. A distributed approach is used to obtain the minimum value of the state-action value function. That is, to solve the problem of approximate state-valued functions.

[0155]

[0156] Among them, each unit's intelligent agent has

[0157] make The problem can be solved under the following standard assumptions.

[0158] Assumption 3: There exists a finite optimal solution P for this optimization problem. t a* It satisfies Slater's constraints, that is, it exists. Make For the optimization problem to be solved, have:

[0159]

[0160] Using the Laplace matrix L and The established constraint optimization problem.

[0161]

[0162] stLΛ=0 N

[0163] With multipliers The augmented Lagrange duality is:

[0164]

[0165] The distributed algorithm for agent i is represented as:

[0166]

[0167] Starting from the Karush-Kuhn-Tucker (KKT) conditions, the equilibrium point of this equation is the optimal solution to the minimization problem. Let one of the equilibrium points, col(P... t a* ,Λ a* Z a* ) is represented as the vector P t a* ,Λ a* Z a* Stacked column vectors, then we can obtain The value of .

[0168] S302. For each generator unit's intelligent agent, there is an action network. Use parameters To approximate the optimal policy, a fully connected multilayer perceptron (MLP) with two hidden layers is selected. The action network of generator unit i is represented as:

[0169]

[0170] in, It is a weight matrix. h1 and h2 are the bias vectors, and h1 and h2 are the number of neurons in the two hidden layers. It is a non-linear activation function; reinforcement learning state With agent actions These are the input and output of the action network, respectively; the weight matrix and bias vector are the parameters to be fitted. The J function is defined, and its gradient is expressed as:

[0171]

[0172] To minimize the approximation error, Along Compared to Update using the negative gradient, i.e.:

[0173]

[0174] This invention designs an action network that uses a neural network to approximate the optimal policy for each unit, thereby improving the generalization ability of the policy. ω t and θt Each entry has an index 't', indicating that data from different times was trained separately. The value functions may differ at different times, hence the separate training for data from different time slots. Similarly, since the optimal policy differs across time periods, the action networks for different time periods also need to be trained separately.

[0175] S4. Obtain feasible power output under constraints;

[0176] S401, Each agent makes decisions regarding the power output of each generator. Perform sampling;

[0177] S402, Due to the power output distribution of direct decision-making For load constraints, power upper and lower limit constraints, and ramp constraints that do not meet the grid requirements, a projection controller is added to the power output distribution before interacting with the actual grid environment; if the constraints are not met, the combined action that meets the constraints is obtained through projection.

[0178]

[0179] Among them, a t =(a 1,t ,...,a N,t ) T It is a combined action after projection.

[0180] The equivalent substitution objective function is:

[0181]

[0182] The problem then transforms into an optimization problem, as follows:

[0183]

[0184] Where, x i h is a local decision variable determined by agent i; i (x i ) is a locally convex function; d i It is a local constant.

[0185] A projection optimization algorithm is used to solve the problem:

[0186]

[0187] Where, λ i and z i It is an auxiliary variable; It is h i (x i (relative to x) i The derivative; It projects a vector onto Ω. i Projection operator on.

[0188] If the communication topology is undirected and connected, then the equilibrium point of this equation is the optimal solution to the optimization problem. The optimal solution for the unit output situation is expressed as:

[0189]

[0190] Where, λ i and z i It is an auxiliary variable. It is a projection operator; the optimal solution for adjusting the unit output is P. t ′=(p′ 1,t ,...,p′ N,t ) T . a i,t The equilibrium point is represented as

[0191] S5. Obtain the total cost of power generation through the average consensus algorithm;

[0192] The reward function for economic dispatch is set to consider the total power generation cost of each generator unit within a time period. After each generator agent makes a power output decision, it returns its local power generation cost. To minimize the total power generation cost, the agents need to communicate with each other to obtain the total power generation cost of the grid, and at the same time set their respective reward functions to the total cost, ensuring that the decision objectives of each agent are not limited to its own cost.

[0193] definition and A local estimate of the average power generation cost at time t, where c is the cost of each agent i. i,t =F(p) i,t Applying the average consistent algorithm, the time costs satisfy:

[0194]

[0195] get The local estimate of the generation cost based on the average converges to the actual average generation cost at time slot t. Therefore, the total generation cost is Nc. t .

[0196] S6. Introduce the ε-greedy algorithm with a greedy strategy to balance "exploitation" and "exploration";

[0197] The decision-making of generator unit agents is based on existing experience, making output decisions. By utilizing existing experience, the agent can maximize the current reward and, to a certain extent, ensure the stability and efficiency of the algorithm, continuously exploring possible optimal solutions. In the basic distributed multi-agent deep deterministic policy gradient algorithm framework, random perturbations are only added to the decision actions of the evaluation network, but not to the decision actions of the action network. This makes the decisions of power grid generator units prone to converge to local optima, making it difficult to explore better solutions. Therefore, it is necessary to reasonably balance utilization and exploration to ensure that the agent can learn the optimal strategy in complex environments. Thus, a greedy policy ε-greedy algorithm is introduced to balance "utilization" and "exploration," where the probability of the optimal action is 1-ε, and the probability of a random exploration action is ε. An exploration strategy is then designed. For the current k-th training iteration, with exploration count k ≤ K0, the agent's decision power is determined by:

[0198]

[0199] After the number of explorations k > K0, the agent's decision power is determined by:

[0200]

[0201] in, Indicates in s t The probability of choosing action 'a'.

[0202] Initially, the agent's decision-making action is given by the evaluation network, which has already added some noise inside the algorithm before outputting the action, and uses a greedy strategy to ensure exploration performance;

[0203] After training, the action network directly makes a deterministic decision, employing a greedy strategy in order to provide the algorithm with a better exploration decision.

[0204] S7. To help the action network make better decisions and obtain the most economical action, define... The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0205] Let p i,t Actions performed in a real-world environment; definition:

[0206]

[0207] Update local operation policy:

[0208]

[0209] in, It is the optimal action generated by the evaluation network; at the same time, p i,t =p i ' ,t It is an exploration of the interaction between action and real power grid environment, which is the power output obtained after the decision made by the neural network following the ε-greedy policy; It is the optimal scheduling record during the training process; only or p i,t When the decision-making output combination is more economical, the optimal scheduling strategy is... Only then will it be updated. This strategy also ensures that the optimal scheduling strategy monotonically decreases, eventually reaching its optimum.

[0210] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0211] In another embodiment of the present invention, a dynamic economic dispatch decision system for power systems aimed at improving reliability is provided. This system can be used to implement the above-mentioned dynamic economic dispatch decision method for power systems aimed at improving reliability. Specifically, the dynamic economic dispatch decision system for power systems aimed at improving reliability includes a construction module, an update module, a calculation module, a strategy module, and an output module.

[0212] The module includes a construction module that builds a Markov decision process for dynamic economic scheduling, defines state transition probabilities through Markov chains, and uses them to calculate the policy and value function for reinforcement learning. It also builds a reinforcement learning environment to provide a simulation scenario for the intelligent agent of the power system generator set. The intelligent agent of the power system generator set makes scheduling decisions based on the system state, and schedules the generator sets to minimize the total power generation cost.

[0213] The update module updates the distributed reinforcement learning neural network, and the power system generator agent adjusts its strategy based on environmental feedback to obtain feasible power output under constraints.

[0214] The computation module obtains the total power generation cost through an average consensus algorithm and returns the total power generation cost as a reward for reinforcement learning to the power system generator agent.

[0215] The strategy module introduces the ε-greedy algorithm, which maximizes the current reward that the power system generator agent can obtain, ensuring that the power system generator agent learns the optimal strategy in complex environments.

[0216] Output module, definition The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0217] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions. The processor described in this embodiment of the present invention can be used for the operation of a dynamic economic dispatch decision-making method for power systems aimed at improving reliability, including:

[0218] A Markov decision process for dynamic economic dispatch is constructed, defining state transition probabilities through Markov chains to compute the policy and value function for reinforcement learning. A reinforcement learning environment is built to provide a simulated scenario for the power system generator agent, which makes dispatch decisions based on the system state to minimize the total generation cost. The distributed reinforcement learning neural network is updated, and the power system generator agent adjusts its policy based on environmental feedback. Feasible power output under constraints is obtained. The total generation cost is obtained through an average consensus algorithm and returned as the reinforcement learning reward to the power system generator agent. An ε-greedy policy algorithm is introduced to maximize the current reward obtained by the power system generator agent, ensuring that the agent learns the optimal policy in complex environments. Definitions are defined... The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0219] Please see Figure 10The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the power system dynamic economic dispatch decision-making method for reliability improvement in this embodiment. To avoid repetition, details are omitted here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the power system dynamic economic dispatch decision-making system for reliability improvement in this embodiment. To avoid repetition, details are omitted here.

[0220] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 10 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0221] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0222] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or RAM of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0223] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0224] Please see Figure 11 The terminal device 600 is an electronic device, which takes the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0225] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 12 The steps are shown in the figure.

[0226] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0227] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0228] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0229] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0230] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor; these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0231] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the power system dynamic economic dispatch decision-making method for reliability improvement in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps:

[0232] A Markov decision process for dynamic economic dispatch is constructed, defining state transition probabilities through Markov chains to compute the policy and value function for reinforcement learning. A reinforcement learning environment is built to provide a simulated scenario for the power system generator agent, which makes dispatch decisions based on the system state to minimize the total generation cost. The distributed reinforcement learning neural network is updated, and the power system generator agent adjusts its policy based on environmental feedback. Feasible power output under constraints is obtained. The total generation cost is obtained through an average consensus algorithm and returned as the reinforcement learning reward to the power system generator agent. An ε-greedy policy algorithm is introduced to maximize the current reward obtained by the power system generator agent, ensuring that the agent learns the optimal policy in complex environments. Definitions are defined... The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

[0233] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0234] Construct two neural network frameworks: an action network and an evaluation network. When k ≤ K0, select the action decided by the action network. Conversely, the evaluation of network decision-making actions is chosen. To balance exploring new strategies with making full use of existing experience, a greedy strategy is used to update the existing power. However, this decision may not meet the required load constraints, power upper and lower limit constraints, and power ramping constraints. Therefore, a constrained projection algorithm is used to adjust the power. When there is a new energy unit agent, the projection optimization algorithm is improved as follows:

[0235]

[0236] Where m represents the number of new energy generating units, and n represents the total number of generating units. This represents the maximum output of the new energy unit i at time t. To ensure the consumption of new energy sources.

[0237] If k > K00 The quadratic function coefficients of the evaluation network are output to the state-action value function optimizer to obtain another set of power output distributions that minimize the state-action value function. The costs of these two unit combinations are compared, and the unit combination with the smaller cost and its state transition variables are selected and stored in the experience set for future use. (Where k > K) 00 At that time, a small portion of the dataset is extracted from the experience set to update the parameters of the neural network, ensuring that the reinforcement learning agent's decisions gradually converge to the optimal result.

[0238] The example considers a system consisting of four power generation units, with the communication topology shown in the figure. (See attached figure) Figure 1 Unit 1 is a photovoltaic generator, unit 2 is a wind turbine generator, and units 3 and 4 are thermal power generators G1 and G2, respectively. This section considers using the IEEE 14 standard topology, which lacks one generator, to achieve average consensus on the total load required by the power grid. The topology structure is as follows: Figure 2 As shown, PVG represents photovoltaic power generation units, WPG represents wind power generation units, and TPG represents thermal power generation units.

[0239] In the experiment, 12 time periods were set, each lasting 2 hours. The total power demand for each time period was set to 359, 350.8, 374.9, 399.3, 426.9, 446.1, 449.0, 491.5, 421.9, 404.0, 363.4, and 352.6 MW, respectively. The total number of time slots was set to T = 12. The exploration rate was set to ε = 0.2. The main parameters of the thermal power unit are shown in Table 1.

[0240] Table 1 Main parameters of the thermal power unit in the example.

[0241]

[0242] The selection of DEDP parameters is shown in Table 2. The selection of parameters is simplified in order to improve computational efficiency.

[0243] Table 2 Key parameters of reinforcement learning

[0244]

[0245] After 1,000 training sessions, with all new energy units operating at full capacity during each time period, the decision-making process for thermal power units is shown in Table 3:

[0246] Table 3 Decision-making for thermal power units

[0247]

[0248] Table 3 (continued) Decision-making for thermal power units (continued)

[0249]

[0250] The entire training process is shown in Figure*. After 130 training iterations, the total cost stabilized relatively, with a corresponding total power generation cost of 32,098.78. The cost function of the example is nonlinear and has a complex form. Using the MILP algorithm with piecewise linearized cost as a control group, the cost calculated by the strategy after relaxing the start-stop constraints was 30,654.38, differing from the calculated cost by only 4.71%, indicating a small error. At this point, the renewable energy source has been fully utilized, and the agent's action decisions tend to explore around this relatively optimal solution, making it difficult to find a better one. It can be inferred that increasing the training step size will lead to the exploration of a better strategy, further reducing the error. Although the accuracy of reinforcement learning did not reach that of the optimization algorithm, it can make rapid decisions when dealing with new situations, reducing time costs.

[0251] Please see Figure 3 As the exploration of the computational method of this invention deepens, the cost of the strategies implemented changes. In addition to the algorithm designed in this invention, Algorithm 1 is also used for comparison. Algorithm 1 is derived from the original MADDPG algorithm, optimization with embedded bias adjustment, an Actor-Critic neural network with 40 neurons and 2 hidden layers, and other optimization methods such as a greedy strategy without ε. The comparison shows that the algorithm designed in this invention converges faster and is superior. Furthermore, during the algorithm execution, it is not difficult to find that the bias adjustment optimization method struggles to achieve a bias σ = 0, and the decisions made may not meet the load demand D. t This demonstrates the effectiveness of the constrained projection algorithm. To better illustrate the evolution of the strategy during training, further research is conducted. Figure 4 and Figure 5 This displays the changes in the output of thermal power units at each moment as the training progresses.

[0252] Plot a line graph of load demand versus maximum renewable energy output, and a stacked bar chart of thermal power unit output, such as... Figure 6 As shown, the analysis of the algorithm's effect on renewable energy absorption reveals that in the example, the thermal power unit's output decision fully adapted to the load and fluctuations generated by renewable energy, successfully completing the task of high-proportion renewable energy absorption. To more clearly demonstrate the algorithm's effect on renewable energy absorption, a 48-hour time step was used, with each hour representing a period of time, and the final output results are shown below. Figure 7 The figure more clearly shows the algorithm's effect on the absorption of new energy units.

[0253] To account for the impact of different ε-greedy strategies on the algorithm results, this invention adopts ε = 1 / k, ε = 0.2, ε = 0.4, ε = 0.6, ε = 0.8, and ε = 0.9 × 0.99. k And ε≥0.1, where k is the number of training steps, such that the operation strategy in the ε-greedy policy has a greedy limit of infinite exploration. For example... Figure 8 As shown, Figure 8 The evolution of the total generation cost over 1,000 training iterations for each update policy is shown. The distributed reinforcement learning optimization algorithm converges fastest and yields a good policy when using ε = 0.2.

[0254] Next, the robustness of the method of this invention under sudden fluctuations in power demand is verified. The maximum training time is set to K = 1000. In the training set, when k = 900, the total power demand at time 12 suddenly changes to 432.60 MW, an increase of 80 MW on the original load. The experiment is conducted using the algorithm of this invention. After 1000 training iterations, the total power generation cost is learned to be 38692.94. Figure 4 The total cost of the policies learned during the training process is shown, along with the evolution of power output learned during training at time 6. The results show that when k = 1500, the method of this invention can quickly adjust the output power distribution to meet power demand; the output of generator G2 abruptly changes from 59.70143MW to 139.70143MW, and the total power output becomes 432.60MW. Figure 9 The cost of generating electricity in China has suddenly increased.

[0255] Experimental results show that the algorithm has strong robustness under fluctuating power demand conditions.

[0256] In summary, this invention presents a dynamic economic dispatch decision-making method and system for power systems aimed at improving reliability. It utilizes reinforcement learning to train a model capable of predicting optimal generator output. During training, the algorithm learns how to adjust generator output based on changes in electricity demand, while also considering generation costs. Through continuous iteration and learning, the reinforcement learning model gradually approaches the optimal power output strategy. This ensures power system stability and reliability while efficiently utilizing renewable energy resources, reducing reliance on traditional fossil fuel power generation, and lowering overall generation costs and environmental impact.

[0257] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0258] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0259] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0260] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0261] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0262] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0263] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random-access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0264] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0265] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0266] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0267] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A dynamic economic dispatch decision-making method for power systems aimed at improving reliability, characterized in that, Includes the following steps: S1. Construct a Markov decision process for dynamic economic scheduling, and define the state transition probability through a Markov chain to calculate the policy and value function for reinforcement learning. S2. Construct a reinforcement learning environment to provide a simulation scenario for the intelligent agent of the power system generator set. The intelligent agent of the power system generator set makes scheduling decisions based on the system state and schedules the generator sets to minimize the total power generation cost. S3. Update the distributed reinforcement learning neural network. The power system generator agent adjusts its strategy based on environmental feedback. Step S3 specifically involves: S301. Design a dual neural network comprising an action network and an evaluation network. The action network is responsible for the output action to be taken under a given power grid condition; the evaluation network assesses the expected cost of the system after taking a certain action. Define... As in the time slot ,state Action output Location, strategic actions The next-state action-value function is used to establish a reinforcement learning value network estimation. These are the network parameters, updated using the mean squared error of the target network as the loss function. The parameters are used to obtain the minimum value of the state-action value function, thus obtaining an approximate solution for economic scheduling; a distributed method is used to obtain the minimum value of the state-action value function. To solve the problem concerning the approximate state action value function; S302, Each generator set's intelligent agent has an action network. Using parameters Approximate optimal strategy; select a fully connected multilayer perceptron with two hidden layers as the approximate optimal strategy; set it to the maximum output of the new energy source. With total load Strengthen the learning state Compared with the best historical strategy These are the input and output of the action network, respectively. along Compared to Update using the negative gradient; S4. Obtain the feasible power output under the constraints, specifically: S401, Each agent makes decisions regarding the power output of each generator. Sampling is performed; S402. Add a projection controller to the power output distribution; if the constraints are not met, obtain the joint action that satisfies the constraints through projection, and transform the problem of adjusting the power output distribution to satisfy the constraints into an optimization problem. Use the projection optimization algorithm to solve it. If the communication topology is undirected and connected, then the equilibrium point of this optimization problem is the optimal solution to the optimization problem. S5. Obtain the total power generation cost through the average consensus algorithm, and return the total power generation cost as a reward for reinforcement learning to the power system generator agent; S6. Introduce a greedy strategy The algorithm leverages the power system generator agent to maximize the current reward, ensuring that the power system generator agent learns the optimal strategy in complex environments. S7, Definition The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

2. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, In step S1, a multi-agent Markov decision process for dynamic economic scheduling of a high-proportion renewable energy system will be designed, defined as follows: in, It is a global state space shared by all intelligent agents, set to the maximum output value of the new energy source. With total load , , The number of new energy sources; It is a node The operating space is designed for the active power output of the generator set. ; For nodes Action reward value; As a discount factor, in multi-agent decision-making, the agent obtains information about the global state space. Predicting local intelligent agents Action strategy At the same time, receive local rewards. .

3. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, In step S2, the reinforcement learning environment includes a state space. Generator output constraints; Each unit agent estimates the total power demand using an average consensus algorithm, which is expressed as follows: in, It is a picture The Laplace matrix of the inter-node communication topology. ,in, Laplace matrix Maximum value of diagonal elements For the first The average power obtained from the next iteration; Derived from the consensus protocol: in, It is 3D column vectors, matrix elements are , for Total grid load at any given time This represents the total number of power grid nodes. definition To represent nodes Estimated communication topology Each node obtains the same [status] through a distributed algorithm. ; After N iterations, there exists , Equal to the degree of communication topology.

4. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, The loss function is defined as: in, This refers to calculating the expected value. As a discount factor, This is an estimate of the action-value function for the current state. For nodes In time The action; renew : in, The learning rate; Indicates the state From time arrive The minimum total cost; yes right The derivative; Generator unit The action network is represented as: in, , , It is a weight matrix. , , It is the deviation vector. and It represents the number of neurons in the two hidden layers; , , It is a non-linear activation function.

5. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, In step S5, define and For time Local estimates of the average generation cost at each node of ; Applying the average consistency algorithm, the time costs satisfy: get The local estimate of the generation cost based on the average converges to the time slot. The actual average cost of electricity generation at the location, and the total cost of electricity generation are .

6. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, Step S6 is as follows: Introducing a greedy strategy The algorithm has the following probability of optimal action: The probability of a random exploration operation is Design exploration strategies For the current first Training sessions; number of explorations The decision-making power of an intelligent agent depends on: Number of explorations Subsequently, the agent's decision-making power is determined by: in, Indicates in Select action The probability of; Initially, the agent's decision-making action is given by the evaluation network, which has already added some noise inside the algorithm before outputting the action, and uses a greedy strategy to ensure exploration performance; After training, the action network makes a deterministic decision.

7. The dynamic economic dispatch decision-making method for power systems oriented towards reliability improvement according to claim 1, characterized in that, In step S7, let Actions performed in a real-world environment; definition: Update local operation policy: in, It is the optimal action generated by the evaluation network; It is an exploration of the interaction between action and a real power grid environment; or When the decision-making output combination is more economical, the optimal scheduling strategy is... It is updated to reach its optimal state.

8. A dynamic economic dispatch decision-making system for power systems aimed at improving reliability, characterized in that, The power system dynamic economic dispatch decision-making method for reliability improvement as described in claim 1 includes: The module constructs a Markov decision process for dynamic economic scheduling, defining state transition probabilities through Markov chains to compute the policy and value function for reinforcement learning. It also constructs a reinforcement learning environment to provide a simulation scenario for the power system generator agent. The generator agent makes scheduling decisions based on the system state, scheduling generators to minimize the total generation cost. Specifically: S301. Design a dual neural network comprising an action network and an evaluation network. The action network is responsible for the output action to be taken under a given power grid condition; the evaluation network assesses the expected cost of the system after taking a certain action. Define... As in the time slot ,state Action output Location, strategic actions The state-action value function is used to establish a reinforcement learning value network estimation. These are the network parameters, updated using the mean squared error of the target network as the loss function. The parameters are used to obtain the minimum value of the state-action value function, thus obtaining an approximate solution for economic scheduling; a distributed method is used to obtain the minimum value of the state-action value function. Solve the problem concerning approximate state-valued functions; S302, Each generator set's intelligent agent has an action network. Using parameters Approximate optimal policy; select a fully connected multilayer perceptron with two hidden layers as the approximate optimal policy; reinforcement learning state. With agent actions These are the input and output of the action network, respectively. along Compared to Update using the negative gradient; The update module updates the distributed reinforcement learning neural network, and the power system generator agent adjusts its strategy based on environmental feedback; it obtains feasible power output under constraints, specifically: Each agent makes decisions regarding the power output of each generator. Sampling is performed; a projection controller is added to the power output distribution; if the constraints are not met, the joint action that satisfies the constraints is obtained through projection, and the problem of adjusting the power output distribution to satisfy the constraints is transformed into an optimization problem, which is solved by the projection optimization algorithm. If the communication topology is undirected and connected, the equilibrium point of the optimization problem is the optimal solution of the optimization problem. The computation module obtains the total power generation cost through an average consensus algorithm and returns the total power generation cost as a reward for reinforcement learning to the power system generator agent. The strategy module introduces a greedy strategy. The algorithm leverages the power system generator agent to maximize the current reward, ensuring that the power system generator agent learns the optimal strategy in complex environments. Output module, definition The local operation strategy is updated to reflect the historical best strategy, so that the optimal scheduling strategy decreases monotonically and eventually reaches the optimal state.

Images