Power system dispatching method considering physical priori and safe projection

Abstract

The application discloses a power system scheduling method considering physical priori and safety projection, relates to the technical field of electric power, and comprises the following steps: fixing discrete states in a joint system scheduling model through an intention locking and conditional projection mechanism; constructing a key line flow linear proxy model according to a DC power flow approximation and a power transfer distribution factor; replacing the nonlinear flow constraint with the key line flow linear proxy model to obtain a target joint system scheduling model; reconstructing the target joint system scheduling model into a target Markov decision process; setting a safety projection layer between the action output end of an Actor network and the bottom layer alternating current flow environment execution end; solving the target Markov decision process through a PI-TD3-Safety algorithm; and scheduling the to-be-scheduled power system. The application can avoid the non-convex problem caused by the heat charging and discharging mutual exclusion of the heat storage system, improve the accuracy of the target scheduling parameters, and accurately schedule the to-be-scheduled power system.

Description

Technical Field

[0001] This invention relates to the field of power technology, and more specifically to a power system dispatching method that takes into account physical priors and security projections. Background Technology

[0002] In the existing power system, the installed capacity of new energy sources, represented by wind power and photovoltaics, continues to expand, and the penetration rate of new energy sources in the power system is constantly increasing. Compared with traditional synchronous generator units, most new energy sources such as wind power and photovoltaics are connected to the grid through power electronic converters. Their output is significantly affected by weather conditions, exhibiting strong randomness, volatility, and intermittency. When new energy sources are connected to the sending-end grid on a large scale, the system operation exhibits stronger uncertainty and dynamic coupling characteristics, posing higher requirements for the safe and stable operation of the power system.

[0003] To address the challenge of solving complex scenarios such as high-proportion renewable energy integration, transient frequency security, and extreme fault disturbances, deep reinforcement learning algorithms like TD3 have been widely applied in the optimal scheduling of power systems and integrated energy systems. The TD3 algorithm can handle continuous action space problems and learns scheduling strategies through agent-environment interaction, thereby reducing online optimization time. To improve the safety of reinforcement learning scheduling, existing methods typically add out-of-bounds penalties to the reward function or prune upper and lower limits on the agent's output actions. Some methods also introduce safety reinforcement learning or optimization correction layers to correct actions that do not meet constraints.

[0004] However, the following technical challenges remain in grid dispatching for high-proportion renewable energy transmission:

[0005] 1. Online safety projection layers typically require input constraints that are convex and continuous algebraic equations. However, the operating mechanisms of power system AC power flow and thermal storage charging / discharging mutual exclusion are nonlinear and nonconvex. In particular, the thermal storage system's charging / discharging mutual exclusion will form mixed integer constraints, which do not meet the direct solution requirements of conventional quadratic programming projection layers, thus affecting the efficiency of online safety correction.

[0006] 2. The original scheduling actions output by the reinforcement learning agent may lead to excessive power flow at critical sections, excessive unit output, excessive thermal storage capacity, or insufficient reserve capacity. If only the reward function is used to punish excessive behavior, the agent needs to repeatedly try and fail during training, making it difficult to ensure that excessive actions are corrected in time before execution. Summary of the Invention

[0007] The purpose of this invention is to provide a power system dispatching method that takes into account physical priors and security projections, so as to solve the technical problems existing in related technologies.

[0008] To achieve the above objectives, the present invention provides a power system dispatching method that considers physical priors and security projections, comprising:

[0009] A joint system scheduling model for the power system to be dispatched, taking into account the security projection layer, is constructed. The joint system scheduling model includes an objective function and constraints. The objective function is constructed with the goal of minimizing the sum of the operating cost, reserve cost, and renewable energy curtailment penalty cost of the power system to be dispatched. The constraints include unit operation constraints, thermal storage system operation constraints, and renewable energy and reserve operation constraints.

[0010] Determine the approximate DC power flow, power transfer distribution factor, and discrete state of the thermal storage system in the power system to be dispatched;

[0011] In the joint system scheduling model, discrete states are fixed by intention locking and conditional projection mechanisms. A critical line power flow linear proxy model is constructed based on DC power flow approximation and power transfer distribution factor. The nonlinear power flow constraints are replaced by the critical line power flow linear proxy model to obtain the target joint system scheduling model.

[0012] The target joint system scheduling model is reconstructed into a target Markov decision process, wherein the target Markov decision process includes a secure projection layer, an Actor network, and an underlying communication flow environment. The secure projection layer is set between the action output end of the Actor network and the execution end of the underlying communication flow environment. The secure projection layer is used to correct the original actions output by the Actor network.

[0013] The target Markov decision process is solved using the PI-TD3-Safety algorithm to obtain the target scheduling parameters, and the power system to be scheduled is then scheduled based on these parameters.

[0014] Optionally, the discrete state is obtained by the following method:

[0015] Obtain the comprehensive command variable of thermal storage power in the thermal storage system of the power system to be dispatched;

[0016] When the comprehensive instruction variable is greater than the preset variable threshold, the discrete state is a heating state; when the comprehensive instruction variable is less than or equal to the preset variable threshold, the discrete state is a heat release state.

[0017] Optionally, the critical path power flow linear surrogate model is expressed by the following calculation formula:

[0018] ;

[0019] ;

[0020] ;

[0021] The security constraints of the critical path power flow linear surrogate model are expressed by the following calculation formula:

[0022] ;

[0023] in, Inject the net active power vector into the node. Here is the nodal susceptance matrix. The node voltage phase angle vector. This is the active power flow vector of the line. The branch diagonal susceptance matrix, This is the branch-node correlation matrix. The power transfer distribution factor matrix, The active power flow vector of the basic line before the action is issued at time t. To correct the change vector of net active power injected into the node caused by the action. This is the upper limit vector for the active power flow of the line's thermal stability.

[0024] Alternatively, the correction process of the safety projection layer to the original motion can be expressed by the following calculation formula:

[0025] ;

[0026] ;

[0027] ;

[0028] in, In the target Markov decision-making process, after correction by the safety projection layer, the actual effect on the underlying AC current environment is... The corrective action at any given moment To represent finding the minimum Euclidean projection distance with candidate action 'a' as the optimization variable, Candidate action variables, for The original action output by the Actor network at the given moment. The linear inequality constraint matrix for the safety polyhedron. Let the upper bound vector be the linear inequality of the safety polyhedron. Let be the lower bound vector of the action variables. Let be the upper bound vector of the action variables. It is a Euclidean norm.

[0029] Optionally, the step of solving the target Markov decision process using the PI-TD3-Safety algorithm to obtain the target scheduling parameters includes:

[0030] The Actor-Critic network is trained using the TD3 algorithm in the PI-TD3-Safety algorithm. The Actor-Critic network is used to receive the system state output by the target Markov decision process and output the original action based on the system state.

[0031] A differentiable safe projection layer is constructed using Cvxpylayers in the PI-TD3-Safety algorithm, and the original action is then subjected to a second planning projection through the differentiable safe projection layer to obtain the corrected action.

[0032] The first-order transient frequency analytical model in the PI-TD3-Safety algorithm is used to transform the correction action into a physical prior gradient and embed it into the Actor network update, guiding the agent to actively reserve system inertia and frequency safety margin.

[0033] The Actor-Critic network is iteratively updated based on the state and reward signals obtained from the feedback of the target Markov decision process and the physical prior gradient to obtain the target scheduling parameters.

[0034] Optionally, the objective function is expressed by the following formula:

[0035] ;

[0036] in, The total operating cost of the power system to be dispatched. For operating costs, For backup costs, The cost of abandoning new energy sources.

[0037] Optionally, the step of converting the correction action into a physical prior gradient and embedding it into the Actor network update using the first-order transient frequency analytical model in the PI-TD3-Safety algorithm includes:

[0038] For each control step, after the Actor network outputs the original action, the safety projection layer solves the quadratic programming problem in real time. The specific solution process is expressed by the following calculation formula:

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] in, In the target Markov decision-making process, after correction by the safety projection layer, the actual effect on the underlying AC current environment is... The corrective action at any given moment for The original action output by the Actor network at the given moment. This is the upper limit vector of the active power flow for thermal stability of the critical path. for The active power flow vector of the basic line before the action is issued. The power transfer distribution factor matrix, To correct the action The resulting vector of net active power injection at the nodes. To correct the primary frequency regulation reserve capacity corresponding to the action, To pre-set the maximum sudden active power deficit, Let be the lower bound vector of the action variables. Let be the upper bound vector of the action variables. To perform safety correction actions The rate of frequency change of the power system to be dispatched later, The maximum rate of frequency change of the power system to be dispatched;

[0045] At each control step, the safety projection layer receives the original action and solves the optimization problem to obtain the corrected action. The projection process of the differentiable safety projection layer is regarded as a parameterized computational operator. After the Actor network outputs the original action, the partial derivatives at the KKT points are obtained through the implicit function theorem, specifically expressed by the following formula:

[0046] ;

[0047] in, This is the accurate local gradient obtained through KKT conditional analysis. The objective function of the Actor network with respect to the network parameters gradient, This refers to the number of experience samples contained in a single microbatch during experience replay. The action value function Q of the Critic network with respect to the modified action gradient, Correction actions for the output of the secure projection layer. The original action output by the Actor network. For policy actions with respect to Actor network parameters The gradient;

[0048] When internalizing the physical gradient based on the first-order transient frequency analytical model, the correction action is embedded into the loss function of the Actor network. For a single disturbance scenario, the extreme value of the frequency drop of the power system to be dispatched after the disturbance is expressed by the following formula:

[0049] ;

[0050] in, This refers to the extreme frequency drop value of the power system to be dispatched. R represents the sudden active power deficit of the power system to be dispatched under the current dispatch action; D represents the equivalent speed governor droop coefficient of the power system to be dispatched; and D represents the load damping constant of the power system to be dispatched. The mechanical power gain constant of the prime mover; The time it takes for the frequency to drop to its lowest point;

[0051] The sudden power deficit vector in the power system to be dispatched is expressed by the following formula:

[0052] ;

[0053] in, For the first The burst of success corresponding to each empirical sample Transpose of a vector;

[0054] By using mapping calculations, the extreme values ​​of frequency drop in the power system to be dispatched corresponding to all states and actions under each empirical sample are obtained. ;

[0055] If the original action causes the absolute value of the predicted frequency drop to exceed the threshold defined by the power grid safety regulations, a one-way asymmetric physical overrun penalty is triggered, and its physical loss function is expressed by the following formula:

[0056] ;

[0057] in, For physical loss function, Thresholds defined by power grid safety regulations, For the i-th experience sample in the original action The absolute value of the frequency drop extremum obtained from the first-order transient frequency analytical model is shown below. The original action output by the Actor network;

[0058] By fusing the physical loss function and based on the deterministic policy gradient theorem, the parameter update gradient in the Actor network is reconstructed as a linear superposition of the purely data-driven action value gradient and the physical prior gradient:

[0059] ;

[0060] in, The loss function of the Actor network Regarding Actor network parameters The update gradient, The action value function output by the Critic network. The Jacobian matrix of the differentiable secure projection layer is obtained from the KKT conditions. For physical penalty adaptive weighting coefficients, For the action value function with respect to the modified action gradient, The physical loss function with respect to the original action gradient, For the Actor network in the target Markov decision process state The output of the deterministic strategy.

[0061] Through the above technical solutions, in the joint system scheduling model, the discrete state is fixed by intention locking and conditional projection mechanisms, which can avoid the non-convex problem caused by the mutual exclusion of thermal storage system charging and discharging during the calculation of target scheduling parameters. Then, based on the DC power flow approximation and power transfer distribution factor, a critical line power flow linear surrogate model is constructed. The nonlinear power flow constraints are replaced by the critical line power flow linear surrogate model to obtain the target joint system scheduling model. This can avoid the problem that it is difficult to solve the AC optimal power flow with nonlinear constraints in real time at the projection layer during the solution process. The target joint system scheduling model is reconstructed into a target Markov decision process. The target Markov decision process is solved by the PI-TD3-Safety algorithm (Physics-Informed Twin Delayed Deep Deterministic Policy Gradient with SafetyProjection, PI-TD3-Safety, a double-delayed deep deterministic policy gradient algorithm that takes into account physical priors and safety projection) to obtain the target scheduling parameters. The power system to be scheduled is then scheduled based on the target scheduling parameters. The target Markov decision process includes a secure projection layer, an Actor network, and an underlying AC power flow environment. The secure projection layer is positioned between the action output of the Actor network and the execution end of the underlying AC power flow environment. This layer corrects the original actions output by the Actor network. Improving the accuracy of the secure projection layer's correction of the original actions enhances the accuracy of the target scheduling parameters, thereby enabling accurate scheduling of the power system to be dispatched.

[0062] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0063] Figure 1 This is a schematic diagram illustrating a power system dispatching method that takes into account physical priors and security projections according to an exemplary embodiment of the present invention.

[0064] Figure 2 This is a framework diagram illustrating a power system dispatching method that takes into account physical priors and security projections according to an exemplary embodiment of the present invention.

[0065] Figure 3 This is a comparison chart of reward convergence curves during the training period of an agent according to an exemplary embodiment of the present invention.

[0066] Figure 4 This is a comparison chart showing the cumulative number of times physical constraints are exceeded during the training process of an intelligent agent, according to an exemplary embodiment of the present invention.

[0067] Figure 5 This is a diagram illustrating the day-ahead scheduling optimization results according to an exemplary embodiment of the present invention.

[0068] Figure 6 This is a power flow comparison diagram of a critical transmission line under a fault scenario, according to an exemplary embodiment of the present invention. Detailed Implementation

[0069] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, so as to provide a better understanding of the concept of the present invention, the technical problem solved, the technical features constituting the technical solution, and the technical effects brought about.

[0070] like Figure 1 As shown, Figure 1 This is a schematic diagram illustrating a power system dispatching method considering physical priors and security projections according to an exemplary embodiment of the present invention, with reference to... Figure 1 The method includes:

[0071] S101: Construct a joint system scheduling model for the power system to be dispatched, taking into account the security projection layer. The joint system scheduling model includes an objective function and constraints. The objective function is constructed with the goal of minimizing the sum of the operating cost, reserve cost, and renewable energy curtailment penalty cost of the power system to be dispatched. The constraints include unit operation constraints, thermal storage system operation constraints, and renewable energy and reserve operation constraints.

[0072] S102: Determine the approximate DC power flow, power transfer distribution factor, and discrete state of the thermal storage system in the power system to be dispatched;

[0073] S103: In the joint system scheduling model, the discrete state is fixed by intention locking and conditional projection mechanism. A critical line power flow linear proxy model is constructed based on the DC power flow approximation and power transfer distribution factor. The nonlinear power flow constraint is replaced by the critical line power flow linear proxy model to obtain the target joint system scheduling model.

[0074] S104: Reconstruct the target joint system scheduling model into a target Markov decision process, wherein the target Markov decision process includes a secure projection layer, an Actor network, and a bottom-level communication flow environment, and the secure projection layer is set between the action output end of the Actor network and the execution end of the bottom-level communication flow environment. The secure projection layer is used to correct the original actions output by the Actor network.

[0075] S105: Solve the target Markov decision process using the PI-TD3-Safety algorithm to obtain the target scheduling parameters, and then schedule the power system to be scheduled based on the target scheduling parameters.

[0076] Through the above technical solution, in the joint system scheduling model, the discrete state is fixed by intention locking and conditional projection mechanisms, which avoids the non-convexity problem caused by the mutual exclusion of thermal repulsion of the thermal storage system during the calculation of target scheduling parameters. Then, based on the DC power flow approximation and power transfer distribution factor, a critical line power flow linear surrogate model is constructed. This critical line power flow linear surrogate model replaces the nonlinear power flow constraints, resulting in the target joint system scheduling model. This avoids the difficulty of real-time solving of AC optimal power flow with nonlinear constraints in the projection layer during the solution process. The target joint system scheduling model is reconstructed into a target Markov decision process. The PI-TD3-Safety algorithm is used to solve the target Markov decision process to obtain the target scheduling parameters, and the power system to be scheduled is then scheduled based on these parameters. The target Markov decision process includes a safety projection layer, an Actor network, and a lower-level AC power flow environment. The safety projection layer is located between the action output end of the Actor network and the execution end of the lower-level AC power flow environment. The safety projection layer is used to correct the original actions output by the Actor network. This improves the accuracy of the safety projection layer's correction of the original actions, thereby improving the accuracy of the target scheduling parameters and enabling accurate scheduling of the power system to be scheduled.

[0077] To enable those skilled in the art to better understand the power system dispatching method considering physical priors and security projections provided by this invention, the above steps are illustrated in detail below.

[0078] For example, the power system to be dispatched can be a power system object that requires dispatching decisions and power / generator output optimization control. The security projection layer can be an outer action security correction module deployed between the Actor network action output end and the underlying AC power flow environment execution end; the joint system dispatching model can be a mathematical optimization model established for the power system to be dispatched, used to describe the optimization objectives and various operating constraints of the power system operation.

[0079] In this embodiment of the invention, the joint system scheduling model includes an objective function and constraints. These constraints may include unit operation constraints, thermal storage system operation constraints, and new energy and reserve operation constraints, all of which are relevant constraints in the prior art. The objective function is expressed by the following formula:

[0080] ;

[0081] in, The total operating cost of the power system to be dispatched. For operating costs, For backup costs, The cost of abandoning new energy sources.

[0082] Operating costs mainly include fuel costs for thermal power units and operation and maintenance costs for solar thermal power plants, which can be expressed as:

[0083] ;

[0084] in, The number of time periods within the scheduling cycle. This refers to the number of thermal power units. For the first Each thermal power unit Contributing effort at all times, , and All of these are consumption characteristic coefficients for thermal power units. For the number of solar thermal power plants, The unit operation and maintenance cost of CSP (Concentrated Solar Power). For thermal power units exist Contributing to the cause at all times.

[0085] Reserve costs reflect the price the system pays for reserving safety support resources to withstand fluctuations in new energy sources and extreme disturbances. Specifically, they include positive spin-off reserves, negative spin-off reserves, and primary frequency regulation reserve costs, namely:

[0086] ;

[0087] in, This refers to the number of thermal power units. For the number of solar thermal power plants, , and The unit capacity price corresponds to the various types of standby capacity. For the unit Provided forward rotation reserve capacity, For the unit Provided negative spin-off reserve capacity, For the unit Provides millisecond-level primary frequency modulation backup capacity.

[0088] In scenarios with a high proportion of renewable energy, ensuring the efficient utilization of photovoltaic and solar thermal resources remains a crucial scheduling objective. Therefore, this invention employs the penalty cost of abandoning renewable energy to characterize the economic losses resulting from underutilization of renewable energy, specifically expressed through the following calculation formula:

[0089] ;

[0090] in, For the number of photovoltaic power plants, For photovoltaic power plant index, and These are the unit photovoltaic curtailment penalty coefficient and the unit solar thermal curtailment penalty coefficient, respectively. For photovoltaic power station The output of ultra-short-term forecasts To contribute to actual dispatching, For solar thermal power plants exist Available heat collection power that is constantly being discarded.

[0091] For example, the DC power flow approximation can be a simplified approximation method for power flow in a power grid. It ignores node voltage amplitude variations and reactive power, considering only the transmission relationship of active power, simplifying the nonlinear AC power flow equations into a linear model. This can replace complex nonlinear AC power flow calculations and achieve model convexity. The power transfer distribution factor can be used to characterize the change in active power flow on each transmission line when a unit of active power is injected into a node in the power grid. The discrete state of the thermal storage system can be a discrete variable representing the operating mode of the thermal storage system. In this embodiment of the invention, after constructing the joint dispatch model, the DC power flow approximation, power transfer distribution factor, and discrete state of the thermal storage system in the power model to be dispatched can be determined.

[0092] For example, after obtaining the DC power flow approximation, power transfer distribution factor, and discrete state of the thermal storage system, the joint system scheduling model can be processed based on these factors. Specifically, to address the non-convex problem caused by the mutual exclusion of thermal charging and discharging in the thermal storage system, an intention-locking and conditional projection mechanism is adopted to fix the discrete state of the thermal storage system. This discrete state includes the charging and discharging states of the thermal storage system, thus allowing the safety projection layer to correct its actions only within a continuous convex subspace. In this embodiment of the invention, the intention-locking and conditional projection mechanism determines the charging or discharging state of the thermal storage system based on the comprehensive command variable of the thermal storage power, and uses the determined discrete state as the fixed boundary condition of the safety projection layer, ensuring that safety projection is performed only within a continuous convex subspace.

[0093] Meanwhile, to address the difficulty of solving the AC optimal power flow with nonlinear constraints in real time at the safety projection layer, a linear proxy model for critical line power flow is constructed using DC power flow approximation and power transfer distribution factor. This linear proxy model for critical line power flow can replace the nonlinear power flow constraints in the joint scheduling model, thus obtaining the target joint system scheduling model.

[0094] The critical path power flow linear surrogate model is ultimately expressed by the line power flow safety polyhedral inequality; its derivation is obtained sequentially from the node net injected power, node phase angle, branch power flow, and PTDF matrix, and the final calculation formula is as follows:

[0095] In this embodiment of the invention, for the security constraints of key transmission sections of the power system to be dispatched, a linear surrogate model of power flow on the critical lines is constructed using the power transfer distribution factor, and the final " This refers to the system of linear algebraic inequalities actually used in the safety projection layer. The specific derivation is as follows.

[0096] Consider a containing Each node and An arbitrary connected transmission network with branches. The linearization of DC power flow is based on the following three core assumptions: neglecting line resistance, i.e. The per-unit voltage amplitude of each node is set to a constant 1.0 pu; it is assumed that the phase angle difference between adjacent nodes is extremely small, i.e. The mathematical derivation process is as follows:

[0097] Define the column vector of net injected active power at system nodes. Its nth element This represents the net active power injection at node n, which is the difference between the sum of the output of all generators at that node and the load at that node.

[0098] ;

[0099] in, Let n be the sum of the active power output of all thermal power plants at node n. The sum of all active power outputs from light and heat at node n. Let n be the sum of the active power output of all photovoltaic cells at node n. Let n be the active load of node n.

[0100] According to Kirchhoff's laws and the DC power flow approximation, the phase angle vector between node injected power and node voltage is... The relationship can be represented as a linear nodal equation:

[0101] ;

[0102] in, Here is the nodal susceptance matrix. This represents the number of nodes. Since the phase angle of the entire network requires a reference point, and the system must have a balancing node, its phase angle is typically specified. To balance the active power loss of the entire network, the original matrix Singularity exists. By removing the rows and columns corresponding to the balancing nodes, a reduced-order and non-singular nodal susceptance matrix can be obtained. Based on this, the rest The phase angle vector of each node can be obtained by inverting the matrix:

[0103] ;

[0104] in, After removing the balancing node, the remaining The node voltage phase angle vector composed of nodes Let be the inverse of the reduced-order and non-singular nodal susceptance matrix. After removing the element corresponding to the balanced node, the rest... The node net injected active power vector consisting of nodes.

[0105] Define the branch-node correlation matrix of the system If the starting point of a branch road is the first The endpoint is the nth node. The nth node, then the nth node in that row Column 1, No. The column is -1, the rest are 0, and the branch diagonal susceptance matrix , This represents the number of branches. Its diagonal elements represent the susceptance of each branch. , For the first Reactance of the branch circuit. Power system to be dispatched. Active power flow vector of the line It can be expressed as the product of the phase angle difference between the two ends of the branch and the branch susceptance:

[0106] ;

[0107] Substituting the phase angle expression into the subsequent equations will completely eliminate the intermediate variable, phase angle. A direct linear mapping from node-injected power to line power flow is established. To maintain matrix dimensionality consistency, the columns corresponding to slack nodes are padded with all-zero vectors, ultimately yielding the complete power transfer distribution factor matrix. :

[0108] ;

[0109] Power transfer distribution factor matrix Middle elements The physical meaning of it is extremely intuitive and universal: it represents the proportionality coefficient that causes a change in the active power flow of the l-th line when the active power injection of node n increases by 1 unit and is absorbed equally by the balancing node.

[0110] Based on the complete linearization derivation above, regardless of the size of the power grid topology, the spatial power flow security constraints of this model can ultimately be expressed as a standard set of polyhedral inequalities, perfectly adapting to the solution format of the external quadratic programming security layer:

[0111] ;

[0112] In the formula, for The active power flow vector of the basic line before the action is issued; The vector representing the change in net active power injected into the node due to the correction action; This represents the upper limit vector of active power flow for thermal stability of the line. This formula corresponds to the system of linear algebraic inequalities in the linear surrogate model of critical path power flow.

[0113] In one possible manner, the discrete state is obtained by the following method:

[0114] Obtain the comprehensive command variable of thermal storage power in the thermal storage system of the power system to be dispatched;

[0115] When the comprehensive instruction variable is greater than the preset variable threshold, the discrete state is a heating state; when the comprehensive instruction variable is less than or equal to the preset variable threshold, the discrete state is a heat release state.

[0116] It should be understood that the comprehensive command variable of thermal storage power can be used to characterize the magnitude of the charging and discharging power of the thermal storage system, comprehensively reflecting the operating power command of the thermal storage device. In this embodiment of the invention, the comprehensive command variable can be obtained when the Actor network in the PI-TD3-Safety algorithm outputs the high-dimensional original action; this embodiment of the invention does not specifically limit this. When the comprehensive command variable is extracted... Then, based on the comprehensive instruction variables To determine the discrete state. In this embodiment of the invention, the preset variable threshold can be 0, and the process of determining the discrete state can be expressed by the following calculation formula.

[0117] when At that time, the discrete state is a heated state;

[0118] when When the discrete state is exothermic, the embodiments of the present invention do not specifically limit this.

[0119] For example, Markov decision processes, a mathematical framework for sequential decision problems, can be used to describe the complete process of an agent interacting with its environment to make decisions. In this embodiment of the invention, a secure projection layer can be placed between the action output end of the Actor network and the execution end of the underlying communication flow environment. This secure projection layer can then correct the original action output by the Actor network to obtain a modified action.

[0120] To enable the agent to anticipate potential limits before issuing actions, the state space of the objective Markov decision process is reconstructed. This state space, in addition to including conventional weather and equipment states, must be expanded to reflect internal characteristics reflecting the degree of grid congestion. The reconstructed state space is shown below. external observation state and internal power grid status It is pieced together:

[0121] ;

[0122] External observation status This primarily reflects objective boundary conditions that are not subject to the subjective control of the scheduler, including the net load demand at the current moment. Normal direct radiation and global horizontal radiation .

[0123] Internal power grid status In addition to including the output level of each unit at the previous moment: , and the current energy state of the thermal storage tank In addition, the power flow load rate of key sections was specifically introduced. .

[0124] ;

[0125] Introduction The core objective is to endow Actor networks with topology awareness, enabling them to proactively avoid high-risk action zones that are highly likely to induce flow deviations during long-term economic optimization. for Active power flow vector of critical transmission lines at all times.

[0126] The safety projection layer is constructed between the action output end of the Actor network and the execution end of the underlying AC power system environment. Its insertion position is after the agent outputs its original action and before the execution end of the underlying AC power flow environment. The quadratic programming solver in the safety projection layer constitutes the action projection layer. When the agent outputs its original action, if the original action does not exceed the bounds, all algebraic inequalities related to power flow, output, and thermal storage are considered. Constructing a safety polyhedron Internally, the corrected action that satisfies the constraints is obtained directly; if the original action goes out of bounds, the QP solver in the safety projection layer will search for it in spatial geometry. The safe alternative point with the shortest Euclidean distance and satisfying the constraints can be expressed by the following formula: The correction process of the safe projection layer to the original action can be expressed by the following calculation:

[0127] ;

[0128] ;

[0129] ;

[0130] in, The corrective actions that, after being corrected by the safety projection layer, actually affect the underlying AC power flow environment during the target Markov decision-making process. To represent finding the minimum Euclidean projection distance with candidate action 'a' as the optimization variable, Candidate action variables, for The original action output by the Actor network at the given moment. The linear inequality constraint matrix for the safety polyhedron. Let the upper bound vector be the linear inequality of the safety polyhedron. Let be the lower bound vector of the action variables. This is the upper bound vector of action variables.

[0131] For example, the PI-TD3-Safety algorithm can be considered an improved reinforcement learning algorithm based on the TD3 algorithm (Twin Delayed Deep Deterministic Policy Gradient). The PI-TD3-Safety algorithm comprises the PI algorithm (Physics-Informed), the TD3 algorithm, and the Safety algorithm (integrated with Cvxpylayers, a differentiable safety projection layer). The PI algorithm is a physical prior gradient algorithm obtained by introducing a first-order transient frequency analytical model. The TD3 algorithm forms the basic framework and can include an Actor-Critic network for policy learning of continuous scheduling actions. The Safety algorithm serves as an outer action safety correction module, performing secondary planning projection correction to ensure that the output scheduling actions meet static operational constraints such as unit, thermal storage, and line power flow.

[0132] In this embodiment of the invention, after obtaining the target Markov decision process, the PI-TD3-Safety algorithm can be used to solve the target Markov decision process to obtain the target scheduling parameters. These target scheduling parameters may include the average daily total operating cost, the extreme value of the NK fault frequency, the maximum power flow on the critical line, the positive / negative reserve shortage, and the decision-making time; however, this embodiment of the invention does not specifically limit these parameters. After obtaining the target scheduling parameters, the corresponding parameters in the power system to be scheduled can be adjusted based on these parameters, thereby improving the stability and accuracy of the operation of the power system to be scheduled.

[0133] In one possible approach, the step of solving the target Markov decision process using the PI-TD3-Safety algorithm to obtain the target scheduling parameters includes:

[0134] like Figure 2 As shown, the Actor-Critic network is trained using the TD3 algorithm in the PI-TD3-Safety algorithm. The Actor network receives the current system state of the target Markov decision process. And output the original action The original action was corrected by the safety projection layer to Afterwards, it acts on the underlying communication flow environment, and the environment returns to the next state. and rewards The Critic network evaluates the value of states, actions, rewards, and the next state from the experience replay pool and provides updated gradients to the Actor network. Figure 2 In It can be represented as the state, action, reward, and next state in the experience replay pool;

[0135] A differentiable safe projection layer is constructed using Cvxpylayers in the PI-TD3-Safety algorithm, and the original action is then subjected to a second planning projection through the differentiable safe projection layer to obtain the corrected action.

[0136] The first-order transient frequency analytical model in the PI-TD3-Safety algorithm is used to transform the correction action into a physical prior gradient and embed it into the Actor network update, guiding the agent to actively reserve system inertia and frequency safety margin.

[0137] The Actor-Critic network is iteratively updated based on the state and reward signals obtained from the feedback of the target Markov decision process and the physical prior gradient to obtain the target scheduling parameters.

[0138] It should be understood that an Actor-Critic network can include both Actor networks and Critic networks. The Actor network can be a policy network, taking the system state of the environment as input and outputting continuous decision actions. The Critic network can be a value network, primarily used to evaluate the merits of performing a certain action in the current state, providing updated gradient signals to the Actor network and guiding the Actor to adjust its policy. After training the Actor-Critic network using the TD3 algorithm in the PI-TD3-Safety algorithm, a differentiable safe projection layer can be constructed using Cvxpylayers in the PI-TD3-Safety algorithm. This differentiable safe projection layer then performs a secondary planning projection on the original action to obtain the corrected action.

[0139] Specifically, such as Figure 2 As shown, where, Figure 2 The policy network in this embodiment is the Actor network, and the value network is the Critic network. During the training of the Actor-Critic network using the TD3 algorithm, the two Target Q networks respectively train the next state from the empirical samples of the target Markov decision process. and target action The target action value is evaluated, and the smaller of the two values ​​is used to construct the TD target value. The two Target Q networks are... Figure 2 The target value network consists of target value network 1 and target value network 2; the TD target value is used to calculate and minimize the Bellman residual of the Critic network, which includes... Figure 2 Current value network 1 and current value network 2 in the data:

[0140] ;

[0141] in, The Bellman residual loss for the Critic network. For the expectation, The goal-oriented Markov decision process in The reward for returning in time As a discount factor, For the next state, For the next state The target action obtained by smoothing the target strategy is as follows. For the next state The parameters of the Critic network are as follows: This is the current state. For experience replay pool and state The corresponding execution action, These are the current Critic network parameters. The number of Target Q networks.

[0142] Furthermore, target policy smoothing regularization is used to enhance policy stability and smooth the Q-function; that is, when calculating the Bellman residual, in the next state... Actions taken Will be selected as:

[0143] ;

[0144] in, This represents the Target policy network, which includes... Figure 2 The current policy network and the target policy network in the text; The noise to be added is usually Gaussian noise, and its amplitude is clipped to limit it to a small range.

[0145] Subsequently, the correction action can be transformed into a physical prior gradient and embedded into the Actor network update through the first-order transient frequency analytical model in the PI-TD3-Safety algorithm, guiding the agent to actively reserve system inertia and frequency safety margin; and the Actor-Critic network is iteratively updated based on the state, reward signal and physical prior gradient obtained from the target Markov decision process feedback to obtain the target scheduling parameters.

[0146] Specifically, the algorithm alternates between end-to-end differentiable projection based on Cvxpylayers and physical gradient internalization based on a first-order transient analytical model. For each control step size t, after the Actor network outputs the original action, the safe projection layer solves the quadratic programming problem in real time. The specific solution process is expressed by the following calculation formula:

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] ;

[0152] in, The corrective action taken by the target Markov decision process, after correction by the safety projection layer, and actually applied to the underlying AC power flow environment at time t. Let t be the original action output by the Actor network. This is the upper limit vector of the active power flow for thermal stability of the critical path. The active power flow vector of the basic line before the action is issued at time t. The power transfer distribution factor matrix, To correct the action The resulting vector of net active power injection at the nodes. To correct the primary frequency regulation reserve capacity corresponding to the action, To pre-set the maximum sudden active power deficit, Let be the lower bound vector of the action variables. Let be the upper bound vector of the action variables. To perform safety correction actions The rate of frequency change of the power system to be dispatched later, The maximum rate of frequency change of the power system to be dispatched;

[0153] At each control step, the security projection layer receives the original action. And solve the optimization problem shown to obtain the corrected action. Simultaneously, the constructed quadratic programming problem is encapsulated as a differentiable optimization layer of Cvxpylayers. The differentiable safe projection layer treats the projection process as a parameterized computational operator, where the Actor network outputs the original action. Then, the partial derivatives at the KKT points are obtained using the implicit function theorem, specifically expressed by the following formula:

[0154] ;

[0155] in, This is the accurate local gradient obtained through KKT conditional analysis. The objective function of the Actor network with respect to the network parameters gradient, This refers to the number of experience samples contained in a single microbatch during experience replay. The action value function Q of the Critic network with respect to the modified action gradient, Correction actions for the output of the secure projection layer. The original action output by the Actor network. For this policy action with respect to the Actor network parameters The gradient;

[0156] When internalizing the physical gradient based on the first-order transient frequency analytical model, the correction action is embedded into the loss function of the Actor network. By utilizing the frequency analytical model with continuous differentiability, a directional prior that conforms to the physical laws of the power grid is provided for the gradient descent of the neural network.

[0157] Based on the first-order frequency transient model of the system that takes into account the governor response, when faced with a sudden active power deficit... At this time, the frequency response of the power system to be dispatched exhibits a continuous dynamic process in the time domain. For a single disturbance scenario, the extreme value of the frequency drop of the power system to be dispatched after the disturbance is expressed by the following formula:

[0158] ;

[0159] in, This refers to the extreme frequency drop value of the power system to be dispatched. R represents the sudden active power deficit of the power system to be dispatched under the current dispatch action; D represents the equivalent speed governor droop coefficient of the power system to be dispatched; and D represents the load damping constant of the power system to be dispatched. The mechanical power gain constant of the prime mover; The time it takes for the frequency to drop to its lowest point;

[0160] In the batch training mechanism of deep reinforcement learning, to support the efficient parallel computation of the automatic differentiation engine, the aforementioned physical scalar model needs to be extended to a high-dimensional space. Assuming a single micro-batch contains B empirical samples, the system burst power deficit vector is defined as... The corresponding time constant vector is The droop coefficient vector is By using mapping calculations, the extreme frequency drop values ​​of the power system to be dispatched corresponding to all states and actions under each empirical sample are solved. ;

[0161] When the absolute value of the frequency drop extreme caused by the original action exceeds the threshold specified in the power grid safety regulations. In this case, a one-way asymmetric physical boundary violation penalty is triggered, and its physical loss function is expressed by the following formula:

[0162] ;

[0163] in, For physical loss function, Thresholds defined by power grid safety regulations, For the i-th experience sample in the original action The absolute value of the frequency drop extremum obtained from the first-order transient frequency analytical model;

[0164] By fusing the physical loss function and based on the deterministic policy gradient theorem, the parameter update gradient in the Actor network is reconstructed as a linear superposition of the purely data-driven action value gradient and the physical prior gradient:

[0165] ;

[0166] in, The loss function of the Actor network Regarding Actor network parameters The update gradient, The action value function output by the Critic network. The Jacobian matrix of the differentiable secure projection layer is obtained from the KKT conditions. For physical penalty adaptive weighting coefficients, For the action value function with respect to the modified action gradient, The physical loss function with respect to the original action gradient, For the Actor network in the target Markov decision process state The output of the deterministic strategy.

[0167] To verify the algorithm, such as Figure 2 As shown, a model simulation environment is set up. Specifically, in the improved IEEE-30 node system, a 200MW photovoltaic cluster, a 100MW solar thermal power plant, a 3500MWh thermal storage system, and a 120MW electric heater are connected to the sending-end collection node, while four conventional thermal power units are retained to provide basic active power support and underlying inertia for the system. Based on the typical damping and governor characteristics of the improved IEEE-30 node system under low-inertia operation, the system load damping constant is set. Equivalent governor droop coefficient Mechanical power gain The time constant at the lowest frequency Set as To simulate the rigid constraints of microscopic equipment limits and macroscopic network topology on scheduling, and in accordance with the "Guidelines for the Safety and Stability of Power Systems," the thermal stability capacity limit of key transmission sections is set at 120MW for main transmission lines 22-24; and the maximum frequency change rate limit is set at 0.5Hz / s, with the safety red line for the lowest frequency drop point at 49.2Hz.

[0168] In terms of extreme test scenario design, to fully verify the rigid cutoff capability of the outer action projection layer and the defensive foresight of the inner physical prior, this invention selects noon, the time when the system's online rotational inertia is lowest and its anti-disturbance capability is weakest, as the extreme Nk fault breakpoint. It is assumed that a sudden compound disturbance occurs at this time: the photovoltaic cluster's output drops by 30% instantaneously due to severe weather, while simultaneously, a single-pole blockage of the ultra-high voltage DC line causes a sudden 15% increase in active power demand at the load center.

[0169] This invention's algorithm is built upon the TensorFlow 2.0 deep learning framework. To adapt to the specific requirements of the two-layer safety architecture, this chapter adaptively adjusts the core hyperparameters of PI-TD3-Safety. Regarding the basic neural network parameters, the learning rate of the Actor network is set to [value missing], the learning rate of the Critic network is set to [value missing], the experience replay pool size remains unchanged at 300,000, and the total number of training epochs is set to 1500.

[0170] like Figure 3 As shown, the convergence trajectories of the economic rewards of each algorithm exhibit clear differences over 1500 training rounds. PI-TD3 shows improved training stability and a more proactive response to frequency risks after introducing physical priors; however, its constraint satisfaction capability remains limited under extreme conditions due to the lack of rigid protection at its execution layer. PI-TD3-Safety, on the other hand, performs better in both convergence speed and final cumulative reward, indicating a strong complementarity between the guiding role of physical priors in policy learning and the corrective role of safety projections on execution boundaries.

[0171] Figure 4This paper demonstrates that the proposed PI-TD3-Safety algorithm exhibits a significant two-stage evolutionary characteristic in terms of the cumulative number of constraint violations. During the initial 200 rounds of random exploration in the early training phase, the Actor network parameters are in a state of random initialization or preliminary fitting, and the agent needs to fully traverse the action space by introducing exploration noise. In this stage, the system inevitably experiences a small number of physical violations, resulting in a slight increase in the cumulative number of violations. This initial exploratory violation truly reflects the objective physical laws of reinforcement learning's trial and error in unknown environments. However, as training progresses, the physical prior gradient of the first-order transient frequency model begins to deeply participate in network updates, and the Actor network gradually internalizes the complex frequency dynamics and safety boundaries. After the exploration phase, the cumulative violation curve of PI-TD3-Safety rapidly flattens out, meaning that almost no new violation events are triggered in subsequent massive training sessions.

[0172] like Figure 5 As shown, the 24-hour day-ahead scheduling results demonstrate a relatively clear multi-energy temporal complementarity relationship while maintaining the basic inertia requirement. Table 1 also presents a comparison of the 24-hour comprehensive scheduling economy and transient safety indicators for different algorithms. Figure 5 Load in the table represents the load of the power system to be dispatched. G2, G5, G11 and G13 represent the conventional thermal power units that intervene at nodes 2, 5, 11 and 13 respectively. PV stands for Photovoltaic, which means photovoltaic power generation unit.

[0173] Table 1 Comparison of 24-hour integrated scheduling economy and transient safety indicators of different algorithms .

[0174] like Figure 6 As shown, TD3-Safety, under the action of the external projection layer, can limit the critical line power flow to 120.0MW and maintain zero reserve shortage, but at the cost of increased daily operating costs to 1.1256 million yuan and single-step decision-making time to 38.6ms. This indicates that when the strategy itself lacks proactive risk avoidance capabilities, the projection layer needs to intervene more frequently to correct actions, resulting in more conservative system operation and increased computational burden on the execution layer. Furthermore, although its algebraic boundary constraints are satisfied, the frequency extreme value still drops to 48.95Hz, indicating that passive correction alone is insufficient to compensate for the prior reservation required for frequency dynamic safety. Figures 3-6As shown, the proposed algorithm PI-TD3-Safety maintains the frequency extreme value at 49.25Hz under the set extreme scenario, which is higher than the 49.2Hz threshold, while controlling the maximum power flow of the critical line at 120.0MW and maintaining zero reserve shortage. Compared with TD3-Safety, its daily operating cost is reduced from RMB 1,125,600 to RMB 988,000, a reduction of approximately 12.2%; the single-step decision-making time is reduced from 38.6ms to 24.6ms, a reduction of 36.3%. This result shows that when physical priors and safety projections work together, the policy network can generate original actions that are closer to the safe feasible region, thereby ensuring that the execution layer constraints are satisfied while reducing the intervention frequency of the external projection layer and improving overall economy and real-time performance.

[0175] Through the above technical solutions, such as Figure 2 As shown, the PI-TD3-Safety framework proposed in this invention adopts a two-layer collaborative structure of physical prior inner guidance and safety projection outer correction. For highly nonlinear, time-scale-spanning, and differentiable dynamic processes such as frequency transients, physical prior gradients are used to provide directional information in advance during the policy update phase. For safety boundaries with a high degree of algebraicity and linearity, such as line power flow, output boundaries, and reserve lower limits, rigid correction is implemented through the external projection layer. The former enhances the policy's proactive risk avoidance capability, while the latter ensures that the currently set algebraic safety constraints are not exceeded during the execution phase.

[0176] To address the safety dispatching requirements of high-proportion renewable energy power grids operating with low inertia and under fault disturbances, this invention proposes a PI-TD3-Safety two-layer dispatching framework. This framework employs a collaborative mechanism between a bottom-layer AC power flow environment and an outer-layer DC-PTDF (DC-Power Transfer Distribution Factor) linear safety projection: the bottom-layer AC power flow environment reflects nonlinear operating characteristics such as reactive power balance, node voltage amplitude, and line network losses. Before action issuance, the safety projection layer utilizes a DC-PTDF linear proxy to rapidly correct algebraic safety boundaries such as critical line power flow, unit output, thermal storage capacity, and reserve lower limit, thus balancing the accuracy of the bottom-layer physical environment with the timeliness of online inference. Simultaneously, this invention embeds the physical prior gradient formed by the first-order transient frequency analytical model into the Actor network parameter update, and combines this with the Jacobian matrix backpropagation mechanism of the differentiable safety projection layer to reduce the value assessment deviation between the original action and the corrected action, thereby improving the safety and executability of the dispatching strategy.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A power system dispatching method considering physical priors and security projections, characterized in that, include: A joint system scheduling model for the power system to be dispatched, taking into account the security projection layer, is constructed. The joint system scheduling model includes an objective function and constraints. The objective function is constructed with the goal of minimizing the sum of the operating cost, reserve cost, and renewable energy curtailment penalty cost of the power system to be dispatched. The constraints include unit operation constraints, thermal storage system operation constraints, and renewable energy and reserve operation constraints. Determine the approximate DC power flow, power transfer distribution factor, and discrete state of the thermal storage system in the power system to be dispatched; In the joint system scheduling model, discrete states are fixed by intention locking and conditional projection mechanisms. A critical line power flow linear proxy model is constructed based on DC power flow approximation and power transfer distribution factor. The nonlinear power flow constraints are replaced by the critical line power flow linear proxy model to obtain the target joint system scheduling model. The target joint system scheduling model is reconstructed into a target Markov decision process, wherein the target Markov decision process includes a secure projection layer, an Actor network, and an underlying communication flow environment. The secure projection layer is set between the action output end of the Actor network and the execution end of the underlying communication flow environment. The secure projection layer is used to correct the original actions output by the Actor network. The target Markov decision process is solved using the PI-TD3-Safety algorithm to obtain the target scheduling parameters, and the power system to be scheduled is then scheduled based on these parameters.

2. The power system dispatching method considering physical priors and security projections according to claim 1, characterized in that, The discrete state is obtained through the following method: Obtain the comprehensive command variable of thermal storage power in the thermal storage system of the power system to be dispatched; When the comprehensive instruction variable is greater than the preset variable threshold, the discrete state is a heating state; when the comprehensive instruction variable is less than or equal to the preset variable threshold, the discrete state is a heat release state.

3. The power system dispatching method considering physical priors and security projections according to claim 1, characterized in that, The critical path power flow linear surrogate model is expressed by the following calculation formula: ； ； ； The security constraints of the critical path power flow linear surrogate model are expressed by the following calculation formula: ； in, Inject the net active power vector into the node. Here is the nodal susceptance matrix. The node voltage phase angle vector. This is the active power flow vector of the line. The branch diagonal susceptance matrix, This is the branch-node correlation matrix. The power transfer distribution factor matrix, for The active power flow vector of the basic line before the action is issued. To correct the change vector of net active power injected into the node caused by the action. This is the upper limit vector of the active power flow for line thermal stability.

4. The power system dispatching method considering physical priors and security projections according to claim 1, characterized in that, The correction process of the safety projection layer to the original motion is expressed by the following calculation formula: ； ； ； in, In the target Markov decision-making process, after correction by the safety projection layer, the actual effect on the underlying AC current environment is... The corrective action at any given moment To represent finding the minimum Euclidean projection distance with candidate action 'a' as the optimization variable, For candidate action variables, for The original action output by the Actor network at the given moment. The linear inequality constraint matrix for the safety polyhedron. Let the upper bound vector be the linear inequality of the safety polyhedron. Let be the lower bound vector of the action variables. Let be the upper bound vector of the action variables. It is a Euclidean norm.

5. The power system dispatching method considering physical priors and security projections according to claim 1, characterized in that, The goal scheduling parameters are obtained by solving the target Markov decision process using the PI-TD3-Safety algorithm, including: The Actor-Critic network is trained using the TD3 algorithm in the PI-TD3-Safety algorithm. The Actor-Critic network is used to receive the system state output by the target Markov decision process and output the original action based on the system state. A differentiable safe projection layer is constructed using Cvxpylayers in the PI-TD3-Safety algorithm, and the original action is then subjected to a second planning projection through the differentiable safe projection layer to obtain the corrected action. The first-order transient frequency analytical model in the PI-TD3-Safety algorithm is used to transform the correction action into a physical prior gradient and embed it into the Actor network update, guiding the agent to actively reserve system inertia and frequency safety margin. The Actor-Critic network is iteratively updated based on the state and reward signals obtained from the feedback of the target Markov decision process and the physical prior gradient to obtain the target scheduling parameters.

6. The power system dispatching method considering physical priors and security projections according to claim 1, characterized in that, The objective function is expressed by the following formula: ； in, The total operating cost of the power system to be dispatched. For operating costs, For backup costs, The cost of abandoning new energy sources.

7. The power system dispatching method considering physical priors and security projections according to claim 5, characterized in that, The process of converting the correction action into a physical prior gradient and embedding it into the Actor network update using the first-order transient frequency analytical model in the PI-TD3-Safety algorithm includes: For each control step, after the Actor network outputs the original action, the safety projection layer solves the quadratic programming problem in real time. The specific solution process is expressed by the following calculation formula: ； ； ； ； ； in, The corrective action taken by the target Markov decision process, after correction by the safety projection layer, and actually applied to the underlying AC power flow environment at time t. for The original action output by the Actor network at the given moment. This is the upper limit vector of the active power flow for thermal stability of the critical path. for The active power flow vector of the basic line before the action is issued. The power transfer distribution factor matrix, To correct the action The resulting vector of net active power injection at the nodes. To correct the primary frequency regulation reserve capacity corresponding to the action, To pre-set the maximum sudden active power deficit, Let be the lower bound vector of the action variables. Let be the upper bound vector of the action variables. To perform safety correction actions The rate of frequency change of the power system to be dispatched later The maximum rate of frequency change of the power system to be dispatched; At each control step, the safety projection layer receives the original action and solves the optimization problem to obtain the corrected action. The projection process of the differentiable safety projection layer is regarded as a parameterized computational operator. After the Actor network outputs the original action, the partial derivatives at the KKT points are obtained through the implicit function theorem, specifically expressed by the following formula: ； in, This is the accurate local gradient obtained through KKT conditional analysis. The objective function of the Actor network with respect to the network parameters gradient, This refers to the number of experience samples contained in a single microbatch during experience replay. The action value function Q of the Critic network with respect to the modified action gradient, Correction actions for the output of the secure projection layer. The original action output by the Actor network. For policy actions with respect to Actor network parameters The gradient; When internalizing the physical gradient based on the first-order transient frequency analytical model, the correction action is embedded into the loss function of the Actor network. For a single disturbance scenario, the extreme value of the frequency drop of the power system to be dispatched after the disturbance is expressed by the following formula: ； in, This refers to the extreme frequency drop value of the power system to be dispatched. R represents the sudden active power deficit of the power system to be dispatched under the current dispatch action; D represents the equivalent speed governor droop coefficient of the power system to be dispatched; and D represents the load damping constant of the power system to be dispatched. The mechanical power gain constant of the prime mover; The time it takes for the frequency to drop to its lowest point; The sudden power deficit vector in the power system to be dispatched is expressed by the following formula: ； in, For the first The burst of success corresponding to each empirical sample To represent the transpose of a vector; By using mapping calculations, the extreme values ​​of frequency drop in the power system to be dispatched corresponding to all state-action pairs under each empirical sample are obtained. ; If the absolute value of the frequency drop extreme caused by the original action exceeds the threshold defined by the power grid safety regulations, a one-way asymmetric physical over-limit penalty is triggered, and its physical loss function is expressed by the following formula: ； in, For physical loss function, Thresholds defined by power grid safety regulations, For the first A sample of experience in the original action The absolute value of the frequency drop extremum obtained from the first-order transient frequency analytical model is shown below. The original action output by the Actor network; By fusing the physical loss function and based on the deterministic policy gradient theorem, the parameter update gradient in the Actor network is reconstructed as a linear superposition of the purely data-driven action value gradient and the physical prior gradient: ； in, For the Actor network loss function Regarding Actor network parameters The update gradient, The action value function output by the Critic network. The Jacobian matrix of the differentiable secure projection layer is obtained from the KKT conditions. For physical penalty adaptive weighting coefficients, For the action value function with respect to the modified action gradient, The physical loss function with respect to the original action gradient, For the Actor network in the target Markov decision process state The output of the deterministic strategy.

Images