Power grid transient frequency emergency control method, device and equipment based on reinforcement learning

Abstract

The invention relates to the technical field of transient frequency and emergency control, in particular to a power grid transient frequency emergency control method, device and equipment based on reinforcement learning, and the method comprises the steps: obtaining the current operation data of a power grid if the power grid is in a fault state; and inputting the current operation data to a target transient frequency emergency control model, so as to obtain an actual emergency control strategy from a plurality of emergency control strategy tables in the target transient frequency emergency control model through matching, the target transient frequency emergency control model being based on a power grid fault data sample, training by using a preset improved depth deterministic strategy gradient algorithm; and performing emergency stability control on the transient frequency of the power grid according to the actual emergency control strategy. Therefore, the problem of frequency fluctuation caused by large-scale power grid disturbance is solved, the reliability and economical efficiency of stable operation of the power grid are improved, generator tripping and load shedding online real-time emergency control strategies are quickly and accurately generated, and quick recovery of transient frequency stability after a fault is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of transient frequency and emergency control, and in particular to a method, device and equipment for emergency control of transient frequency of a power grid based on reinforcement learning. Background Art

[0002] New energy is gradually replacing traditional fossil energy to occupy a dominant position, and the frequency stability problem has gradually become an important factor limiting its access ratio. In a high-proportion renewable energy power system, the volatility and intermittent characteristics of new energy such as wind power and photovoltaics increase the randomness and volatility of the power grid, exacerbating the frequency disturbance and active power imbalance of the system. When a large disturbance or fault occurs in the power system, resulting in an imbalance in the active output and load power of the system, the system frequency will be greatly offset, thereby affecting the safe and stable operation of the generator set, the power grid and user equipment. If the grid frequency is lower than the action frequency of the generator set protection device at this time, it may cause the generator to trip, the system to be disconnected, and even cause the collapse of the power grid, which will have a bad impact on social production. In addition, most new energy units operate in maximum power tracking mode and do not provide spinning reserve capacity. Large-scale renewable energy access and replacement of the original thermal power units will lead to a decrease in the equivalent inertia of the power grid and a reduction in the primary frequency regulation reserve capacity, further exacerbating the frequency safety risk of multi-DC feed-in power grids under severe power disturbances such as DC blocking.

[0003] With the development of artificial intelligence technology and the improvement of computer hardware performance, artificial intelligence-based modeling, analysis and decision-making methods are widely used in the power industry and have solved a large number of complex practical problems. Machine learning methods based on data mining technology can extract abstract features from a large amount of high-dimensional complex data, and have significant advantages in the timeliness of data analysis and processing. At the same time, with the advancement of wide-area measurement technology, relying on the rich wide-area measurement data of the power system, including SCADA and PMU data, a post-disturbance frequency response analysis model based on machine learning methods is established to predict the steady-state frequency after the disturbance and the minimum and maximum frequency values, which has become a new idea for emergency control of transient frequency stability of the power system.

[0004] Therefore, it is urgent to explore the technical research of emergency control methods for transient frequency in power grids with a high proportion of renewable energy based on reinforcement learning, to achieve fast and accurate integrated online real-time emergency coordinated control of active and reactive power control, to respond to faults quickly and accurately, and to achieve stable recovery of transient frequency after faults, which is beneficial to the economic operation of the power grid. Summary of the invention

[0005] The present application provides a method, device and equipment for emergency control of transient frequency of a power grid based on reinforcement learning to solve problems such as frequency fluctuations caused by large-scale power grid disturbances.

[0006] The first aspect of the present application provides a reinforcement learning-based method for emergency control of transient frequency of a power grid, comprising the following steps: determining whether the power grid is in a fault state; if the power grid is in the fault state, obtaining the current operating data of the power grid; inputting the current operating data into a target transient frequency emergency control model to match an actual emergency control strategy from a plurality of emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on a power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm; and performing emergency stability control on the transient frequency of the power grid according to the actual emergency control strategy.

[0007] Optionally, before inputting the current operating data into the target transient frequency emergency control model, the process includes: obtaining the power grid fault data sample, wherein the power grid fault data sample includes historical fault data; based on the historical fault data, training the target transient frequency emergency control model using a preset improved deep deterministic policy gradient algorithm to obtain a plurality of machine cutting and load shedding emergency control strategies for the power grid; and obtaining the plurality of emergency control strategy tables based on the plurality of machine cutting and load shedding emergency control strategies.

[0008] Optionally, the improved deep deterministic policy gradient algorithm preset based on the historical fault data trains the target transient frequency emergency control model to obtain a variety of machine cutting and load shedding emergency control strategies for the power grid, including: based on the historical fault data, utilizing the current network of Actors in the target transient frequency emergency control model to interact with the power grid to obtain an action of superimposing normal distribution random variable disturbances; controlling the power grid to execute the action of superimposing normal distribution random variable disturbances to obtain a new state and new reward of the power grid, and determining whether the new state is a terminal state; if the new state is a terminal state, obtaining a variety of machine cutting and load shedding emergency control strategies for the power grid.

[0009] Optionally, after determining whether the new state is a terminal state, the method includes: if the new state is not a terminal state, calculating a current Q value based on the action of superimposing normal distribution random variable disturbance, the new state and the new reward, and updating the Critic current network in the target transient frequency emergency control model based on the current Q value; calculating a target Q value using the updated Critic current network, updating the Actor current network based on the target Q value, and continuing to use the updated Actor current network to interact with the power grid to obtain an action of superimposing normal distribution random variable disturbance, controlling the power grid to execute the action of superimposing normal distribution random variable disturbance, and obtaining a new state and a new reward for the power grid, until the new state is the terminal state, and obtaining a variety of machine cutting and load cutting emergency control strategies for the power grid.

[0010] Optionally, the target transient frequency emergency control model includes an objective function, a static safety constraint, a transient stability constraint, an action space and a reward mechanism, wherein:

[0011] The objective function is

[0012]

[0013] Among them, N G is the number of switchable generator nodes, N L is the number of nodes that can cut loads, c G,i is the cost of switching off the i-th generator node, c L,j is the load shedding cost of the jth load node, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable,

[0014] The static safety constraints include equality constraints and inequality constraints, wherein the equality constraints are:

[0015]

[0016] Among them, P Gi is the active power generated by the generator, Q Gi is the reactive power of the generator, P Li is the active power consumed by the load, Q Li Reactive power consumed by the load, U i is the magnitude of the node voltage, θ ij is the phase angle difference of the node voltage, G ij is the conductance of the line, B ij is the electrical conductance of the line,

[0017] The inequality constraints are:

[0018] P imin <P<P imax ,i=1,2,...N G

[0019] Q imin <Q<Q imax ,i=I,2,...N G

[0020] U jmin <U<U jmax ,j=I,2,...N L

[0021] fjmin <f<f jmax ,j=1,2,...N L

[0022] u imin i imax ,i=1,2,...N G

[0023] u jmin j jmax ,j=I,2,...N L ;

[0024] Among them, P imax and P imin is the upper and lower limits of the generator active power, Q imax and Q imin is the upper and lower limits of the generator reactive power, U imax and U jmin is the upper and lower limits of bus voltage, f jmax and fjmin are the upper and lower limits of frequency,

[0025] The transient stability constraint is:

[0026] f(X,u i ,u j )>M stable

[0027] Among them, f(X) is the transient stability rule expression after cutting the generator and shedding the load, X is the system operation state after cutting the generator and shedding the load, M stable is the transient stability threshold of the action space, M stable >0,

[0028] The action space includes machine shedding strategy and load shedding strategy.

[0029] The reward mechanism is:

[0030]

[0031] Among them, I F is the transient frequency stability risk index value, I F max is the maximum frequency safety index value, c1, c2, c3 are all weight coefficients, is the total cutting capacity, P G,i (t) is the capacity of the generator node i removed at time t, m invalid It is the invalid action flag, and Step is the number of steps for cutting the machine and load.

[0032] ​​​​The second aspect of the present application provides a reinforcement learning-based grid transient frequency emergency control device, including: a judgment module, used to judge whether the grid is in a fault state; an acquisition module, used to obtain the current operating data of the grid if the grid is in the fault state; a matching module, used to input the current operating data into a target transient frequency emergency control model to match the actual emergency control strategy from a plurality of emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on grid fault data samples and is trained using a preset improved deep deterministic policy gradient algorithm; a control module, used to perform emergency stability control on the transient frequency of the grid according to the actual emergency control strategy.

[0033] Optionally, before inputting the current operating data into the target transient frequency emergency control model, the matching module is also used to: obtain the power grid fault data samples, wherein the power grid fault data samples include historical fault data; based on the historical fault data, train the target transient frequency emergency control model using a preset improved deep deterministic policy gradient algorithm to obtain a variety of machine cutting and load shedding emergency control strategies for the power grid; and obtain the variety of emergency control strategy tables based on the variety of machine cutting and load shedding emergency control strategies.

[0034] Optionally, the matching module is also used to: based on the historical fault data, utilize the current network of Actors in the target transient frequency emergency control model to interact with the power grid to obtain an action of superimposing normal distribution random variable disturbances; control the power grid to execute the action of superimposing normal distribution random variable disturbances to obtain a new state and new reward of the power grid, and determine whether the new state is a terminal state; if the new state is a terminal state, obtain a variety of machine cutting and load cutting emergency control strategies for the power grid.

[0035] Optionally, after determining whether the new state is a terminal state, the matching module is also used to: if the new state is not a terminal state, calculate the current Q value based on the action of the superimposed normal distribution random variable disturbance, the new state and the new reward, and update the Critic current network in the target transient frequency emergency control model based on the current Q value; calculate the target Q value using the updated Critic current network, and update the Actor current network based on the target Q value, and continue to use the updated Actor current network to interact with the power grid to obtain the action of the superimposed normal distribution random variable disturbance, control the power grid to execute the action of the superimposed normal distribution random variable disturbance, and obtain the new state and new reward of the power grid, until the new state is the terminal state, and obtain a variety of machine cutting and load cutting emergency control strategies for the power grid.

[0036] Optionally, the target transient frequency emergency control model includes an objective function, a static safety constraint, a transient stability constraint, an action space and a reward mechanism, wherein:

[0037] The objective function is

[0038]

[0039] Among them, N G is the number of switchable generator nodes, N L is the number of nodes that can cut loads, c G,i is the cost of switching off the i-th generator node, c L,j is the load shedding cost of the jth load node, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable,

[0040] The static safety constraints include equality constraints and inequality constraints, wherein the equality constraints are:

[0041]

[0042] Among them, P Gi is the active power generated by the generator, Q Gi is the reactive power of the generator, P Li is the active power consumed by the load, Q Li Reactive power consumed by the load, U i is the magnitude of the node voltage, θ ij is the phase angle difference of the node voltage, G ij is the conductance of the line, B ij is the electrical conductance of the line,

[0043] The inequality constraints are:

[0044] P imin <P<P imax ,i=1,2,...N G

[0045] Q imin <Q<Q imax ,i=I,2,...N G

[0046] U jmin <U<U jmax ,j=I,2,...N L

[0047] f jmin <f<f jmax,j=1,2,...N L

[0048] u imin i imax ,i=1,2,...N G

[0049] u jmin j jmax ,j=I,2,...N L ;

[0050] Among them, P imax and P imin is the upper and lower limits of the generator active power, Q imax and Q imin is the upper and lower limits of the generator reactive power, U imax and U jmin is the upper and lower limits of bus voltage, f jmax and fjmin are the upper and lower limits of frequency,

[0051] The transient stability constraint is:

[0052] f(X,u i ,u j )>M stable

[0053] Among them, f(X) is the transient stability rule expression after cutting the generator and shedding the load, X is the system operation state after cutting the generator and shedding the load, M stable is the transient stability threshold of the action space, M stable >0,

[0054] The action space includes machine shedding strategy and load shedding strategy.

[0055] The reward mechanism is:

[0056]

[0057] Among them, I F is the transient frequency stability risk index value, I F max is the maximum frequency safety index value, c1, c2, c3 are all weight coefficients, is the total cutting capacity, P G,i (t) is the capacity of the generator node i removed at time t, m invalid It is the invalid action flag, and Step is the number of steps for cutting the machine and load.

[0058] ​​​​The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the reinforcement learning-based power grid transient frequency emergency control method as described in the above embodiment.

[0059] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the reinforcement learning-based power grid transient frequency emergency control method as described in the above embodiment.

[0060] In the above implementation, it is determined whether the power grid is in a fault state. If the power grid is in a fault state, the current operation data of the power grid is obtained, and the current operation data is input into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on the power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm; emergency stability control is performed on the transient frequency of the power grid according to the actual emergency control strategy. In this way, the problem of frequency fluctuations caused by large-scale power grid disturbances is solved, the reliability and economy of stable operation of the power grid are improved, and by preparing before the fault and responding in real time after the fault, an online real-time emergency coordination control strategy for cutting the machine and shedding the load is quickly and accurately generated, so as to achieve rapid recovery of transient frequency stability after the fault.

[0061] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0063] Figure 1 A flowchart of a method for emergency control of transient frequency of a power grid based on reinforcement learning according to an embodiment of the present application;

[0064] Figure 2 The figure is a general schematic diagram of a method for emergency control of transient frequency of a power grid based on reinforcement learning according to an embodiment of the present application;

[0065] Figure 3 It is a flowchart of a method for emergency control of transient frequency of a power grid based on reinforcement learning according to one embodiment of the present application;

[0066] Figure 4 This is an example diagram of a power grid transient frequency emergency control device based on reinforcement learning according to an embodiment of the present application;

[0067] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0069] The following describes the method, device and equipment for emergency control of transient frequency of power grid based on reinforcement learning of the embodiment of the present application with reference to the accompanying drawings. In view of the problem of frequency fluctuation caused by large-scale power grid disturbance mentioned in the above background technology, the present application provides a method for emergency control of transient frequency of power grid based on reinforcement learning, in which it is judged whether the power grid is in a fault state. If the power grid is in a fault state, the current operation data of the power grid is obtained, and the current operation data is input into the target transient frequency emergency control model, so as to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on the power grid fault data sample and trained by the preset improved deep deterministic policy gradient algorithm; the transient frequency of the power grid is emergency stabilized according to the actual emergency control strategy. Thus, the problem of frequency fluctuation caused by large-scale power grid disturbance is solved, the reliability and economy of stable operation of the power grid are improved, and the online real-time emergency coordination control strategy of cutting machine and cutting load is generated quickly and accurately by preparing before the fault and responding in real time after the fault, so as to realize the rapid recovery of transient frequency stability after the fault.

[0070] In view of the deficiencies of the prior art, the technical solution adopted in the embodiments of the present application is: using reinforcement learning methods to train and solve based on the improved deep deterministic policy gradient algorithm (DDPG), wherein the improved deep deterministic policy gradient algorithm (DDPG) model will generate actions based on the action superimposed with normal distribution random variable disturbances during the training process, wherein the action strategy includes machine cutting and load cutting actions; the basic framework of the reinforcement learning method includes five parts: agent, environment, state, action and reward, which is a kind of learning that maps from environmental state to action, and the goal is to enable the controller to obtain the maximum cumulative reward in the process of interaction with the environment. In order to apply reinforcement learning in the transient frequency emergency control optimization model, the embodiments of the present application construct a reinforcement learning environment model suitable for transient frequency emergency control optimization and regulation based on the characteristics of transient frequency emergency control.

[0071] Specifically, Figure 1 A flowchart of a method for emergency control of transient frequency of a power grid based on reinforcement learning provided in an embodiment of the present application.

[0072] Before specifically introducing the process of the network transient frequency emergency control method, it is necessary to introduce the overall process of the embodiment of the present application, including constructing "offline training module, online budget module, real-time matching module", such as Figure 2 As shown, the work is carried out in stages according to the division of key tasks, so that after an actual power grid fault occurs, a fast and accurate emergency coordination control solution can be implemented based on the online budget module and the real-time matching module.

[0073] The specific division of labor and workflow of the three modules are as follows:

[0074] 1) Offline training module

[0075] The key tasks are: offline generation of a large number of various fault data samples for training, and training the transient frequency emergency control strategy model based on the improved deep deterministic policy gradient (DDPG), and finally obtaining a trained offline transient frequency emergency control strategy.

[0076] 2) Online budget module

[0077] The key task is to solve the emergency coordination control scheme under various fault conditions online; the workflow is to aggregate the emergency coordination control strategies under various fault conditions into an emergency control strategy table online and embed it into 3) the real-time matching module as a matching resource;

[0078] 3) Real-time matching module

[0079] The key task is to match the emergency control strategy table in 2) according to the actual faults and working conditions. The workflow is to send the real-time control quantity of each control means to the corresponding stability control system according to the matched emergency coordination control strategy table.

[0080] like Figure 1 As shown, the grid transient frequency emergency control method based on reinforcement learning includes the following steps:

[0081] In step S101, it is determined whether the power grid is in a fault state.

[0082] In step S102, if the power grid is in a fault state, current operation data of the power grid is acquired.

[0083] In step S103, the current operating data is input into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on the power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm.

[0084] In step S104, emergency stability control is performed on the transient frequency of the power grid according to the actual emergency control strategy.

[0085] Optionally, before inputting the current operating data into the target transient frequency emergency control model, it includes: obtaining power grid fault data samples, wherein the power grid fault data samples include historical fault data; based on the historical fault data, using a preset improved deep deterministic policy gradient algorithm to train the target transient frequency emergency control model to obtain a variety of power grid cutting and load shedding emergency control strategies; based on the multiple cutting and load shedding emergency control strategies, obtaining a variety of emergency control strategy tables.

[0086] Optionally, in some embodiments, an improved deep deterministic policy gradient algorithm preset based on historical fault data is used to train a target transient frequency emergency control model to obtain a variety of machine cutting and load shedding emergency control strategies for the power grid, including: based on historical fault data, utilizing the current network of the Actor in the target transient frequency emergency control model to interact with the power grid to obtain an action of superimposing normal distribution random variable disturbances; controlling the power grid to execute the action of superimposing normal distribution random variable disturbances to obtain a new state and new reward of the power grid, and determining whether the new state is a terminal state; if the new state is a terminal state, obtaining a variety of machine cutting and load shedding emergency control strategies for the power grid.

[0087] Optionally, in some embodiments, after determining whether the new state is a terminal state, the process includes: if the new state is not a terminal state, calculating the current Q value based on the action of superimposing the normal distribution random variable disturbance, the new state and the new reward, and updating the Critic current network in the target transient frequency emergency control model based on the current Q value; calculating the target Q value using the updated Critic current network, updating the Actor current network based on the target Q value, and continuing to use the updated Actor current network to interact with the power grid to obtain the action of superimposing the normal distribution random variable disturbance, controlling the power grid to execute the action of superimposing the normal distribution random variable disturbance, and obtaining the new state and new reward of the power grid, until the new state is a terminal state, and obtaining a variety of machine cutting and load cutting emergency control strategies for the power grid.

[0088] Optionally, in some embodiments, the target transient frequency emergency control model includes an objective function, a static safety constraint, a transient stability constraint, an action space, and a reward mechanism.

[0089] Among them, the objective function is

[0090]

[0091] Among them, N G is the number of switchable generator nodes, N L is the number of nodes that can cut loads, c G,iis the cost of switching off the i-th generator node, c L,j is the load shedding cost of the jth load node, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable,

[0092] Static safety constraints include equality constraints and inequality constraints, where the equality constraints are:

[0093]

[0094] Among them, P Gi is the active power generated by the generator, Q Gi is the reactive power of the generator, P Li is the active power consumed by the load, Q Li Reactive power consumed by the load, U i is the magnitude of the node voltage, θ ij is the phase angle difference of the node voltage, G ij is the conductance of the line, B ij is the electrical conductance of the line,

[0095] The inequality constraints are:

[0096] P imin <P<P imax ,i=1,2,...N G

[0097] Q imin <Q<Q imax ,i=I,2,...N G

[0098] U jmin <U<U jmax ,j=I,2,...N L

[0099] f jmin <f<f jmax ,j=1,2,...N L

[0100] u imin i imax ,i=1,2,...N G

[0101] u jmin j jmax ,j=I,2,...N L ;​​​​

[0102] Among them, P imax and P imin is the upper and lower limits of the generator active power, Q imax and Q imin is the upper and lower limits of the generator reactive power, U imax and U jmin is the upper and lower limits of bus voltage, f jmax and fjmin are the upper and lower limits of frequency,

[0103] The transient stability constraint is:

[0104] f(X,u i ,u j )>M stable

[0105] Among them, f(X) is the transient stability rule expression after cutting the generator and shedding the load, X is the system operation state after cutting the generator and shedding the load, M stable is the transient stability threshold of the action space, M stable >0,

[0106] The action space includes machine shedding strategy and load shedding strategy.

[0107] The reward mechanism is:

[0108]

[0109] Among them, I F is the transient frequency stability risk index value, I F max is the maximum frequency safety index value, c1, c2, c3 are all weight coefficients, is the total cutting capacity, P G,i (t) is the capacity of the generator node i removed at time t, m invalid It is the invalid action flag, and Step is the number of steps for cutting the machine and load.

[0110] Specifically, the transient frequency emergency control optimization model environment is constructed based on reinforcement learning.

[0111] (1) Objective function

[0112] Select cutting off the machine and load as the cut-in, and set the optimization objective function to minimize the cutting off of the machine and load of the power grid. It can be expressed as:

[0113]

[0114] Where N G is the number of switchable generator nodes, N L is the number of nodes that can be cut off, cG,i is the cost of cutting the i-th generator node, cL,j is the load shedding cost of the jth load node, both are set to 1, that is, the cost of machine shedding is the same as that of load shedding, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable. Since the whole generator is shut down in the project, u i The value is a discrete integer. Under the premise of maintaining the transient stability constraint and considering the power supply reliability requirements, the load shedding amount can be cut in a certain proportion according to the degree of transient frequency drop caused by the fault. Therefore, u i is a discrete variable in (0,1).

[0115] (2) State space

[0116] It includes the output of conventional units before the disturbance, spinning reserve, unit capacity, unit regulation power, inertia time constant, output of each new energy unit, system damping coefficient, load size, DC power and disturbance amount; and should meet the following constraints:

[0117] 1) Static safety constraints:

[0118] Static safety constraints are divided into equality constraints and inequality constraints. The equality constraints are power flow constraints and should satisfy power conservation, which can be expressed as:

[0119]

[0120] Where P Gi , Q Gi is the active and reactive power generated by the generator, P Li , Q Li is the active and reactive power consumed by the load, U i ,θi j is the amplitude and phase difference of the node voltage, G ij , B ij are the conductance and susceptance of the line.

[0121] The inequality constraints are state variable and control variable constraints. The state variables should meet the voltage, power, and frequency requirements of the grid during steady-state operation, which are determined by the grid operation conditions. The control variables should meet the equipment's own and emergency control requirements, which are determined by the equipment's technical conditions and the grid operation conditions. They can be expressed as:

[0122] P imin <P<P imax ,i=1,2,...N G

[0123] Q imin <Q<Qimax ,i=I,2,...N G

[0124] U jmin <U<U jmax ,j=I,2,...N L

[0125] f jmin <f<f jmax ,j=1,2,...N L

[0126] u imin i imax ,i=1,2,...N G

[0127] u jmin j jmax ,j=I,2,...N L

[0128] Where P imax , Pimin is the upper and lower limits of the generator active power, Q imax , Q imin U is the upper and lower limits of the generator reactive power; imax , U jmin is the upper and lower limits of bus voltage, f jmax 、f jmin The upper and lower limits of the frequency.

[0129] 2) Transient stability constraints

[0130] Transient stability constraints are used to determine the stability level of the system after the generator and load are cut off, and are the basic conditions for the safe and stable operation of the power system. A reasonable generator and load cutting strategy should keep the unstable system transiently stable, and the stability margin should be greater than a certain threshold, which can be expressed as:

[0131] f(X,u i ,u j )>M stable

[0132] Where f(X) is the transient stability rule expression after the machine is cut off and the load is shed, X is the system operation state after the machine is cut off and the load is shed; M stable is the transient stability threshold, M stable >0.

[0133] (3) Action Space

[0134] The action space is an adjustable variable used to describe the specific actions of controlling the state of the transient frequency emergency control model, including the emergency control action strategy and the cutting machine A​​​​G , Load shedding A L Control measures;

[0135] (4) Reward mechanism

[0136] The reward mechanism takes into account the transient frequency stability risk index value I F , including the maximum transient frequency deviation Δf max , maximum frequency change rate and quasi-steady-state frequency deviation Δf ss Three frequency safety indicators, on this basis, the amount of machine and load shedding should be kept as small as possible. Combining the above two aspects to measure the performance of the control action, the reward function r is designed t Such as:

[0137]

[0138] In the formula, after the machine and load shedding actions are executed, the transient frequency stability risk index value I F If it is less than the preset maximum frequency safety index value I F max , the system is judged to be unstable, the reward is set to a large negative number; when the system is transiently stable after the machine and load shedding actions are executed, the action is further comprehensively evaluated from three aspects:

[0139] (1) The number of machine and load shedding steps in the current episode;

[0140] (2) Total cutting capacity P G,i (t) represents the capacity of the generator node i removed at time t;

[0141] (3) Invalid action flag bit m invalid :The capacity of the generator is limited. When the capacity of a generator has been completely removed at the previous moment, considering that the agent may still choose to remove the generator at the current moment, this is an invalid action, m invalid =1, should be punished.

[0142] c1, c2, and c3 are the weight coefficients of the above three parts. They are set according to empirical values ​​during initial training and can be adjusted through training to ensure good performance of the model.

[0143] The improved deep deterministic policy gradient (DDPG) algorithm is used to solve the transient frequency emergency control strategy problem. First, the current critic network, the target critic network and the current Q network of DDQN are basically similar to the target Q network in terms of functional positioning, but the deep deterministic policy gradient (DDPG) algorithm has an Actor policy network, so there is no need for a selection method such as the ∈-greedy method; the next step is to use the greedy method to select the action A′ for the next state S′ sampled in the experience replay pool. Since this part of the work is used to estimate the target Q value, it can be completed in the Actor target network. Based on the S′ and A′ provided by the experience replay pool and the target Actor network, part of the target Q value is calculated. Since this part is an evaluation, it is still completed in the Critic target network. After the Critic target network calculates part of the target Q value, the current critic network will calculate the target Q value, update the network parameters, and regularly copy the network parameters to the Critic target network. In addition, the Actor current network will also update the network parameters based on the target Q value calculated by the Critic current network, and regularly copy the network parameters to the Actor target network. In the training process, the improved deep deterministic policy gradient algorithm (DDPG) model will generate action A with superimposed normal distribution random variable perturbations based on action A. t , where the action strategies include machine shedding and load shedding actions.

[0144] This algorithm is used to train the reinforcement learning model. The specific process is as follows:

[0145] Input: Actor current network, Actor target network, Critic current network, Critic target network, parameters θ, θ′, w, w′, decay factor γ, soft update coefficient τ, number of samples of batch gradient descent m, target Q network parameter update frequency C, maximum number of iterations Τ, random noise function

[0146] Output: optimal Actor current network parameter θ, Critic current network parameter w;

[0147] (1) Initialize θ, θ′, w, w′, and clear the set D of experience replays;

[0148] (2) for i from 1 to T, iterate;

[0149] a) Initialize S as the first state of the current state sequence and get its feature vector φ(S)

[0150] b) Use the current Actor network to interact with the power system and obtain actions based on the current operating state S of the power system

[0151] A=π θ (φ(S))+N, based on action A, generates action A with superimposed normal distribution random variable perturbation t .

[0152] c) Control the power system to perform action A t , get the new state S′ reward R, whether to terminate the state: is_end

[0153] d) Substitute φ(S), A t , R, φ(S′), is_end This five-tuple is stored in the experience playback set D

[0154] e) Sample m samples φ(S) from the experience replay set D j ), A j , R j ,φ(S j ′), is_end, j = 1.2...m, calculate

[0155] Current Q value y j :

[0156]

[0157] f) Use mean square error loss function Update all parameters w of the Critic's current network through the gradient back propagation of the neural network;

[0158] g) Use Update all parameters of the Actor's current network through the gradient back propagation of the neural network

[0159] h) If T%C=1, update the Critic target network and Actor target network parameters:

[0160] w′←τw+(1-τ)w′

[0161] θ′←τθ+(1-τ)θ′

[0162] i) If S′ is in the terminal state, the current round of iteration is completed, otherwise go to step b)

[0163] (3) After the iteration is completed, the transient frequency emergency control strategy is obtained.

[0164] After training the transient frequency emergency control strategy model based on the improved deep deterministic policy gradient (DDPG), the transient frequency emergency control strategy table is finally obtained, where the state s represents the grid operation state after optimized control, including the normal power flow level, voltage level, DC power, etc. Action Pool A MIt represents the set of actions taken to achieve the optimal state, that is, to control the power grid to cut off the generator and load according to the control strategy to ensure the safe and stable operation of the power grid.

[0165] Finally, the current operating data of the power grid is input into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, and the power grid is cut off and load is controlled according to the actual emergency control strategy.

[0166] It should be noted that the emergency control strategy table is: the successive and final control quantities of the control means required in the control process corresponding to various fault conditions. Due to the difference in the weight of machine cutting and load cutting, the priority of each control means is different, and some control means with lower priority may not need to respond.

[0167] In order to enable those skilled in the art to further understand the power grid transient frequency emergency control method based on reinforcement learning in the embodiment of the present application, it is described in detail below in conjunction with specific embodiments. Figure 3 shown.

[0168] If the power grid is in a fault state, the online operation data of the power grid is obtained.

[0169] Input the online operation data into the target transient frequency emergency control model, determine the objective function through static safety constraints and transient stability constraints, and obtain the machine shedding and load shedding control strategies;

[0170] The obtained machine cutting and load shedding control strategy is matched with the preset emergency control strategy table. If the match is successful, the optimal control strategy is output. If the match is unsuccessful, the preset emergency control strategy table is updated based on the output machine cutting and load shedding control strategy to obtain a new emergency control strategy table, so as to match the control strategy for the next time period according to the new emergency control strategy table.

[0171] Therefore, the transient frequency emergency control method based on reinforcement learning proposed in the embodiment of the present application has the following advantages:

[0172] (1) A transient frequency emergency control model is constructed based on the improved deep deterministic policy gradient (DDPG) reinforcement learning method. By solving the model, the emergency control strategy of transient frequency emergency control can be obtained. By cutting generators and loads, the frequency fluctuation caused by large-scale power grid disturbances can be effectively solved, thereby improving the reliability and economy of stable operation of the power grid.

[0173] (2) Through the three modules, the phased division of labor and process connection are clarified. By preparing before the failure and responding in real time after the failure, the online real-time emergency coordination control strategy of machine shedding and load shedding is quickly and accurately generated to achieve rapid recovery of transient frequency stability after the failure.

[0174] According to the reinforcement learning-based grid transient frequency emergency control method proposed in the embodiment of the present application, it is determined whether the grid is in a fault state. If the grid is in a fault state, the current operation data of the grid is obtained, and the current operation data is input into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on grid fault data samples and trained using a preset improved deep deterministic policy gradient algorithm; emergency stability control is performed on the transient frequency of the grid according to the actual emergency control strategy. Thus, the problem of frequency fluctuations caused by large-scale grid disturbances is solved, the reliability and economy of stable operation of the grid are improved, and by preparing before the fault and responding in real time after the fault, an online real-time emergency coordination control strategy for cutting the machine and cutting the load is quickly and accurately generated, so as to achieve rapid recovery of transient frequency stability after the fault.

[0175] Next, a power grid transient frequency emergency control device based on reinforcement learning proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0176] Figure 4 It is a block diagram of a power grid transient frequency emergency control device based on reinforcement learning according to an embodiment of the present application.

[0177] like Figure 4 As shown, the power grid transient frequency emergency control device 10 based on reinforcement learning includes: a judgment module 100, an acquisition module 200, a matching module 300 and a control module 400.

[0178] Among them, the judgment module 100 is used to judge whether the power grid is in a fault state; the acquisition module 200 is used to obtain the current operation data of the power grid if the power grid is in a fault state; the matching module 300 is used to input the current operation data into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on the power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm; the control module 400 is used to perform emergency stability control on the transient frequency of the power grid according to the actual emergency control strategy.

[0179] Optionally, in some embodiments, before inputting the current operating data into the target transient frequency emergency control model, the matching module 300 is also used to: obtain power grid fault data samples, wherein the power grid fault data samples include historical fault data; based on the historical fault data, train the target transient frequency emergency control model using a preset improved deep deterministic policy gradient algorithm to obtain a variety of power grid cutting and load shedding emergency control strategies; obtain a variety of emergency control strategy tables based on the multiple power grid cutting and load shedding emergency control strategies.

[0180] Optionally, in some embodiments, the matching module 300 is also used to: based on historical fault data, utilize the interaction between the Actor current network and the power grid in the target transient frequency emergency control model to obtain the action of superimposing the normal distribution random variable disturbance; control the power grid to execute the action of superimposing the normal distribution random variable disturbance, obtain the new state and new reward of the power grid, and determine whether the new state is a terminal state; if the new state is a terminal state, obtain a variety of power grid emergency control strategies for cutting machine and load.

[0181] Optionally, in some embodiments, after determining whether the new state is a terminal state, the matching module 300 is also used to: if the new state is not a terminal state, calculate the current Q value based on the action of superimposing the normal distribution random variable disturbance, the new state and the new reward, and update the Critic current network in the target transient frequency emergency control model based on the current Q value; calculate the target Q value using the updated Critic current network, and update the Actor current network based on the target Q value, and continue to execute the steps of using the updated Actor current network to interact with the power grid to obtain the action of superimposing the normal distribution random variable disturbance, control the power grid to execute the action of superimposing the normal distribution random variable disturbance, and obtain the new state and new reward of the power grid, until the new state is a terminal state, and obtain a variety of machine cutting and load cutting emergency control strategies for the power grid.

[0182] Optionally, in some embodiments, the target transient frequency emergency control model includes an objective function, a static safety constraint, a transient stability constraint, an action space, and a reward mechanism.

[0183] Among them, the objective function is

[0184]

[0185] Among them, N G is the number of switchable generator nodes, N L is the number of nodes that can cut load; c G,i is the cost of switching off the i-th generator node, c L,j is the load shedding cost of the jth load node, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable,

[0186] Static safety constraints include equality constraints and inequality constraints, where the equality constraints are:

[0187]

[0188] Among them, P Giis the active power generated by the generator, Q Gi is the reactive power of the generator, P Li is the active power consumed by the load, Q Li Reactive power consumed by the load, U i is the magnitude of the node voltage, θ ij is the phase angle difference of the node voltage, G ij is the conductance of the line, B ij is the electrical conductance of the line,

[0189] The inequality constraints are:

[0190] P imin <P<P imax ,i=1,2,...N G

[0191] Q imin <Q<Q imax ,i=I,2,...N G

[0192] U jmin <U<U jmax ,j=I,2,...N L

[0193] f jmin <f<f jmax ,j=1,2,...N L

[0194] u imin i imax ,i=1,2,...N G

[0195] u jmin j jmax ,j=I,2,...N L ;

[0196] Among them, P imax and P imin is the upper and lower limits of the generator active power, Q imax and Q imin is the upper and lower limits of the generator reactive power, U imax and U jmin is the upper and lower limits of bus voltage, f jmax and fjmin are the upper and lower limits of frequency,

[0197] The transient stability constraint is:

[0198] f(X,u i ,u j )>M​​​​stable

[0199] Among them, f(X) is the transient stability rule expression after cutting the generator and shedding the load, X is the system operation state after cutting the generator and shedding the load, M stable is the transient stability threshold of the action space, M stable >0,

[0200] The action space includes machine shedding strategy and load shedding strategy.

[0201] The reward mechanism is:

[0202]

[0203] Among them, I F is the transient frequency stability risk index value, I F max is the maximum frequency safety index value, c1, c2, c3 are all weight coefficients, is the total cutting capacity, P G,i (t) is the capacity of the generator node i removed at time t, m invalid It is the invalid action flag, and Step is the number of steps for cutting the machine and load.

[0204] It should be noted that the above explanation of the embodiment of the method for emergency control of transient frequency of a power grid based on reinforcement learning is also applicable to the emergency control device for transient frequency of a power grid based on reinforcement learning of this embodiment, and will not be repeated here.

[0205] According to the reinforcement learning-based grid transient frequency emergency control device proposed in the embodiment of the present application, it is determined whether the grid is in a fault state. If the grid is in a fault state, the current operation data of the grid is obtained, and the current operation data is input into the target transient frequency emergency control model to match the actual emergency control strategy from the multiple emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on grid fault data samples and trained using a preset improved deep deterministic policy gradient algorithm; emergency stability control is performed on the transient frequency of the grid according to the actual emergency control strategy. Thus, the problem of frequency fluctuations caused by large-scale grid disturbances is solved, the reliability and economy of stable operation of the grid are improved, and by preparing before a fault and responding in real time after a fault, an online real-time emergency coordination control strategy for cutting the machine and cutting the load is quickly and accurately generated, so as to achieve rapid recovery of transient frequency stability after a fault.

[0206] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0207] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0208] When the processor 502 executes the program, the power grid transient frequency emergency control method based on reinforcement learning provided in the above embodiment is implemented.

[0209] Furthermore, the electronic device further comprises:

[0210] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0211] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0212] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0213] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0214] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0215] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0216] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned reinforcement learning-based power grid transient frequency emergency control method.

[0217] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0218] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0219] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0220] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0221] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0222] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0223] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0224] The computer-readable storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for emergency control of transient frequency of power grid based on reinforcement learning, characterized in that: The following steps are involved: Determine whether the power grid is in a fault state; If the power grid is in the fault state, obtaining current operation data of the power grid; Inputting the current operating data into a target transient frequency emergency control model to match an actual emergency control strategy from a plurality of emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on a power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm; Emergency stability control is performed on the transient frequency of the power grid according to the actual emergency control strategy.

2. The method according to claim 1, characterized in that Before inputting the current operating data into the target transient frequency emergency control model, the method includes: Acquire the power grid fault data sample, wherein the power grid fault data sample includes historical fault data; Based on the historical fault data, the target transient frequency emergency control model is trained using a preset improved deep deterministic policy gradient algorithm to obtain a variety of machine cutting and load cutting emergency control strategies for the power grid; The multiple emergency control strategy tables are obtained based on the multiple machine cutting and load cutting emergency control strategies.

3. The method according to claim 2, characterized in that The improved deep deterministic policy gradient algorithm preset based on the historical fault data trains the target transient frequency emergency control model to obtain multiple machine cutting and load shedding emergency control strategies of the power grid, including: Based on the historical fault data, the current network of actors in the target transient frequency emergency control model is used to interact with the power grid to obtain an action of superimposing a normal distribution random variable disturbance; Controlling the power grid to execute the action of superimposing the normal distribution random variable disturbance, obtaining a new state and a new reward of the power grid, and determining whether the new state is a terminal state; If the new state is a terminal state, multiple machine-cutting and load-cutting emergency control strategies of the power grid are obtained.

4. The method according to claim 3, characterized in that After determining whether the new state is a terminal state, the method further comprises: If the new state is not a terminal state, a current Q value is calculated based on the action of the superimposed normal distribution random variable disturbance, the new state and the new reward, and a Critic current network in the target transient frequency emergency control model is updated based on the current Q value; The target Q value is calculated using the updated Critic current network, and the Actor current network is updated based on the target Q value, and the updated Actor current network is continued to interact with the power grid to obtain the action of superimposing the normal distribution random variable perturbation, and the power grid is controlled to execute the action of superimposing the normal distribution random variable perturbation to obtain the new state and new reward of the power grid, until the new state is the termination state, and a variety of machine cutting and load cutting emergency control strategies of the power grid are obtained.

5. The method according to claim 1, characterized in that The target transient frequency emergency control model includes an objective function, a static safety constraint, a transient stability constraint, an action space and a reward mechanism, wherein: The objective function is Among them, N G is the number of switchable generator nodes, N L is the number of nodes that can cut loads, c G,i is the cost of switching off the i-th generator node, c L,j is the load shedding cost of the jth load node, ΔP G,i is the amount of power cut at the ith generator node, ΔP L,j is the load shedding amount of the jth load node, u i 、u j is the control variable, The static safety constraints include equality constraints and inequality constraints, wherein the equality constraints are: Among them, P Gi is the active power generated by the generator, Q Gi is the reactive power of the generator, P Li is the active power consumed by the load, Q Li Reactive power consumed by the load, U i is the magnitude of the node voltage, θ ij is the phase angle difference of the node voltage, G ij is the conductance of the line, B ij is the electrical conductance of the line, The inequality constraints are: P imin <P<P imax ,i=1,2,...N G Q imin <Q<Q imax ,i=I,2,...N G U jmin <U<U jmax ,j=I,2,...N L f jmin <f<f jmax ,j=1,2,...N L u imin i imax ,i=1,2,...N G ​​ u jmin <u j <u jmax ,j=I,2,...N L ; Among them, P imax and P imin is the upper and lower limits of the generator active power, Q imax and Q imin is the upper and lower limits of the generator reactive power, U imax and U jmin is the upper and lower limits of bus voltage, f jmax and fjmin are the upper and lower limits of frequency, The transient stability constraint is: f(X,u i ,u j )>M stable Among them, f(X) is the transient stability rule expression after cutting the generator and shedding the load, X is the system operation state after cutting the generator and shedding the load, M stable is the transient stability threshold of the action space, M stable >0, The action space includes machine shedding strategy and load shedding strategy. The reward mechanism is: Among them, I F is the transient frequency stability risk index value, I F max is the maximum frequency safety index value, c1, c2, c3 are all weight coefficients, is the total cutting capacity, P G,i (t) is the capacity of the generator node i removed at time t, m invalid It is the invalid action flag, and Step is the number of steps for cutting the machine and load.

6. A power grid transient frequency emergency control device based on reinforcement learning, characterized in that: include: A judgment module, used to judge whether the power grid is in a fault state; an acquisition module, configured to acquire current operation data of the power grid if the power grid is in the fault state; A matching module, used for inputting the current operation data into a target transient frequency emergency control model to match an actual emergency control strategy from a plurality of emergency control strategy tables in the target transient frequency emergency control model, wherein the target transient frequency emergency control model is based on a power grid fault data sample and is trained using a preset improved deep deterministic policy gradient algorithm; A control module is used to perform emergency stability control on the transient frequency of the power grid according to the actual emergency control strategy.

7. The device according to claim 6, characterized in that Before inputting the current operation data into the target transient frequency emergency control model, the matching module is further used to: Acquire the power grid fault data sample, wherein the power grid fault data sample includes historical fault data; Based on the historical fault data, the target transient frequency emergency control model is trained using a preset improved deep deterministic policy gradient algorithm to obtain a variety of machine cutting and load cutting emergency control strategies for the power grid; The multiple emergency control strategy tables are obtained based on the multiple machine cutting and load cutting emergency control strategies.

8. The device according to claim 7, characterized in that The matching module is further used for: Based on the historical fault data, the current network of actors in the target transient frequency emergency control model is used to interact with the power grid to obtain an action of superimposing a normal distribution random variable disturbance; Controlling the power grid to execute the action of superimposing the normal distribution random variable disturbance, obtaining a new state and a new reward of the power grid, and determining whether the new state is a terminal state; If the new state is a terminal state, multiple machine-cutting and load-cutting emergency control strategies of the power grid are obtained.

9. An electronic device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the reinforcement learning-based power grid transient frequency emergency control method as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the reinforcement learning-based power grid transient frequency emergency control method as described in any one of claims 1 to 5.

Images