Power grid stability control method and device, terminal equipment and storage medium

Abstract

The invention discloses a power grid stability control method and device, terminal equipment and a storage medium, and belongs to the technical field of power distribution network stability control, and the method comprises the steps: constructing a state space, an action space and a reward function of a power grid, and abstracting a power grid stability control strategy generation problem into a Markov decision process; then, aiming at the system stability, power balance and recovery capability of the power grid after the abnormal event, providing a safety constraint condition related to generator tripping operation in a stability control strategy; and finally, solving the constructed Markov decision model to generate a power grid stability control strategy. The problems that a traditional stability control method is poor in adaptability and lacks an effective security constraint and recovery mechanism are solved, and the adaptability and flexibility of the power grid stability control method when a complex abnormal event occurs are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power distribution network stabilization control, and in particular to a power distribution network stabilization control method, device, terminal equipment and storage medium. Background Art

[0002] With the transformation of the global energy mix and the rapid development of modern power grids, the scale of power grids continues to expand, and the proportion of renewable energy generation continues to rise. The randomness and volatility of renewable energy integration have significantly increased the complexity and uncertainty of grid operations, posing unprecedented challenges to grid stability control. At the same time, the frequency and scope of abnormal events such as extreme weather, equipment failures, and large-scale load fluctuations are gradually increasing, placing higher demands on the response speed, accuracy, and adaptability of grid stability control strategies.

[0003] Most traditional grid stabilization methods ignore the need to comprehensively consider factors such as system stability, power balance and recovery capability after an abnormal event occurs. In addition, the safety constraints are single, resulting in poor adaptability and lack of flexibility of the stabilization methods. Summary of the Invention

[0004] The present invention provides a power grid stabilization method, apparatus, terminal equipment and storage medium to solve the problem of lack of flexibility caused by single security constraint conditions in traditional power grid stabilization technology.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a power grid stabilization method, including:

[0006] A Markov decision model is constructed using grid frequency, generator output power, load demand, grid voltage, and energy storage device charge and discharge status as the state space, and generator power regulation, load transfer, and energy storage device charge and discharge regulation operations as the action space.

[0007] A reward function of the Markov decision model is constructed using grid frequency stability, grid power balance, grid resilience, and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to grid frequency stability, grid power balance, grid resilience, and grid voltage stability;

[0008] Under the established constraints, the Markov decision model is solved with maximizing the cumulative reward as the objective function to generate a target action strategy. The constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints, and load shedding operation time limit constraints. The target action strategy includes: generator power target adjustment amount, load target transfer amount, and energy storage device charging and discharging target adjustment operation.

[0009] The power of the generator is adjusted according to the generator power target adjustment amount; the load is transferred according to the load target transfer amount; and the charge and discharge state of the energy storage device is set according to the energy storage device charge and discharge target adjustment operation.

[0010] As a preferred solution, the state space expression of the Markov decision model is:

[0011] S={f,P g ,P L ,V,E s};

[0012] Where S represents the state space of the Markov decision model; f represents the grid frequency; P g Indicates the generator output power; P L Indicates load demand; V indicates grid voltage; E s Indicates the charging and discharging status of the energy storage device;

[0013] The action space expression of the Markov decision model is:

[0014] A={ΔP g ,ΔP L ,ΔE s};

[0015] Where A represents the action space of the Markov decision model; ΔP g Indicates the generator power regulation; ΔP L Indicates the load transfer amount; ΔE s Indicates the charge and discharge regulation operation of the energy storage device;

[0016] The reward function of the Markov decision model is:

[0017] R(s,a)=r1(s,a)+r2(s,a)+r3(s,a)+r4(s,a);

[0018] r1(s,a)=-α|ff target |;

[0019] r2(s,a)=-β|∑P g -P L |;

[0020] r3(s,a)=-γ|VV target |;

[0021] r4(s,a)=δ·C;

[0022] Where R(s,a) represents the reward function of the Markov decision model; r1(s,a) represents the grid frequency stability reward signal function; r2(s,a) represents the grid power balance reward signal function; r3(s,a) represents the grid voltage stability reward signal function; r4(s,a) represents the grid restoration capability reward signal function; α is the weighting coefficient of the grid frequency stability reward signal; f target represents the target frequency of the grid voltage; β is the weighting coefficient of the grid power balance reward signal; γ is the weighting coefficient of the grid voltage stability reward signal; V target represents the target voltage of the power grid; δ is the weighting coefficient of the power grid restoration capability reward signal; C represents the quantitative index of the power grid restoration capability.

[0023] As a preferred solution, the formula of the objective function is:

[0024] π*(s)=arg max a Q(s,a);

[0025] Where, π * (s) represents the optimal strategy; Q(s,a) represents the Q function of the DQN algorithm.

[0026] As a preferred solution, the expression of the grid frequency stability constraint is:

[0027] f min ≤f≤f max ;

[0028] Where, f represents the grid frequency; f min Indicates the minimum value of the grid frequency deviation range; f max Indicates the maximum value of the grid frequency deviation range;

[0029] The expression of the power balance constraint of the power grid is:

[0030] |P g -P L |≤ΔP max ;

[0031] Where, P g Indicates the generator output power; P L Indicates load demand; ΔP max Indicates the maximum acceptable power deviation;

[0032] The expression of the grid voltage stability constraint is:

[0033] V min ≤V≤V max ;

[0034] Where, V represents the grid voltage; V minIndicates the minimum value of the grid voltage deviation range; V max Indicates the maximum value of the grid voltage deviation range;

[0035] The expression of the load shedding operation time limit constraint is:

[0036]

[0037] Where, T cut Indicates load shedding operation time; Indicates the maximum shutdown time.

[0038] As a preferred solution, the Markov decision model is solved with maximizing the cumulative reward as the objective function to generate the target action strategy, including:

[0039] Constructing a main Q network and a target Q network, and initializing parameters of the main Q network and the target Q network; wherein the target Q network is regularly updated according to the main Q network;

[0040] Repeat the main Q network training operation until the number of training iterations reaches a preset threshold, and a trained main Q network is obtained;

[0041] Solving the Markov decision model based on the trained main Q network to generate a target action strategy;

[0042] The main Q network training operation includes:

[0043] According to the target Q network, a Q function is constructed by using the Bellman equation;

[0044] For each experience sample in the experience replay pool, calculate the target Q value of the experience sample according to the Q function; wherein the experience pool replay is used to store all experience samples in the solution process, and each experience sample includes: state data, action data, reward data and next state data;

[0045] Calculating a loss function value based on the target Q values ​​of all the experience samples and the main Q network;

[0046] According to the loss function value, the parameters of the main Q network are updated so that the main Q network converges to the Q function.

[0047] As a preferred solution, the formula of the Q function is:

[0048] y i =R(s,a)+γmax a′ Q(s′,a′;θ - );

[0049] Where yi represents the target Q value of the i-th experience sample in the experience replay pool; R(s,a) represents the current reward; γ represents the discount factor; s′ represents the next state; max a′ Q(s′,a′,θ - ) represents the maximum future Q value of the next state s′; θ - Represents the parameters of the target Q network.

[0050] As a preferred solution, the calculation formula of the loss function value is:

[0051]

[0052] Where L(θ) represents the loss function value; B represents the experience replay pool; y i represents the target Q value of the i-th experience sample in the experience replay pool; s i Represents the state data of the i-th experience sample in the experience replay pool; a i represents the action data of the i-th experience sample in the experience replay pool; θ represents the parameters of the main Q network.

[0053] Based on the above embodiment, another embodiment of the present invention provides a power grid stabilization device, including: a model building module, a model solving module and a power grid regulation module;

[0054] The model construction module is used to construct a Markov decision model using the grid frequency, generator output power, load demand, grid voltage and energy storage device charge and discharge status as the state space, and the generator power adjustment amount, load transfer amount and energy storage device charge and discharge adjustment operation as the action space; and construct a reward function of the Markov decision model using the grid frequency stability, grid power balance, grid recovery capability and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to the grid frequency stability, the grid power balance, the grid recovery capability and the grid voltage stability;

[0055] The model solving module is used to solve the Markov decision model under the established constraints and take maximizing the cumulative reward as the objective function to generate a target action strategy; wherein the constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints, and load shedding operation time limit constraints; the target action strategy includes: generator power target adjustment amount, load target transfer amount, and energy storage device charging and discharging target adjustment operation;

[0056] The grid regulation module is used to regulate the power of the generator according to the generator power target regulation amount; transfer the load according to the load target transfer amount; and set the charge and discharge state of the energy storage device according to the energy storage device charge and discharge target regulation operation.

[0057] Based on the above embodiments, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the power grid stabilization method described in the above embodiments of the invention.

[0058] Based on the above embodiments, another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the power grid stabilization method described in the above invention embodiment.

[0059] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0060] This invention abstracts the problem of generating a grid stabilization strategy into a Markov decision process by constructing the grid's state space, action space, and reward function. Subsequently, it proposes safety constraints for generator tripping in the stabilization strategy, targeting the grid's system stability, power balance, and recovery capability after an abnormal event. Finally, it solves the constructed Markov decision model to generate a grid stabilization strategy. This invention overcomes the poor adaptability and lack of effective safety constraints and recovery mechanisms of traditional stabilization methods, significantly improving the grid's intelligent decision-making capabilities, operational stability, and recovery efficiency during complex abnormal events, and providing strong technical support and solutions for the safe operation of modern power grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of a power grid stabilization method provided by one embodiment of the present invention;

[0062] Figure 2 It is an improved IEEE39 node new energy topology diagram;

[0063] Figure 3 This is a comparison chart of system frequency recovery curves of the traditional method and the method of the present invention;

[0064] Figure 4 This is a comparison chart of the decision reward function values ​​of the traditional method and the method of the present invention;

[0065] Figure 5 A schematic structural diagram of a power grid stabilization device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0067] Example 1

[0068] Please refer to Figure 1 , which is a flow chart of a power grid stabilization method provided by one embodiment of the present invention, including:

[0069] S1. A Markov decision model is constructed with grid frequency, generator output power, load demand, grid voltage, and energy storage device charge and discharge status as the state space, and generator power regulation, load transfer, and energy storage device charge and discharge regulation operation as the action space.

[0070] S2. Construct a reward function of the Markov decision model with grid frequency stability, grid power balance, grid recovery capability, and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to the grid frequency stability, proportional to the grid power balance, proportional to the grid recovery capability, and proportional to the grid voltage stability.

[0071] In a preferred embodiment, the state space expression of the Markov decision model is:

[0072] S={f,P g ,P L ,V,E s};

[0073] Where S represents the state space of the Markov decision model; f represents the grid frequency; P g Indicates the generator output power; P L Indicates load demand; V indicates grid voltage; E s Indicates the charging and discharging status of the energy storage device;

[0074] The action space expression of the Markov decision model is:

[0075] A={ΔP g ,ΔP L ,ΔE s};

[0076] Where A represents the action space of the Markov decision model; ΔP g Indicates the generator power regulation; ΔP L Indicates the load transfer amount; ΔEs Indicates the charge and discharge regulation operation of the energy storage device;

[0077] The reward function of the Markov decision model is:

[0078] R(s,a)=r1(s,a)+r2(s,a)+r3(s,a)+r4(s,a);

[0079] r1(s,a)=-α|ff target |;

[0080] r2(s,a)=-β|∑P g -P L |;

[0081] r3(s,a)=-γ|VV target |;

[0082] r4(s,a)=δ·C;

[0083] Where R(s,a) represents the reward function of the Markov decision model; r1(s,a) represents the grid frequency stability reward signal function; r2(s,a) represents the grid power balance reward signal function; r3(s,a) represents the grid voltage stability reward signal function; r4(s,a) represents the grid restoration capability reward signal function; α is the weighting coefficient of the grid frequency stability reward signal; f target represents the target frequency of the grid voltage; β is the weighting coefficient of the grid power balance reward signal; γ is the weighting coefficient of the grid voltage stability reward signal; V target represents the target voltage of the power grid; δ is the weighting coefficient of the power grid restoration capability reward signal; C represents the quantitative index of the power grid restoration capability.

[0084] In this embodiment, the state space of the power grid is a multi-dimensional vector that reflects the state of the power grid system at every moment. Specifically, the state space can include the following main variables:

[0085] (1) Grid frequency f: at the grid frequency lower limit f min and grid frequency f max Between the upper and lower limits, frequency deviation will affect system stability;

[0086] (2) Generator output power P g : Each generator's power may have upper and lower limits;

[0087] (3) Load demand P L : That is, the power demand at the user end;

[0088] (4) Grid voltage V: at the lower limit of grid voltage V min and the upper limit of the grid voltage Vmax between;

[0089] (5) Energy storage device charging and discharging state E s : Related to the stability and power balance of the power grid.

[0090] The action space specifically includes:

[0091] (1) Motor power adjustment ΔP g : The power output of each generator can be adjusted. Assume there are generators, each with an adjustment range;

[0092] (2) Load transfer ΔP L : In an emergency, the load can be cut off or transferred to reduce the burden on the power grid;

[0093] (3) Energy storage device charge and discharge regulation operation ΔE s : Charging or discharging operation of energy storage equipment.

[0094] The reward function measures the benefit of each state-action pair. The reward signal of the reward function of the present invention specifically includes:

[0095] (1) Grid frequency stability: The closer the grid frequency is to the target value, the higher the reward. Set the target frequency to f target , then the frequency deviation |ff target |Negative factors as rewards;

[0096] (2) Grid power balance: The better the system power balance, the higher the reward. The power balance index can be defined as the difference between generator power and load ΔP = ∑P g -P L , the closer ΔP is to 0, the higher the reward;

[0097] (3) Grid recovery capability: Grid recovery capability can be measured by the system’s response time and the stable state after recovery. For example, the faster the system recovers to a stable state, the higher the reward.

[0098] (4) Grid voltage stability: voltage deviation from target voltage V target The smaller it is, the higher the reward;

[0099] It should be noted that, in addition to this, the Markov decision model also includes state transition probabilities. This state transition probability represents the probability of transitioning to the next state s′ after executing action a in state s. This transition depends on the dynamic characteristics of the power grid, such as load changes, generator regulation, and the charging and discharging of energy storage devices. Transition probabilities can be modeled using data acquisition methods. The state transition probability is given by the following formula:

[0100] P(s′|s,a)=P(f′,P′g ,P′ L ,V′,E′ s |f,P g ,P L ,V,E s ,a);

[0101] Where s′=(f′,P′ g ,P′ L ,V′,E′ s ) is the state at the next moment. P(s′|s,a) describes the transition law of the power grid after executing an action in a certain state

[0102] Finally, the Markov decision process model of the power grid stability problem is: (S, A, P, R).

[0103] S3. Under the established constraints, the Markov decision model is solved with maximizing the cumulative reward as the objective function to generate a target action strategy; wherein the constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints and load shedding operation time limit constraints; the target action strategy includes: generator power target adjustment amount, load target transfer amount and energy storage device charging and discharging target adjustment operation.

[0104] In step S3, considering the system stability, power balance, and system recovery capability after an abnormal event occurs in the power grid, the safety constraints involved in the stable control and generator tripping action in the stabilization control strategy are proposed, including:

[0105] (1) Grid frequency stability constraint: Grid frequency is an important indicator of grid stability. When an abnormal event occurs, the grid frequency may deviate significantly, so load shedding is necessary to prevent excessive frequency deviation.

[0106] (2) Grid power balance constraint: The power balance constraint ensures the balance between the power supply and the load of the grid. In the event of an abnormal event, the generator tripping action should be executed according to the change of the power balance.

[0107] (3) Grid voltage stability constraint: Voltage is a key indicator of grid operation. Voltage deviation from the target voltage can lead to equipment damage or system instability. When an abnormal event occurs, load shedding needs to consider changes in grid voltage.

[0108] (4) Load shedding operation time limit constraint: Load shedding operation time should be completed within the specified time limit to ensure that the system will not be in an unstable state for a long time. Load shedding operation time limit constraint ensures that the load shedding operation is completed in time, avoids the power grid from being in an unstable state for a long time, and thus accelerates the recovery process.

[0109] In a preferred embodiment, the objective function is formulated as follows:

[0110] π * (s)=arg max a Q(s,a);

[0111] Where, π * (s) represents the optimal strategy; Q(s,a) represents the Q function of the DQN algorithm.

[0112] It should be noted that the present invention uses the DQN algorithm to solve the Markov decision model. The goal of the DQN algorithm is to learn an optimal strategy π * , so that the cumulative reward is maximized.

[0113] In a preferred embodiment, the expression of the grid frequency stability constraint is:

[0114] f min ≤f≤f max ;

[0115] Where, f represents the grid frequency; f min Indicates the minimum value of the grid frequency deviation range; f max Indicates the maximum value of the grid frequency deviation range;

[0116] The expression of the power balance constraint of the power grid is:

[0117] |P g -P L |≤ΔP max ;

[0118] Where, P g Indicates the generator output power; P L Indicates load demand; ΔP max Indicates the maximum acceptable power deviation;

[0119] The expression of the grid voltage stability constraint is:

[0120] V min ≤V≤V max ;

[0121] Where, V represents the grid voltage; V min Indicates the minimum value of the grid voltage deviation range; V max Indicates the maximum value of the grid voltage deviation range;

[0122] The expression of the load shedding operation time limit constraint is:

[0123]

[0124] Where, T cut Indicates load shedding operation time; Indicates the maximum shutdown time.

[0125] In a preferred embodiment, the Markov decision model is solved with maximizing the cumulative reward as the objective function to generate the target action strategy, including:

[0126] Constructing a main Q network and a target Q network, and initializing parameters of the main Q network and the target Q network; wherein the target Q network is regularly updated according to the main Q network;

[0127] Repeat the main Q network training operation until the number of training iterations reaches a preset threshold, and a trained main Q network is obtained;

[0128] Solving the Markov decision model based on the trained main Q network to generate a target action strategy;

[0129] The main Q network training operation includes:

[0130] According to the target Q network, a Q function is constructed by using the Bellman equation;

[0131] For each experience sample in the experience replay pool, calculate the target Q value of the experience sample according to the Q function; wherein the experience pool replay is used to store all experience samples in the solution process, and each experience sample includes: state data, action data, reward data and next state data;

[0132] Calculating a loss function value based on the target Q values ​​of all the experience samples and the main Q network;

[0133] According to the loss function value, the parameters of the main Q network are updated so that the main Q network converges to the Q function.

[0134] It should be noted that DQN uses a neural network to approximate the Q function, namely:

[0135] Q(s,a;θ)≈Q * (s,a);

[0136] Where: θ is the parameter of the main Q network; the goal is to make Q(s,a;θ) converge to the true Q through continuous training * (s,a).

[0137] In a preferred embodiment, the formula of the Q function is:

[0138] y i =R(s,a)+γmax a′ Q(s′,a′;θ - );

[0139] Where y i represents the target Q value of the i-th experience sample in the experience replay pool; R(s,a) represents the current reward; γ represents the discount factor; s′ represents the next state; max a′ Q(s′,a′,θ - ) represents the maximum future Q value of the next state s′; θ - Represents the parameters of the target Q network.

[0140] It should be noted that the Q function is updated based on the Bellman equation.

[0141] In a preferred embodiment, the calculation formula of the loss function value is:

[0142]

[0143] Where L(θ) represents the loss function value; B represents the experience replay pool; y i represents the target Q value of the i-th experience sample in the experience replay pool; s i Represents the state data of the i-th experience sample in the experience replay pool; a i represents the action data of the i-th experience sample in the experience replay pool; θ represents the parameters of the main Q network.

[0144] It should be noted that the experience replay pool B stores the sample set B∈(s,a,r,s′) to break the time correlation. The present invention also designs an action selection strategy, and the action selection formula is:

[0145]

[0146] Where ε represents the exploration probability, which is used to control the probability of randomly selecting an action.

[0147] S4. Adjust the power of the generator according to the generator power target adjustment amount; transfer the load according to the load target transfer amount; set the charge and discharge state of the energy storage device according to the energy storage device charge and discharge target adjustment operation.

[0148] It should be noted that the present invention abstracts the problem of generating a power grid stabilization strategy into a Markov decision process by constructing the state space, action space, and reward function of the power grid; then, in view of the system stability, power balance, and recovery capability of the power grid after an abnormal event, safety constraints involving the power generation operation in the stabilization strategy are proposed; finally, the constructed Markov decision model is solved to generate a power grid stabilization strategy. The present invention overcomes the problems of poor adaptability, lack of effective safety constraints, and recovery mechanisms of traditional stabilization methods, significantly improves the intelligent decision-making capability, operational stability, and recovery efficiency of the power grid when complex abnormal events occur, provides strong technical support and solutions for the safe operation of modern power grids, and significantly improves the adaptability and flexibility of power grid stabilization methods when complex abnormal events occur.

[0149] The following is an experimental verification of the implementation effect of the present invention:

[0150] Based on the IEEE 10-machine 39-node distribution network system, generators G3 and G9 are replaced with wind turbine units of equal capacity, and synchronous units G2, G6, and G8 are replaced with photovoltaic power generation units in equal quantities, ensuring that the new energy penetration rate remains between 40% and 60%. The network structure after the transformation of the IEEE 39-node new energy structure diagram is as follows Figure 2 shown.

[0151] Assume that at 0.5s, the G1, G4 and G8 motors of this system fail simultaneously, resulting in a loss of 1650MW of active power. The system frequency recovery curve is shown in Figure 1. Figure 3 As shown in the figure, the comparison of decision reward function values ​​is shown in Figure 4 shown.

[0152] Depend on Figure 3 It can be seen that compared with the traditional method, the method proposed in the present invention has better system frequency recovery, which verifies the correctness of the established mathematical model.

[0153] Depend on Figure 4 It can be seen that the reward function of the method of the present invention reaches 14 after 300 rounds of iteration. After about 500 rounds of iteration, the reward function gradually converges to 16. The reward function value increase and training efficiency of the present invention are significantly better than those of the traditional method.

[0154] Example 2

[0155] Please refer to Figure 2 , is a schematic structural diagram of a power grid stabilization device provided by an embodiment of the present invention, comprising: a model building module, a model solving module and a power grid regulation module;

[0156] The model construction module is used to construct a Markov decision model using the grid frequency, generator output power, load demand, grid voltage and energy storage device charge and discharge status as the state space, and the generator power adjustment amount, load transfer amount and energy storage device charge and discharge adjustment operation as the action space; and construct a reward function of the Markov decision model using the grid frequency stability, grid power balance, grid recovery capability and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to the grid frequency stability, the grid power balance, the grid recovery capability and the grid voltage stability;

[0157] The model solving module is used to solve the Markov decision model under the established constraints and take maximizing the cumulative reward as the objective function to generate a target action strategy; wherein the constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints, and load shedding operation time limit constraints; the target action strategy includes: generator power target adjustment amount, load target transfer amount, and energy storage device charging and discharging target adjustment operation;

[0158] The grid regulation module is used to regulate the power of the generator according to the generator power target regulation amount; transfer the load according to the load target transfer amount; and set the charge and discharge state of the energy storage device according to the energy storage device charge and discharge target regulation operation.

[0159] Example 3

[0160] Accordingly, an embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the power grid stabilization method described in the above embodiment of the invention.

[0161] Example 4

[0162] Accordingly, an embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the power grid stabilization method described in the above embodiment of the invention.

[0163] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0164] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0165] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0166] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, connecting various parts of the entire device using various interfaces and lines.

[0167] The memory can be used to store the computer program, and the processor realizes various functions of the device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0168] The storage medium is a storage medium, and the computer program is stored in the storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0169] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for stabilizing a power grid, characterized in that: include: A Markov decision model is constructed using grid frequency, generator output power, load demand, grid voltage, and energy storage device charge and discharge status as the state space, and generator power regulation, load transfer, and energy storage device charge and discharge regulation operations as the action space. A reward function of the Markov decision model is constructed using grid frequency stability, grid power balance, grid resilience, and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to grid frequency stability, grid power balance, grid resilience, and grid voltage stability; Under the established constraints, the Markov decision model is solved with maximizing the cumulative reward as the objective function to generate a target action strategy. The constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints, and load shedding operation time limit constraints. The target action strategy includes: generator power target adjustment amount, load target transfer amount, and energy storage device charging and discharging target adjustment operation. The power of the generator is adjusted according to the generator power target adjustment amount; the load is transferred according to the load target transfer amount; and the charge and discharge state of the energy storage device is set according to the energy storage device charge and discharge target adjustment operation.

2. The power grid stabilization method according to claim 1, wherein: The state space expression of the Markov decision model is: S={f,P g ,P L ,V,E s }; Where S represents the state space of the Markov decision model; f represents the grid frequency; P g Indicates the generator output power; P L Indicates load demand; V indicates grid voltage; E s Indicates the charging and discharging status of the energy storage device; The action space expression of the Markov decision model is: A={ΔP g ,ΔP L ,NO s }; Where A represents the action space of the Markov decision model; ΔP g Indicates the generator power regulation; ΔP L Indicates the load transfer amount; ΔE s Indicates the charge and discharge regulation operation of the energy storage device; The reward function of the Markov decision model is: R(s,a)=r1(s,a)+r2(s,a)+r3(s,a)+r4(s,a); r1(s,a)=-α|f-f target |; r2(s,a)=-β|∑P g -P L |; r3(s,a)=-γ|VV target |; r4(s,a)=δ·C; Where R(s,a) represents the reward function of the Markov decision model; r1(s,a) represents the grid frequency stability reward signal function; r2(s,a) represents the grid power balance reward signal function; r3(s,a) represents the grid voltage stability reward signal function; r4(s,a) represents the grid restoration capability reward signal function; α is the weighting coefficient of the grid frequency stability reward signal; f target represents the target frequency of the grid voltage; β is the weighting coefficient of the grid power balance reward signal; γ is the weighting coefficient of the grid voltage stability reward signal; V target represents the target voltage of the power grid; δ is the weighting coefficient of the power grid restoration capability reward signal; C represents the quantitative index of the power grid restoration capability.

3. The power grid stabilization method according to claim 1, wherein: The formula of the objective function is: p * (s)=arg max a Q(s,a); Where, π * (s) represents the optimal strategy; Q(s,a) represents the Q function of the DQN algorithm.

4. The power grid stabilization method according to claim 1, wherein: The expression of the grid frequency stability constraint is: f min ≤f≤f max ; Where, f represents the grid frequency; f min Indicates the minimum value of the grid frequency deviation range; f max Indicates the maximum value of the grid frequency deviation range; The expression of the power balance constraint of the power grid is: |P g -P L |≤ΔP max ; Where, P g Indicates the generator output power; P L Indicates load demand; ΔP max Indicates the maximum acceptable power deviation; The expression of the grid voltage stability constraint is: In min ≤V≤V max ; Where, V represents the grid voltage; V min Indicates the minimum value of the grid voltage deviation range; V max Indicates the maximum value of the grid voltage deviation range; The expression of the load shedding operation time limit constraint is: Where, T cut Indicates load shedding operation time; Indicates the maximum shutdown time.

5. The power grid stabilization method according to claim 1, wherein: The Markov decision model is solved with maximizing the cumulative reward as the objective function to generate the target action strategy, including: Constructing a main Q network and a target Q network, and initializing parameters of the main Q network and the target Q network; wherein the target Q network is regularly updated according to the main Q network; Repeat the main Q network training operation until the number of training iterations reaches a preset threshold, and a trained main Q network is obtained; Solving the Markov decision model based on the trained main Q network to generate a target action strategy; The main Q network training operation includes: According to the target Q network, a Q function is constructed by using the Bellman equation; For each experience sample in the experience replay pool, calculate the target Q value of the experience sample according to the Q function; wherein the experience pool replay is used to store all experience samples in the solution process, and each experience sample includes: state data, action data, reward data and next state data; Calculating a loss function value based on the target Q values ​​of all the experience samples and the main Q network; According to the loss function value, the parameters of the main Q network are updated so that the main Q network converges to the Q function.

6. The power grid stabilization method according to claim 5, characterized in that: The formula of the Q function is: y i =R(s,a)+γmax a′ Q(s′,a′;θ - ); Where y i represents the target Q value of the i-th experience sample in the experience replay pool; R(s,a) represents the current reward; γ represents the discount factor; s′ represents the next state; max a′ Q(s′,a′,θ - ) represents the maximum future Q value of the next state s′; θ - Represents the parameters of the target Q network.

7. The power grid stabilization method according to claim 5, characterized in that: The calculation formula of the loss function value is: Where L(θ) represents the loss function value; B represents the experience replay pool; y i represents the target Q value of the i-th experience sample in the experience replay pool; s i Represents the state data of the i-th experience sample in the experience replay pool; a i Represents the action data of the i-th experience sample in the experience replay pool; θ represents the parameters of the main Q network.

8. A power grid stabilization device, characterized in that: include: Model building module, model solving module and power grid regulation module; The model building module is used to build a Markov decision model with grid frequency, generator output power, load demand, grid voltage and energy storage device charge and discharge status as the state space, and generator power adjustment amount, load transfer amount and energy storage device charge and discharge adjustment operation as the action space; A reward function of the Markov decision model is constructed using grid frequency stability, grid power balance, grid resilience, and grid voltage stability as reward signals; wherein the function value of the reward function is proportional to grid frequency stability, grid power balance, grid resilience, and grid voltage stability; The model solving module is used to solve the Markov decision model under the established constraints and take maximizing the cumulative reward as the objective function to generate a target action strategy; wherein the constraints include: grid frequency stability constraints, grid power balance constraints, grid voltage stability constraints, and load shedding operation time limit constraints; the target action strategy includes: generator power target adjustment amount, load target transfer amount, and energy storage device charging and discharging target adjustment operation; The grid regulation module is used to regulate the power of the generator according to the generator power target regulation amount; transfer the load according to the load target transfer amount; and set the charge and discharge state of the energy storage device according to the energy storage device charge and discharge target regulation operation.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for stabilizing the power grid according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute the power grid stabilization method according to any one of claims 1 to 7.

Images