Method for constructing emergency load shedding decision model and emergency load shedding decision method

Abstract

The present invention discloses a method for constructing an emergency load shedding decision model and an emergency load shedding decision method. The method for constructing an emergency load shedding decision model includes simulating data of a first bus voltage and a first generator power angle according to any fault and a corresponding first emergency load shedding control; fusing the first emergency load shedding action, the first bus voltage and the first generator power angle data, and using a BDQ agent to decide a second emergency load shedding action; simulating data of a second bus voltage and a second generator power angle according to the new emergency load shedding action, and calculating a reward; using the first bus voltage, the first generator power angle, the first emergency load shedding control, the second bus voltage, the second generator power angle, the second emergency load shedding control and the reward under different faults as training samples, and training the BDQ agent, and repeating the above operations to obtain an emergency load shedding decision model. The training efficiency and decision-making ability of the BDQ agent incorporating the first emergency load shedding action are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system stability control, and more specifically, relates to a method for constructing an emergency load shedding decision model and an emergency load shedding decision method. Background Art

[0002] Emergency load shedding control is an important method to ensure the transient stability of the power system after a transient voltage instability fault occurs. In an actual power grid, the emergency load shedding control generally adopts an off-line pre-decision and real-time working condition matching mode. That is, power grid experts conceive a large number of actual fault conditions according to possible actual operating conditions, and combine transient stability simulations to obtain a large amount of transient data of system operation. Power grid experts analyze these operation data and formulate emergency load shedding control measures.

[0003] With the expansion of the scale of power grid interconnection, the amount of power grid simulation data has increased sharply. This manual decision-making method has the problems of low work efficiency and time-consuming and laborious. To reduce the manual workload, some studies use optimization algorithms to solve the optimal decision-making quantity. However, when the controllable load increases, there may be a problem of non-convergence in actual applications, and the calculation amount is large. The rapid development of the Deep Reinforcement Learning (DRL) algorithm provides a new idea for emergency load shedding control. Applying DRL to solve the emergency load shedding control problem is expected to reduce labor costs and improve the efficiency of decision-making. However, the learning efficiency and decision-making ability of its deep reinforcement learning still need to be improved. Summary of the Invention

[0004] In view of the above defects or improvement requirements of the prior art, the present invention provides a method for constructing an emergency load shedding decision model and an emergency load shedding decision method, aiming to improve the learning efficiency and decision-making ability of deep reinforcement learning.

[0005] To achieve the above object, according to one aspect of the present invention, a method for constructing an emergency load shedding decision model is provided. The emergency load shedding decision model is a branch competition Q-network agent based on action fusion, and the model construction method includes:

[0006] Step S1: Select a transient voltage instability fault of the power grid;

[0007] Step S2: According to the current transient voltage instability fault of the power grid and the corresponding first emergency load shedding action, simulate the first bus voltage and the first generator power angle for the entire time period;

[0008] Step S3: Input the first emergency load shedding action and the simulated first bus voltage and the first generator power angle into the branch competition Q-network agent to make a decision on the second emergency load shedding action;

[0009] Step S4: Based on the current power grid transient voltage instability fault and the corresponding second emergency load shedding action, simulate to obtain the second bus voltage and the second generator power angle for the entire time period, calculate the corresponding reward, and form an interaction sample with the first emergency load shedding action, the first bus voltage, the first generator power angle, the second emergency load shedding action, the second bus voltage, the second generator power angle, and the reward obtained under the current power grid transient voltage instability fault, and store it in the capacity pool;

[0010] Step S5: Determine whether the iteration termination condition is triggered. If not, use the second emergency load shedding action as the first emergency load shedding action and jump to Step S2 to continue the iteration. Otherwise, use the interaction sample acquisition process under the current power grid transient voltage instability fault as one round, end the current round, and execute Step S6. The iteration termination condition includes that the second emergency load shedding action can make the voltage return to stability or the iteration number reaches the upper limit:

[0011] Step S6: Determine whether the cumulative number of rounds reaches the termination value. If so, end the modeling. If not, execute Step S7;

[0012] Step S7: Determine whether the cumulative number of rounds reaches the training node value. If not, update the current power grid transient voltage instability fault and jump to Step S2. If so, execute Step S8:

[0013] Step S8: Randomly select multiple interaction samples from the capacity pool to train the branch competition Q-network agent, and then jump to Step S1.

[0014] In one embodiment,

[0015] In Step S3, input the first bus voltage and the first generator power angle obtained by simulation into the branch competition Q-network agent. Specifically, intercept the first bus voltage and the first generator power angle for a partial time period near the fault point and input them into the branch competition Q-network agent.

[0016] In one embodiment, the entire time period is 10 seconds, and the intercepted time period is the first 6 seconds.

[0017] In one embodiment, call the PSASP simulation software to simulate and obtain the first bus voltage and the first generator power angle for the entire time period.

[0018] In one embodiment, the branch competition Q-network agent includes a feature preprocessing module, a mapping module, a state value extraction module, and a decision module. Among them,

[0019] The feature preprocessing module includes a voltage feature extraction network, an angle-of-power feature extraction network, a first emergency load shedding action feature extraction network, and a splicing network. The voltage feature extraction network is used to convert the data of the first bus voltage into a voltage matrix and then extract voltage features through a convolutional layer and a fully connected layer. The angle-of-power feature extraction network is used to convert the data of the first generator's angle of power into an angle-of-power matrix and then extract angle-of-power features through a convolutional layer and a fully connected layer. The first emergency load shedding action feature extraction network is used to extract action features from the data of the first emergency load shedding action through a convolutional layer and a fully connected layer. The splicing network is used to splice the voltage features, angle-of-power features, and action features to obtain shared features;

[0020] The mapping module includes a convolutional layer with N channels. The shared features are respectively input into the convolutional layers of each channel in the mapping module to obtain the advantages of the sub-actions of the load for N dominant branches. The advantage of the sub-action of the load for dominant branch d is A d (s,a d ), where each dominant branch corresponds to a load;

[0021] The state extraction module includes a convolutional layer. The shared features are used to extract the state value V(s) through the convolutional layer;

[0022] The decision-making module obtains the advantage A d (s,a d ) of the sub-action of the load for dominant branch d and calculates the Q value Q d (s,a) of dominant branch d by combining it with the state value V(s).

[0023] In one embodiment, the calculation formula for the Q value Q d (s,a) is:

[0024]

[0025] Where, represents the action of dominant branch d, represents the action space, represents the action space and represents the dimension of the action space.

[0026] In one embodiment, the calculation formula for the target Q value Y d of dominant branch d is:

[0027]

[0028] Where, γ is a set discount factor, and s′ represents the state represented by the second bus voltage, the second generator's angle of power, and the second emergency load shedding action.

[0029] In one embodiment, the discount factor γ is 0.1.

[0030] In one embodiment, the loss function L for training the branch competition Q-network agent is defined as:

[0031]

[0032] According to another aspect of the present invention, an emergency load shedding decision method is provided, including:

[0033] Based on the current power grid transient voltage instability fault and the corresponding first emergency load shedding action, the first bus voltage and the first generator power angle for the entire time period are simulated.

[0034] Input the first emergency load shedding action and the first bus voltage and the first generator power angle obtained by simulation into the emergency load shedding decision model, and the final emergency load shedding action is determined.

[0035] Wherein, the emergency load shedding decision model is the emergency load shedding decision model constructed by using the above-mentioned emergency load shedding decision model construction method.

[0036] Generally speaking, compared with the prior art through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0037] First of all, the present invention selects the branch competition Q-network agent as the emergency load shedding decision model architecture. The number of output neurons of the branch competition Q-network agent increases linearly with the increase of controllable loads, while the number of output neurons of the traditional non-branched agent only increases exponentially with the increase of controllable loads. The significant reduction in the number of output neurons can greatly reduce the training difficulty and improve the training efficiency. At the same time, when the branch competition Q-network agent determines the second emergency load shedding action, the information of the first emergency load shedding action is incorporated. While the traditional deep reinforcement learning agent only selects the state as the input of the agent when determining the second action, ignoring the first action given by the agent itself. The training efficiency and decision-making ability of the BDQ agent incorporating the information of the first emergency load shedding action are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic structural diagram of the BDQ agent in one embodiment;

[0039] Figure 2 It is a flowchart of the steps for constructing an emergency load shedding decision model in one embodiment;

[0040] Figure 3 It is a schematic diagram of the feature preprocessing module in one embodiment;

[0041] Figure 4Topology diagram of an 8-machine 36-node system for an embodiment;

[0042] Figure 5(a) shows the voltage state before the BDQ agent's action for an embodiment;

[0043] Figure 5(b) shows the voltage state after the BDQ agent's action for an embodiment;

[0044] Figure 6 Schematic diagram for comparing the total rewards, total number of iterations, and total number of successes of three agent algorithms for an embodiment. Detailed implementation manners

[0045] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] The branching dueling Q-network (BDQ) agent is a common deep reinforcement learning neural network. As Figure 1 shown in the structure diagram of the BDQ agent, it can be mainly divided into a feature preprocessing module, a mapping module, and a decision-making module. The state s is input into the feature preprocessing module for feature preprocessing and then outputs a shared feature. On the one hand, the shared feature is input into the mapping module to obtain the advantages of N advantage branches, and on the other hand, it is input into the state extraction module to extract the state value V(s). The advantage of the advantage branch d and the state value V(s) are calculated to obtain the Q value Q d (s,a) of the advantage branch d, and the corresponding action a is determined according to the Q value. The operation processes in each module of the BDQ agent can be implemented by conventional methods. For example, the operation process is introduced in detail in the literature "Action Branching Architectures for Deep Reinforcement Learning", which will not be elaborated here.

[0047] The present invention constructs an emergency load shedding decision model based on the above BDQ agent. First, a preset number of interaction samples are obtained according to different power grid transient voltage instability faults. Each power grid transient voltage instability fault can generate one or more interaction samples, and then the interaction samples are used to train the BDQ agent. The trained BDQ agent is the constructed emergency load shedding decision model.

[0048] As Figure 2 shown in the flow chart of the steps for constructing the emergency load shedding decision model, it mainly includes:

[0049] Step S110: Select a power grid transient voltage instability fault.

[0050] After determining the topology of the power grid system, all operating conditions of the power grid system and all possible fault conditions under each operating condition can be determined, and all possible instability faults that may occur in the power grid system are combined. Select some of the instability faults as the training set, and randomly select an instability fault from the training set.

[0051] Step S120: According to the current power grid transient voltage instability fault and the corresponding first emergency load shedding action, simulate the first bus voltage and the first generator power angle for the entire time period.

[0052] After randomly selecting an instability fault, input the operating condition, fault condition, and the first emergency load shedding action of the current instability fault into the Power System Analysis Software Package (PSASP), and simulate the bus voltage and the current generator power angle state for the entire time period, that is, the first bus voltage and the first generator power angle. Among them, the initial first emergency load shedding action is 0, that is, no action.

[0053] Step S130: Input the first emergency load shedding action, the first bus voltage, and the first generator power angle obtained by simulation into the branch competition Q-network agent to make a decision to obtain the second emergency load shedding action.

[0054] The simulation results for the entire time period can be input into the BDQ agent, or a part of the first bus voltage and the first generator power angle can be intercepted and input into the BDQ agent. In this embodiment, the entire time period is 10 seconds, and the simulation results for the first 6 seconds are intercepted and input into the BDQ agent, that is, the intercepted first bus voltage, the first generator power angle state, and the first load shedding action are input into the pre-set BDQ network agent, and the shared representation f is obtained after feature preprocessing.

[0055] Such as Figure 3The following is a schematic structural diagram of the feature preprocessing module in this embodiment. The feature preprocessing module includes a voltage feature extraction network, an angle-of-power feature extraction network, a first emergency load shedding action feature extraction network, and a splicing network. The voltage feature extraction network is used to convert the data of the first bus voltage into a voltage matrix and then extract voltage features through a convolutional layer and a fully connected layer. There are multiple buses and multiple generators in the power grid system. The bus voltage obtained by simulation is the voltage of different buses changing with time. After being converted into a voltage matrix, the elements in the matrix represent the voltages of the corresponding buses at the corresponding sampling points. The angle-of-power feature extraction network is used to convert the data of the first generator's angle of power into an angle-of-power matrix and then extract angle-of-power features through a convolutional layer and a fully connected layer. The generator's angle of power obtained by simulation is the angle of power of different generators changing with time. After being converted into an angle-of-power matrix, the elements in the matrix represent the angles of power of the corresponding generators at the corresponding sampling points. The first emergency load shedding action feature extraction network is used to extract action features from the data of the first emergency load shedding action through a convolutional layer and a fully connected layer. The data of the load shedding action specifically represents the proportion of the load shedding, such as shedding 10%, 30%, 50%, 70% of the load. The splicing network is used to splice the voltage features, angle-of-power features, and action features to obtain a shared feature f.

[0056] After obtaining the shared feature f, the shared feature f is respectively input into a mapping module and a state extraction module. Among them, the mapping module includes a convolutional layer with N channels. Each channel's convolutional layer corresponds to a load and forms a dominant branch. There are a total of N loads, so there are N dominant branches. After the shared feature f passes through the convolutional layer of the mapping module, the advantages of the load sub-actions of the N dominant branches are obtained. The advantage of the load sub-action of the dominant branch d is A d (s,a d ), d ∈ 1, 2, ……, N. The state extraction module also includes a convolutional layer. The shared feature f is extracted by the convolutional layer in the state extraction module to obtain a state value V(s).

[0057] The decision-making module obtains the advantage A d (s,a d ) of the load sub-action of the dominant branch d and calculates the Q value Q d (s,a) of the dominant branch d with the state value V(s). Finally, the second emergency load shedding action is obtained through the decision-making of each branch.

[0058] Among them, in the BDQ network agent, the action action space of the dominant branch d has a dimension of n, indicating that there are n choices for the proportion of the load shedding. The Q value Q d (s,a) is calculated in the way of the competitive Q network:

[0059]

[0060] Step S140: According to the current power grid transient voltage instability fault and the corresponding second emergency load shedding action, simulate to obtain the second bus voltage and the second generator power angle for the entire time period and calculate the corresponding reward. Use the first emergency load shedding action, the first bus voltage, the first generator power angle, the second emergency load shedding action, the second bus voltage, the second generator power angle, and the reward obtained under the current power grid transient voltage instability fault to form an interaction sample and store it in the capacity pool.

[0061] After obtaining the second emergency load shedding action through step S130, based on the current power grid transient voltage instability fault (the current operating condition and the corresponding fault conditions) and the corresponding second emergency load shedding action, using the Power System Analysis Software Package (PSASP), simulate to obtain the current bus voltage and the current generator power angle data for the entire time period of the corresponding fault conditions of this operating condition, that is, the second bus voltage and the second generator power angle data. At the same time, calculate the reward under the second emergency load shedding action. Use the selected power grid transient voltage instability fault and the corresponding first emergency load shedding action, the first bus voltage, the first generator power angle, the second emergency load shedding action, the second bus voltage, the second generator power angle, and the reward to form an interaction sample and store it in the capacity pool.

[0062] Step S150: Determine whether the iteration termination condition is triggered. If not, use the second emergency load shedding action as the first emergency load shedding action and jump to step S200 to continue the iteration. Otherwise, use the interaction sample acquisition process under the current power grid transient voltage instability fault as one round, end the current round and execute step S160.

[0063] Among them, the iteration termination conditions include that the second emergency load shedding action can make the voltage return to stability (for example, if the simulation is for 10 s, then if all voltage amplitudes between 9 s and 10 s are greater than 0.8 p.u., it means the voltage has returned to stability) or the iteration number reaches the upper limit.

[0064] The effect of the second emergency load shedding action can be reflected by the voltage. When the voltage returns to stability, it indicates that the corresponding second emergency load shedding action has a good effect and there is no need to continue the iteration under the current instability fault. At the same time, to avoid excessive iterations, an iteration upper limit is set. When the iteration number reaches the upper limit, there is no need to continue the iteration either. When the reward does not meet the preset conditions and the iteration number has not reached the upper limit, use the second emergency load shedding action as the first emergency load shedding action, jump to step S120 to continue the iteration, and repeat steps S120 - S150 until the iteration number reaches the upper limit or the reward meets the preset conditions.

[0065] Step S160: Determine whether the cumulative number of rounds has reached the termination value. If so, end the modeling. If not, execute step S170.

[0066] Set the round termination value. For example, set the total number of rounds to 30,000 rounds, and the iteration process of one training sample is 1 round. After operating 30,000 rounds in total, it ends. The training samples are selected from the training set, and one sample can be reused. For example, the training set contains 150 training samples. The training samples are selected in sequence from the front to the back to obtain interactive samples. When the 150th training sample is selected, the next time it starts to select from the first sample again, and so on in a cycle until it is selected 30,000 times, and it ends after operating 30,000 rounds.

[0067] In one embodiment, the capacity of the capacity pool is limited. When the interactive sample data stored in the capacity pool reaches the upper limit of the capacity pool and new interactive samples need to be stored continuously, the sample stored at the beginning is removed.

[0068] Step S170: Determine whether the cumulative number of rounds has reached the training node value. If not, update the current power grid transient voltage instability fault and jump to step S120. If so, execute step S180.

[0069] After setting the round termination value, the training node value can be set. For example, set a training node every 200 rounds, and use 200, 400, 600... as the training node values in sequence. If the training node value is not reached, a new training sample is selected from the training set, that is, after updating the power grid transient voltage instability fault, jump to step S120 to continue the next round until the training node value is reached and then perform step S180.

[0070] Step S180: Randomly select multiple interactive samples from the capacity pool to train the branch competition Q-network agent, and then jump to step S110.

[0071] Once the training node is reached, randomly select multiple interactive samples from the capacity pool to train the branch competition Q-network agent, update the parameters of the branch competition Q-network agent, and then continue to obtain interactive samples based on the updated agent.

[0072] In one embodiment, when the round termination value is reached, multiple interactive samples are also randomly selected from the capacity pool to train the branch competition Q-network agent first, and then the entire modeling process ends.

[0073] Specifically, when updating the BDQ network, the target Q value Y of branch d can be calculated in the way of the Double Deep Q Network (DDQN). d :

[0074]

[0075] Among them, γ is the discount factor, θ is the BDQ policy network parameter, θ- For the BDQ target network parameters, s′ represents the state represented by the second bus voltage, the second generator power angle data, and the second emergency load shedding control action in the interaction sample. The loss function L of BDQ is defined as the mean square error between the Q values of all branches and the target Q value:

[0076]

[0077] Update the parameters of the agent for the loss function according to the gradient descent method.

[0078] Through steps S110 to S180, use the finally trained branch competition Q-network agent as the emergency load shedding decision model.

[0079] Correspondingly, the present invention also relates to an emergency load shedding decision method, which includes:

[0080] According to the current power grid transient voltage instability fault and the corresponding first emergency load shedding action (initially 0, no control measure), simulate the first bus voltage and the first generator power angle for the entire time period;

[0081] Input the first emergency load shedding action and the simulated first bus voltage and first generator power angle into the emergency load shedding decision model to make a decision on the final emergency load shedding action.

[0082] Wherein, the emergency load shedding decision model is the emergency load shedding decision model constructed by using the emergency load shedding decision model construction method in the above text.

[0083] During the operation of the power grid, when a certain fault condition in the operating condition is detected, use the optimal emergency load shedding control action corresponding to the certain fault condition in the operating condition in the corresponding relationship for emergency load shedding control.

[0084] Taking the 8-machine 36-node system of China Electric Power Research Institute as an example, the method in this embodiment is specifically described, and the system topology diagram is as Figure 4 shown. Generate the operating condition and fault condition sets by using the settings shown in Table 1. The power flow level value represents the multiple under the rated state, which is used to characterize the operating condition.

[0085] Table 1 Operating condition and fault condition sets

[0086] Type Specific setting Quantity Fault type Three-phase metallic short-circuit fault 1 Faulty line All non-transformer lines 26 Fault duration 0.05s, 0.15s 2 Fault location 2％，50％，98％ 3 Power flow level (generator, load) (0.9,0.9)，(1.0,1.0)，(1.1,1.1) 3

[0087] A total of 166 transient voltage instability conditions are generated, 127 of which are randomly selected for the model training of the agent, and the remaining 39 are used for the performance test of the agent.

[0088] Agent Training Parameter Settings: The total number of training rounds for the agent's neural network is 30000, the capacity of the experience pool is 10000, the capacity of the batch of samples drawn from the experience pool is 128, and the discount factor is 0.1. The total time period is 10 seconds, and the intercepted time period is the first 6 seconds. The agent's neural network is written using the PyTorch framework and enables CUDA-accelerated training. The agent executes the load shedding decision by calling the PSASP simulation software through Python.

[0089] BDQ Agent Load Shedding Decision Example: An actual transient voltage instability condition is used to verify the load shedding decision effect of the BDQ agent. Refer to Figure 5(a). Select a transient voltage instability condition in the test set with a power flow level of [1.0, 1.0]. A three-phase short-circuit grounding fault occurs at 98% of line 25-26 near node 25, and the fault lasts for 0.15 s. Before applying the load shedding decision, the system voltage becomes unstable. The agent gives the decision based on the power angle, voltage, and load shedding action information: Cut 15% at node BUS9, 50% at node BUS16, 50% at node BUS18, 10% at node BUS19, 15% at node 20, 10% at node 21, 5% at node 22, 5% at node 23, and 40% at node 29. Refer to Figure 5(b). After applying the load shedding decision, the system finally restores voltage stability. And because the BDQ network has the ability to make decisions for multiple nodes simultaneously, it can give an effective decision at one time, reducing the number of load shedding attempts and the time consumption.

[0090] Agent Decision Performance Metrics: To evaluate the decision performance of the agent after training, three performance metrics, namely the total number of iterations, the total number of successful decisions, and the total reward, are used for evaluation. The meanings of the metrics and their relationships with the decision performance are referred to Table 2.

[0091] Table 2 Meanings of the Metrics and Their Relationships with the Decision Performance

[0092]

[0093] The total number of iterations reflects the rapidity of the agent's decision. The fewer the total number of iterations, the faster the agent's decision speed. The total number of successes reflects the agent's decision-making ability. The more the total number of successes, the stronger the agent's decision-making ability. The magnitude of the total reward further reflects the agent's decision-making ability. On the premise that the decision is effective, the smaller the total reward obtained, the greater the total reward, and the higher the decision quality of the agent.

[0094] Three deep reinforcement learning agents are defined here to verify the impact of the presence or absence of branches and the presence or absence of action fusion on the performance of the agents. Agent A1 is a branchless deep reinforcement learning agent, Agent A2 is a BDQ agent with branches, and Agent A3 is a BDQ agent with branches and action fusion (the solution of the present invention). Performance evaluation is carried out on all test sets, and performance evaluation is carried out every 200 training rounds. The experiment is repeated 3 times, and the results are as Figure 6 shown. The solid line represents the average value of multiple results, and the light-colored blocks represent the variance of the performance metrics. It can be seen that compared with the branchless Agent A1, the multi-branch BDQ Agent A2 is superior in all performance metrics, indicating that the multi-branch BDQ agent structure can reduce the training difficulty and improve the decision-making quality. Compared with Agent A2 without action fusion, the BDQ Agent A3 with action fusion is superior in all performance metrics, indicating that action fusion can further improve the training efficiency and decision-making quality.

[0095] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for constructing an emergency load shedding decision model, characterized in that The emergency load shedding decision-making model is a branch competition Q-network agent based on action fusion, and the model construction method includes: Step S1: Select a power grid transient voltage instability fault; Step S2: According to the current power grid transient voltage instability fault and the corresponding first emergency load shedding action, simulate the first bus voltage and the first generator power angle for the entire time period; Step S3: Input the first emergency load shedding action and the simulated first bus voltage and the first generator power angle into the branch competition Q-network agent to make a decision on the second emergency load shedding action; Step S4: According to the current power grid transient voltage instability fault and the corresponding second emergency load shedding action, simulate the second bus voltage and the second generator power angle for the entire time period and calculate the corresponding reward, and form an interaction sample with the first emergency load shedding action, the first bus voltage, the first generator power angle, the second emergency load shedding action, the second bus voltage, the second generator power angle and the reward obtained under the current power grid transient voltage instability fault and store it in the capacity pool; Step S5: Judge whether the iteration termination condition is triggered. If not, use the second emergency load shedding action as the first emergency load shedding action and jump to Step S2 to continue the iteration. Otherwise, take the interaction sample acquisition process under the current power grid transient voltage instability fault as a round, end the current round and execute Step S6, where the iteration termination condition includes that the second emergency load shedding action can make the voltage return to stability or the iteration number reaches the upper limit: Step S6: Judge whether the cumulative number of rounds reaches the termination value. If so, end the modeling. If not, execute Step S7; Step S7: Judge whether the cumulative number of rounds reaches the training node value. If not, update the current power grid transient voltage instability fault and jump to Step S2. If so, execute Step S8: Step S8: Randomly select multiple interaction samples from the capacity pool to train the branch competition Q-network agent and then jump to Step S1.

2. The emergency load shedding decision-making model construction method according to claim 1, characterized in that In Step S3, inputting the simulated first bus voltage and the first generator power angle into the branch competition Q-network agent specifically means: intercepting the first bus voltage and the first generator power angle of a partial time period near the fault point and inputting them into the branch competition Q-network agent.

3. The method for constructing an emergency load shedding decision model according to claim 2, wherein, The entire time period is 10 seconds, and the intercepted time period is the first 6 seconds.

4. The method for constructing an emergency load shedding decision model according to claim 1, wherein Call the PSASP simulation software to simulate the first bus voltage and the first generator power angle for the entire time period.

5. The method for constructing an emergency load shedding decision model according to claim 1, wherein The branch competition Q-network agent includes a feature preprocessing module, a mapping module, a state value extraction module and a decision-making module, where The feature preprocessing module includes a voltage feature extraction network, an angle-of-power feature extraction network, a first emergency load shedding action feature extraction network, and a splicing network. The voltage feature extraction network is used to convert the data of the first bus voltage into a voltage matrix and then extract voltage features through a convolutional layer and a fully connected layer. The angle-of-power feature extraction network is used to convert the data of the first generator's angle of power into an angle-of-power matrix and then extract angle-of-power features through a convolutional layer and a fully connected layer. The first emergency load shedding action feature extraction network is used to extract action features from the data of the first emergency load shedding action through a convolutional layer and a fully connected layer. The splicing network is used to splice the voltage features, angle-of-power features, and action features to obtain shared features; The mapping module includes a convolutional layer with N channels. After the shared features are respectively input into the convolutional layers of each channel in the mapping module, the advantages of the load sub-actions of N dominant branches are obtained. The advantage of the load sub-action of the dominant branch d is A d (s,a d ), where each dominant branch corresponds to a load; The state extraction module includes a convolutional layer, and the shared features are used to extract the state value V(s) through the convolutional layer; The decision-making module obtains the advantage A of the load sub-action of the advantageous branch d d (s,a d ) and calculates the Q-value Q of the advantageous branch d by combining it with the state value V(s) d (s,a).

6. The method for constructing an emergency load shedding decision model according to claim 5, wherein Q-value Q d (s,a)'s calculation formula is: Among them, represents the action of the dominant branch d, represents the action space, represents the action space dimension.

7. The method for constructing an emergency load shedding decision model according to claim 6, wherein Target Q-value Y of the advantageous branch d d The calculation formula is as follows: where γ is a set discount factor, and s′ is the state represented by the second bus voltage, the second generator's angle of power, and the second emergency load shedding action.

8. The method for constructing an emergency load shedding decision model according to claim 7, wherein, The discount factor γ is 0.

1.

9. The method for constructing an emergency load shedding decision model according to claim 7, wherein The loss function L for training the branch competition Q-network agent is defined as:

10. An emergency load shedding decision method, characterized in that, including: According to the current power grid transient voltage instability fault and the corresponding first emergency load shedding action, the first bus voltage and the first generator's angle of power for the entire time period are simulated; The first emergency load shedding action and the simulated first bus voltage and first generator's angle of power are input into the emergency load shedding decision model, and the final emergency load shedding action is determined; wherein, the emergency load shedding decision model is an emergency load shedding decision model constructed by using the emergency load shedding decision model construction method according to any one of claims 1 to 9.

Images