Emergency load shedding decision method and decision model training method and training equipment

Abstract

The present invention discloses an emergency load shedding decision model training method and its training device as well as an emergency load shedding decision method. The emergency load shedding decision model is an intelligent agent with a branch competition Q network. The model training method includes: k Input the agent and make the updated emergency load shedding action a k+1 And get the corresponding status s k+1 and reward r k , get the interaction sample {s k , a k+1 , s k+1 , r k In this approach, the agent randomly selects a strategy from a greedy strategy or an exploration strategy to decide on an emergency load shedding action. As the number of decision rounds increases, the probability of the greedy strategy increases, while the probability of the exploration strategy decreases. The exploration strategy randomly selects an emergency load shedding action from the load portion with low bus voltage. Multiple iterations are performed to obtain multiple interaction samples under the current grid transient voltage instability fault. These interaction samples are then used to update the agent parameters. This approach combines a constrained greedy strategy with an exploration strategy to improve the training efficiency and decision-making performance of the BDQ agent.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system stability control, and more specifically, relates to an emergency load shedding decision model training method and training equipment thereof, as well as an emergency load shedding decision method. Background Art

[0002] Event-driven emergency load shedding is a crucial measure for ensuring transient voltage stability in power systems after a severe fault causes transient voltage instability. Currently, emergency load shedding measures are typically developed offline by power grid experts. These experts continuously adjust and develop emergency load shedding measures based on a wide range of actual operating conditions and transient stability simulations. The mapping between operating conditions and emergency load shedding measures is documented in the operator manual. During online operation, emergency load shedding measures are rapidly implemented based on operating conditions.

[0003] As interconnected power grids expand and become more complex, relying on grid experts to devise emergency load shedding control measures becomes time-consuming and labor-intensive. To reduce the workload of grid experts, some studies have used optimization methods to determine emergency control measures. However, as the scale of grid interconnections increases, optimization methods are time-consuming and can sometimes fail to converge.

[0004] In recent years, the rapid development of branching dueling Q-network (BDQ) algorithms in artificial intelligence has provided a new approach for formulating emergency load shedding control measures. BDQ algorithms can directly process high-dimensional decision spaces, potentially reducing reliance on experts and improving the efficiency of formulating emergency load shedding control measures. However, when existing BDQ-based algorithms are applied to event-based emergency load shedding decision-making, an event-driven interaction process between the agent and the environment has not yet been established. Furthermore, the lack of incorporation of proprietary knowledge about power systems results in low agent training efficiency and decision-making performance. How to quickly and effectively formulate emergency load shedding control measures is a technical problem that urgently needs to be addressed. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides an emergency load shedding decision model training method and its training equipment, as well as an emergency load shedding decision method, the purpose of which is to improve the training efficiency and decision-making performance of the branch competition Q network intelligent agent in deciding emergency load shedding control measures.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for training an emergency load shedding decision model is provided. The emergency load shedding decision model is an intelligent agent having a branched competitive Q network. The model training method includes:

[0007] Step S11: According to the current grid transient voltage instability fault and the current emergency load shedding action ak , the bus voltage and generator power angle state s are obtained by simulation k ;

[0008] Step S12: The current bus voltage and generator power angle state s k Input the agent and make the updated emergency load shedding action a k+1 , wherein the agent randomly selects a strategy from the greedy strategy and the exploration strategy to decide the emergency load shedding action, and as the number of decision rounds increases, the probability of the greedy strategy being selected gradually increases, and the probability of the exploration strategy being selected gradually decreases. The exploration strategy randomly selects the emergency load shedding action under the set constraints, and the constraints include randomly selecting the emergency load shedding action from the part of the load with lower bus voltage;

[0009] Step S13: Based on the current grid transient voltage instability fault and the updated emergency load shedding action a k+1 , the simulation obtains the updated bus voltage and generator power angle state s k+1 , and calculate the corresponding reward r k , get an interaction sample {s k , a k+1 , s k+1 , r k};

[0010] Step S14: Update the bus voltage and generator power angle state s k+1 As the current bus voltage and generator power angle state s k , jump to step S12 to iterate, iterate multiple times to obtain multiple interaction samples under the current power grid transient voltage instability fault;

[0011] Step S15: Extract interaction samples to update agent parameters.

[0012] In one embodiment, when the agent makes an initial decision, the probability of the greedy strategy being selected is no more than 10%, and the probability of the exploration strategy being selected is no less than 90%.

[0013] In one embodiment, the probability of the exploration strategy being selected decays exponentially.

[0014] In one embodiment, the constraint condition of the exploration strategy further includes constraining the range of the emergency load shedding action to be a portion of larger values ​​among the optional load shedding amounts.

[0015] In one embodiment, the constraint condition is expressed as:

[0016]

[0017] V1≤…≤Vi ≤V i+1 ≤…≤V l

[0018]

[0019]

[0020]

[0021] Where l represents the number of loads, V i represents the voltage of the i-th load, Indicates the position of the i-th load after sorting according to the bus voltage, d p represents the set of allowed actions for each output branch of the agent, Indicates the allowable emergency load shedding capacity of the bth load, d max and d min Indicates the maximum and minimum allowable load shedding amounts.

[0022] In one embodiment, the simulation is to call PSASP simulation software to simulate and obtain the bus voltage and generator power angle state.

[0023] In one embodiment, after step S14 and before step S15, the method further includes:

[0024] The grid transient voltage instability fault is updated, the updated grid transient voltage instability fault is used as the current grid transient voltage instability fault and the process jumps to step S11 for iteration. Multiple iterations are performed to obtain interaction samples under multiple different grid transient voltage instability faults.

[0025] In one embodiment, after step S15, the method further includes:

[0026] Step S16: Randomly select a grid transient voltage instability fault and jump to step S11 for iteration, and iterate multiple times to achieve multiple updates of the agent parameters.

[0027] According to another aspect of the present invention, an emergency load shedding decision model training device is provided for executing the above-mentioned emergency load shedding decision model training method.

[0028] According to another aspect of the present invention, there is provided an emergency load shedding decision method, comprising:

[0029] Step S21: According to the current grid transient voltage instability fault and the current emergency load shedding action a k , the bus voltage and generator power angle state s are obtained by simulation k ;

[0030] Step S22: The current bus voltage and generator power angle state s k Input the intelligent agent with branch competition Q network in the emergency load shedding decision model and make the updated emergency load shedding action a k+1 The emergency load shedding decision model is obtained by training according to the above-mentioned emergency load shedding decision model training method;

[0031] Step S23: Based on the current grid transient voltage instability fault and the updated emergency load shedding action a k+1 , the simulation obtains the updated bus voltage and generator power angle state s k+1 ;

[0032] Step S24: Determine whether the current bus voltage has recovered and stabilized or whether the iteration has reached the upper limit. If so, perform the emergency load shedding action a with the updated value. k+1 As the final decision action under the current power grid transient voltage instability fault, if no, jump to step S22.

[0033] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0034] (1) For event-driven emergency load shedding control, an event-driven agent-environment interaction process is proposed. This interaction process can effectively guide the parameter update of the BDQ agent. The power system environment directly calls the transient stability simulation to update its state according to each emergency event, while the BDQ agent provides emergency load shedding control measures based on the state feedback from the environment.

[0035] (2) The BDQ agent's action selection adopts a combination of greedy and exploration strategies, and the action selection under the exploration strategy is subject to specific constraints. Analysis shows that when the agent adopts the exploration strategy, it guides the BDQ agent to select actions at multiple load nodes with low voltage. The BDQ agent is more likely to obtain effective emergency load shedding control measures, thereby improving the training efficiency and decision-making performance of the BDQ agent.

[0036] Furthermore, by limiting the value of each strategy probability, the specific type of change trend, and further setting the load shedding range in the constraint conditions, the training efficiency and decision-making performance of the BDQ agent can be more effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of the steps of a training method for an emergency load shedding decision model according to one embodiment;

[0038] Figure 2 is a schematic diagram of the interaction process between a BDQ agent and an environment according to an embodiment;

[0039] Figure 3 A schematic diagram of a BDQ agent subject to constraints according to an embodiment;

[0040] Figure 4 A topological diagram of a power system according to an embodiment;

[0041] FIG5( a ) shows the voltage status before load shedding decision according to an embodiment;

[0042] FIG5( b ) shows the voltage status after the load shedding decision is made according to an embodiment;

[0043] FIG6( a ) is a schematic diagram showing a comparison of the total number of iterations of a BDQ agent with and without knowledge fusion according to an embodiment;

[0044] FIG6( b ) is a schematic diagram showing a comparison of the total rewards of a BDQ agent with and without knowledge fusion according to an embodiment;

[0045] FIG6( c ) is a schematic diagram showing a comparison of the total number of successes of BDQ agents with and without knowledge fusion according to an embodiment. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0047] The branching dueling Q-network (BDQ) agent is a common deep reinforcement learning neural network. Given a state s as input, the agent is multi-branched. Each action is chosen for each load, allowing it to simultaneously decide on actions for multiple loads. The paper "Action Branching Architectures for Deep Reinforcement Learning" provides a detailed introduction to its structure, so I will not elaborate on it here.

[0048] The present invention constructs an emergency load shedding decision model based on the above-mentioned BDQ intelligent agent.

[0049] like Figure 1 The following is a flow chart of the training method steps for the emergency load shedding decision model: Figure 2 Shown is a schematic diagram of the interaction process between the corresponding BDQ agent and the environment.

[0050] The training steps of the emergency load shedding decision model mainly include:

[0051] Step S110: According to the current grid transient voltage instability fault and the current emergency load shedding action a k , the bus voltage and generator power angle state s are obtained by simulation k .

[0052] After determining the grid system topology, we can determine all operating conditions of the grid system and all possible fault conditions under each operating condition. This combination yields all possible instability faults that may occur in the grid system. A subset of these instability faults is selected as a training set, and one of these instability faults is randomly selected from the training set as the current grid transient voltage instability fault.

[0053] According to the current grid transient voltage instability fault and the current emergency load shedding action a k , call the transient stability simulation program, simulate the bus voltage and generator power angle state s k Specifically, the Power System Analysis Software Package (PSASP) is called to simulate the bus voltage and generator power angle state s k And feedback to the BDQ agent.

[0054] Among them, the initial current emergency load shedding action a k When it is 0, there is no action, that is, no load shedding action is performed.

[0055] Step S120: The bus voltage and the generator power angle state s k Input the agent and make the updated emergency load shedding action a k+1 , in which the intelligent agent randomly selects a strategy from the greedy strategy and the exploration strategy to decide the emergency load shedding action, and with the increase of decision rounds, the probability of the greedy strategy being selected gradually increases, and the probability of the exploration strategy being selected gradually decreases. The exploration strategy is to randomly select the emergency load shedding action under the set constraints. The constraints include randomly selecting the emergency load shedding action from the part of the load with lower bus voltage.

[0056] The agent can randomly select one of the greedy and exploration strategies to decide on an emergency load shedding action. The sum of the selection probabilities of the exploration and greedy strategies equals 1. In the initial decision-making process, the agent's decision-making ability is weaker and exploration is more needed. Therefore, the probability of the exploration strategy is higher. The initial probability of the exploration strategy can be set to 90% or above. As the number of decision rounds increases, the agent's decision-making ability becomes stronger and the need for exploration becomes weaker. Therefore, the probability of the exploration strategy gradually decreases to less than 10%, and the probability of the greedy strategy gradually increases. At the same time, constraints are set in the exploration strategy (i.e., the exploration strategy is limited by knowledge). When the exploration strategy is randomly selected, it must comply with these constraints. That is, the emergency load shedding action is randomly selected from the loads with lower bus voltages. For example, if there are 20 loads in total, 10 loads with lower bus voltages can be selected as the target set, and load shedding actions are selected from this target set. The other 10 loads with higher bus voltages are not subjected to load shedding actions. In other words, the output load shedding action is a load shedding amount of 0 (effectively, no load shedding). Among them, the interval span range of the target set can be optimized according to debugging.

[0057] In one embodiment, the probability of the exploration strategy being selected decays exponentially, while the probability of the greedy strategy being selected increases exponentially. ε Strategy:

[0058]

[0059] in, Indicates the neuron selected by the b-th output branch of the BDQ agent at the k-th decision step. k represents the bus voltage and generator power angle state at the kth step decision, d p represents the set of allowed actions for each output branch of the BDQ agent, ε k represents the probability of exploring the strategy, It is the sequence number value of the b-th branch action in the k-th step decision. is the load shedding amount of the b-th branch at the k-th decision step (each branch represents a load, e.g. Indicates that the load shedding amount of the fifth load is 10% at the kth decision), l inte represents the load shedding step (e.g. 5%, i.e. the load shedding amount is a multiple of 5%), ε0 represents the initial exploration probability, ε e represents the minimum exploration probability, n k is the total number of iterations, n decay is the attenuation factor, a k Represents the load shedding action at the k-th decision step.

[0060] When adopting the greedy strategy, the BDQ agent inputs the state information into its strategy network according to the state information fed back by the power system environment, and then obtains the current emergency load shedding control measure a k .

[0061] In one embodiment, if Figure 3 As shown in Figure 2, when adopting the exploration strategy, the random selection actions of the BDQ agent are subject to the following specific constraints:

[0062] Sort the load bus voltage from low to high:

[0063]

[0064] Among them, V i represents the voltage of the i-th load, Indicates the position of the i-th load after sorting by voltage.

[0065] Define the set of actions allowed:

[0066]

[0067] in, Indicates the permissible action of the bth load, d max and d min Indicates the maximum and minimum allowed actions. In one embodiment, the range of the emergency load shedding action is the larger value of the optional load shedding amount. For example, the optional load shedding amount is 50%, 45%, 35%, 30%, 25%, 20%, 15%, 10%, and 5%. The load shedding amount of 30% to 50% can be selected as the constraint condition. The span of the specific interval can be optimized through debugging. For example, Figure 3 As shown in the figure, there are three loads: load 1, load 2, and load 3. The voltage of each load is collected at the sampling point and sorted. When the exploration strategy is selected for decision-making, only loads 2 and load 3 with lower voltages are selected to be removed. For load 1, the load shedding action is not performed.

[0068] Conventional BDQ agents generally choose a greedy strategy with a constant probability to make decisions. However, the present invention optimizes the agent's decision-making strategy, combines the greedy strategy with a varying probability and the exploration strategy, and imposes constraints on the exploration strategy, thereby improving the training efficiency and decision-making performance of the BDQ agent.

[0069] Step S130: Based on the current grid transient voltage instability fault and the updated emergency load shedding action a k+1 , the simulation obtains the updated bus voltage and generator power angle state s k+1 , and calculate the corresponding reward r k , get an interaction sample {s k , ak+1 , s k+1 , r k}.

[0070] Step S140: Update the bus voltage and generator power angle state s k+1 As the current bus voltage and generator power angle state s k , jump to step S110 to iterate, and iterate multiple times to obtain multiple interaction samples under the current power grid transient voltage instability fault.

[0071] Each time the updated bus voltage and generator power angle state s is obtained k+1 Then, the updated bus voltage and generator power angle state s k+1 As the current bus voltage and generator power angle state and jump to step S120, repeat steps S120 to S140 to achieve multiple iterations, and multiple interactive samples under the current grid transient voltage instability fault can be obtained.

[0072] In one embodiment, in order to obtain more samples, after obtaining multiple interactive samples under the current power grid transient voltage instability fault, the following steps may be further performed:

[0073] Step S141: Update the grid transient voltage instability fault, use the updated grid transient voltage instability fault as the current grid transient voltage instability fault and jump to step S110 for iteration, repeat steps S110 to S141 to implement multiple iterations, and obtain interaction samples under multiple different grid transient voltage instability faults.

[0074] Step S150: Extract interaction samples to update agent parameters.

[0075] The agent periodically extracts a batch of experience samples and calculates the temporal difference error between the policy network and the target network. This error is used to update the parameters of the BDQ agent's policy network. The target network parameters are periodically synchronized with the policy network.

[0076] First, calculate the actual value Q d (s,a):

[0077]

[0078] Among them, s represents the current bus voltage and generator power angle state, d represents a certain dominant branch, a d Indicates the current emergency load shedding action of branch d, a′ d Indicates the updated emergency load shedding action of branch d. Action of branch d represents the action space, Representing the action space The dimension of V(s) represents the state value of the shared feature extracted by the convolution layer, A d (s,a d ) represents the dominance of the load sub-action of the dominant branch d.

[0079] Secondly, the target value of Q is Y d The calculation method is as follows:

[0080]

[0081] Among them, r represents the reward obtained by the agent, γ is the set discount coefficient, θ is the BDQ strategy network parameter, and θ - is the BDQ target network parameter, and s′ represents the updated bus voltage and generator power angle state.

[0082] Finally, the timing differential error is defined as:

[0083]

[0084] Among them, e TD is the temporal difference error, and N is the total number of branches of the BDQ agent.

[0085] The policy network parameter update method of the BDQ agent is as follows.

[0086] θ w+1 =θ w -α▽ θ e TD

[0087] Among them, θ w is the policy network parameter of the BDQ agent after the wth parameter update, α is the learning rate, ▽ θ represents the derivative with respect to the parameter θ.

[0088] In one embodiment, after performing one round of parameter updates, training may be continued to perform multiple parameter updates, that is, further comprising:

[0089] Step S160: Randomly select a transient voltage instability fault in the power grid and jump to step S110 for iteration. Multiple iterations are performed to achieve multiple updates of the agent parameters.

[0090] Specifically, S110 - S160 may be repeatedly executed until the BDQ agent, after parameter update, makes a decision to obtain an effective emergency load shedding action.

[0091] Correspondingly, the present invention also relates to an emergency load shedding decision model training device, which is used to execute the above-mentioned emergency load shedding decision model training method.

[0092] Accordingly, the present invention also relates to an emergency load shedding decision method based on an emergency load shedding decision model, which mainly includes:

[0093] Step S210: According to the current grid transient voltage instability fault and the current emergency load shedding action a k , the bus voltage and generator power angle state s are obtained by simulation k .

[0094] Initially, the current emergency load shedding action is 0, that is, no load shedding action is performed.

[0095] Step S220: The current bus voltage and generator power angle state s k Input the intelligent agent with branch competition Q network in the emergency load shedding decision model and make the updated emergency load shedding action a k+1 .

[0096] Among them, the emergency load shedding decision model is trained according to the emergency load shedding decision model training method mentioned above. When making decisions using the trained emergency load shedding decision model, the intelligent agent in the emergency load shedding decision model adopts a combination of greedy strategy and exploration strategy to make decisions.

[0097] Step S230: Based on the current grid transient voltage instability fault and the updated emergency load shedding action a k+1 , the simulation obtains the updated bus voltage and generator power angle state s k+1 ;

[0098] Step S240: Determine whether the current bus voltage has recovered and stabilized or whether the iteration has reached the upper limit. If so, perform the emergency load shedding action a according to the updated emergency load shedding action. k+1 As the final decision action under the current grid transient voltage instability fault, if no, jump to step S220 to iterate and make a new decision.

[0099] When the intelligent agent outputs an emergency load shedding action, that is, the specific load shedding amount of each load, it judges through simulation whether the current action can restore the voltage to stability. If it cannot be restored and the number of iterations has not reached the upper limit, the intelligent agent continues to make decisions and repeats the state-action-state-action cycle until the current action can restore the voltage to stability or the number of iterations reaches the upper limit. The current action is used as the emergency load shedding action under the current transient voltage instability fault of the power grid.

[0100] Taking the 8-machine 36-node system of China Electric Power Research Institute as an example, the method in this embodiment is specifically described. The system topology diagram is as follows: Figure 4As shown in Table 1, a large number of transient voltage instability operating conditions were generated by setting different fault types, fault lines, fault durations, fault locations, and power flow levels to train and test the BDQ agent. A total of 166 transient voltage instability operating conditions were generated, 127 of which were randomly selected for agent model training, and the remaining 39 were used for performance testing.

[0101] Table 1

[0102]

[0103] The agent training parameters were set as follows: 30,000 total neural network training rounds, 10,000 experience pool capacity, 128 batches of samples drawn from the experience pool, a discount factor of 0.1, a learning rate of 0.0001, 9 branches (i.e., making load shedding decisions for 9 loads at a time), a minimum allowed action of 0.2 (i.e., a minimum load shedding of 20%), a maximum allowed action of 0.5 (i.e., a maximum load shedding of 50%), an initial exploration probability of 0.9, and a minimum exploration probability of 0.1. The agent's neural network was developed using the PyTorch framework and trained with CUDA acceleration. The agent executed load shedding decisions using the PSASP simulation software, which was called from Python.

[0104] Verification of the effectiveness of the event-based interaction process: A real-world transient voltage instability condition was used to validate the load shedding decision-making of the BDQ agent after event-based interaction. A transient voltage instability condition with a power flow level of [1.0, 1.0] was selected from the test set. A three-phase short-circuit to ground fault occurred at 98% of line 27-28 near node 27, lasting 0.15 seconds. As shown in Figure 5(a), before the load shedding decision was made, the system experienced transient voltage instability. Based on the power angle and voltage information, the agent made the following decisions: 50% shedding of nodes BUS16, 50% of BUS18, and 40% of node 29. As shown in Figure 5(b), after the emergency load shedding decision was made, the system regained transient stability. This demonstrates that the event-driven interaction process effectively guides the BDQ agent in event-based load shedding.

[0105] In order to verify the influence of the combination of exploration strategy and greedy strategy on the training efficiency and decision-making performance of the agent in the present invention, the decision-making strategy of the agent in the present invention is defined as a combination strategy with knowledge fusion, and the conventional agent equal probability greedy strategy is defined as a combination strategy without knowledge fusion. The improved combination strategy with knowledge fusion and the combination strategy without knowledge fusion before improvement are evaluated respectively. The performance of the BDQ agent on all test sets, that is, the performance of the trained agent is compared. A performance evaluation is performed every 200 training rounds. The total number of iterations, the total reward and the total number of successes are used to evaluate the performance of the agent. The fewer the total number of iterations, the faster the decision-making speed of the agent. Under the premise of effective decision-making, the smaller the total reward, the greater the total reward obtained, and the higher the quality of the agent's decision. The more total successes, the stronger the decision-making ability of the agent. The experiment was repeated 3 times, and the results are as follows Figures 6(a) to 6(b) As shown, the lines represent the average of multiple results, the light-colored blocks represent the variance of performance indicators, the horizontal axis represents the training rounds in the agent training phase, and the vertical axis represents the performance of various decisions made by the trained agent. As shown in Figure 6(a), after the same training rounds, the BDQ agent with knowledge fusion requires fewer total iterations in actual decision-making, indicating that knowledge fusion can improve the decision-making speed of the agent. As shown in Figure 6(b), after the same training rounds, the BDQ agent with knowledge fusion has more total rewards in actual decision-making, indicating that knowledge fusion can improve the decision-making ability of the agent. As shown in Figure 6(c), after the same training rounds, the BDQ agent with knowledge fusion has more total successes in actual decision-making, indicating that knowledge fusion can improve the decision-making ability of the agent. On the other hand, Figures 6(a) to 6(b) In the data, the performance indicators of the knowledge-fused BDQ agent improve faster, indicating that knowledge fusion can improve the training efficiency of the BDQ agent.

[0106] It will be easily understood by those skilled in the art that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for training an emergency load shedding decision model, characterized in that: The emergency load shedding decision model is an intelligent agent with a branched competitive Q network, and the model training method includes: Step S11: According to the current grid transient voltage instability fault and the current emergency load shedding action , the bus voltage and generator power angle state are obtained by simulation ; Step S12: Current bus voltage and generator power angle state Input the agent and make the updated emergency load shedding action , wherein the agent randomly selects a strategy from the greedy strategy and the exploration strategy to decide the emergency load shedding action, and as the number of decision rounds increases, the probability of the greedy strategy being selected gradually increases, and the probability of the exploration strategy being selected gradually decreases. The exploration strategy randomly selects the emergency load shedding action under the set constraints, and the constraints include randomly selecting the emergency load shedding action from the part of the load with lower bus voltage; Step S13: Based on the current grid transient voltage instability fault and the updated emergency load shedding action , the simulation obtains the updated bus voltage and generator power angle state , and calculate the corresponding reward , get an interactive sample { , , , }; Step S14: Update the bus voltage and generator power angle state As the current bus voltage and generator power angle status , jump to step S12 to iterate, iterate multiple times to obtain multiple interaction samples under the current power grid transient voltage instability fault; Step S15: extract interaction samples and update agent parameters; The constraints of the exploration strategy also include constraining the range of the emergency load shedding action to be a larger value among the optional load shedding amounts. The constraint is expressed as follows: in, Indicates the amount of load, represents the voltage of the i-th load, It represents the position of the i-th load after being sorted by bus voltage, represents the set of allowed actions for each output branch of the agent, Indicates the allowable emergency load shedding capacity of the bth load, and Indicates the maximum and minimum allowable load shedding amounts.

2. The emergency load shedding decision model training method according to claim 1, characterized in that: When the agent makes an initial decision, the probability of the greedy strategy being selected is no more than 10%, and the probability of the exploration strategy being selected is no less than 90%.

3. The emergency load shedding decision model training method according to claim 2, characterized in that: The probability of the exploration strategy being selected decays exponentially.

4. The emergency load shedding decision model training method according to claim 1, characterized in that: The simulation is to call the PSASP simulation software to simulate and obtain the bus voltage and generator power angle state.

5. The emergency load shedding decision model training method according to claim 1, characterized in that: After step S14 and before step S15, the method further includes: The grid transient voltage instability fault is updated, the updated grid transient voltage instability fault is used as the current grid transient voltage instability fault and the process jumps to step S11 for iteration. Multiple iterations are performed to obtain interaction samples under multiple different grid transient voltage instability faults.

6. The emergency load shedding decision model training method according to claim 5, characterized in that: After step S15, the method further includes: Step S16: Randomly select a grid transient voltage instability fault and jump to step S11 for iteration, and iterate multiple times to achieve multiple updates of the agent parameters.

7. An emergency load shedding decision model training device, characterized in that: Used to execute the emergency load shedding decision model training method according to any one of claims 1 to 6.

8. An emergency load shedding decision method, characterized in that: include: Step S21: According to the current grid transient voltage instability fault and the current emergency load shedding action , the bus voltage and generator power angle state are obtained by simulation ; Step S22: The current bus voltage and generator power angle state Input the intelligent agent with branch competition Q network in the emergency load shedding decision model and make the updated emergency load shedding action The emergency load shedding decision model is obtained by training according to the emergency load shedding decision model training method according to any one of claims 1 to 6; Step S23: Based on the current grid transient voltage instability fault and the updated emergency load shedding action , the simulation obtains the updated bus voltage and generator power angle state ; Step S24: Determine whether the current bus voltage has returned to stability or whether the iteration has reached the upper limit. If so, perform the updated emergency load shedding action. As the final decision action under the current power grid transient voltage instability fault, if no, jump to step S22.

Images