Typhoon-resistant power distribution network resilience improvement method and terminal based on improved DQN

Abstract

The application discloses a typhoon-based power distribution network resilience improvement method and terminal based on improved DQN, wherein the fault probability of a power grid line is evaluated according to typhoon meteorological prediction data, and an emergency power generator vehicle is selected and sized in advance according to the probability; then, the emergency power generator vehicle and the tie line are adjusted in the disaster according to the real-time state of the power distribution network line. According to the resilience improvement method before and after the disaster, a Markov decision model is adaptively established, then the DQN is improved by using k-cross sampling, the optimal control strategy of the Markov decision model is solved by using the improved DQN algorithm, and the application performance of the DQN is evaluated by using a quantitative index. In this way, the resilience improvement effect of the power distribution network can be improved according to the prevention situation of the power distribution network before the disaster and the corresponding situation of the power distribution network in the disaster, the optimal control strategy of the resilience improvement is solved without modeling the power distribution network, the calculation amount of the power distribution network modeling is avoided, and real-time dynamic decision can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power distribution network scheduling, and particularly relates to a typhoon-based power distribution network resilience improvement method and terminal based on an improved DQN. BACKGROUND

[0002] As a basic component of the power system, the power distribution network has a complex structure and a fragile operating environment. In addition, the frequent occurrence of extreme weather disasters makes the power distribution network face serious and complex challenges. The mutual influence of this complexity and fragility makes it an urgent task to ensure the stable operation of the power distribution network in today's power system management.

[0003] Typhoon disasters have the characteristics of small probability and high risk. They not only cause a large number of power equipment to be damaged in a short period of time, but also significantly increase the difficulty of power system recovery, thereby causing a large-scale and long-duration power outage. The occurrence of this event chain not only poses a serious challenge to the reliability of the power system, but also puts higher requirements on the emergency response and repair process of the system. The concept of "resilience" is proposed in the field of electric power, which aims to enhance the sensing and coordination capabilities of the power distribution network to improve its resistance and recovery capabilities in the face of extreme disasters. This concept aims to make the power system more adaptive, effectively adapt to and quickly respond to unpredictable extreme events, and ensure the reliability of power supply.

[0004] Under typhoon disasters, the disaster process of the power distribution network can be divided into three stages: before the disaster, during the disaster, and after the disaster, which reflect the prevention ability of the power distribution network before the disaster, the resistance ability during the disaster, and the power restoration ability after the disaster. Before the disaster, a series of prevention and reinforcement measures can be taken to improve the resilience of the power system, including performing vulnerability assessment and targeted power distribution network reinforcement to reduce the potential failure rate, and deploying emergency power generation vehicles in advance to provide emergency power supply. During the disaster, emergency power supply can be achieved through tie lines and emergency power generation vehicles, and scheduling decisions can be made by introducing a dynamic programming algorithm. After the disaster, temporary power supply can be achieved by optimizing the location of power generation vehicles and reasonably utilizing distributed energy sources; and fast power restoration can be achieved by optimizing the resources of power restoration repair teams.

[0005] Although the above research effectively improves resilience at different stages, they mainly focus on their respective stages and lack comprehensive cross-stage optimization. The disaster stage is difficult to achieve real-time traffic scheduling through fixed-capacity emergency resources due to rapid changes, high uncertainty, and high timeliness requirements, so the resilience improvement effect is limited. SUMMARY

[0006] The technical problem solved by the present application is to provide a typhoon-based power distribution network resilience improvement method and terminal based on improved DQN, which can improve the resilience improvement effect of the power distribution network according to the pre-disaster power distribution network prevention and the corresponding situation of the power distribution network in the disaster, and ensure the real-time of the resilience improvement decision.

[0007] To solve the above technical problems, the technical scheme adopted by the present application is:

[0008] A typhoon-based power distribution network resilience improvement method based on improved DQN, comprising the steps of:

[0009] According to the typhoon meteorological prediction data, the fault probability of the power distribution network line is evaluated, and the site and capacity of the emergency power car are selected before the disaster;

[0010] According to the real-time state of the power distribution network line, the emergency power car and the tie line are adjusted in the disaster;

[0011] For the pre-disaster and disaster power distribution network resilience improvement method, the control action, system state and reward function of the intelligent agent interactive environment are defined to establish a Markov decision model;

[0012] The improved DQN algorithm is used for k-cross sampling, the improved DQN algorithm is used to solve the Markov decision model, the resilience improvement strategy of the power distribution network is obtained, and the resilience of the power distribution network is evaluated using the quantitative index.

[0013] To solve the above technical problems, another technical scheme adopted by the present application is:

[0014] A typhoon-based power distribution network resilience improvement terminal based on improved DQN, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to realize each step of the above typhoon-based power distribution network resilience improvement method based on improved DQN.

[0015] The beneficial effects of the present application are that the failure probability of the power grid line is evaluated according to the typhoon meteorological prediction data, and the emergency power generation vehicle is selected and sized in advance according to the probability; and then the emergency power generation vehicle and the tie line are adjusted in the disaster according to the real-time state of the distribution network line. According to the above pre-disaster-post-disaster resilience improvement method, the control action, the system state and the reward function of the adaptive definition intelligent agent interaction environment are established to establish a Markov decision model, then the improved DQN is used for k-cross sampling, the optimal control strategy of the Markov decision model is solved by using the improved DQN algorithm, and the application performance of the DQN is evaluated by using the quantitative index. Among them, the improved DQN using k-cross sampling strategy explores the action space by purposeful search instead of random action. In this way, the resilience improvement effect of the distribution network can be improved according to the pre-disaster distribution network prevention and the corresponding situation of the distribution network in the disaster, the optimal control strategy of the resilience improvement is solved without modeling the distribution network, the calculation amount of the distribution network modeling is avoided, and real-time dynamic decision can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flow chart of a typhoon-based distribution network resilience improvement method based on an improved DQN according to an embodiment of the present application;

[0017] Figure 2 A schematic diagram of a typhoon-based distribution network resilience improvement terminal based on an improved DQN according to an embodiment of the present application;

[0018] Figure 3 A state curve diagram of a distribution network during a typhoon according to an embodiment of the present application;

[0019] Figure 4 A flow chart of a pre-disaster, disaster and post-disaster distribution network resilience improvement method according to an embodiment of the present application;

[0020] Figure 5 A dynamic decision diagram of a DCLR problem according to an embodiment of the present application;

[0021] Figure 6 An input-output structure diagram of a DQN according to an embodiment of the present application;

[0022] Figure 7 A training process diagram of a DQN value network according to an embodiment of the present application;

[0023] Figure 8 An architecture diagram of a k-cross sampling strategy according to an embodiment of the present application;

[0024] Figure 9 An IEEE 33-node distribution system and a typhoon path diagram according to an embodiment two of the present application;

[0025] Figure 10A typhoon path map for an IEEE 141 node power distribution system of embodiment two of the present application;

[0026] Label explanation:

[0027] 1. A typhoon power distribution network resilience improvement terminal based on improved DQN; 2. Memory; 3. Processor. DETAILED DESCRIPTION

[0028] In order to explain the technical content, the purpose and effect of the present application in detail, the following will be described in conjunction with the embodiments and the accompanying drawings.

[0029] Please refer to Figure 1 The embodiment of the present application provides a typhoon power distribution network resilience improvement method based on improved DQN, which comprises the following steps:

[0030] According to the typhoon meteorological prediction data, the fault probability of the power distribution network line is evaluated, and the site selection and capacity of the emergency power car are determined before the disaster;

[0031] According to the real-time state of the power distribution network line, the emergency power car and the tie line are adjusted in the disaster;

[0032] For the pre-disaster and disaster power distribution network resilience improvement method, the control action, system state and reward function of the intelligent agent interactive environment are defined to establish a Markov decision model;

[0033] The improved DQN algorithm is used for k-cross sampling, the improved DQN algorithm is used to solve the Markov decision model, the resilience improvement strategy of the power distribution network is obtained, and the resilience of the power distribution network is evaluated by using the quantitative index.

[0034] From the above description, the beneficial effects of the present application are as follows: according to the typhoon meteorological prediction data, the fault probability of the power distribution network line is evaluated, and the site selection and capacity of the emergency power car are determined before the disaster; then according to the real-time state of the power distribution network line, the emergency power car and the tie line are adjusted in the disaster. According to the above pre-disaster-post-disaster resilience improvement method, the control action, system state and reward function of the intelligent agent interactive environment are adaptively defined to establish a Markov decision model, then k-cross sampling is used to improve DQN, the improved DQN algorithm is used to solve the optimal control strategy of the Markov decision model, and the application performance of DQN is evaluated by using the quantitative index. The improved DQN using k-cross sampling strategy explores the action space by purposeful search instead of random action. In this way, the resilience improvement effect of the power distribution network can be improved according to the pre-disaster power distribution network prevention and the corresponding situation of the disaster, the optimal control strategy of the resilience improvement is solved without modeling the power distribution network, the calculation amount of the power distribution network modeling is saved, and real-time dynamic decision can be realized.

[0035] Further, the pre-disaster emergency generator vehicle site selection and capacity determination includes:

[0036] A target function of the pre-disaster emergency generator vehicle site selection and capacity determination is established with the minimum total cost of the power distribution network as the target:

[0037]

[0038]

[0039] In the formula, Ω ξ represents a scene set of samples, p ξ represents a probability of the scene ξ appearing in the sampling, T represents the total time of the power distribution network being affected by disasters, C t,ξ represents the total cost of the scene ξ at t under the strategy adopted in the disaster;

[0040] C EG represents the emergency generator vehicle configuration cost, Ω N represents a set of all nodes, d n represents the distance from the node to the resource station, represents a state variable of whether the node n is preset with the emergency generator vehicle, C D represents the empty vehicle dispatch unit cost, represents the active capacity of the emergency generator vehicle at the node n, C P represents the capacity dispatch unit cost.

[0041] As can be seen from the above description, for the power distribution network affected by severe disasters, the emergency vehicle configuration cost is one order of magnitude smaller than the disaster cost. Under the premise of ensuring load supply, considering the configuration distance and the total capacity of the generator loaded by the emergency vehicle, the lowest pre-disaster transportation cost can be achieved under the same effect.

[0042] Further, the processing strategy and the tie line of the emergency generator vehicle in the disaster are adjusted, including:

[0043] A target function of the adjustment in the disaster is established with the maximum mathematical expectation of the weighted load recovery total amount during the recovery period as the target:

[0044] maxE s (β1P R -β2C G -β3C L );

[0045]

[0046]

[0047]

[0048] In the formula, E s (·) represents the mathematical expectation in state s, β1, β2, β3 all represent weight coefficients, P R represents the total amount of weighted load recovery, C G represents the operation cost of the emergency generator car, C L represents the operation cost of the tie line; ω m represents the weight coefficient of the load of node m, o t,m,φ represents the load p t,m,φ of the node, and whether to recover the state variable; represents the active power of the emergency generator car; represents the unit cost of the emergency generator car of node m; represents the state variable of line m-m', and represents the unit cost of the operation of tie line m-m', ω m-M represents the node set directly connected to node m.

[0049] As can be seen from the above description, in the case of considering the operation cost of the emergency generator car and the tie line, the target function of the disaster adjustment is established with the maximum mathematical expectation of the total amount of weighted load recovery during the recovery period as the target, which can fully consider the corresponding situation of the distribution network in the disaster to achieve the improvement of the disaster resilience.

[0050] Further, the distribution network system operation constraint condition, the line operation state constraint condition, and the emergency generator car output constraint condition are configured for the distribution network resilience improvement method before and in the disaster.

[0051] As can be seen from the above description, if the tie line fails, the closed line will be disconnected, and the unclosed line will not be closed; if the normally closed line fails, the line will be disconnected, so that the line operation state constraint condition is set to ensure the reliability of the line. And the emergency generator car output constraint condition is set to constrain the configuration, capacity and quantity of the emergency generator car, so as to ensure the reliability of the emergency resource configuration.

[0052] Further, the definition control action includes:

[0053] The control action taken by the intelligent agent at time t is expressed as:

[0054]

[0055] In the formula, represents the state variable of whether there is an emergency generator car for the operation of switch m at time t, represents the active power of the emergency generator car for switch m at time t, represents the reactive power of the emergency generator car for switch m at time t, and M represents the node set.

[0056] From the above description, it can be seen that, under the premise of guaranteeing the maximum recovery load, the control of active power and reactive power can ensure the efficient use of emergency generators, because the number and capacity of emergency generators are limited.

[0057] Further, defining the system state comprises:

[0058] Defining the system state according to the operation constraints of the distribution network and the constraints of the emergency generators:

[0059]

[0060] In the formula, p t,m,φ represents the active power of node m at time t, q t,m,φ represents the reactive power of node m at time t, represents the state variable of whether there is an emergency generator for switch m at time t.

[0061] From the above description, it can be seen that, because the number and capacity of emergency generators are limited, it is necessary to define the power constraints of the emergency generators to meet the actual operation and use conditions.

[0062] Further, defining the reward function comprises:

[0063] r t =-o hc,t c1+(1-o hc,t )c2∑(β1P t R -β2C t G -β3C t L );

[0064] In the formula, o hc,t represents the state variable of whether the action at time t violates the hard constraints; c1 and c2 both represent positive coefficients, and c1 is much larger than c2.

[0065] Further, the use of the quantitative index to evaluate the resilience of the distribution network comprises:

[0066] The resistance, load reduction and recovery capabilities of the distribution network are evaluated by calculating the load loss rate, maximum load retention rate and load recovery rate;

[0067] The overall resilience of the distribution network is evaluated by calculating the difference between the disaster state and the normal state.

[0068] From the above description, the load loss rate, the maximum load retention rate, the load recovery rate and the comprehensive toughness index can characterize the disaster resistance, post-disaster recovery and overall toughness level of the power distribution network, and can intuitively evaluate the overall toughness of the power distribution network in multiple dimensions.

[0069] Please refer to Figure 2 Another embodiment of the application provides a typhoon-based power distribution network toughness improvement terminal based on improved DQN, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize each step of the above-mentioned typhoon-based power distribution network toughness improvement method based on improved DQN.

[0070] The above-mentioned typhoon-based power distribution network toughness improvement method and terminal based on improved DQN are suitable for improving the toughness improvement effect of the power distribution network according to the pre-disaster prevention situation of the power distribution network and the corresponding situation of the power distribution network in the disaster, and ensuring the real-time of the toughness improvement decision, which will be described below through specific implementation ways.

[0071] Embodiment one

[0072] First, please refer to Figure 3 The state change of the power distribution network under typhoon disaster can be divided into the following three stages:

[0073] 1) Pre-disaster preparation stage (T pre ): Before the typhoon arrives, the power distribution network pre-allocates the position of the emergency generator car and the number of emergency generators of different capacities, P1 is the total load of the power distribution network.

[0074] 2) Disaster resistance stage (T res ): From the time t0 when the typhoon affects to the time t1 when the load is cut, the system performs load shedding operation. At time t2, the load loss reaches the maximum value, P2 is the total load at the maximum load loss.

[0075] 3) Post-disaster recovery stage (T rec ): At time t3, the typhoon leaves, at time t4, the power distribution network starts to recover the load, and at time t5, the power distribution network load supply is fully recovered.

[0076] In order to improve the pre-disaster prediction ability and disaster response ability of the power distribution network, in this embodiment, a power distribution network toughness improvement method is proposed based on the tie line and the emergency generator car.

[0077] Please refer to Figure 1 A typhoon-based power distribution network toughness improvement method based on improved DQN, comprising the steps of:

[0078] S1, according to the typhoon meteorological prediction data, the fault probability of the power distribution network line is evaluated, and the site selection and capacity of the emergency generator car are determined before the disaster.

[0079] Specifically, please refer to Figure 4 , the pre-disaster allocation strategy mainly covers 3 elements: first, obtain typhoon prediction information; second, use typhoon wind field model and line fault model to respectively evaluate the wind speed in the predicted typhoon affected area and the fault probability of each line at different time points; finally, according to the evaluation results, pre-allocate the capacity and arrangement position of the emergency power generation vehicle.

[0080] In this embodiment, under typhoon conditions, the distribution network may not be able to obtain main network power through the substation or feeder for a period of time, during which the distribution system operator (DSO) can decompose the distribution system into multiple island microgrids based on sectional switches, tie lines and emergency power generation vehicles, thereby continuing to supply power to critical loads and improving the resilience of the distribution network.

[0081] Please refer to Figure 5 , the recovery process is discretized and denoted as a set T. The DSO will make load recovery decisions at each time point t∈T. The distribution system is denoted as a graph G=(M,L), where M and L represent the node set and line set respectively. The sectional switches are denoted as a set S, and the active power and reactive power of each node m(∈M) are denoted as p t,m,φ and q t,m,φ , respectively, where φ∈{a,b,c} represents the phase.

[0082] Considering the configuration cost and disaster operation cost of the emergency power generation vehicle, by simulating the time sequence influence of the typhoon on the line, the scheme that minimizes the total cost of the distribution network is found. That is, the objective function of the pre-disaster allocation strategy is:

[0083]

[0084]

[0085] In the formula, Ω ξ represents the scenario set of the sample, p ξ represents the probability of the occurrence of scenario ξ in the sample, T represents the total time of the disaster of the distribution network, C t,ξ represents the total cost of scenario ξ at time t based on the strategy adopted in the disaster;

[0086] C EG represents the configuration cost of the emergency power generation vehicle, Ω N represents the set of all nodes, d n represents the distance from the node to the resource station, represents the state variable of whether the node n is preset with an emergency power generation vehicle (1 represents yes and 0 represents no), C D represents the unit cost of empty vehicle scheduling, represents the active capacity of the emergency generator car at node n, C P represents the capacity scheduling unit cost.

[0087] For the distribution network affected by severe disasters, the emergency generator car configuration cost C EG One order of magnitude smaller than the disaster cost, under the premise of ensuring load supply, considering the configuration distance of the emergency generator car and the total capacity of the loaded generator, the lowest pre-disaster transportation cost can be realized under the same effect.

[0088] S2, according to the real-time state of the distribution network line, adjusting the emergency generator car and the tie line in the disaster.

[0089] Specifically, please refer to Figure 4 , the real-time scheduling strategy in the disaster includes two key steps: first, the state sensing device of the distribution network obtains the line fault condition and the repair progress in real time; second, according to the fault and power restoration condition, the output power of the emergency generator car and the reconstruction strategy of the distribution network line are adjusted.

[0090] In this embodiment, the target function of the disaster adjustment is established by taking the mathematical expectation maximum of the weighted load recovery total amount during the recovery period as the target:

[0091] maxE s (β1P R -β2C G -β3C L );

[0092]

[0093]

[0094]

[0095] In the formula, E s (·) represents the mathematical expectation under the state s, β1, β2, β3 all represent weight coefficients, P R represents the weighted load recovery total amount, C G represents the emergency generator car operation cost, C L represents the tie line operation cost; ω m represents the weight coefficient of the node m load, o t,m,φ represents the state variable of whether the node load p t,m,φ recovers (1 represents yes, 0 represents no); represents the active power of the emergency generator car; represents the unit cost of the emergency generator car power generation at node m; represents the state variable of the line m-m' (1 is closed, 0 is disconnected), Ω m-M Ω

[0096] Further, the power distribution network system operation constraint condition, line operation state constraint condition and emergency generator car output constraint condition are configured for the pre-disaster and in-disaster power distribution network resilience improvement method.

[0097] 1) Power distribution network system operation constraint condition:

[0098] The power distribution system load recovery should satisfy the three-phase unbalanced power flow equation:

[0099]

[0100] In the formula, p t,m,φ Ω t,m,φ Ω Ω Ω m Ω mn,φφ' Ω

[0101] The voltage and current of each node of the power distribution system should be maintained within a feasible range:

[0102]

[0103] In the formula, U t,m,φ Ω t,m,φ Ω l,min Ω l,max Ω

[0104] 2) Line operation state constraint condition

[0105] The line equipped with a remote control switch is a tie line, and the line without a remote control switch is a normally closed line. If the tie line fails, the line that has been closed will be disconnected, and the line that has not been closed will not be able to be closed. If the normally closed line fails, the line will be disconnected. The line operation state constraint is as follows:

[0106]

[0107] In the formula, σ m-m',t is a 0-1 state variable used to represent the line fault state, which can be determined by the line state information obtained at each time; Ω denotes the set of tie lines.

[0108] 3) Emergency generator car output constraint condition

[0109] To ensure that the emergency generator car is connected to the distribution network, each node is configured with at most one emergency generator car, and the total constraint of the number of emergency generator cars is:

[0110]

[0111] In the formula, N V denotes the number of emergency generator cars, denotes the state variable of whether the node m has an emergency generator car.

[0112] Only the node configured with an emergency generator car can be configured with an emergency generator, and the condition needs to be met:

[0113]

[0114] In the formula, denotes the active power capacity of the emergency generator car at node m, P G denotes the capacity of the emergency generator.

[0115] The relationship between the active and reactive power capacity of the emergency generator car and the loading of the emergency generator is as follows:

[0116]

[0117]

[0118] In the formula, denotes the number of emergency generator cars of the i-th capacity at node m, P i G denotes the active power capacity of the i-th capacity emergency generator; denotes the reactive power capacity of the emergency generator car at node m, denotes the power factor angle of the i-th capacity emergency generator, and I(i∈I) denotes the set of emergency generators of different capacities.

[0119] The number constraint of each capacity emergency generator is:

[0120]

[0121] The number constraint of the generator loaded by the emergency generator car is:

[0122]

[0123] In the formula, denotes the upper limit of the emergency generator car.

[0124] S3, for the pre-disaster and disaster distribution network resilience improvement method, by defining the control action, system state and reward function of the interactive environment of the agent, to establish a Markov decision model.

[0125] Specifically, the agent will assume the role of DSO, and formulate load restoration strategies from the global perspective of the distribution system. The environment interacting with the agent is a distribution system simulator with action execution, state analysis and reward feedback functions.

[0126] S31, the control action taken by the agent at time t is expressed as:

[0127]

[0128] In the formula, Indicates the state variable of whether there is an emergency generator for switch m at time t, Indicates the closing operation, Indicates the opening operation. Since the action space A is composed of limited binary state variables, A has the characteristics of "discrete". Indicates the active power of the emergency generator for switch m at time t, Indicates the reactive power of the emergency generator for switch m at time t, and M indicates the node set.

[0129] S32, at time t, the system state is composed of 2 types of parameters related to DCLR (dynamic critical load recovery) decision, i.e. distribution system operation constraints and emergency generator constraints. The system state is expressed as:

[0130]

[0131] In the formula, p t,m,φ Indicates the active power of φ phase of node m at time t, q t,m,φ Indicates the reactive power of φ phase of node m at time t, Indicates the state variable of whether there is an emergency generator for switch m at time t. Since the state space S contains continuous variables of the power type, S has the characteristics of "continuous". In addition, s t Only part of the observable parameters, while the influence of other parameters (such as node voltage, line current) on decision-making can be assumed that the agent can actively learn in the process of interaction with the environment.

[0132] S33, reward function:

[0133] The reward function directly affects the agent's decision-making behavior. Therefore, the design of the reward function needs to comprehensively consider the impact of the recovery objective and constraints on the decision-making. In this embodiment, the constraints are divided into two categories: hard constraints and soft constraints. During the recovery period, if the agent's decision violates a soft constraint, the agent will be penalized (i.e., receive a negative reward), but the load recovery can continue; if a hard constraint is violated, the agent will be severely penalized, and the load recovery will fail.

[0134] The reward function is expressed as follows:

[0135] r t =-o hc,t c1+(1-o hc,t )c2∑(β1P t R -β2C t G -β3C t L );

[0136] In the formula, o hc,t The state variable o represents whether the action at time t violates the hard constraint. hc,t =1 indicates yes, o hc,t =0 indicates no; c1 and c2 are both positive coefficients, and c1 is much larger than c2.

[0137] The DCLR simulation environment, which interacts with the intelligent agent, has two basic functions: three-phase unbalanced power flow calculation and topology analysis. t and r t The parameters are obtained by reading the power flow calculation results.

[0138] In this embodiment, a deterministic DCLR simulation environment is constructed based on Python and OpenDSS, further forming ADSEII. First, a three-phase unbalanced power flow calculation model of the test system is built in OpenDSS. Furthermore, the network topology of the distribution system is stored in graph data format. Based on this, the topological constraints are verified for the agent's decisions using a depth-first search algorithm. Then, through the OpenDSS component interface, a Python-based DCLR simulation is implemented. After topology analysis and power flow calculation are completed, the system state and rewards are generated and fed back to the agent.

[0139] By adopting an "offline training-online decision-making" model, intelligent agents can fully leverage the speed advantage of neural network input-output mapping, thereby effectively improving the computational efficiency of application processes.

[0140] S4, using k-cross sampling to improve the DQN algorithm, using the improved DQN algorithm to solve the Markov decision model, obtaining the resilience improvement strategy of the power distribution network, and using the quantitative index to evaluate the resilience of the power distribution network.

[0141] Specifically, reinforcement learning is a general framework for solving MDP. In the MDP model of the established DCLR, the optimal load restoration strategy can be expressed as the control action with the highest value. For the characteristics of the DCLR problem "continuous state space-discrete action space", the DQN algorithm is adopted to search for the control action with the highest value through value iteration. As a model-free deep reinforcement learning algorithm, DQN does not require additional prior knowledge in the process of searching for the optimal strategy.

[0142] Deep Q-network (DQN) algorithm combines convolutional neural network and traditional Q-learning method. Its input and output structure is shown in Figure 6 DQN uses neural network to approximate the value function. The goal based on the value function method is to maximize the reward value by optimizing the value network, i.e.

[0143]

[0144] In the formula, θ is the parameter of the neural network.

[0145] Figure 7 The training process of DQN value network is shown. DQN uses deep neural network to represent the current value function and the target value function respectively. Q(a t ,s t ; θ) represents the output of the current value network, and Q(a′ t ,s′ t ; θ′) represents the output of the current target value network. The error function of DQN is:

[0146]

[0147] In the formula, γ is the discount factor.

[0148] Then update θ by stochastic gradient descent (SDG):

[0149]

[0150] In the formula, η is the update rate.

[0151] The training process of DQN can be summarized as follows:

[0152] (1) Initialize the experience pool and neural network parameters;

[0153] (2) Reset the environment and observe the initial state;

[0154] (3) Select an action at each time step and observe the next state and reward returned by the environment;

[0155] (4) Store these information in the experience pool;

[0156] (5) Use a randomly selected batch of experience to train the neural network;

[0157] (6) Update the policy network using the target network;

[0158] Repeat (2) to (6) until convergence or maximum training times are reached.

[0159] In this embodiment, an improved DQN method using k-crossover sampling strategy is used to solve the problem that the ε-greedy exploration in traditional DQN cannot effectively explore the action space of power distribution system network and cannot update Q value network parameters in time. The improved DQN explores the action space by purposeful search instead of random action. On the one hand, the reuse of high-quality historical data can be realized by using k-sampling strategy, and the access frequency of low reward value action is reduced. On the other hand, by combining the uniform crossover operator with the SoftMax roulette selection, the possibility of falling into local optimum can be avoided. The detailed architecture of the proposed k-crossover sampling strategy is shown in Figure 8 In the data preprocessing stage, the data is divided into training set and test set according to different paths and intensity of typhoon damage to different lines, and normalized. In the training stage, the Q network takes s t as input, calculates the Q value of all actions in the action space; similarly, the state s t+1 is input into the target Q network to calculate the target Q value of all actions, and the loss function is constructed by using the target Q value and the actual Q value to update the Q network. In the test stage, different states are input into the Q network, and the Q network calculates the Q value of all actions in the action space, and selects the optimal action as the strategy for improving the resilience of power distribution network.

[0160] The resistance, load reduction and recovery capabilities of the power distribution network are evaluated by calculating the load loss rate, maximum load holding rate and load recovery rate:

[0161] The load loss rate R loss represents the speed of the power distribution network from normal operation to load reduction operation state, reflecting the resistance of the power distribution network:

[0162]

[0163] The maximum load holding rate K load_maxCharacterize the pressure limit of the distribution network under the influence of typhoon, reflect the absorption capacity of the distribution network: K load_max = P2 / P1.

[0164] Relative load recovery rate L rec Indicates the speed of the distribution network quickly recovering to the normal load level after the extreme disaster leaves, related to the repair resources and dispatching strategy of the distribution network, reflects the power restoration capability of the distribution network after the disaster:

[0165]

[0166] The overall resilience of the distribution network is evaluated by calculating the difference between the disaster state and the normal state:

[0167] Comprehensive resilience index R com The difference between the disaster state and the normal state curve is calculated, and the resilience performance of the distribution network during the entire disaster process is considered, reflecting the overall resilience performance of the distribution network:

[0168]

[0169] Example two

[0170] The main difference between this embodiment and example one is that a specific application scenario is provided for example analysis, specifically:

[0171] The proposed method is applied to the improved IEEE 33, 141 node system to verify its effectiveness. The simulation configuration environment is shown in Table 1, and the cost and dispatching period parameter settings are shown in Table 2.

[0172] Table 1 Simulation configuration environment

[0173]

[0174]

[0175] Table 2 Cost and dispatching period parameter settings

[0176]

[0177] Based on the improved IEEE 33 node distribution network system, the system single line diagram and the typhoon path are shown in Figure 9 The parameter settings are as follows:

[0178] 1) Key load nodes, importance is set to 3; other nodes are general load nodes, importance is set to 1.

[0179] 2) Emergency generators include 0.1MW and 0.2MW two capacities, each 5.

[0180] 3) 5 emergency power generation vehicles, each vehicle is loaded with 6 emergency power generators.

[0181] To verify the effectiveness of the method proposed in this embodiment, this embodiment sets up 6 different test scenarios for comparison and analysis in terms of whether there is pre-disaster resource allocation, disaster reconstruction of distribution network and emergency power generation vehicle. Please refer to Table 3. Among them, the strategy without pre-disaster allocation is to allocate emergency resources to each important load node.

[0182] Table 3 Resilience improvement strategies under different test scenarios

[0183]

[0184]

[0185] The resilience indicators of each scenario are shown in Table 4.

[0186] In terms of comprehensive resilience index indicators:

[0187] 1) The scenario S5 adopting the two-stage strategy proposed in this embodiment improves by 54.84% compared to S0 without taking any measures, and improves by 2.59% compared to S4 without pre-disaster allocation;

[0188] 2) Among the two schemes that only adopt the emergency power generation vehicle strategy, S3 with pre-disaster allocation improves by 1.55% compared to S1 without pre-disaster allocation;

[0189] 3) S1 and S2 that only use emergency power generation vehicles and distribution network reconstruction respectively improve by 47.78% and 9.17% compared to S0 without taking any measures.

[0190] In terms of pre-disaster indicators, the loss of load rate, the two-stage resilience improvement strategy S5 can significantly reduce the loss of load rate of the distribution network, which is 86.09% lower than S0 without taking any measures, and 3.90% lower than S4 without pre-disaster allocation. In terms of disaster indicators, the maximum load retention rate, the two-stage resilience improvement strategy S5 can significantly improve the maximum load retention rate, which is 127.30% higher than S0 without taking any measures, and 6.75% higher than S4 without pre-disaster allocation. In terms of post-disaster indicators, the relative load recovery rate, scenario S5 can improve the relative load recovery rate of the distribution network, which is 38.40% higher than S0. The results show that the strategy proposed in this embodiment can effectively improve the resilience of the distribution network in the resistance stage, load reduction stage and recovery stage.

[0191] Table 4 Resilience index results of IEEE 33-node system under different test scenarios

[0192]

[0193] The test is based on an IEEE 141-node distribution system, and a single-line diagram and a typhoon path are shown in FIG. 1. Figure 10 The parameter settings are as follows.

[0194] 1) The emergency generators include three capacities of 0.1 MW, 0.2 MW and 0.3 MW, and each capacity has nine generators.

[0195] 2) There are nine emergency generator vehicles, and the upper limit of the emergency generators loaded on each vehicle is six.

[0196] 3) The remaining parameters are the same as those of the IEEE 33-node test system.

[0197] The resilience indicators of each scenario are shown in Table 5.

[0198] In terms of the comprehensive resilience index indicator:

[0199] 1) The two-stage resilience improvement strategy S5 of the present embodiment improves by 60.04% compared with not taking any measures S0, and improves by 2.27% compared with only taking disaster-time measures without pre-disaster allocation S4;

[0200] 2) Among the two schemes of only using the emergency generator vehicle strategy, S3 with pre-disaster allocation improves by 2.17% compared with S1 without pre-disaster allocation.

[0201] It can be seen that the pre-disaster allocation strategy proposed in the present embodiment can effectively improve the resilience of the distribution network while ensuring economy in a large-scale node system.

[0202] In terms of the pre-disaster index load loss rate, the two-stage improvement strategy S5 can significantly reduce the load loss rate of the distribution network, which is reduced by 84.44% compared with S0 without taking any measures, and is reduced by 6.44% compared with S4 without pre-allocation; in terms of the disaster-time index maximum load retention rate, the two-stage improvement strategy S5 can significantly improve the maximum load retention rate, which is improved by 115.86% compared with S0 without taking any measures, and is improved by 5.24% compared with S4 without pre-allocation; in terms of the post-disaster index relative load recovery rate, the scenario S5 can improve the relative load recovery rate of the distribution network, which is improved by 38.76% compared with S0. The results show that the strategy proposed in the present embodiment can effectively improve the resilience of the distribution network in the resistance stage, the load reduction operation stage and the recovery stage.

[0203] Table 5 Resilience indicator results of different test scenarios of the IEEE 141-node system

[0204]

[0205] To verify whether the model built in the embodiment meets the dispatching requirements of the power distribution network, the two-stage optimization problem before and during the disaster is solved based on the IEEE 33 and 141 node systems, and the time and convergence accuracy are recorded. It can be seen that the comprehensive resilience index of the IEEE 33 and 141 node systems is improved by 54.84% and 60.03% respectively, and the average decision time is only 1.6ms and 2.1ms respectively, which meets the real-time decision-making requirements.

[0206] Embodiment three

[0207] Please refer to Figure 2 A typhoon-based power distribution network resilience improvement terminal 1 based on improved DQN, comprising a memory 2, a processor 3 and a computer program stored on the memory 2 and executable on the processor 3, wherein the processor 3 implements each step of the typhoon-based power distribution network resilience improvement method based on improved DQN of embodiment one or two when executing the computer program.

[0208] In summary, the typhoon-based power distribution network resilience improvement method and terminal based on improved DQN provided by the present application is based on deep reinforcement learning (DRL) technology, and proposes a pre-disaster-disaster resilience improvement method for power distribution systems, which supports solving complex optimization decision problems in a model-free manner to realize real-time dynamic decision-making. First, the pre-disaster-disaster resilience improvement process can be divided into flexible pre-allocation of emergency resources before the disaster, and coordinated scheduling of tie lines and emergency power output during the disaster. Then, a Markov decision process is constructed, and a simulation environment for pre-disaster-disaster resilience improvement is constructed based on OpenDSS. The DQN algorithm is used to solve the optimal control strategy for resilience improvement, and four power system resilience evaluation indexes are defined to evaluate the application performance of DQN. Finally, simulation experiments are carried out based on the IEEE 33 node and 141 node power distribution systems. The simulation results show that the pre-disaster-disaster resilience improvement strategy proposed in the present application improves the comprehensive resilience index of the IEEE 33 and 141 node systems by 54.84% and 60.03% respectively, and the average decision time is only 1.6ms and 2.1ms respectively, which meets the real-time decision-making requirements.

[0209] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent transformation or direct or indirect application in related technical fields based on the content of the present application specification and drawings is also included in the patent protection scope of the present application.

Claims

1. A typhoon-based power distribution network resilience improvement method based on improved DQN, characterized by, The method comprises the steps of: According to the typhoon meteorological prediction data, the fault probability of the power distribution network line is evaluated, and the selection and capacity of the emergency power generation vehicle are determined before the disaster; According to the real-time state of the power distribution network line, the emergency power generation vehicle and the tie line are adjusted during the disaster; For the pre-disaster and disaster power distribution network resilience improvement method, the Markov decision model is established by defining the control action, system state and reward function of the agent interaction environment; The improved DQN algorithm is used to solve the Markov decision model, and the resilience improvement strategy of the power distribution network is obtained, and the resilience of the power distribution network is evaluated using quantitative indicators; The selection and capacity of the emergency power generation vehicle before the disaster comprises: The objective function of the selection and capacity of the emergency power generation vehicle before the disaster is established with the minimum total cost of the power distribution network as the target: ； ； In the formula, Represents the set of scenarios for the samples. Indicates the scenarios that occurred during sampling. The probability, T This indicates the total time the power distribution network was affected by the disaster. This indicates the strategy adopted during the disaster. t Scene at any time Total cost; C EG represents the configuration cost of the emergency generator car, represents the set of all nodes, d n represents the distance of a node to a resource station, represents the node n whether the state variable of the emergency generator car is preset, C D represents the empty car dispatch unit cost, represents the node n active capacity of the emergency generator car at the node C P represents the capacity dispatch unit cost; The processing strategy of the emergency power generation vehicle and the tie line during the disaster comprises: The objective function of the adjustment during the disaster is established with the maximum mathematical expectation of the weighted load recovery total amount during the recovery period as the target: ； ； ； ； In the formula, represents the mathematical expectation in the state s , , , all represent weight coefficients, P R represents the total amount of weight load recovery, C G represents the operation cost of the emergency generator car, C L represents the operation cost of the tie line; represents the weight coefficient of the node m load, represents the state variable of whether the node load is recovered; represents the active power of the emergency generator car; represents the unit cost of the node m emergency generator car power generation; represents the state variable of the line m-m' , represents the unit cost of the tie line m-m' operation, represents the node set directly connected to the node m ; Further comprising configuring the power distribution network system operation constraint condition, the line operation state constraint condition and the emergency power generation vehicle output constraint condition for the pre-disaster and disaster power distribution network resilience improvement method; The definition of the control action comprises: The agent is at time t The control action taken is expressed as: ； In the formula, indicates the time t for the switch m whether there is an emergency generator car state variable, indicates the time t for the switch m active power of the emergency generator car, indicates the time t for the switch m reactive power of the emergency generator car, M indicates the node set.

2. The typhoon-resilient power distribution network based on improved DQN according to claim 1, wherein, The definition of the system state comprises: According to the power distribution network operation constraint and the emergency power generation vehicle constraint, the system state is defined: ； In the formula, indicates the time t node m phase active power, indicates the time t node m phase reactive power, indicates the time t for the switch m whether there is an emergency generator car state variable.

3. The typhoon-resilient power distribution network based on improved DQN according to claim 1, characterized in that, The definition of the reward function comprises: ； wherein o hc,t representing the time instant t a state variable indicating whether the action at the time instant c 1、 c 2 both represent positive coefficients, and c 1 is much larger than c 2.

4. The typhoon-resilient power distribution network based on improved DQN according to claim 1, wherein, The resilience of the power distribution network is evaluated using quantitative indicators, which comprises: The resistance, load reduction and recovery capabilities of the power distribution network are evaluated by calculating the loss of load rate, maximum load holding rate and load recovery rate; The overall resilience of the power distribution network is evaluated by calculating the difference between the disaster state and the normal state.

5. A typhoon-based power distribution network resilience improvement terminal based on improved DQN, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize each step of the typhoon-based power distribution network resilience improvement method based on the improved DQN according to any one of claims 1 to 4.

Images