Method and device for evaluating reliability of power distribution network

Abstract

The invention discloses a power distribution network reliability evaluation method and device. The method comprises the following steps: S1, constructing a two-state fault outage model of each element in a power distribution network based on historical fault data; s2, simulating a state transition process of the system in a given time span based on a sequential Monte Carlo algorithm according to the two-state fault shutdown model, and generating a possible state of the system; s3, quantifying a reliability evaluation result in each system state; s4, calculating the reliability index of the system according to the reliability evaluation result of each system state; s5, judging whether the evaluation result meets a convergence standard or not; wherein the variance coefficient of the simulation time or the reliability index is adopted as a convergence standard, if the convergence standard is met, a reliability evaluation result is output, and otherwise, the step S2 is returned to continue simulation. By adopting the technical scheme of the invention, the evaluation accuracy and the calculation efficiency are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of distribution network evaluation, and in particular relates to a distribution network reliability evaluation method and device. Background Art

[0002] Distribution network reliability assessment is a key process to ensure that the power system can continuously supply electricity to users at acceptable quality standards and required quantities, providing the necessary reliability support for distribution network planning and operation. At present, the assessment methods are mainly divided into analytical methods and simulation methods. Among them, Monte Carlo simulation (MCS) is widely used in distribution network reliability assessment due to its flexibility in simulating complex distribution network operation strategies and random characteristics. However, with the access of flexible resources such as distributed power sources, the operating status of the distribution network has become more complex, and the efficiency of distribution network reliability assessment cannot be improved. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a distribution network reliability assessment method and device to improve the accuracy and calculation efficiency of the assessment.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A distribution network reliability assessment method, comprising:

[0006] Step S1, constructing a two-state fault outage model for each component in the distribution network based on historical fault data;

[0007] Step S2: According to the two-state fault outage model, simulate the state transition process of the system within a given time span to generate possible states of the system;

[0008] Step S3, quantifying the reliability evaluation results in each system state;

[0009] Step S4, calculating the reliability index of the system according to the reliability evaluation result of each system state;

[0010] Step S5, determine whether the evaluation result meets the convergence standard; wherein, the simulation time or the variance coefficient of the reliability index is used as the convergence standard, if it meets the standard, the reliability evaluation result is output, otherwise, return to step S2 to continue the simulation.

[0011] Preferably, in step S1, the historical fault data includes the failure rate and mean repair time of the component.

[0012] Preferably, in step S2, according to the two-state fault outage model, based on the sequential Monte Carlo algorithm, the state transition process of the system within a given time span is simulated to generate possible states of the system.

[0013] Preferably, in step S3, a strategy combining imitation learning and exploration learning is adopted by multi-agents to analyze the consequences of distribution network failures, and the reliability assessment results under each system state are quantified.

[0014] The present invention also provides a distribution network reliability assessment device, comprising:

[0015] A construction module for constructing a two-state fault outage model of each component in the distribution network based on historical fault data;

[0016] A generation module is used to simulate the state transition process of the system within a given time span according to a two-state fault outage model and generate possible states of the system;

[0017] An analysis module to quantify the reliability assessment results at each system state;

[0018] A calculation module, used for calculating the reliability index of the system according to the reliability evaluation result of each system state;

[0019] The judgment module is used to judge whether the evaluation result meets the convergence standard; wherein the simulation time or the variance coefficient of the reliability index is used as the convergence standard. If it meets the standard, the reliability evaluation result is output, otherwise, the simulation is returned to step S2 to continue.

[0020] Preferably, the historical failure data includes the failure rate and mean repair time of the components.

[0021] Preferably, the generation module generates possible states of the system by simulating the state transition process of the system within a given time span based on a two-state fault outage model and a sequential Monte Carlo algorithm.

[0022] Preferably, the analysis module uses a strategy combining imitation learning and exploration learning through multi-agents to analyze the consequences of distribution network failures and quantify the reliability assessment results under each system state.

[0023] The present invention first constructs an evaluation framework based on reinforcement learning, and realizes system state generation through sequential Monte Carlo simulation; for the fault consequence analysis part, a multi-agent model is established, and a training algorithm combining imitation learning and exploration learning is proposed, so that the agent can learn the optimal strategy through interaction with the expert experience model. The technical solution of the present invention improves the efficiency of topological analysis and fault feature description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0025] Figure 1 The present invention is a flow chart of a method for evaluating the reliability of a power distribution network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Embodiment 1:

[0029] like Figure 1 As shown, an embodiment of the present invention provides a distribution network reliability assessment method, comprising:

[0030] Step S1: construct a two-state fault outage model for each component (such as a line, a transformer, etc.) in the distribution network based on historical fault data; the historical fault data includes the failure rate and the mean repair time of the component;

[0031] Step S2: According to the two-state fault outage model, the state transition process of the system within a given time span is simulated based on the sequential Monte Carlo algorithm to generate possible states of the system;

[0032] Step S3, using a multi-agent reinforcement learning method to simulate the fault recovery process, and quantify the reliability evaluation results in each system state; wherein, the multi-agent adopts a strategy combining imitation learning and exploration learning to perform distribution network fault consequence analysis, and quantify the reliability evaluation results in each system state;

[0033] Step S4, calculating the reliability index of the system according to the reliability evaluation result of each system state;

[0034] Step S5, determine whether the evaluation result meets the convergence standard; wherein, the simulation time or the variance coefficient of the reliability index is used as the convergence standard, if it meets the standard, the reliability evaluation result is output, otherwise, return to step S2 to continue the simulation.

[0035] As an implementation method of the present invention, in step S2, the sequential Monte Carlo simulation algorithm can simulate the system timing operation and effectively realize the reliability evaluation of the distribution network. A two-state outage model is established for the lines, transformers and other equipment in the system, and its normal and fault states are respectively composed of the component failure rate γ and the repair rate μ, and the mean time between failures M TTF and mean maintenance time M TTR , the following relationship exists:

[0036] γ=1 / M TTF

[0037] μ=1 / M TTR

[0038] The "operation-fault-operation" time series of a component can be obtained by simulating the fault-free continuous working time and the repair time. Assuming that the duration of the component in each state follows an exponential distribution, the system state duration is obtained by the inverse transformation method:

[0039] t ttf =-lnα / γ

[0040] t ttr =-lnβ / μ

[0041] Among them, t ttf and t ttr are the trouble-free continuous working time and the repair time respectively; α and β are random numbers generated by 0 / 1 uniform distribution respectively.

[0042] As an implementation method of an embodiment of the present invention, in step S3, a distribution network optimal load shedding model of mixed integer linear programming is established based on optimization theory to provide a simulation environment for the fault consequence analysis of other reinforcement learning, and a multi-agent reinforcement learning algorithm is further adopted to solve the problem of huge action space. At the same time, through imitation learning and exploration learning, the model convergence speed is improved, which specifically includes:

[0043] Step S31: Constructing an optimal load shedding model for distribution network

[0044] In the analysis of the consequences of distribution network failures, the goal is to minimize the load reduction of the load node importance, and the objective function is:

[0045]

[0046] Among them, w i,trepresents the load level weight coefficient at time t; represents the load required by node i at time t; T represents the fault recovery time; N B Indicates the number of load nodes in the distribution network.

[0047] The constraints are as follows: Equations (1)-(2) represent node active power balance constraints and reactive power balance constraints; Equation (3) represents line voltage drop balance constraints; Equations (4)-(8) represent node voltage amplitude, line active power, line reactive power, substation active power, and substation reactive power upper and lower limit constraints. Equations (9)-(10) represent virtual power flow constraints; Constraints (11) and (12) ensure that the actual power grid is consistent with the virtual topology and that the subgraph satisfies connectivity; The constraint satisfies that the number of closed lines is equal to the number of network nodes minus the number of subgraphs, thereby ensuring the radial constraint of the distribution network.

[0048]

[0049]

[0050] Among them, P ij,t and represents the active power of the line and the substation at time t; Q ij,t and It represents the reactive power of the line and the reactive power injected by the substation at time t; Indicates power factor; I ij,t Represents the square of the line current at time t; parameter r ij and x ij Represents line impedance; y ij,t Indicates the line protection action state at time t 0 / 1 variable, 0 means open, 1 means closed; represents the square of the node voltage at time t; V i max and V i min Respectively represent the upper and lower limits of the node voltage; and Respectively represent the maximum active power and reactive power of the line; M is a large integer; F ij,t represents the virtual power of line ij at time t; γ i,t represents a 0 / 1 variable at time t; in the virtual network, the substation node and γ i,t The nodes with the value equal to 1 are regarded as source nodes, and the virtual power output power is set to be unlimited. The model takes the substation and the first and last nodes of the fault line as the potential root node set Ω Sr , other nodes set a virtual load with a load value of 1; u ij,t is a 0 / 1 variable representing the line state at time t; α j and β jThey represent the line head node set with node j as the end node and the line end node set with node j as the head node respectively; ij represents the line index; Ω L and Ω S Represent the line and substation collection respectively.

[0051] Step S32: Multi-agent failure consequence analysis

[0052] In order to achieve accurate and efficient analysis of the consequences of distribution network failures, a multi-agent-based fault consequence analysis model is adopted. In this model, the distribution system is divided into multiple sub-areas, each of which is managed by an agent. Each agent runs in a distributed environment, shares state information with each other, and implements the global optimal load recovery strategy through collaborative decision-making.

[0053] The multi-agent fault consequence analysis model is essentially an enhanced method based on the single-agent fault consequence analysis model, that is, the fault consequence analysis problem is expressed as a Markov decision process. First, the DDPG algorithm is used to establish a single agent, which interacts with the environment to learn and make decisions. DDPG is a deep reinforcement learning algorithm based on policy gradients, suitable for continuous action spaces, and adopts an actor-critic architecture. Based on deep neural networks as approximations of policy networks and action value functions, the stochastic gradient method is used to train the parameters in the policy network and value network models. Among them, the policy network is used to maximize the action value function to solve the optimal strategy based on the state s of the external environment at time t. t Make an action t ,The value network is used to evaluate the action value function to evaluate the ,quality of the action at the current moment.

[0054] For this reason, a single agent is responsible for the switch and load recovery in the area, and its policy and training process are as follows:

[0055] Step S321: State collection: the state vector s of each agent i i , including the current load status, switch status and status information of adjacent intelligent agents.

[0056] Step S322: Action selection: The agent selects an action based on the policy network (actor network) in are the parameters of the policy network and N is the exploration noise.

[0057] Step S323: Strategy evaluation: Evaluate the value of the action through the Q network (critic network) in are the parameters of the value network.

[0058] Step S324: Environment interaction: Agent performs selected action a i, observe the next state s′ i and timely rewards i .

[0059] Step S325: Experience playback: (s i ,a i ,r i ,s′ i ) is stored in the experience pool.

[0060] Step S326: Network update

[0061] 1) Value network update

[0062] The value network update goal is to minimize the error between the true Q value and the predicted Q value, and the target Q value y i The calculation is as follows:

[0063] y i =r i +γQ′(s′ i ,π′(s′ i |θ π′ ))

[0064] Where γ is the discount factor. Q′ and π′ represent the target value network and the target policy network, respectively.

[0065] The value network loss function is defined as:

[0066]

[0067] Use stochastic gradient descent to minimize the loss function L and update the parameters θ of the value network Q :

[0068]

[0069] Among them, η Q is the value network learning rate.

[0070] 2) Policy network update

[0071] The optimization goal of the policy network is the cumulative discounted reward:

[0072]

[0073] Optimizing the deterministic policy function is to maximize the objective function J(θ).

[0074] Silver et al. proved that in a deterministic environment, the gradient of J(θ) with respect to the parameter θ is equivalent to the expectation of the gradient of the Q-value function. According to the deterministic strategy a=π(s|θ), applying the chain rule, we get

[0075]

[0076] Randomly obtain N small batches of data from the experience replay pool, and use the gradient of the value network to guide the update of the policy network:

[0077]

[0078] in, is the gradient of the value network with respect to the action, is the gradient of the policy network with respect to the parameters.

[0079] Policy network parameters θ π Update via gradient ascent:

[0080]

[0081] Among them, η π is the policy network learning rate.

[0082] 3) Soft update of target value network and target policy network In order to stabilize the training process, a target network is introduced, and its parameters are obtained from the main network parameters through soft update:

[0083] θ Q′ ←τθ Q +(1-τ)θ Q′

[0084] θ π′ ←τθ π +(1-τ)θ π′

[0085] Where τ is the soft update coefficient, usually

[0086] For the multi-agent failure consequence analysis model, each agent achieves collaborative decision-making by sharing state information. Each agent not only considers its own state and action, but also the behavior of other agents and their impact on the global system. The specific collaboration mechanism includes:

[0087] 1. State sharing. n agents share local state information to form a global state representation S = {s1, s2, ..., s n}.

[0088] 2. Joint action. n agents together form a global action space A = {a1, a2, ..., a n}.

[0089] 3. Joint Strategy Optimization Agents collaborate to optimize joint strategies through joint reward functions To evaluate the global decision-making effect.

[0090] 4. Distributed decision-making: Each agent makes a local optimal decision based on shared information and joint strategies to ensure the optimality of overall system recovery.

[0091] This framework takes advantage of the DDPG algorithm and improves the recovery efficiency and stability of complex power systems after faults through continuous strategy optimization and collaboration between agents. Compared with traditional methods, the multi-agent model based on the DDPG algorithm can better cope with the complexity and dynamics of the distribution system and ensure fast and efficient load recovery under different fault scenarios.

[0092] Among them, the reward function r i It is composed of the normalized objective function and the limit-crossing penalty term, and the reward function target is the maximum value, so it takes a negative value:

[0093]

[0094] The penalty items are:

[0095]

[0096] Among them, N ζ Indicates the number of action variables; i represents the i-th action variable; and represents the maximum and minimum values ​​of the i-th action variable; M represents the penalty coefficient of the action variable, which is a large integer.

[0097] Step S33: Training based on imitation learning and exploration learning

[0098] In order to improve the training efficiency of the model, an imitation learning and exploration learning mechanism is proposed. In the early stage of model training, the imitation learning mechanism is adopted, based on expert experience data, that is, supervised learning is performed to fit the expert experience data based on the mixed integer linear programming model data. The intelligent agent uses imitation learning to learn from the expert experience data to obtain the initial strategy, reducing the continuous interactive exploration process between the model and the environment. In the actual application process, actual power grid operation data or expert experience data can be used to improve the model fitting ability.

[0099] In the later stage of model training, strategy optimization is achieved through exploration learning. The agent further optimizes the strategy through exploration learning, that is, the conventional reinforcement learning training process, to cope with more complex and random failure scenarios. The agent interacts autonomously with the external environment to explore scenarios that do not exist in historical data. At the same time, it can avoid erroneous samples in historical data, reduce the agent's dependence on historical data, and improve the model's autonomous decision-making ability.

[0100] The advantages of the proposed imitation learning and exploration learning strategies are that, in the initial stage, better strategies can be quickly acquired through imitation learning, thus reducing training time; in the subsequent stages, exploration learning is used to continuously optimize and improve the robustness and generalization ability of the strategies.

[0101] To this end, in the imitation learning stage, an expert experience dataset D is established e , including state-action pairs (s, a), where state s is the network parameters such as system line status and load; action a is the decision variable of the optimal load shedding model of the distribution network.

[0102] Use supervised learning to train the policy network π(s|θ π ), so that it can output the action a that is closest to the expert experience under a given state s. * , the loss function is:

[0103]

[0104] In the multi-agent fault consequence analysis model, in order to achieve effective analysis of the consequences of distribution network faults, the embodiment of the present invention proposes a training algorithm based on imitation learning and exploration learning. The algorithm guides the behavior of the agent by imitating the strategy of expert experience and optimization model in the early stage, and then further optimizes the strategy through exploration learning, thereby improving the training efficiency and evaluation accuracy. The input is: expert experience data D e , the maximum number of training steps N T , experience replay buffer size N M , batch size N B , soft update coefficient τ, discount factor γ, exploration noise σ, number of imitation learning steps N IL , target network update frequency N TN ; Output: trained multi-agent neural network; specifically includes:

[0105] 1. Initialize the strategy network π and the value network Q;

[0106] 2. Initialize the target network π′ and the value network Q′;

[0107] 3. Initialize the experience playback buffer;

[0108] 4. Initialize state s. Each episode (training step number: represents a complete interaction process from beginning to end) starts from 1 to N. T Finish;

[0109] 5. episode ≤ N IL This is the imitation learning stage: select action a = π(s | θ π), without adding exploration noise; perform action a, obtain a new state s′ and reward r; store the new state and reward (s, a, r, s′) in the experience replay buffer;

[0110] 6. Use expert experience data to optimize the strategy network: From the expert experience data D e Sample a batch of samples from; traverse the batch N B Each sample (s, a) is trained and the policy network π(s|θ) is optimized by minimizing the supervised learning loss function π );

[0111] 7. episode>N IL It is the exploration learning stage: select action a = π(s|θ π )+N(0,σ) adds exploration noise; execute action a, obtain new state s′ and reward r; store the new state and reward (s,a,r,s′) in the experience replay buffer; if the size of the experience replay buffer exceeds the set value N M , then randomly sample a batch of samples N from the experience replay buffer B ; For each sample (s, a, r, s′), calculate the target value, the value network loss function, the policy network cumulative discounted reward and update the value network parameters;

[0112] 8. The current episode is the target network update frequency N TN When the value is an integer multiple of , the condition for updating the target network is met. Then the target value network and target strategy network are updated based on the formula;

[0113] 9. Update state s→s';

[0114] 10. Repeat steps 5 to 9 until the maximum number of training steps N is reached T .

[0115] As an implementation method of the present invention, in step S4, the annual average failure rate λ of each load node in the distribution network is quantified according to the failure consequence analysis part. i and the annual average failure time U i :

[0116]

[0117] Among them, Ω B Node collection; c i,t It indicates the load i reduction at time t 0 / 1 variable, 1 indicates load shedding; T indicates the simulation time

[0118] The reliability of the distribution network is expressed by the system average interruption frequency index (SAIFI), system average interruption duration index (SAIDI), average service availability index (ASAI) and average energy not supplied (AENS). The details are as follows:

[0119]

[0120] Among them, N i Number of users at node i; represents the average load of node i.

[0121] The embodiment of the present invention adopts a distribution network reliability assessment framework based on reinforcement learning, covering component modeling, system state generation, fault consequence analysis, reliability index calculation and assessment result convergence judgment steps; the embodiment of the present invention adopts a distribution network fault consequence analysis method based on multi-agent to solve the limitations of traditional single-agent methods when facing complex distribution network systems, especially the problem of huge equipment action space; the embodiment of the present invention adopts a strategy of combining imitation learning with exploration learning, quickly obtains better strategies through initial imitation learning, reduces training time, and continuously optimizes through exploration learning in the later stage to improve the robustness and generalization ability of the model.

[0122] Embodiment 2:

[0123] The embodiment of the present invention further provides a distribution network reliability assessment device, comprising:

[0124] A construction module for constructing a two-state fault outage model of each component in the distribution network based on historical fault data;

[0125] A generation module is used to simulate the state transition process of the system within a given time span according to a two-state fault outage model and generate possible states of the system;

[0126] An analysis module to quantify the reliability assessment results at each system state;

[0127] A calculation module, used for calculating the reliability index of the system according to the reliability evaluation result of each system state;

[0128] The judgment module is used to judge whether the evaluation result meets the convergence standard; wherein the simulation time or the variance coefficient of the reliability index is used as the convergence standard. If it meets the standard, the reliability evaluation result is output, otherwise, the simulation is returned to step S2 to continue.

[0129] As an implementation of the embodiment of the present invention, the historical fault data includes the failure rate and the mean repair time of the component.

[0130] As an implementation of the embodiment of the present invention, the generation module generates possible states of the system by simulating the state transition process of the system within a given time span based on the two-state fault outage model and the sequential Monte Carlo algorithm.

[0131] As an implementation method of an embodiment of the present invention, the analysis module uses a strategy combining imitation learning and exploration learning through multi-agents to analyze the consequences of distribution network failures and quantify the reliability assessment results under each system state.

[0132] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A distribution network reliability assessment method, characterized in that: include: Step S1, constructing a two-state fault outage model for each component in the distribution network based on historical fault data; Step S2: According to the two-state fault outage model, simulate the state transition process of the system within a given time span to generate possible states of the system; Step S3, quantifying the reliability evaluation results in each system state; Step S4, calculating the reliability index of the system according to the reliability evaluation result of each system state; Step S5, determine whether the evaluation result meets the convergence standard; wherein, the simulation time or the variance coefficient of the reliability index is used as the convergence standard, if it meets the standard, the reliability evaluation result is output, otherwise, return to step S2 to continue the simulation.

2. The distribution network reliability assessment method according to claim 1, characterized in that: In step S1, the historical fault data includes the failure rate and mean repair time of the components.

3. The distribution network reliability assessment method according to claim 2, characterized in that: Step S2: According to the two-state fault outage model, the state transition process of the system within a given time span is simulated based on the sequential Monte Carlo algorithm to generate possible states of the system.

4. The distribution network reliability assessment method according to claim 3, characterized in that: In step S3, a strategy combining imitation learning and exploration learning is used by multi-agents to analyze the consequences of distribution network failures and quantify the reliability assessment results under each system state.

5. A distribution network reliability assessment device, characterized in that: include: A construction module for constructing a two-state fault outage model of each component in the distribution network based on historical fault data; A generation module is used to simulate the state transition process of the system within a given time span according to a two-state fault outage model and generate possible states of the system; An analysis module to quantify the reliability assessment results at each system state; A calculation module, used for calculating the reliability index of the system according to the reliability evaluation result of each system state; The judgment module is used to judge whether the evaluation result meets the convergence standard; wherein the simulation time or the variance coefficient of the reliability index is used as the convergence standard. If it meets the standard, the reliability evaluation result is output, otherwise, the simulation is returned to step S2 to continue.

6. The distribution network reliability assessment device according to claim 5, characterized in that: Historical failure data includes component failure rates and mean time to repair.

7. The distribution network reliability assessment device according to claim 6, characterized in that: The generation module simulates the state transition process of the system within a given time span based on the two-state fault outage model and the sequential Monte Carlo algorithm to generate the possible states of the system.

8. The distribution network reliability assessment device according to claim 7, characterized in that: The analysis module uses a strategy combining imitation learning and exploration learning through multi-agent to analyze the consequences of distribution network failures and quantify the reliability assessment results under each system state.

Images