Power distribution network dynamic reconstruction method and system based on dynamic space screening

Abstract

This disclosure provides a dynamic reconfiguration method and system for distribution networks based on dynamic space filtering. The method includes acquiring real-time load and DG output data of distribution network nodes as a dataset; initializing the distribution network environment; obtaining the initial DN load state based on the current moment; constructing a DDNR model with the objective function of minimizing the daily operating cost of DNs; expressing the DDNR model as a Markov data decision process; using the DQN solution mode for calculation; predicting the probability of switch action; selecting action switches to form the DNQ action space; and filtering the action space to update the feasible topology of the distribution network. This disclosure aims to reduce losses and equalize voltage, thereby saving daily operating costs of DNs.

Description

Technical Field

[0001] This disclosure relates to the field of distribution network reconfiguration technology, specifically to a dynamic reconfiguration method and system for distribution networks based on dynamic spatial filtering. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Dynamic reconfiguration of distribution networks involves the coordinated optimization of multiple time periods within the distribution network (DN), aiming to obtain the optimal decision across the entire time frame. CNN-based reconfiguration algorithms can effectively solve static reconfiguration problems, but because they do not establish connections between different time periods, they are not suitable for dynamic reconfiguration scenarios. Furthermore, traditional dynamic reconfiguration algorithms are mostly based on the physical model of the DN, and their optimization performance is highly dependent on the overall observability of the system. They have poor applicability to distribution systems with missing parameters and their decision-making process is time-consuming. Summary of the Invention

[0004] To address the aforementioned issues, this disclosure proposes a dynamic reconfiguration method and system for distribution networks based on dynamic space filtering. Combining reinforcement learning principles, the value-based Dueling DQN method further breaks free from the constraints of the distribution network model, increasing scalability. To solve the DQN convergence problem caused by large-scale action spaces, the action space filtering of a CNN model is utilized.

[0005] According to some embodiments, the present disclosure adopts the following technical solutions:

[0006] A dynamic reconfiguration method for distribution networks based on dynamic spatial filtering includes:

[0007] Acquire real-time load and DG output data of distribution network nodes as a dataset;

[0008] Initialize the distribution network environment, obtain the initial DN load status based on the current time, and construct the DDNR model with the objective function of minimizing the daily operating cost of DN;

[0009] The DDNR model is expressed as a Markov decision process, and the DQN solution mode is used for calculation to predict the probability of switch action. The action switches are selected to form the action space of DNQ, and the action space is screened to update the feasible topology of the distribution network.

[0010] According to other embodiments, the present disclosure adopts the following technical solutions:

[0011] The distribution network dynamic reconfiguration system based on dynamic spatial filtering includes:

[0012] The acquisition module is used to acquire real-time load and DG output data of distribution network nodes as a dataset;

[0013] The model building module is used to initialize the distribution network environment, obtain the initial DN load status based on the current time, and build the DDNR model with the objective function of minimizing the daily operating cost of DN.

[0014] The reconstructing module is used to express the DDNR model as a Markov decision process, calculate it using the DQN solution mode, predict the probability of switch action, select the action switches to form the action space of DNQ, and filter the action space to update the feasible topology of the distribution network.

[0015] According to other embodiments, the present disclosure adopts the following technical solutions:

[0016] An electronic device, characterized in that it includes a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps described in the method.

[0017] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0018] The action space optimization method based on CNN prediction proposed in this disclosure promotes the training and convergence performance of the DQN model to a certain extent; the Dueling DQN method based on double Q learning has advantages in both convergence performance and reconstruction optimization effect; the dynamic reconstruction decision given by DQN can reduce losses and equalize voltage, save the daily operating cost of DN, and has a better effect on the allocation of DG output; it can be extended to the larger-scale PG&E69 node distribution system, and the optimization performance of dynamic reconstruction is not significantly different from that of the IEEE 33 node system, so the method proposed in this disclosure has universality; it also has the advantage of decision speed of data-driven methods. Attached Figure Description

[0019] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0020] Figure 1 This refers to the DNN structure in a Dueling network.

[0021] Figure 2 For experience storage and playback processes;

[0022] Figure 3 This is based on the dual-Q learning principle;

[0023] Figure 4 The structure and operating principle of DQN;

[0024] Figure 5 Here is the algorithm flowchart;

[0025] Figure 6 This represents the probability distribution of switch actions.

[0026] Figure 7 For training effect;

[0027] Figure 8 (a) shows the node voltage distribution before and after DNR in the unoptimized original system;

[0028] Figure 8 (b) shows the node voltage distribution after DQN decision;

[0029] Figure 9 Voltage distribution under different DG penetration rates;

[0030] Figure 10 This represents the lowest voltage at each time period;

[0031] Figure 11 The power distribution system structure for PG&E69 node;

[0032] Figure 12 For the intraday node voltage of the unoptimized DN;

[0033] Figure 13 Intraday node voltage for DQN decision;

[0034] Figure 14 This refers to intraday node voltage exceeding limits;

[0035] Figure 15 For intraday network losses;

[0036] Figure 16 Comparison of DN operating costs. Detailed implementation method:

[0037] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0039] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0040] Example 1

[0041] One embodiment of this disclosure provides a dynamic reconfiguration method for distribution networks based on dynamic spatial filtering, including:

[0042] Acquire real-time load and DG output data of distribution network nodes as a dataset;

[0043] Initialize the distribution network environment, obtain the initial DN load status based on the current time, and construct the DDNR model with the objective function of minimizing the daily operating cost of DN;

[0044] The DDNR model is expressed as a Markov decision process, and the DQN solution mode is used for calculation to predict the probability of switch action. The action switches are selected to form the action space of DNQ, and the action space is screened to update the feasible topology of the distribution network.

[0045] Furthermore, as one embodiment, the method for constructing a Dueling DQN model based on double Q learning includes:

[0046] Decisions are made by establishing and maintaining a Q-table relating each action to a state. For the current state s... t The Q-value is used to measure the return and determine the action, that is:

[0047]

[0048] In the formula, s t a t Let A represent the state at time step t and the action taken by the agent; let A be the action space of the agent in this environment; and Q be the action space of the agent in this environment. * (s t a) is the optimal action value function under this environment.

[0049] The purpose of Q-learning is also to learn from the behavior of intelligent agents, enabling artificially constructed... The function gradually converges to the optimal function Q. * Combining the optimal Bellman equation, the iterative formula for Q-learning can be derived as follows:

[0050]

[0051] In the formula, p(·|s t ,a t ) is the state transition function; It is the action value function with respect to S t+1 Expectations at any moment; R t (r t ) is the action A taken by the intelligent agent. t (a t The reward received in response to the environment afterward; It is an expectation The Monte Carlo approximation; γ is the discount factor; Let Q be the time t. * (s t ,a t The estimate of α is the learning rate.

[0052] When Q→Q * At that time, the corresponding optimal control strategy is also determined, that is:

[0053]

[0054] In the formula, π * Q π* These are the optimal control strategy and the value function, respectively, meaning the optimal decision is based on the maximum value obtained.

[0055] For dynamic programming problems with high-dimensional state spaces and complex actions, manually building and maintaining Q-tables is impractical. Therefore, researchers combine deep neural networks (DNNs) with Q-learning. Leveraging the powerful data fitting capabilities of DNNs, they reduce the dimensionality of the high-dimensional state space and establish a connection between the state and the action domain. In this way, the Q-values ​​of each state-action pair can be derived from the DNN, greatly expanding the applicability of Q-learning. Correspondingly, the action value function is also represented by the DNN, i.e.:

[0056]

[0057] In the formula, θ is the parameter matrix of the DNN.

[0058] DQN parameter update target To some extent, it is based on the DQN's own estimates, the so-called "bootstrapping." In this case, if the estimated value... For the true value Q*(s) t ,a t If a bias is introduced (θ), this bias will spread among the samples as the parameters are updated. Furthermore, the distribution of action samples in the experience pool is not uniform, resulting in inconsistent estimation biases of A by the final DQN, which will directly lead to a significant deviation of the decision π from the expected π. * .

[0059] To overcome the bootstrap phenomenon, the expectation calculation and error backpropagation processes need to be separated to some extent. Therefore, DQN sets up two DNNs: the estimation network (Evalnet) participating in training and the target network (Targetnet) with its parameters frozen.

[0060] In double-Q learning, Evalnet is responsible for action selection and error backpropagation, while Targetnet is responsible for solving the expectation. After a certain number of training iterations, the evaluation network will update the weights θ.- The target network is then passed on to the target network. While eliminating the "bootstrapping" phenomenon, the static target network in dual-Q learning can provide a relatively stable estimate of the target value, which also facilitates the convergence of DQN.

[0061] The formula for calculating a single parameter update in a DNN is as follows:

[0062]

[0063] In the formula, The gradient of the DNN parameters.

[0064] By establishing a Dueling DQN model structure based on double Q learning, an improved method based on Dueling network, experience replay, double Q learning, and greedy strategy was adopted.

[0065] The DQN model consists of several parts: an agent, an environment, and an experience pool. The agent reacts to the state of the environment and gives corresponding actions. The environment receives the actions fed back by the agent and outputs action rewards and new states. The experience pool stores the interaction experience between the agent and the environment, which can be retrieved by the agent later.

[0066] The DQN model execution flow is as follows:

[0067] (1) The agent reacts to the state of the environment and gives corresponding actions;

[0068] (2) The environment receives the action feedback from the agent and outputs action reward and new state;

[0069] (3) The experience pool stores the interaction experience between the agent and the environment, so that the agent can use it again.

[0070] To improve network performance, this disclosure employs an improved method based on the basic DQN model, incorporating Dueling networks, experience replay, double-Q learning, and a greedy strategy, such as... Figure 1 The aforementioned competition network, based on the original DNN structure, adds branches to calculate the state value V. s and the advantage value V of each action in the state. s,a Finally, the combined output is Q for this state. s,a Value. The output of Dueling DQN is shown below:

[0071]

[0072] In the formula, V s (s;θ s V s,a (s,a;θ s,a Z represents the state value function and the action advantage function, respectively.s,a With meanZ s,a This represents the output of the CNN and its mean.

[0073] Equation (6) shows that the advantage function can be defined as the advantage of each action value in state s relative to the average value, V s,a The higher the value, the stronger its superiority. Combining equation (4), it can be seen that, compared to traditional DQN, the error backpropagation of the Dueling network will first be based on the state value V. s The total value of actions Q for s s,a Update, and then use V s,a By defining a separate update direction for each action, the Dueling network ensures that each error propagation is directed towards the overall A, rather than updating only the single action 'a' corresponding to arg max Q(s,a). This results in stronger convergence properties for the Dueling network. For ease of explanation, DQN as used below refers to Dueling DQN.

[0074] For Q-learning and DPG algorithms, the behavioral policy can differ from the target policy, a process known as "off-policy." Similar to the dataset shuffle operation in DNN training, experience replay primarily aims to break the temporal correlation of input sequences, enhancing the generalization ability of DQN. Furthermore, high-quality behavioral experiences stored in the experience pool can be extracted multiple times for DQN training. This allows for the acquisition of sufficient training data for DQN using fewer native samples.

[0075] Historical experiences generated during the interaction between the agent and the environment are stored in the experience pool in the form of quintuples, as follows:

[0076] (s,a,r,s - ,d) (7)

[0077] In the formula, d is a Boolean variable. The environment returns d = 1 only when the state sequence ends. At this time, the agent calculates the total reward obtained in this round; if state s is extracted in the experience replay... d＝1 Then its expected value is directly calculated as That is, in this state

[0078] In the DQN algorithm, the depth of the experience pool is a hyperparameter that is set manually, and its changes will also affect the training results. Before the number of experiences reaches the upper limit, historical experiences are stored in the experience pool in a data structure as shown in equation (7); when the capacity of the pool reaches the upper limit, new experiences are used to replace old experiences in turn; when using experience, data is extracted in a uniform sampling manner according to the preset batch size.

[0079] Value-based learning methods are model-free, with agents accumulating experience through trial and error. Therefore, in the early stages of learning, agents need to be encouraged to explore the unknown environment to gain more comprehensive behavioral experience; in the later stages, the trust in the agent's decisions needs to be gradually increased to obtain optimized decisions. Based on these requirements, an ε-greedy policy is introduced into DQN to dynamically adjust the ratio between the two, as shown in the following formula:

[0080]

[0081] In the formula, ε is the greed coefficient, ε ≥ 0, Δε represents its initial value, upper limit, and change, respectively; t ep denoted as the total number of iterations; p represents the confidence index.

[0082] According to equation (8), ε is the trigger threshold of the greedy policy, and p can be regarded as the position of the current action in the probability space. When p ≥ ε, a greedy policy is triggered, and the current action selection is a random sample from the entire action space; otherwise, the action selection follows the agent's decision. Furthermore, ε will gradually increase to... That is, the level of trust in the agent's decisions increases as the learning process progresses.

[0083] In summary, the constructed DQN model and its operation process are as follows: Figure 4 As shown, it mainly consists of five parts: environment, experience pool, model library, evaluation network, and target network. The latter two are based on the Dueling architecture.

[0084] As another implementation, DQN learning is divided into two parts: data generation and network training. The specific process is as follows:

[0085] (1) Data generation phase: In this phase, the evaluation network in DQN is not trained, but rather explored in the unknown environment.

[0086] ① At the start of a new round, the environment is initialized to an initial state s0;

[0087] ② Introduce this state into the agent and determine whether the greedy strategy is effective according to equation (8). If p≤ε, proceed to step ③; otherwise, proceed to step ④.

[0088] ③ The greedy strategy determines that action 'a' follows the evaluation network's decision. The state matrix is ​​input into the evaluation network to obtain a = argmaxQ(s, a; θ).

[0089] ④ Action selection is a random sampling of the entire action space;

[0090] ⑤ The agent feeds back action 'a' to the environment, calculates the reward 'r' for the action, and generates a new state 's'. - ;

[0091] ⑥ Determine s - Is it a final state? If yes, proceed to step 9; otherwise, proceed to step 7.

[0092] ⑦ Feedback d=0, experience pool with {s,a,r,s - Data is received in the form of ,d}, t ep ←t ep +1, update ε according to equation (8);

[0093] ⑧If t ep ≥t set If the first step is not completed, proceed to step 1 (2); otherwise, proceed to step 2.

[0094] ⑨ Feedback d=1, experience pool stores data, t ep ←t ep +1, update ε according to equation (8);

[0095] ⑩ If t ep ≥t set , transition to stage (2) step ①, otherwise, the current round ends, the total number of rounds increases by one, and transition to step ①.

[0096] (2) Network training phase: data volume or t ep Once the training requirements are met, DQN trains Evalnet while generating data.

[0097] ① Accept the hyperparameter batch size, extract a batch of data from the experience pool based on the uniform sampling rule, and input it into the agent;

[0098] ② In the historical data tuple, s is input into Evalnet and obtained Targetnet receives s - Combine the data {a,r,d} and calculate based on equation (4).

[0099] ③ Calculate the mean square loss based on equation (5) and perform backpropagation on the Evalnet network;

[0100] ④ Determine whether the cumulative number of training iterations meets the weight transfer condition. If so, the weights of Evalnet will cover the original weights of Targetnet; otherwise, no weight transfer will be performed.

[0101] ⑤ Determine if the total number of rounds has reached the limit. If yes, proceed to step ⑦; otherwise, proceed to step ⑥.

[0102] ⑥ If d = 0, proceed to step ② of stage (1); otherwise, proceed to step ① of stage (1).

[0103] ⑦ Save the trained model to the model library, save the training iteration data, output the corresponding charts, and end the learning process.

[0104] Dynamic reconfiguration of distribution networks can be viewed as a sequential decision-making process with temporal continuity. By controlling the topology of the distribution network (DN) state at each time interval, an optimized decision chain is obtained, minimizing the operating cost of the entire state sequence. Based on this characteristic, the DDNR problem can be represented as a Markov decision process and solved using the DQN method.

[0105] (1) Objective function

[0106] To address the dynamic reconfiguration characteristics of distribution networks, the power supply quality of DN (Digital Network) is mapped to voltage over-limit costs. Combining daily DN network losses and switching operation costs during the reconfiguration process, a DDNR (Distributed Direct Voltage Reduction) model is constructed with the objective function of minimizing the daily operating cost of DN.

[0107] minf = C f +C v +C s (9)

[0108] In the formula, C f C is the daily operating network loss cost for DN; v Cost of DN node voltage exceeding limits; C s The switching operation cost is the cost of the dynamic reconstruction strategy. The calculation methods of each sub-item in the objective function are shown in equations (10), (11), and (12).

[0109] 1) Daily network operation loss cost

[0110]

[0111] In the formula, T = 24, which represents the number of time periods within a day; t represents a single time period; c f For electricity price; K b,t , Let t represent the on / off state of branch b and the current flowing through it, respectively.

[0112] 2) Costs of exceeding daily voltage limits

[0113]

[0114] In the formula, c v Cost per unit voltage exceeding limits; v i,t Let be the per-unit voltage value of node i at time t; The node voltage penalty threshold for DN operation is set to 1.05 pu and 0.95 pu in this disclosure.

[0115] 3) Daily reconfiguration switch operation cost

[0116]

[0117] In the formula, c s Cost of a single switching operation; K t Let be the on / off state matrix of the entire system branches at time t.

[0118] (2) Constraints

[0119] During dynamic reconfiguration, the DN must meet constraints on power flow, node voltage, branch capacity, and number of switching operations throughout the entire time period, as detailed below:

[0120] 1) Current constraints

[0121]

[0122] In the formula, and Let δ represent the load and the power demand and output of the DG at time t, respectively; ij,t Let be the voltage phase angle difference between nodes ij at time t.

[0123] 2) Node voltage constraints

[0124]

[0125] In the formula, v i,max v i,min These are the upper and lower limits of the node voltage.

[0126] 3) Branch capacity constraints

[0127]

[0128] In the formula, S b,t Let be the apparent power flowing through branch b at time t.

[0129] 4) Network topology constraints

[0130]

[0131] In the formula, Let be the topology of DN at time t.

[0132] 5) Switching frequency constraint

[0133]

[0134] In the formula, n s , This refers to the number of switching operations per day and its upper limit.

[0135] Dynamically Reconstructed DQN Solution

[0136] This disclosure requires that DDNR be expressed as a Markov decision process to adapt to the solution mode of DQN. To this end, starting from the principle of MDP, the constructed DQN is used to complete the transformation of the dynamic reconstruction model into MDP.

[0137] (1) MDP principle

[0138] The MDP problem generally consists of a state space, an action space, a state transition function, a reward function, a discount factor, and a time series. The required elements can be expressed as tuples as shown below:

[0139] M = (S, A, P, r, γ, T) p (18)

[0140] In the formula, S is the state space; P is the state transition probability; and T is the time series.

[0141] In each discrete time step t∈T, the agent makes an action decision a∈A based on the state s∈S and transmits it to the environment. Then, the environment provides a reward r=r(s,a) based on the above information and determines s according to the state transition function p=P(s-|s,a). - This continues until the time step reaches the end of the sequence. Thus, in the time series T, a sequence of the form τ = (s, a, s) is obtained. - ,a - ,...,s t＝T ,a t＝T The agent's movement trajectory. The discounted reward for the trajectory is calculated as follows:

[0142]

[0143] In the formula, τ is the trajectory of the intelligent agent; U(τ) is the discounted reward of the trajectory.

[0144] The trajectory τ is determined by the agent's decision π, and its purpose is to find π to maximize E. π (U(τ)). Therefore, two functions are established to characterize the value of a specific strategy, as follows;

[0145]

[0146] In the formula, Q π V is the state value function; π This is the action value function.

[0147] In equation (11), the function Q π V π The solution method is detailed in Example 1 and will not be repeated here.

[0148] (2) MDP Representation of Dynamic Reconstruction Model

[0149] Here, the function shown in equation (18) is mapped one-to-one with each parameter of the distribution network, and the specific definitions are as follows:

[0150] Environment: The environment provides an initial state s0, accepts the agent's action decision a, and provides feedback with a reward r and the state s. - In the DDNR problem, the environment is built based on the gym reinforcement learning framework. Based on the current DN state, the power flow program is invoked to calculate and feedback the network loss, number of switching actions, and node voltage obtained from the actions.

[0151] Agent: Responsible for responding to state inputs with actions. The agent is configured using DuelingDQN based on double Q-learning.

[0152] State Space: The state is the input of DQN. The state is set as the simplified input matrix A, removing information about DG from the matrix and incorporating the DG output into the active and reactive power demands of the node loads. The state space is then defined as the set of all-time power matrices of each node in the system.

[0153] Action space: Each action corresponds to a feasible topology of the system. The action space is defined as the set of system topologies optimized by CNN. The optimization effect is given in the dynamic reconstruction model.

[0154] State transition matrix: For the DDNR problem, the state sequence is given, meaning that for each state transition there exists p = P(s). - |s)=1.

[0155] Discount factor: γ = 0.9.

[0156] Time series: For intraday dynamic reconstruction, therefore T = [1, 2, ..., 24].

[0157] Reward function: The reward value can be set as the negative of the daily operating cost of DN, so that the optimization direction of DQN is consistent with the direction expected by equation (9). However, for dynamic reconstruction, its objective function can be regarded as a soft constraint for optimization, and the constraint condition is a hard constraint for optimization. Therefore, the DQN reconstruction decision must make the operation of DN satisfy the constraint conditions described in equations (13)-(17). For this purpose, a constraint index r1 is introduced. If DN satisfies the constraint, then r1 = 0, and DQN decision is not interfered with; otherwise, feedback r1 = -2000, d = 1, interrupting the current round of decision. Finally, the reward function is set as a comprehensive consideration of the DDNR objective function and the operating constraints, that is:

[0158] r ＜s,a＞ =r1-C f -C v -C s (twenty one)

[0159] In the formula, r ＜s,a＞ Let r1 be the reward function corresponding to the state-action pair <s,a>; r1 is the DN running constraint index.

[0160] Algorithm Flowchart

[0161] The methods and procedures presented in this chapter are as follows: Figure 5 As shown, the specific steps are given below. Since the training and optimization principles of DQN have been explained in detail in Example 1, they will not be repeated here.

[0162] Step 1: The program starts and loads the DQN model and distribution network dataset;

[0163] Step 2: Initialize the distribution network environment, set the current time t to zero, and give the initial DN load state s. t ;

[0164] Step 3: Greedy policy determination. Determine the agent's behavior according to equation (8). If the greedy policy is triggered, proceed to step 4; otherwise, proceed to step 5.

[0165] Step 4: Select agent action a based on random sampling t ∈A;

[0166] Step 5: Based on DN state s t Using DQN_Evalnet to predict agent actions a t ;

[0167] Step 6: DN environment accepts s t a t The flow program is invoked to calculate the action reward r as shown in equation (4.21). t Feedback r t With s t+1 ∈S;

[0168] Step 7: Motor experience with {s t ,a t ,r t ,s t+1 Stored in the format ,t} for use during training, where t = t + 1;

[0169] Step 8: If DQN training is not performed, proceed to step 9; otherwise, proceed to step 10.

[0170] Step 9: Determine the current time. If t < 24, then provide feedback s. t If t = 24, proceed to step 3. If t = 24, the day ends, proceed to step 2.

[0171] Step 10: Train DQN based on equations (4) and (5);

[0172] Step 11: If the learning is complete, proceed to Step 12; otherwise, proceed to Step 9.

[0173] Step 12: Save the DQN model and the program terminates.

[0174] Example 2

[0175] One embodiment of this disclosure provides a dynamic reconfiguration system for a distribution network based on dynamic spatial filtering, including:

[0176] The acquisition module is used to acquire real-time load and DG output data of distribution network nodes as a dataset;

[0177] The model building module is used to initialize the distribution network environment, obtain the initial DN load status based on the current time, and build the DDNR model with the objective function of minimizing the daily operating cost of DN.

[0178] The reconstructing module is used to express the DDNR model as a Markov decision process, calculate it using the DQN solution mode, predict the probability of switch action, select the action switches to form the action space of DNQ, and filter the action space to update the feasible topology of the distribution network.

[0179] To verify the effectiveness of the proposed CNN-based action space optimization and Dual Q-learning-based Dueling DQN dynamic reconstruction method, performance analysis was performed on the standard IEEE 33-bus distribution system, and the applicability of the method on the PG&E 69-bus system was tested. The reinforcement learning environment was built using Gym.env. The training hyperparameter settings of the DRL method for the two test systems are shown in Table 1.

[0180] Table 1. Partial parameter settings for the DQN method.

[0181]

[0182]

[0183] CNN-based action space optimization

[0184] The increase in the number of nodes in the power distribution system will cause the action space of the MDP to increase exponentially. For common power distribution systems, the initial action space and the action space scale of the topology are shown in Table 2.

[0185] Table 2. Operating Space Dimensions of Common Power Distribution Systems

[0186]

[0187] As shown in Table 2, even with a relatively simple 16-node system, considering only the switching state, the action space is about 65,000. Even considering the radial constraints of the DN topology, the action space of the IEEE 33-node system, which is often used as a basic test case for DNR, still reaches about 51,000.

[0188] In the DQN solution to the dynamic DNR problem, the number of neurons in the output layer of the two DNN networks is equivalent to the scale of the action space A. However, training the network under the A shown in Table 1 is unacceptably difficult in terms of fitting and time cost.

[0189] The constructed loop-based CNN model achieves static reconstruction of DN. Based on this, A is simplified using the CNN model, as follows:

[0190] (1) Call the trained CNN model and extract its probability output;

[0191] (2) Perform a topology prediction on the IEEE 33-node dataset;

[0192] (3) Collect the action probabilities of each switch predicted by CNN, and select the switches with the dominant action probability in each loop to form the action space of DQN.

[0193] The summary results of CNN predictions are as follows Figure 6 As shown in the figure. To ensure readability, system node numbers and DG access information are not labeled in the figure, and switches with an operation rate of less than 0.5% are omitted from the statistical data.

[0194] comprehensive Figure 6 Based on the probability distribution and the original tie switch branches of the system, the final selection of the action branches for each loop in the IEEE 33-node system is shown in Table 3. After arranging and combining the loop switches, 1440 actions were obtained. Exhaustive search was used to eliminate topologically infeasible solutions, and the final number of actions obtained was 918.

[0195] Table 3 Loop Action Switches for IEEE 33-Node System

[0196]

[0197]

[0198] Algorithm performance testing

[0199] To verify the positive effect of the proposed method and action space optimization strategy on solving the DDNR problem, tests were conducted on the IEEE 33-node system. Six test configurations were set up for different DQN methods, action space optimization strategies and objective functions, as shown in Table 4.

[0200] Table 4 IEEE 33-node system dynamic reconfiguration test configuration

[0201]

[0202] The above six test configurations can be divided into three control groups: configurations 1, 2, and 4 verify the learning ability of different algorithms under the same action space; configurations 3 and 4 verify the effectiveness of action space optimization strategies; and configurations 4, 5, and 6 verify the impact of optimization under different objective functions (distribution network optimization environment). The trend of the daily operating cost of reconstruction under each configuration with the number of training rounds is as follows: Figure 7 As shown.

[0203] Depend on Figure 7 It can be seen that configurations 2, 3, 4, and 5 can make dynamic reconfiguration decisions by learning from reconfiguration experience, thereby reducing the daily operating cost of the distribution network, while the curve corresponding to configuration 1 oscillates at a level higher than the historical average operating cost.

[0204] Comparing curves 1, 2, and 4, we can see that under the same action space, the daily operating cost of the network is ranked as configuration 1 > 2 > 4. In this example, the reward function is the negative of the operating cost, r. ＜s,a；4＞ >r ＜s,a；2＞ >r ＜s,a；1＞ This corresponds to the ability of each algorithm to eliminate "bootstrapping" as mentioned earlier, i.e., public method > Dueling DQN > DQN. The stronger the "bootstrapping" rejection capability of an algorithm, the more conducive it is for the agent to escape local optima and make high-quality decisions.

[0205] Comparing the convergence trends of curves 3 and 4 reveals the necessity of action space optimization: under the same algorithm, configuration 4, whose action space has undergone dual screening through CNN optimization and topological feasibility, converges to a lower running cost about 800 rounds earlier than configuration 3.

[0206] Furthermore, the training data for each configuration is summarized in Table 5, and the impact of decisions on the DN operating status is analyzed in detail. In the table, the node voltage over-limit rate is calculated as the ratio of over-limit voltages to the total voltage data collected within 24 hours.

[0207] Table 5 Summary of Algorithm Optimization Results

[0208]

[0209] Configuration 1 failed to optimize operating costs and is therefore considered a decision failure, excluded from the case study analysis. Furthermore, corresponding to the reward function settings in Example 1, configurations 2, 3, and 4 weigh network losses, node voltages, and switching frequency, providing their respective decisions. Based on the data in the table:

[0210] In terms of total network loss, all three have seen varying degrees of reduction, with the highest reduction being 26.32% for configuration 4.

[0211] Regarding the voltage at each node, the typical daily minimum node voltage per unit value in historical data is 0.9133, while approximately one-quarter of the nodes experience voltage exceedances. Comparing configurations 3 and 4, the former yields a superior minimum node voltage. min =0.9374pu; the latter discarded some optimizations for the minimum voltage value and instead chose to further suppress the voltage over-limit rate of all nodes to 4.29%.

[0212] In the reward function, the number of switching operations is a pure penalty. In this example, the electricity price is set at $0.13 / kW·h, and the cost of a switching operation is $4.6 per operation. Assuming that the DN performs one switching operation per hour, the loss reduction threshold for this period is approximately 4.6 / 0.13 ≈ 35.39 kW. The loss reduction effect of the switching operation needs to be higher than this value for the DQN to obtain positive feedback. However, in actual calculations, the impact of node voltage fluctuations on costs is considered, and this value will fluctuate to some extent. Therefore, configurations 3 and 4 both select 4 switching operations to avoid reducing the overall economic efficiency of the decision due to excessive switching penalties.

[0213] As seen in configuration 5, in an optimized environment where voltage over-limit costs are disregarded, the reduction in DN operating costs after optimization is minimal; under this decision, the proportion of each DN operating cost is approximately C. f :C v :C s = 5.22:4.41:0.37, and the ratio of the unit costs of the three is c. f :c v :c s = 0.13:130:4.6; For DN, C f With C v The correlation is positive, therefore C is taken into account. v This allows the agent to receive relatively positive feedback in the initial stages of learning; conversely, c f With c s The gap between these factors can negatively impact the agent and affect the final optimization result; therefore, in a dynamically reconfigured DQN environment, C must be taken into account. v It is necessary.

[0214] For discrete static reconfiguration scenarios, although configuration 6 achieves the lowest operating network loss and voltage over-limit rate, its unrestricted switching operations cause a sharp increase in total cost. Data analysis shows that switching costs account for approximately 45% of the daily operating cost in this scenario. Excessively frequent switching changes are unacceptable in terms of both economy and system stability.

[0215] Based on the data above, it can be seen that configurations 2, 3, 4, 5, and 6 all satisfy the reward function r. ＜s,a＞Decisions regarding the optimization direction. Configuration 2 is constrained by inherent algorithmic limitations, resulting in a lower superiority of its reconstructed solution compared to the latter two. As for configurations 3 and 4, both employ the same algorithm, yielding similar network losses and switching counts; their divergence lies in the choice of voltage optimization direction. Configuration 5, lacking C... v This leads to passive decision-making by the agent, resulting in poor performance. As shown in configuration 6, dynamic reconfiguration requires restrictions on switching operations. Cost statistics indicate that the reconfiguration strategy in configuration 4 is more economical in this example.

[0216] The impact of DG access on DDNR

[0217] This study tests the impact of DG grid connection at different penetration rates on the daily operation and dynamic reconfiguration decisions of DN. Since algorithm comparisons are not involved, DQN in the following text refers to the algorithm. Based on historical data and decisions made under Configuration 4, node voltage variations of DN were monitored over 24 hours, and the relevant data were plotted. Figure 8 .

[0218] Depend on Figure 8 It can be seen that the voltages of system nodes 18 and 33 were relatively low in historical data. After dynamic reconfiguration by DQN, the system voltage improved year-on-year, but the voltages of the aforementioned nodes remained inferior compared to the previous period. To strengthen the system voltage support role of DG, this example selects wind power access nodes 18, 22, 25, and 33, and conducts data research with different DG penetration rates. The penetration rates p... dg The full-time node voltage trend and the minimum voltage of each period are listed below. Figure 9 and Figure 10 middle.

[0219] The summaries of voltage and operating parameters in Tables 6 and 7 provide data support for the above conclusions. Considering the supporting role of DG grid connection on voltage, a 97% voltage over-limit rate indicator is added here to more intuitively measure the optimization effect.

[0220] Table 6 Node Voltage Indicators

[0221]

[0222] Table 7 Summary of Operating Parameters

[0223]

[0224] Based on the data in the table, the following conclusions can be drawn:

[0225] First, under the same conditions, the voltage parameters and operating costs of the DQN reconstructed solution are superior to those of the original network structure.

[0226] Secondly, a common trend can be derived between the original network topology and the DQN reconstructed topology: the minimum node voltage and p dg There is a positive correlation between network loss and daily operating cost and p.dg It shows a negative correlation.

[0227] Finally, regarding voltage over-limit rate: the original network structure achieved a 95% over-limit rate compared to p dg The changes are clearly correlated, but the 97% over-limit rate is not; after DQN reconstruction, the 97% over-limit rate of the voltage varies with p. dg The increase shows a clear downward trend. This indicates that the original network structure limited part of the voltage support capability of the DG, while the reconfiguration solved the power distribution problem of the DG.

[0228] Furthermore, it should be noted that Table 7 shows the decrease in network loss and the decrease in total operating cost resulting from DQN reconstruction, which vary with p. dg The trends of change are inconsistent. This is because, as p... dg As the voltage increases, the optimization space for the 95% voltage exceedance rate of the distribution network decreases. The lower limit of the voltage penalty threshold set by the reward function is 0.95pu. Therefore, the optimization of the 97% voltage exceedance rate of DN nodes by DQN is not included in the cost and reward function calculation, which leads to the above phenomenon.

[0229] PG&E 69-node system case study analysis

[0230] To study the applicability of the algorithm to large-scale power distribution systems, the PG&E 69-bus system was selected as a test case, and its structure is as follows: Figure 11 As shown.

[0231] To leverage the advantages of distributed generation (DG) grid connection, the intraday voltage variations of the DN node were first monitored based on an unoptimized control strategy. This monitoring served as the basis for selecting the DG grid connection node. The relevant results are plotted in [the graph]. Figure 12 middle.

[0232] according to Figure 12 Comparing the node voltages of the DN system throughout the entire time period, two voltage-weak areas can be identified: the areas corresponding to nodes 16-27 and 47-54. Therefore, in selecting the DG grid connection point, representative nodes 27 and 54 are first selected from these voltage-weak areas; additionally, considering... Figure 11 The DN structure shown selects common nodes 16 and 48 in loops II and V and loops IV and V, respectively, to ensure flexible allocation of DG output during reconfiguration. The final selected wind power grid connection nodes are 16, 27, 48, and 54, as marked on [the diagram / document / reference]. Figure 11 middle.

[0233] The selection of operating switches for each loop of DN is shown in Table 8. After topology feasibility screening, the operating space scale of this system is 833.

[0234] Table 8 Loop Action Switches for PG&E69 Node System

[0235]

[0236] The dynamic reconfiguration effects of the PG&E69 node system are listed below: Table 9 summarizes the DN operating parameters under each scenario. Figure 13 The intraday node voltage curve of DN based on DQN dynamic reconfiguration decision operation; Figure 14 This refers to the full-time and space-time voltage over-limit conditions under various scenarios; Figure 15 Comparison of network loss for DN at different time periods.

[0237] Table 9 Summary of operating parameters of PG&E69 node system

[0238]

[0239] Combination Figure 12 and 13 It can be seen that the intraday node voltage curve of DN based on DQN reconfiguration decision is flatter; Figure 14 This more intuitively demonstrates the optimization effect of DDNR on node voltage: As can be seen from the figure, after dynamic reconfiguration and DG grid connection, the 95% (dark blue area) and 97% over-limit area (light blue area) of DN voltage show a significant reduction trend. The voltage data given in Table 9 corresponds to the above viewpoint.

[0240] Figure 15 The intraday network loss trend and the DN operating parameters shown in Table 9 indicate that the optimization effect of the DQN method on the 69-node system is not significantly different from that on the IEEE 33-node system. The analysis of related phenomena and data trends has been given in the algorithm performance test and the impact of DG access on DDNR.

[0241] Efficiency Analysis

[0242] Based on the impact of DG access on DDNR and the DQN model trained using the PG&E 69-node system case study, continuous dynamic reconfiguration was performed on the IEEE 33-node and PG&E 69-node systems for 60 days. The DN operating cost was calculated, and the advantages of the method presented in this chapter on DDNR decision-making efficiency were analyzed. The daily operating cost trend of the system's dynamic reconfiguration is as follows: Figure 16 As shown in the figure; Table 10 shows a comparison of the decision-making speed of different methods, among which the mixed integer method is based on the Yamip and Cplex toolbox programming in the MATLAB environment.

[0243] Table 10 Comparison of DDNR decision speeds using different methods

[0244]

[0245]

[0246] Depend on Figure 16It can be seen that the proposed DQN dynamic reconfiguration method significantly reduces the operating cost of the distribution network during a 60-day continuous reconfiguration. The algorithm decision time in Table 10 shows that the decision speed of both DQN methods is much better than that of the mixed integer method, and the method has an advantage in decision time compared to Nature DQN; and since the 69-node system is larger in scale, its decision time should also be higher than that of the IEEE 33-node system.

[0247] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0248] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0249] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A dynamic reconfiguration method for distribution networks based on dynamic spatial filtering, characterized in that, include: Acquire real-time load and DG output data of distribution network nodes as a dataset; Initialize the distribution network environment, obtain the initial DN load status based on the current time, and construct the DDNR model with the objective function of minimizing the daily operating cost of DN; The DN power supply quality is mapped to voltage over-limit cost. Combining the daily DN network loss and the switching operation cost during the reconfiguration process, a DDNR model is constructed with minimizing the daily DN operating cost as the objective function. In the formula, This refers to the daily network loss cost of DN (Network Grid). Cost of DN node voltage exceeding limit; The switching operation cost of the dynamic reconfiguration strategy; The DDNR model is expressed as a Markov decision process, and the DQN solution mode is used for calculation. The CNN model is used to predict the probability of switch action, and the action switches are selected to form the action space of DQN. The action space is then filtered to update the feasible topology of the distribution network. The DQN model is improved by adding branches to calculate the state value and the advantage value of each action in the state separately, and then merging the outputs, specifically: In the formula, , These are the state value function and the action advantage function, respectively. and The output of the CNN and its mean; The process of optimizing the action space using a CNN model is as follows: Call the trained CNN model and extract its probability output; Perform a topology prediction on the node dataset; Collect the action probabilities of each switch predicted by the CNN, and then select the switches with the dominant action probability in each loop to form the action space of DQN.

2. The distribution network dynamic reconfiguration method based on dynamic spatial filtering as described in claim 1, characterized in that, The DQN model comprises an agent, an environment, and an experience pool. The agent reacts to the state in the environment and gives corresponding actions. The environment receives the actions fed back by the agent and outputs action rewards and new states. The experience pool stores the interaction experience between the agent and the environment, which can be retrieved by the agent later.

3. The distribution network dynamic reconfiguration method based on dynamic spatial filtering as described in claim 2, characterized in that, Before the amount of experience reaches its limit, historical experience generated during the interaction between the agent and the environment is stored in the experience pool in the form of 5-tuples. When the capacity of the pool reaches its limit, new experience is used to replace old experience in turn. When using experience, data is extracted in a uniform sampling manner according to the preset batch size.

4. The distribution network dynamic reconfiguration method based on dynamic spatial filtering as described in claim 1, characterized in that, In DQN, a greedy policy is introduced, and a greedy policy trigger threshold is set. If the position of the current action in the probability space is greater than or equal to the threshold, the current action is selected as a random sample from the entire action space. Otherwise, the action selection follows the agent's decision.

5. The distribution network dynamic reconfiguration method based on dynamic spatial filtering as described in claim 1, characterized in that, Define constraints for the entire time period of the DN, including power flow constraints, node voltage constraints, branch capacity constraints, and switching operation count constraints.

6. The distribution network dynamic reconfiguration method based on dynamic spatial filtering as described in claim 5, characterized in that, The power flow constraint is: In the formula, , and , They are respectively t Real-time load and power demand and output of DG; for t Time Node ij The voltage phase angle difference between them.

7. A distribution network dynamic reconfiguration system based on dynamic spatial filtering, characterized in that, include: The acquisition module is used to acquire real-time load and DG output data of distribution network nodes as a dataset; The model building module is used to initialize the distribution network environment, obtain the initial DN load status based on the current time, and build the DDNR model with the objective function of minimizing the daily operating cost of DN. The DN power supply quality is mapped to voltage over-limit cost. Combining the daily DN network loss and the switching operation cost during the reconfiguration process, a DDNR model is constructed with minimizing the daily DN operating cost as the objective function. In the formula, This refers to the daily network loss cost of DN (Network Grid). Cost of DN node voltage exceeding limit; The switching operation cost of the dynamic reconfiguration strategy; The reconstruction module is used to express the DDNR model as a Markov decision process, use the DQN solution mode for calculation, use the CNN model to predict the probability of switch action, select the action switches to form the action space of DQN, and filter the action space to update the feasible topology of the distribution network. The DQN model is improved by adding branches to calculate the state value and the advantage value of each action in the state separately, and then merging the outputs, specifically: In the formula, , These are the state value function and the action advantage function, respectively. and The output of the CNN and its mean; The process of optimizing the action space using a CNN model is as follows: Call the trained CNN model and extract its probability output; Perform a topology prediction on the node dataset; Collect the action probabilities of each switch predicted by the CNN, and then select the switches with the dominant action probability in each loop to form the action space of DQN.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

Images