Branch double deep q network-based real-time dynamic reconfiguration method and system for distribution network

Abstract

The disclosure provides a distribution network real-time dynamic reconstruction method and system based on branch double deep Q network, the reconstruction method comprising the following processes: acquiring real-time node load and distributed power output of a distribution network; transmitting the acquired data to a dynamic DNR model constructed based on Markov decision process (MDP); the dynamic DNR model taking minimization of network loss cost and switch action cost as an objective function; obtaining a branch double deep Q network based on loop decomposition of the distribution network, solving the dynamic DNR model by using a Q learning algorithm, obtaining a switch action set that maximizes the output reward of the branch double deep Q network, and updating the topology structure of the distribution network according to the switch action set. The disclosure can greatly improve the operation performance of the distribution network by mining the time sequence dynamic change law between the dynamic DNR decision variable and the decision result, without power flow modeling and segmented decision making in online application, and without relying on load and distributed power output prediction in advance.

Description

Technical Field

[0001] This disclosure relates to the technical field of distribution network reconfiguration, specifically to a method and system for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q networks. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Distribution Network Reconfiguration (DNR) is an important function in distribution management systems, typically aiming to minimize network losses and improve power quality and reliability. DNR can be divided into two categories: static reconfiguration and dynamic reconfiguration. Dynamic reconfiguration ensures the safe, high-quality, and economical operation of the distribution network and, compared to static reconfiguration, better meets the actual operational and scheduling needs of the distribution network (DN).

[0004] The dynamic reconfiguration algorithm based on the Long-Short Term Memory (LSTM) network model and switching action function can effectively solve the dynamic distribution network reconfiguration (DNR) problem. However, this method requires two steps to solve the dynamic reconfiguration problem, and power flow calculation is required when executing the algorithm. Real-time power flow modeling of some low-sensitivity distribution networks is often quite difficult. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a method and system for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q networks. This method mines the temporal dynamic change patterns between dynamic DNR decision variables and decision results. When applied online, it eliminates the need for power flow modeling and segmented decision-making, and does not rely on day-ahead load and distributed generation output forecasts, thereby significantly improving the operational performance of the distribution network.

[0006] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0007] One or more embodiments provide a method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q networks, including the following processes:

[0008] Obtain real-time node loads and distributed generation outputs in the power distribution network;

[0009] The acquired data is transmitted to a dynamic DNR model constructed based on Markov Decision Process (MDP); the dynamic DNR model takes minimizing network loss cost and switching action cost as its objective function.

[0010] A branched dual-depth Q-network is obtained based on the distribution network loop decomposition, and a method is adopted. Q The learning algorithm solves the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branched dual-depth Q network, and updates the topology of the distribution network based on the set of switching actions.

[0011] One or more embodiments provide a real-time dynamic reconfiguration system for distribution networks based on branch dual-depth Q networks, including:

[0012] Acquisition module: configured to acquire real-time node loads and distributed generation outputs in the distribution network;

[0013] Markov Decision Building Module: Configured to transmit acquired data to a dynamic DNR model built on Markov Decision Process (MDP); the dynamic DNR model has the objective function of minimizing network loss cost and switching action cost;

[0014] Distribution network dynamic reconfiguration module: configured to obtain branched dual-depth Q-networks based on distribution network loop decomposition, employing... Q The learning algorithm solves the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branched dual-depth Q network, and updates the topology of the distribution network based on the set of switching actions.

[0015] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps described in the above method.

[0016] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0017] This disclosure improves the Q-learning algorithm network structure based on the loop decomposition of the distribution network, resulting in a deep reinforcement learning algorithm, Branch Double Deep Q Network (BDDQN), to solve the dynamic DNR model. The BDDQN algorithm can iteratively seek the optimal decision for the distribution network Markov dynamic reconfiguration model, and its execution process does not require power flow calculations. The dynamic reconfiguration solution provided by BDDQN generates lower operating costs, significantly improving system performance.

[0018] Advantages of this disclosure in additional aspects will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0019] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0020] Figure 1 This is a flowchart of the reconstruction method of Embodiment 1 of this disclosure;

[0021] Figure 2 This is a diagram of the Markov dynamic reconfiguration decision framework for the distribution network in Embodiment 1 of this disclosure;

[0022] Figure 3 It is the dual-depth Q-network prior to the improvement of Embodiment 1 of this disclosure;

[0023] Figure 4 This is the branched dual-depth Q-network obtained based on the improvement of the distribution network loop in Embodiment 1 of this disclosure;

[0024] Figure 5(a) shows the node load data of the IEEE 33-node system simulation test set in the example analysis of Embodiment 1 of this disclosure;

[0025] Figure 5(b) shows the DG output sequence data of the IEEE 33-bus system simulation test set in the example analysis of Embodiment 1 of this disclosure;

[0026] Figure 6 This is a comparison of the simulation results of the IEEE 33-node system in the example analysis of Embodiment 1 of this disclosure;

[0027] Figure 7(a) is a comparison of the overall operating costs of the BDDQN algorithm and the static reconstruction algorithm after reconstruction in the IEEE 33-node system simulation of Embodiment 1 of this disclosure;

[0028] Figure 7(b) is a comparison of network losses generated by the BDDQN algorithm and the static reconstruction algorithm in the simulation of the IEEE 33-node system of Embodiment 1 of this disclosure;

[0029] Figure 7(c) is a comparison of the lowest node voltage after reconstruction using the BDDQN algorithm and the static reconstruction algorithm in the simulation of the IEEE 33-bus system of Embodiment 1 of this disclosure;

[0030] Figure 8(a) shows the node load data of the 185-node system simulation test set in the example analysis of Embodiment 1 of this disclosure;

[0031] Figure 8(b) shows the DG output sequence data of the 185-node system simulation test set in the example analysis of Embodiment 1 of this disclosure;

[0032] Figure 9(a) is a comparison of the overall operating costs of the BDDQN algorithm and the static reconstruction algorithm after reconstruction in the simulation of the 185-node system of Embodiment 1 of this disclosure;

[0033] Figure 9(b) is a comparison of network losses generated by the BDDQN algorithm and the static reconstruction algorithm in the simulation of the 185-node system of Embodiment 1 of this disclosure;

[0034] Figure 9(c) is a comparison of the number of switching actions after reconstruction by the BDDQN algorithm and the static reconstruction algorithm in the simulation of the 185-node system of Embodiment 1 of this disclosure. Detailed implementation method:

[0035] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used in these embodiments have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0037] It should be noted that the terminology used herein is for descriptive purposes only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0038] This disclosure first establishes a Markov dynamic reconfiguration decision process for distribution networks based on a mathematical model of distribution network dynamic reconfiguration and incorporating a Markov decision process to minimize network loss costs and switching operation costs. Then, considering the characteristics of distribution networks with "multi-ring network structure and radial operation," the traditional Double Deep Q Network (DDQN) algorithm is improved by proposing a deep reinforcement learning algorithm based on loop decomposition and a Branch Double Deep Q Network (BDDQN) to solve for dynamic DNR. Compared to traditional methods, the proposed method can mine the temporal dynamic change patterns between dynamic DNR decision variables and decision results using the BDDQN algorithm. In online applications, it eliminates the need for power flow modeling and segmented decision-making, and does not rely on day-ahead load and distributed generation (DG) output forecasting. Case studies demonstrate that the proposed method can effectively improve system performance and economy. Specific embodiments are described below.

[0039] Example 1

[0040] In one or more of the technical solutions disclosed in the embodiments, such as Figure 1 As shown, the real-time dynamic reconfiguration method for distribution networks based on branched dual-depth Q-networks includes the following process:

[0041] Step 1: Obtain real-time node loads and distributed generation outputs of the distribution network;

[0042] Step 2: Transmit the acquired data to a dynamic DNR model constructed based on Markov Decision Process (MDP); the dynamic DNR model takes minimizing network loss cost and switching action cost as its objective function.

[0043] Step 3: Improve the structure of the Q-learning algorithm network based on the distribution network loop decomposition to obtain the branch double-depth Q-network. Use the Q-learning algorithm to solve the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branch double-depth Q-network. Update the topology of the distribution network based on the set of switching actions.

[0044] In this embodiment, based on the loop decomposition of the distribution network, the network structure of the Q-learning algorithm is improved, resulting in a deep reinforcement learning algorithm, Branch Double Deep Q Network (BDDQN), to solve the dynamic DNR model. The BDDQN algorithm can iteratively seek the optimal decision for the distribution network Markov dynamic reconfiguration model, and its execution process does not require power flow calculations. The dynamic reconfiguration solution provided by BDDQN generates lower operating costs, significantly improving system performance.

[0045] The following is a detailed explanation.

[0046] (1) Construct a dynamic DNR model based on Markov Decision Process (MDP).

[0047] The principle of Markov Decision Process (MDP) is as follows:

[0048] MDP is a decision-making process for stochastic dynamic systems based on Markov process theory. It mainly consists of five elements:

[0049] (1)

[0050] In the formula, The state set represents the set of all states of the environment that the agent can perceive. The action set represents the set of all possible actions that the Agent can take. The reward set is the set of immediate feedbacks that the environment provides to the agent based on the state and actions. It is an indicator for evaluating the quality of actions. The state transition probability matrix is ​​determined by the environment. During the MDP process, the state transition probability satisfies the Markov property, which means that the current state and the action taken only affect the state at the next time step. This is the attenuation factor.

[0051] In MDP, state and optimal action The mapping relationship between them is called the strategy. If the action If it is certain, then For a given strategy If the action If it follows a probability distribution, then For a random policy The ultimate goal of MDP is to find an optimal strategy to maximize cumulative returns. :

[0052] (2)

[0053] In the formula, express t Instant rewards at any moment.

[0054] In mathematical expectation form, i.e., the state-value function:

[0055] (3)

[0056] Among them, strategy This determines the distribution of actions and their impact on state transition probabilities and immediate rewards. In the state... s Agent performs actions a The mathematical expectation of the subsequent reward is defined as the action value. ,Right now:

[0057] (4)

[0058] According to equations (3) and (4), the strategy The action potential function can be written as:

[0059] (5)

[0060] This function represents the state. s The system takes action. a The resulting reward. Note that this reward differs from an immediate reward; it is a long-term reward. Theoretically, it can be proven that an optimal strategy exists within an MDP. , It will produce a time series of actions This causes the system to change from state The goal of the Reinforcement Algorithm (RL) is to find the optimal policy by maximizing the cumulative reward obtained from starting the process. This involves continuous interaction between the agent and the environment. The Bellman optimality equation provides the theoretical basis for finding this optimal strategy:

[0061] (6)

[0062] (7)

[0063] In the formula, Indicates the system in state s At that time, perform the action a After that, the system state changed The probability of. For the system in state s At that time, perform the action a The immediate reward generated afterward.

[0064] A dynamic DNR model is constructed based on Markov Decision Process (MDP). The construction process is as follows:

[0065] 1.1 Construct the state set and action set of the MDP.

[0066] The node load and DG output are integrated into the node injected power, and the set of node injected power and the open switches are used as the state set in the MDP. ,Right now:

[0067] (8)

[0068] In the formula, and They represent in t At any given moment, the injected active and reactive power of all nodes in the distribution network.

[0069] Define the set of on / off actions as the action set. In this case, the action switch is coded according to its position in the basic loop.

[0070] 1.2 Construct the distribution network state transition probability function matrix based on the state set and action set;

[0071] Based on the above set of states and set of actions, a current state can be defined. and the next state The state transition probability function matrix between:

[0072] (9)

[0073] 1.3 In this embodiment, optionally, the reciprocal of the distribution network operating cost is set as the immediate return of the MDP, and a penalty term is added. The penalty term is used to penalize switching action strategies that do not comply with system security constraints, i.e.:

[0074] (10)

[0075] In the formula, express t The state at any given moment. express t The action switch at any given moment. It is a binary variable that, when the action switch satisfies system constraints, ,on the contrary, . M If the value is a large positive real number, the denominator term in the equation will be reduced when the action does not meet the system safety constraints. M Adjust to a larger number so that the return approaches 0. It is a positive integer.

[0076] Through the above transformation, the decision problem of dynamic reconfiguration can be converted into an MDP, and the optimal Bellman equation for dynamic reconfiguration of the distribution network is the constructed model, which can be expressed as:

[0077] (11)

[0078] (12)

[0079] in, Indicates the system in state At that time, perform the action After that, the system state changed The probability of; For the system in state At that time, perform the action The immediate reward generated afterward.

[0080] like Figure 2 As shown, this is a Markov dynamic reconfiguration decision framework for a distribution network. The training agent interacts with the distribution network to obtain action and state information and selects the optimal switching action to achieve the preset goal. After making a decision, the system provides feedback on the execution status of the current action. The agent learns from the quality of this feedback signal and ultimately discovers the optimal strategy.

[0081] (2) Based on the distribution network loop decomposition, adopt Q The learning algorithm solves the dynamic DNR model.

[0082] QThe learning algorithm is a typical value-function-based reinforcement learning algorithm, which aims to maximize the expected reward obtained after taking an action in a given state. Q Learning algorithms to maximize Q The value is used to learn the optimal policy, where Q The formula for updating the value is as follows:

[0083] (13)

[0084] In the formula, express t The state at any given moment, express t The actions performed at any given moment. Indicates the learning rate. Indicates the discount factor. Indicates in t Perform actions at all times The immediate reward obtained afterward Indicates the execution of an action Post-state The resulting Estimated value.

[0085] Tradition Q Learning algorithms are updated Use tables to store However, when solving dynamic reconstruction problems, the number of states and actions is often enormous, therefore... Q The learning efficiency of learning algorithms may be very low.

[0086] In the dynamic reconfiguration (DNR) problem of distribution network, the operating states of the switches are interconnected, and the switches in each loop coordinate their actions to achieve the optimal system operating state.

[0087] This embodiment employs a dual-depth approach. Q Network (Double deep) Q The algorithm (DDQN) uses two identical network structures. Q The network, a network for the present Q network One is used to select actions and update model parameters, and the other is the target. Q network Used to calculate the target Q Value. By decoupling the target Q Value action selection and target Q The two steps of calculating the value are used to eliminate the overestimation problem.

[0088] Dual Depth QNetworks Figure 3 As shown, the experience pool is used to store the state features, action vectors, and immediate rewards generated by the Agent's interaction with the environment during the algorithm iteration process.

[0089] This embodiment is based on distribution network loop decomposition and adopts... Q The learning algorithm solves the dynamic DNR model. First, it improves the structure of the Q-learning algorithm network based on the distribution network loop decomposition, and then solves the model based on the improved network. The specific steps include:

[0090] 2.1 Using the number of loops in the distribution network as the output dimension of the Q-learning algorithm network, and the dimension of the output vector of each dimension being the number of switches in the corresponding loop, the structure of the Q-learning algorithm network is improved to obtain a branched double-depth Q-network.

[0091] Optionally, in this embodiment, the one-dimensional output is... Q The network was improved to have multi-dimensional output. Q The network has an output dimension equal to the number of basic loops in the DNR problem. Q Each dimension of the network output is a vector, and the dimension of each vector is determined by the number of switches that can potentially operate within the loop. For example... Figure 4 As shown, this is the improved version based on the number of distribution network loops. Q The network structure is a branched double-depth Q-network.

[0092] Suppose a certain distribution network has L A loop, or L branches, has a maximum number of switching operations. d ,but Q Network output one L × d The state value matrix. It has the same dimension as the switching action matrix (FLM). Each value in the FLM represents the value generated by the corresponding switch action at that location. If an element in the FLM is 0, then the value will be... The element at the same position can be set to 0.

[0093] In this context, FLM refers to a basic loop matrix in a radial distribution network, consisting of a tie switch and several sectionalizing switches as a basic loop. Each row of FLM contains all the tie switches and sectionalizing switches included in a loop.

[0094] 2.2 Based on the improved branching dual-depth Q-network solution, the set of switching actions that maximizes the output reward of each dimension of the Q-learning algorithm network is obtained. .

[0095] according to And greedy selection strategy to select The switch with the highest return in each dimension serves as the decision-making action switch, based on... Greedy selection method for choosing actions, value function for:

[0096] (14)

[0097] In the formula, A set of switching action strategies is formed by the action with the largest value in each row of the state value matrix.

[0098] value function You can select the action that produces the highest value from each row of the state value matrix to form a complete reconstruction strategy.

[0099] The improved Q-learning algorithm network, namely the branched dual-depth Q-network, decomposes the original one-dimensional complex decision problem into multi-dimensional simple decisions.

[0100] Furthermore, it also includes the step of training the branched dual deep Q-network, where the training process can be separated from the usage process, including the following:

[0101] Step 3.1: Obtain historical operating data of the distribution network and network structure parameters of the distribution network, and construct a training set;

[0102] Historical operating data includes the historical node load of the distribution network and the output of distributed power sources;

[0103] Step 3.2: Initialize the structure and parameters of the branched dual-depth Q-network, generate the initial experience pool, and begin algorithm iteration;

[0104] Step 3.3: Initialize the state vector, collect samples from the experience pool and input them into the branch dual-depth Q network;

[0105] Step 3.4: Based on the set of switching actions in the collected samples Calculate the output and loss function of the branched dual-depth Q network, and update the experience pool;

[0106] After selecting the action switch with the highest reward in each dimension, the action... Enter to , to calculate The output is then added to the reward of the current action. Then you can get the goal. Value, that is:

[0107] (15)

[0108] In the formula, It is a one-dimensional row vector with the number of columns equal to the number of cycles, and all its elements are 1.

[0109] at this time, and It is no longer just a numerical value, but a one-dimensional vector. Therefore, in order to calculate the average loss, the loss function is improved as follows:

[0110] (16)

[0111] Step 3.5: Optimize the parameters of the branched dual-depth Q-network using gradient descent and proceed to the next iteration until the training set data has been traversed.

[0112] In this embodiment, the initial data of the experience pool can be directly replaced by the historical dataset of the distribution network, i.e., the DNR dataset, which can speed up the convergence speed of the algorithm.

[0113] In step 3.4, the experience pool is updated. In each iteration, assuming the experience pool is reached... t At any given time, the steps to update the experience pool of a Branch Dual Deep Q Network (BDDQN) are as follows:

[0114] Step 1: Use the current state in the branched dual-depth Q-network As input, using Greedy selection method selects the corresponding switching action. :

[0115] (17)

[0116] in, for The parameters. It is a random number. The probability of a greedy choice. This action selection method uses... Probability choice The strategy with the highest value is the optimal action. Or, use 1- The probability is used to randomly select an action to achieve diversity in strategy search.

[0117] Step 2: In the current state Execution action correspondence Then, the current flow is calculated, and the instantaneous return is derived. The state at the next moment is obtained by indexing the node load and DG output data of the distribution network. .

[0118] Step 3: Obtain These 5 tuples are stored in the experience pool. It is then determined whether the experience pool has reached its capacity limit. If so, the first old data in the experience pool is deleted, that is, the oldest data is deleted according to the storage time of the experience pool data.

[0119] Case Analysis

[0120] To verify the effectiveness of the proposed algorithm, this section presents simulations on both the IEEE 33-node and 185-node systems.

[0121] Since the DRL method solves the dynamic DNR problem through the interaction between the decision-maker and the environment, MATLAB is used to simulate the training environment via power flow calculations, and Python is used to train the algorithm. The interaction can be achieved through the interface between the two. The test equipment has a CPU of "quad-core i5-8250U-1.6GHz" and a GPU of "Nvidia GeForce GTX 1060". Furthermore, the hyperparameters of the proposed DRL algorithm in this embodiment are set as follows: discount factor 0.99, learning rate 0.001, experience pool size 50000, number of training epochs 9000, batch size 128, and greedy selection probability 0.8. The historical data and parameters of the distribution network were set according to the following principles: the load data came from the literature "Real-time powersystem state estimation and forecasting via deep unrolled neural networks", and the DG output data came from the "2014 Global Energy Forecasting Competition". The DNR dataset was obtained by simulating system operation using the DNR method with second-order cone programming and heuristic algorithm. 8760 sets of continuous load and DG data were taken in hourly units to generate the training set, and 4000 sets of continuous load and DG data were taken to generate the test set.

[0122] IEEE 33-node system

[0123] To analyze the learning effect of the model, the weights of the policy neural network were recorded every 30 training rounds during the algorithm's training process, and used to evaluate the algorithm's decision convergence performance on the test set. The test set consisted of 100 sets of real-time continuous node load and DG output sequences, as shown in Figure 5. The system operating cost in the test set was used as the evaluation criterion, and the results were compared with the traditional DDQN algorithm. Figure 6 As shown.

[0124] from Figure 6As can be seen, BDDQN and DDQN begin to converge around 110 and 200 iterations, respectively. Because this embodiment sets the output of the Q-network to a multi-dimensional form, it simplifies the representation of the reconstructed solution during the decision-making process, thus improving the search efficiency of the optimal decision in traditional DDQN. Figure 6 In comparison, BDDQN converges to the optimal decision faster. The combined operating cost of BDDQN and DDQN upon reaching convergence is 1.2076 × 10⁻⁶. 3 USD and 1.2155×10 3 USD. Based on the statistics of all possible action switches in Table 1, it can be concluded that if the policy is encoded using the one-dimensional output method of DDQN, there are 1200 candidate policies. However, using the multi-dimensional encoding method of BDDQN, the candidate policies are decomposed into 5, 2, 6, 4, and 5 options, with each dimension being converted into a simple policy combination. Therefore, there is a significant difference in the policy search space between the two algorithms, and DDQN struggles to find the optimal dynamic reconstruction policy. Thus, the BDDQN algorithm ultimately achieves a lower operating cost.

[0125] Table 1. Statistics of Reconstruction Results

[0126]

[0127] To further verify the effectiveness of the method proposed in this embodiment, under the load conditions shown in Figure 5, the network loss, minimum node voltage, and overall operating cost generated by the static reconfiguration and the reconfiguration scheme determined by BDDQN for each time period were compared.

[0128] First, as shown in Figure 7(a), static reconfiguration minimizes network losses through frequent switching operations, leading to significant fluctuations in the cost curve. BDDQN, considering the switching costs, proposes a dynamic reconfiguration strategy that aims to minimize operating costs. Under the load conditions shown in Figure 5, BDDQN's reconfiguration strategy disconnects branches S33, S14, S9, S36, and S27. Therefore, the switching costs are zero during these 100 reconfiguration periods. Consequently, the cost curves of BDDQN and the unreconfigured system show the same trend, without significant fluctuations. The sum of the operating costs for dynamic, static, and unreconfigured systems in Figure 7(a) is 1.2076 × 10⁻⁶. 3 USD, 1.6377 × 10 3 USD and 1.7737×10 3 As can be seen from the total operating cost, static reconstruction, despite frequent on / off actions, still costs less than the initial cost without reconstruction. In contrast, the dynamic reconstruction solution provided by BDDQN can significantly reduce operating costs.

[0129] As can be seen from Figure 7(b), the network loss of the system after static reconstruction is less than that after dynamic reconstruction, but the difference between the two network loss curves is not large. The sum of the network losses of static and dynamic reconstruction is 7.3858 × 10⁻⁶. 3 kW and 7.5494×10 3 The difference is only 163.6kW. Therefore, after considering the cost of switching operations, the operating cost of dynamic reconfiguration is 26.38% less than that of static reconfiguration.

[0130] As shown in Figure 7(c), the lowest node voltage after dynamic reconfiguration is greater than 0.95 pu, satisfying the system operation constraints. However, since this embodiment does not set reducing voltage deviation as the objective function, the lowest node voltage after dynamic reconfiguration will be higher than that after static reconfiguration in some periods.

[0131] To compare the reconstruction optimization effects of the dynamic reconstruction method based on the LSTM model and the switching action function and the dynamic reconstruction algorithm based on the BDDQN model proposed in this embodiment, this embodiment takes the first 24 time periods in Figure 5 as the dynamic reconstruction optimization period and compares the four different algorithms. The results are shown in Table 2.

[0132] Table 2

[0133]

[0134] As shown in Table 2, although static reconfiguration has the lowest total network loss during this optimization period, its operating cost is the highest due to its frequent switching actions, even exceeding that of the original unreconfigured state. Among all dynamic reconfiguration strategies, the MISOCP algorithm has the lowest loss reduction rate, but it requires 4 switching actions for its reconfiguration strategy, resulting in a slightly higher overall operating cost than BDDQN. However, BDDQN offers the highest cost reduction ratio because its reconfiguration scheme only disconnects branches [S33, S14, S9, S36, S27], ensuring a low total network loss throughout the optimization period with the fewest switching actions. Therefore, it can be seen that the BDDQN-based dynamic reconfiguration method for distribution networks proposed in this embodiment can effectively reduce the overall operating cost of the IEEE 33-node system and is superior to existing algorithms.

[0135] 185-node system

[0136] The proposed algorithm was validated on a real-world 185-node system. First, tests were conducted under the load conditions shown in Figure 8.

[0137] The overall operating costs, network losses, and number of switching actions generated by the static reconfiguration and the optimal reconfiguration scheme determined by BDDQN for each time period were compared, as shown in Figure 9.

[0138] First, as shown in Figure 9(a), in this large system, because static reconfiguration involves more frequent switching actions, the resulting losses from these actions cause significant fluctuations in the cost curve, sometimes even exceeding the cost of non-reconfiguration. Furthermore, it can be observed that in certain time periods, such as 12, 14-16, 60, 61, 82, and 83, the cost of dynamic reconfiguration is higher than that of static reconfiguration. This is because, during dynamic reconfiguration, to reduce the number of switching actions within the optimization cycle, the dynamic reconfiguration strategy cannot guarantee minimum network loss in certain periods. If the switching action cost of static reconfiguration is less than the loss reduction benefit during these periods, then the operating cost of static reconfiguration will be lower than that of dynamic reconfiguration. In Figure 9(a), the sum of the operating costs of dynamic, static, and non-reconfiguration is 1.6772 × 10⁻⁶. 3 USD, 1.7841 × 10 3 USD and 2.1663×10 3 As can be seen from the total operating cost, static reconfiguration, despite frequent switch operations, can still significantly reduce operating costs, but this affects the lifespan of the switches. The dynamic reconfiguration solution provided by BDDQN, however, generates even lower operating costs, thus improving the system's economic efficiency.

[0139] Then, by observing Figure 9(b), it can be seen that although the network loss of the system after static reconstruction is less than that after dynamic reconstruction, the difference between the two network loss curves is not large. The sum of the network losses of static and dynamic reconstruction is 1.0083 × 10⁻⁶. 4 kW and 9.7145×10 3 The difference is only 368.5kW. Furthermore, due to the system's complexity with 20 loops, the number of switching actions for static and dynamic reconfiguration in Figure 7(b) are 120 and 32, respectively. After considering the switching action cost, the operating cost of dynamic reconfiguration is 5.99% lower than that of static reconfiguration. Although the cost reduction rate of the BDDQN algorithm is smaller in this complex system than in the IEEE 33-node system, the number of switching actions and the frequency of action after dynamic reconfiguration are significantly lower than those after static reconfiguration. Therefore, it can be concluded that the dynamic reconfiguration strategy decided by BDDQN can effectively extend the switch lifespan and reduce system operating costs.

[0140] To further verify the superiority of the algorithm proposed in this embodiment, the first 24 time periods in Figure 8 were taken as a dynamic reconstruction optimization cycle, and the three different algorithms were compared. The results are shown in Table 3.

[0141] Table 3 Comparison of Dynamic Reconstruction Effects

[0142]

[0143] As shown in Table 3, static reconfiguration also resulted in the lowest total network loss during the optimization cycle of this complex system. Because the initial network loss was too high, static reconfiguration, despite frequent switching actions, did not result in the operating cost exceeding the original unreconfigured cost, as seen in Table 2. Furthermore, it can be seen that the reconfiguration scheme proposed by BDDQN also achieved the highest cost reduction ratio. However, since this decision scheme involves 8 switching actions, the increase in the cost reduction ratio is less than that of the algorithm's decision scheme on the IEEE 33-node system.

[0144] Finally, to verify the superior computational speed of the method proposed in this embodiment, the dynamic reconstruction decision time of several different algorithms was tested. Since both algorithms proposed in this embodiment were trained offline, only their decision time is counted in the table. The results are shown in Table 4.

[0145] Table 4 Comparison of computational efficiency of different dynamic reconstruction methods

[0146]

[0147] The ISOCP algorithm yields three different dynamic reconstruction time periods. T =1、 T =5 and T The computation time is 24. Firstly, it can be seen that the two data-driven dynamic reconfiguration methods proposed in this embodiment have the shortest decision-making time, and the advantages of the data-driven algorithm become more apparent as the distribution network scale increases. The dynamic reconfiguration method based on the LSTM model requires two power flow calculations, so its computational efficiency is slightly lower than BDDQN.

[0148] Furthermore, it can be seen that the computation time of MISOCP increases exponentially with the number of reconstruction periods. T When =1, it is equivalent to a single-time static reconstruction. When T When =5, the computation time of the 33-node system is relative to T =1 increased by 96.67 times. And in the 185-node system... T When the constraint dimension is 5, the computation time increases by 591.71 times. This is because the constraint dimension of MISOCP increases with the increase of the dynamic reconstruction period. When the constraint dimension is too high, commercial solvers will not be able to effectively obtain the optimal solution. Furthermore, when... TWhen the constraint dimensions are 24, the constraint dimensions of the 33-node and 185-node systems are 11160 and 61440, respectively. Therefore, the MISOCP algorithm takes 3.37 hours to obtain the optimal solution on the 33-node system, while it will not be able to obtain the optimal solution in an acceptable time on the 185-node system.

[0149] Therefore, in summary, the data-driven dynamic reconfiguration method is significantly more efficient than traditional methods in terms of computation. It is precisely because of this advantage that this type of method can make reconfiguration decisions quickly after obtaining the real-time operating status of the distribution network, thereby reducing the dependence on high prediction accuracy of load and DG output.

[0150] Simulations were conducted on two distribution networks of different sizes. The results demonstrate that the proposed BDDQN algorithm can effectively learn dynamic reconfiguration strategies and provide an ideal dynamic reconfiguration scheme for the current reconfiguration period within milliseconds, based on the real-time operating status of the system. Compared to traditional day-ahead dynamic reconfiguration methods, the proposed method significantly improves the cost reduction ratio and directly utilizes the real-time system status for decision-making. It does not rely on day-ahead forecasting of load and DG output, and therefore does not require high accuracy in load and DG output forecasting.

[0151] Example 2

[0152] Based on Example 1, this example provides a real-time dynamic reconfiguration system for distribution networks based on a branch dual-depth Q network, including:

[0153] Acquisition module: configured to acquire real-time node loads and distributed generation outputs in the distribution network;

[0154] Markov Decision Building Module: Configured to transmit acquired data to a dynamic DNR model built on Markov Decision Process (MDP); the dynamic DNR model uses minimizing network loss cost and switching action cost as its objective function.

[0155] Distribution network dynamic reconfiguration module: configured to obtain branched dual-depth Q-networks based on distribution network loop decomposition, employing... Q The learning algorithm solves the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branched dual-depth Q network, and updates the topology of the distribution network based on the set of switching actions.

[0156] Example 3

[0157] This embodiment discloses an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it performs the steps described in the above method.

[0158] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0159] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A method for real-time dynamic reconfiguration of distribution networks based on branched dual-depth Q-networks, characterized by including: The process is as follows: Obtain real-time node loads and distributed generation outputs in the power distribution network; The acquired data is transmitted to a dynamic DNR model constructed based on Markov Decision Process (MDP); the dynamic DNR model takes minimizing network loss cost and switching action cost as its objective function; where MDP represents the decision process of a stochastic dynamic system based on Markov process theory; the dynamic DNR model is a distribution network reconfiguration model, specifically represented as follows: ； ； in, Indicates the system in state At that time, perform the action After that, the system state changed The probability of; Indicates the system in state At that time, perform the action The immediate reward generated afterward; express t The state at any given moment, express t+ The state at time 1 express t The action switch at any moment; Based on the improved distribution network loop decomposition, a branched dual-depth Q-network is obtained, which adopts... Q The learning algorithm solves the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branched dual-depth Q network. The distribution network topology is then updated based on this set of switching actions. Specifically: Based on the distribution network loop decomposition, the structure of the resulting branched dual-depth Q-network is as follows: the number of loops in the distribution network is used as the output dimension of the Q-learning algorithm network, and the dimension of each output vector is the number of switches in its respective loop; a greedy selection strategy is used to select the switch with the largest reward in the output dimension of the Q-learning algorithm network as the decision action switch.

2. The method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q-networks as described in claim 1, characterized in that: A dynamic DNR model is constructed based on Markov decision processes. Specifically, node load and distributed power output are integrated into node injected power, and the set of node injected power and disconnected switches is used as the state set of the Markov decision process.

3. The method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q-networks as described in claim 1, characterized in that: A dynamic DNR model is constructed based on the Markov decision process. Specifically, the reciprocal of the distribution network operating cost is set as the immediate reward of the Markov decision process. The immediate reward also includes a penalty term, which is used to penalize switching action strategies that do not comply with system security constraints.

4. The method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q-networks as described in claim 1, characterized in that: It also includes the step of training the branched dual-depth Q-network, including the following: Obtain historical operating data of the distribution network and its network structure parameters to construct a training set; Initialize the structure and parameters of the branched dual-depth Q-network, generate the initial experience pool, and begin algorithm iteration; Initialize the state vector, collect samples from the experience pool and input them into the branch dual-depth Q network; Based on the set of switching actions in the collected samples, calculate the output and loss function of the branched dual-depth Q network, and update the experience pool; The parameters of the branched dual-depth Q-network are optimized using gradient descent, and the next iteration is performed until the training set data has been traversed.

5. The method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q-networks as described in claim 4, characterized in that: The initial experience pool data is replaced by historical datasets from the distribution network.

6. The method for real-time dynamic reconfiguration of distribution networks based on branch dual-depth Q-networks as described in claim 4, characterized in that: The method to update the experience pool, iterating through... t At that time, the steps are as follows: Using the current state in a branched dual-depth Q-network As input, using Greedy selection method selects the corresponding switching action. ; In the current state Execution action correspondence Then, the current flow is calculated, and the instantaneous return is derived. The state at the next moment is obtained by indexing the node load and DG output data of the distribution network. ; Will get These 5 tuples are stored in the experience pool. It is then determined whether the experience pool has reached its capacity limit. If so, the oldest data is deleted according to the storage time of the experience pool data.

7. A distribution network real-time dynamic reconfiguration system based on a branched dual-depth Q-network, characterized in that, include: Acquisition module: configured to acquire real-time node loads and distributed generation outputs in the distribution network; The Markov Decision Process (MDP) module is configured to transmit acquired data to a dynamic DNR model built on Markov Decision Process (MDP). The dynamic DNR model aims to minimize network loss costs and switching operation costs. MDP represents the decision-making process of a stochastic dynamic system based on Markov process theory. The dynamic DNR model is a distribution network reconfiguration model, specifically represented as follows: ； ； in, Indicates the system in state At that time, perform the action After that, the system state changed The probability of; Indicates the system in state At that time, perform the action The immediate reward generated afterward; express t The state at any given moment, express t+ The state at time 1 express t The action switch at any moment; Distribution network dynamic reconfiguration module: configured to obtain branched dual-depth Q-networks based on distribution network loop decomposition, employing... Q The learning algorithm solves the dynamic DNR model to obtain the set of switching actions that maximizes the output reward of the branched dual-depth Q network. The distribution network topology is then updated based on this set of switching actions. Specifically: Based on the distribution network loop decomposition, the structure of the resulting branched dual-depth Q-network is as follows: the number of loops in the distribution network is used as the output dimension of the Q-learning algorithm network, and the dimension of each output vector is the number of switches in its respective loop; a greedy selection strategy is used to select the switch with the largest reward in the output dimension of the Q-learning algorithm network as the decision action switch.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the steps of any one of claims 1-6.

Images