Power distribution network planning method and device based on deep learning, equipment and medium

Abstract

The invention discloses a power distribution network planning method and device based on deep learning, equipment and a medium, and relates to the field of power distribution network planning, and the method comprises the steps: determining a situation awareness tuple of a power distribution network at a current moment; the situation awareness tuple comprises an awareness state set, a response state set and an environment understanding function; inputting the sensing state set and the response state set in the situation sensing tuple at the current moment into the trained competition depth Q network to obtain a Q value at the current moment; determining a power distribution network planning scheme at the current moment according to the Q value at the current moment and the environment understanding function in the situation awareness tuple at the current moment; the power distribution network planning scheme comprises node voltage, branch current, power flow, load demand, equipment state and network topology, and the capability of the novel power distribution network for resisting external natural disasters can be improved to the greatest extent while safe and stable operation of the novel power distribution network is ensured.

Description

Technical Field

[0001] The present application relates to the field of distribution network planning, and in particular to a distribution network planning method, device, equipment and medium based on deep learning. Background Art

[0002] With the development of clean energy technology, the current energy structure is undergoing tremendous changes. One of the external manifestations is the increase in the penetration rate of distributed energy such as wind power and photovoltaics in the power grid. Distributed energy such as wind power and photovoltaics are volatile, intermittent and unpredictable. Their large-scale access has gradually caused the distribution network to present a new form, such as operating voltage exceeding the limit and reduced system frequency regulation ability. The distribution network is facing dual internal and external pressures. The internal level needs to effectively absorb the volatility of distributed power output and adapt to the adverse effects brought about by the access of massive distributed energy. The external level needs to enhance its emergency disaster resistance under extreme disasters. Traditional power systems usually use power generation side resources as backup balancing power, but the power generation side resources are slow to start and the regulation rate is insufficient, making it difficult to track the rapid fluctuations of renewable energy. In addition, due to extreme disasters, the system may be disconnected, and the power of the power generation side is difficult to deliver to the demand side.

[0003] Under the above background, the traditional distribution network is gradually transformed into a new distribution network. The optimization planning of the new distribution network takes flexibility resources as the research object, which not only needs to effectively absorb the volatility of distributed energy output, but also needs to have the ability to cope with external extreme disasters. Therefore, how to maximize the ability of the new distribution network to resist external natural disasters while ensuring the safe and stable operation of the new distribution network has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of this application is to provide a distribution network planning method, device, equipment and medium based on deep learning, which can ensure the safe and stable operation of the new distribution network while maximizing the ability of the new distribution network to resist external natural disasters.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a distribution network planning method based on deep learning, comprising:

[0007] Determine the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environmental understanding function; the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment;

[0008] Input the perception state set and response state set in the current situation awareness tuple into the trained competitive deep Q network to obtain the Q value at the current moment;

[0009] The distribution network planning scheme at the current moment is determined according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

[0010] In a second aspect, the present application provides a distribution network planning device based on deep learning, comprising:

[0011] A situation awareness tuple determination module is used to determine the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environmental understanding function; the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment;

[0012] The network prediction module is used to input the perception state set and the response state set in the current situation awareness tuple into the trained competitive deep Q network to obtain the Q value at the current moment;

[0013] The distribution network planning scheme determination module is used to determine the distribution network planning scheme at the current moment according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

[0014] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described deep learning-based distribution network planning methods.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the deep learning-based distribution network planning methods described above.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects:

[0017] The present application provides a distribution network planning method, device, equipment and medium based on deep learning. Based on the situation awareness tuple of the distribution network at the current moment, a trained dueling deep Q network (DDQN) is used to determine the distribution network planning scheme at the current moment, which can ensure the safe and stable operation of the new distribution network while maximizing the ability of the new distribution network to resist external natural disasters. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is an application environment diagram of a distribution network planning method based on deep learning in one embodiment of the present application;

[0020] Figure 2 A flowchart of a distribution network planning method based on deep learning provided in one embodiment of the present application;

[0021] Figure 3 A schematic diagram of a DQN structure provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a hierarchical learning mechanism of DDQN provided in one embodiment of the present application;

[0023] Figure 5 A schematic diagram of a calculation process of DDQN provided in an embodiment of the present application;

[0024] Figure 6 A framework diagram of a DDQN network provided in an embodiment of the present application for distribution network optimization planning;

[0025] Figure 7 A schematic diagram of functional modules of a distribution network planning device based on deep learning provided in another embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] This application proposes a distribution network planning method based on deep learning, and adopts the Dueling DeepQ Network (DDQN) algorithm to construct a new distribution network optimization planning framework to carry out research on the optimization planning of new distribution network flexibility resources. On the one hand, it ensures the safe, economical and stable operation of the new distribution network, and on the other hand, it maximizes the ability of the new distribution network to resist external natural disasters. It has important academic and practical significance.

[0030] The distribution network planning method based on deep learning provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the situational awareness tuple to the server 104. After the server 104 receives the situational awareness tuple, for the situational awareness tuple, the server 104 inputs the perception state set and the response state set in the situational awareness tuple at the current moment into the trained competitive deep Q network to obtain the Q value at the current moment; determine the distribution network planning scheme at the current moment according to the Q value at the current moment and the environmental understanding function in the situational awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

[0031] The server 104 may feed back the obtained distribution network planning scheme to the terminal 102. In addition, in some embodiments, the distribution network planning method based on deep learning may also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 may directly process the situation awareness tuple, or the server 104 may obtain the situation awareness tuple from the data storage system and process the situation awareness tuple.

[0032] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0033] In an exemplary embodiment, Figure 2 As shown, a distribution network planning method based on deep learning is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 208. Among them:

[0034] Step 201, determining the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environment understanding function.

[0035] Among them, the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment.

[0036] Step 202: Input the perception state set and the response state set in the situation awareness tuple at the current moment into the trained competitive deep Q network to obtain the Q value at the current moment.

[0037] Step 203, determine the distribution network planning scheme at the current moment according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

[0038] The implementation of the above steps 201 to 203 can ensure the safe and stable operation of the new distribution network while maximally improving the ability of the new distribution network to resist external natural disasters.

[0039] In another exemplary embodiment of the present application, this embodiment first introduces the adopted deep learning network.

[0040] Among the deep reinforcement learning (DRL) structures based on Q-learning self-learning optimization, the most representative ones are the deep Q network (DQN) structure and the competitive deep Q network (DDQN) structure. Figure 3 As shown in the figure, the Q value calculation process of the DQN structure is as follows: the perception state S (i.e., the environment state) enters the convolutional layer as the input of the DQN structure neural network, and after the nonlinear simulation function of the activation function and the Q-learning self-learning optimization process of the neural network in the fully connected layer, the state action valuation function is output. The idea of ​​DQN is to parameterize the Q value, that is, to use the neural network to approximate the Q value.

[0041] In DQN, the Q value represents both the value of the perception state S and the value of the response state space A. When the decision problem is relatively simple, the Q function of DQN can still accurately optimize the decision. However, when the dimension of the optimization problem increases, the dimension of the response state space increases sharply. At this time, the Q value of DQN cannot accurately express the value of the perception state S and the response state space A at the same time.

[0042] In the process of selecting the optimal response state space, in order to better judge the current perception state S and evaluate the impact of the response state space A on the environmental understanding function R, this application decomposes the original perception state and response state coupled neural network in DQN into two neural networks: value function V(s) and action advantage function A(s,a), forming DDQN. Through hierarchical learning of perception state and response state, the problem of reducing algorithm solving efficiency and performance due to inconsistent perception state and response state dimensions in the actual operation of DQN in the distribution network is solved. The hierarchical learning mechanism of DDQN is as follows: Figure 4 As shown, Figure 4 Medium S t represents the perceived state at time t, a t Indicates the response status at time t.

[0043] The calculation formula of the Q value of the Q function is as follows:

[0044] Q(s,a)=V(s)+A(s,a);

[0045] The perception state S selects a set of response states in the response state space A, so there is a Q value corresponding to it. Here, the Q value can be derived from the unique V(s) and A(s,a) in the forward direction, but when it is reversely disassembled, the uniqueness of V(s) and A(s,a) cannot be guaranteed. Therefore, A(s,a) is de-freed to ensure the uniqueness of V(s) and A(s,a) in the reverse disassembly, as shown in the following formula:

[0046]

[0047] Where A' represents the set of all responsive states, and |A'| represents the total number of responsive states. The expression of V(s) is V(s) = f(x i ), the expression of A(s,a) is A(s,a)=f(x j ), f is the activation function, x i and x j is the input feature parameter of the neural network.

[0048] like Figure 5 As shown, the calculation process of DDQN is as follows:

[0049] (1) Set the initial parameters of the DDQN network (experience pool capacity N and observation value M).

[0050] Among them, the observation value M refers to the current state information obtained by the agent from the environment. This information is used to guide the behavior of the agent and make value predictions and decisions through neural networks.

[0051] (2) Initialize the DDQN network activation function weight ω and bias b.

[0052] Since the learning ability of linear models is relatively limited, most real-world problems cannot be easily reflected by linear models, so nonlinearity is crucial in deep neural networks. The nonlinearity of neural networks is reflected through activation functions. The choice of activation functions should be considered according to the specific application scenario. There are four common activation functions: sigmoid function, tanh function, and rectified linear unit. This application takes into account that the application scenario is a new type of distribution network and requires fast convergence speed, and selects the rectified linear unit (Rectified Linear Unit, ReLU) as the activation function. The ReLU function is the most commonly used activation function in neural network calculation fitting. Its definition is shown in the following formula:

[0053]

[0054] The ReLU function will not have the problem of gradient saturation and gradient vanishing in the region of x>0. Although the ReLU function still has the problem of non-zero mean, due to its low computational complexity, the overall convergence speed is still faster than the sigmoid function and tanh function.

[0055] (3) Initialize the experience pool.

[0056] During the training process, each state update will generate new training samples. The increasing number of samples will slow down the training speed and training accuracy of DRL. Therefore, an experience pool mechanism is set up to speed up the training speed and accuracy of the DRL algorithm by continuously updating the samples in the experience pool. Among them, when planning the distribution network, the training samples can include: load data (for example, power demand in different time periods), grid status (for example, equipment operating status, fault information and maintenance records), power quality (such as parameters such as voltage, frequency and harmonics), and operating strategies (for example, operating or control strategies adopted in different situations). The reason why new samples are generated by state updates is that when the state of the distribution network changes (such as load fluctuations, equipment failures or the addition of new equipment) using the experience pool mechanism, the system will record these changes and form new samples. These new samples can help the system better understand the current operating environment and improve the accuracy of decision-making.

[0057] The experience pool settings are as follows:

[0058] 1) The experience pool sets a capacity upper limit to eliminate the strong correlation caused by the close collection time of samples. When the number of samples generated exceeds the capacity of the experience pool, the earliest historical observation samples are removed and the latest trained samples are added to keep the sample status real-time.

[0059] 2) The experience pool sets the observation value. When the number of training times is less than the observation value, no training samples are extracted to update the algorithm parameters. When the number of samples in the experience pool exceeds the observation value, a small batch of observation samples are randomly extracted from it to carry out manual training to avoid overfitting and other phenomena.

[0060] (4) Fort = 1, T, do:

[0061] (5) Randomly select the initial state s t .

[0062] The voltage of distribution network nodes and the response power of flexibility resource access nodes are taken as the perception quantities, and the distribution network state and the state of flexibility resource access nodes constitute the perception state set S.

[0063] (6) Select an optimization strategy.

[0064] Considering the complexity of the distribution network system environment, and in order to prevent the DDQN algorithm from converging prematurely and falling into the local optimum, this application adopts the ε-greedy strategy. This strategy means that in each state, the self-learning agent randomly selects an action with a certain probability ε, and selects the greedy strategy with a probability of 1-ε, that is, the action with the largest Q value of the current actual sample is selected. Therefore, the ε-greedy strategy can not only use the current strategy according to the current state, but also has a certain probability of exploring new and better strategies.

[0065] Specifically, a random number β is set during the training process. In the initial stage of training, β<(Tt)*ε / T, and a response action is randomly selected from the response state space. When β>(Tt)*ε / T, the optimal response state is selected from the maximum predicted valuation function of each action in the current state. The ε-greedy strategy is shown in the following formula:

[0066]

[0067] T is the total time of self-learning training, t is the current time, β is a random number between [0,1], and ε is a fixed constant.

[0068] (7) Get the next state s t+1 and environmental understanding function r t+1 .

[0069] In order to achieve the goal of optimizing flexibility resource planning, an environmental understanding function R is established. The environmental understanding function R includes the flexibility resource operation profit function, the response state function and the voltage operation return function.

[0070] (8) If the number of samples is greater than N: eliminate the earliest sample.

[0071] (9) Store the latest samples in the experience pool.

[0072] (10) If the number of samples is greater than M: extract some samples for batch training.

[0073] (11) Calculate V(s) and A(s,a) respectively.

[0074] (12) Obtain the calculated Q value and the predicted Q value.

[0075] (13) Calculate the strategy optimization function.

[0076] In the actual application of deep reinforcement learning, there is a gap between the actual sample Q value and the training sample Q value. This gap is reflected in the fact that compared with the target optimization strategy, the current optimization strategy still needs to be improved. Therefore, in the process of deep reinforcement learning, a mechanism must be introduced to quantify whether the current optimization strategy has approached or reached the target optimization strategy. This mechanism is the strategy optimization mechanism.

[0077] The policy optimization mechanism is mainly implemented through the policy optimization function. Specifically, in the process of solving the optimization strategy, multiple training iterations will generate multiple groups of training sample Q values ​​(preset values) and actual sample Q values ​​(calculated values). If only two Q values ​​of a single training are compared, it may fall into the dilemma of local optimality. Therefore, in order to better reflect whether the current optimization strategy is the optimal strategy, the policy optimization function is defined as the variance of multiple groups of training sample Q values ​​and actual sample Q values. In addition, in order to ensure the speed, real-time and accuracy of training and avoid overfitting, the policy optimization function should avoid selecting all training sample Q values ​​and actual sample Q values, but should select the latest samples in the training process based on the experience pool mechanism. In summary, the policy optimization function is defined as follows:

[0078]

[0079] In the formula, is the preset Q value (i.e., preset value) of sample k in the training sample, is the predicted Q value (i.e., calculated value) of sample k in the training sample. N is the number of samples in the selected training sample.

[0080] (14) Update the DDQN network activation function weight ω and bias b based on the mini-batch gradient descent method.

[0081] (15)t=t+1.

[0082] (16)END For.

[0083] In summary, for the t-th training process, the main steps are as follows: 1. Input: input the perception state at time t and the response state at time t into the network. 2. Output: the network will output the corresponding Q value Q(s t ,a t ). 3. Calculate the variance: By comparing the calculated Q value and the actual Q value, calculate the variance between them to determine whether to stop the iteration. 4. Iteration stop condition: If the variance is less than the preset threshold, stop the iteration, and finally determine the weight ω and bias b to obtain the trained DDQN network. Among them, training sample: The t-th training sample is indeed used to determine the perception state at time t and the response state at time t to input into the network. Usually these training samples come from the previous experience pool and are used to train the model to improve the decision quality.

[0084] When the trained DDQN network is used in actual applications, it is used to determine the Q value of the target sample, and the Q value provides a decision-making basis for the distribution network planning scheme. The Q value provides an evaluation criterion for each possible action, and the system can select the action with the greatest expected benefit based on these Q values. The final distribution network planning scheme is usually determined based on the evaluation results of the Q value. For example, under different operating conditions, the system may choose different power supply strategies, load scheduling, or fault recovery schemes, all of which are based on the analysis of the Q value. Therefore, although the DDQN network itself does not directly generate a distribution network planning scheme, it guides and optimizes the decision-making process by calculating the Q value, which ultimately affects the operation and management strategy of the distribution network. The framework for using the DDQN network for distribution network optimization planning is as follows: Figure 6 shown.

[0085] Based on the above introduction, when the DDQN network is used for the distribution network planning scheme, in step 202, the trained competitive deep Q network is obtained by training the competitive deep Q network using training samples with the goal of minimizing the strategy optimization function.

[0086] The expression of the strategy optimization function is:

[0087]

[0088] Where L(ω,b) represents the policy optimization function; ω represents the activation function weight in the competitive deep Q network; b represents the bias in the competitive deep Q network; represents the preset Q value of sample k in the training sample; represents the predicted Q value of sample k in the training sample output by the competitive deep Q network (i.e., the calculated value after the distribution network data is imported); s t represents the perceptual state at time t; a t represents the response state at time t; N represents the number of training samples (i.e., the capacity of the experience pool); p represents the control strategy; constitute N samples constitute

[0089]

[0090] in,

[0091] γ is the discount factor, and α is the learning rate (0≤α≤1). It should be noted here that the learning rate is an important hyperparameter in the DRL algorithm in the competitive deep Q network. Too large a learning rate will make the model difficult to converge, and too small a learning rate may make the model in a local optimum. It is necessary to set a suitable value for α. In the DRL algorithm based on the competitive deep Q network, the value of α is generally [0.001, 0.01].

[0092] The selection of algorithm hyperparameters is shown in Table 1. The discount factor γ, constant ε, experience pool capacity, total number of training times, and observation value are all set according to the recommended values ​​of the neural network, and the learning rate α is selected according to the convergence analysis of the strategy optimization function.

[0093] Table 1 Algorithm hyperparameters

[0094] parameter Numeric Discount factor γ 0.7 Learning rate α 0.007 ε-greedy strategy constant ε 1 Total experience pool 1000 Total number of training times T 3000 Observations 100

[0095] The expression is as follows:

[0096]

[0097] Where V(s t ,ω,b) and A(s t ,a t ,ω,b) are as follows:

[0098] V(s t ,ω,b)=Relu(x*ω0+b0)*ω1+b1;

[0099] A(s t ,a t ,ω,b)=Relu(x*ω2+b2)*ω3+b3;

[0100] A' is the set of executed actions, |A'| is the total number of executed actions, Relu(x)=max(0,x) is the linear rectification function, ω0 is the parameter related to the distribution network state in the value function, ω1 is the structural parameter in the value function, ω2 is the parameter related to the distribution network state in the advantage function, ω3 is the parameter related to the response state in the advantage function, and b0~b3 are biases.

[0101] In another exemplary embodiment of the present application, step 201 specifically includes:

[0102] (1) Constructing a flexibility resource optimization planning model; the flexibility resource optimization planning model includes an objective function and constraints; the objective function is constructed with the goal of minimizing the number of power grid economic compensation and flexibility resource response state switching; the constraints include: power flow equation constraints, power balance constraints, branch capacity constraints and node phase angle constraints.

[0103] Specifically, under the premise of ensuring that the voltage of the flexible resource access node operates within a safe range after the response, the economic compensation given by the power supply company to the user will affect its electricity sales profit, and because the number of flexible resource switching times should not be too frequent, the optimization model is to minimize the number of grid economic compensation and flexible resource response state switching times. The expression of the objective function is:

[0104]

[0105] The first term in the first formula is the economic compensation function of the power supply company, and the second term is the power outage penalty function. Since the two have different dimensions and the values ​​have an order of magnitude difference, they are normalized. Considering that when the flexible resource access node responds, the grid node voltage should operate within a reasonable range, the voltage operation penalty function is shown in the second formula.

[0106] Where f represents the objective function; T represents a daily scheduling cycle; n represents the number of flexible resource access nodes; λ t represents the electricity price at time t; P t j represents the approved reduction power of the flexible resource access node j at time t; represents the response status of the flexible resource access node j at time t; represents the response state of the flexible resource access node j at time t+1; [] represents the rounding function; represents the voltage per unit value of the flexible resource access node j at time t+1; U e Indicates the rated voltage per unit value.

[0107] The optimization target is achieved through combined optimization control of flexible resource access nodes, and the optimization variable expression is:

[0108]

[0109] Where A represents the response state set; N FR Access node set for flexibility resources; Indicates the response status of the flexible resource access node j at time t; this application defines the response status set as {response, non-response}, The value is 0 (flexible resource access node responds) or 1 (flexible resource access node does not respond).

[0110] The expression of the power flow equation constraint is:

[0111] P m =V m ∑V n (G mn cosθ mn +B mn sinθ mn );

[0112] Q m =V m ∑V n (G mn cosθ mn -B mn sinθ mn);

[0113] Among them, P m represents the active power of node m; Q m Represents the reactive power of node m; V m Represents the voltage amplitude of node m; V n represents the voltage amplitude of node n; θ mn G represents the phase angle difference between the voltage phasors at node m and node n; mn represents the conductance of the branch between node m and node n; B mn Represents the susceptance of the branch between node m and node n.

[0114] The expression of the power balance constraint is:

[0115] ∑P PV (t)+∑P FR (t)+∑P L (t) = P0(t);

[0116] Among them, P PV (t) represents the grid-connected output power of distributed photovoltaic at time t; P FR (t) represents the output power of the flexible resource at time t. When the flexible resource is in the charging state, P FR (t)<0, when the flexibility resource is in the discharge state, P FR (t)>0;P L (t) represents the node load power at time t; P0(t) represents the total power fed into the distribution network from the upper power grid at time t.

[0117] The expression of the branch capacity constraint is:

[0118]

[0119] -P max ≤P mn ≤P max ;

[0120] Among them, P mn represents the branch power flow between node m and node n; P max Represents the transmission capacity of the branch. The expression of the node phase angle constraint is:

[0121] θ m-min ≤θ m ≤θ m-max ;

[0122] Among them, θ m represents the voltage phase angle of node m; θ m-min Represents the minimum value of the voltage phase angle at node m; θ m-maxIndicates the maximum value of the voltage phase angle at node m.

[0123] In the daily scheduling cycle T, there are n flexible resource access nodes (i.e., power observable nodes). After their response, the voltage of c nodes is observed. The operation mode at each time section satisfies the above power flow constraints. For the voltage of c nodes, it is necessary to n Moreover, if there are w time sections in the daily scheduling cycle T, considering the number of response state switching times of adjacent time sections and the optimization goal of flexible resource operation economy, the state space in one cycle is as many as (2 n ) w Therefore, the randomness of the grid operation state will lead to a surge in scenario combinations. The complexity of the solution increases exponentially with the number of solution periods. It is difficult for the optimization model to find the optimal solution. Therefore, the following step (2) is performed.

[0124] (2) Mapping the flexibility resource optimization planning model to the situation awareness tuple to obtain the situation awareness tuple of the distribution network at the current moment.

[0125] Specifically, the flexibility resource optimization planning model is mapped to the situation awareness tuple based on the Markov Decision Process (MDP), and the DRL algorithm in the DDQN network is used to adapt to the uncertainty of the power grid operation status.

[0126] The voltage of the distribution network node and the response power of the flexible resource access node are used as the perception quantity, and the distribution network state and the state of the flexible resource access node constitute the perception state set S. The expression of the perception state set in the situation awareness tuple is:

[0127]

[0128] Among them, S t represents the set of perceptual states; represents the voltage of the observable node i at time t, represents the response power of the flexible resource access node j at time t; N END is the set of observable voltage nodes; N FR Represents a collection of flexible resource access nodes;

[0129] The expression of the response state set in the situation awareness tuple is:

[0130]

[0131] Among them, A represents the response state set; Represents the response status of the flexible resource access node j at time t.

[0132] In order to achieve the goal of flexible resource optimization planning, an environment understanding function R is established. The expression of the environment understanding function in the situation awareness tuple is:

[0133]

[0134] Among them, R t+1 represents the environment understanding function; represents the voltage operation reward function; Represents the response state function; Represents the profit function of operating flexibility resources.

[0135] Run the reward function for the voltage When the flexible resource access node responds and the observable node voltage is within a reasonable range, the voltage operation reward function takes the positive incentive F u , otherwise it is 0. The expression of voltage operation feedback function is as follows:

[0136]

[0137] For the response status function If the switch states are consistent at adjacent moments, then Take 0, otherwise take F a(j) The expression of the response state function is as follows:

[0138]

[0139] Run profit function for flexible resources When the flexibility resource action makes the profit greater than the cost, then Take 0, otherwise take F p(j) The expression of the profit function of flexible resource operation is as follows:

[0140]

[0141] in, is the operating income of flexible resources at time t+1, is the operating cost of the flexible resources at time t+1.

[0142] The parameter settings of the environment understanding function in the algorithm are shown in Table 2. Specifically, in the operation cycle of the distribution network, the goal of energy storage optimization scheduling is to ensure that the voltage returns to the safe operating range, so that the economic benefits of energy storage scheduling are optimal, while limiting the frequent switching of energy storage. Therefore, in this application, the voltage operation reward positive incentive F u , response state penalty F a and economic returns p The weights are set to 0.5, -0.1, and -0.4 respectively.

[0143] Table 2 Model parameters

[0144] parameter Numeric <![CDATA[F u ]]> 0.5 <![CDATA[F a ]]> -0.1 <![CDATA[F p ]]> -0.4

[0145] In another exemplary embodiment of the present application, step 203, specifically: using the environmental understanding function in the situation awareness tuple at the current moment as the planning target (i.e., ensuring that the voltage returns to the safe operating range, the economic benefits of energy storage scheduling are optimized, while limiting the frequent switching of energy storage), select the action with the largest Q value at the current moment, which corresponds to the distribution network planning plan at the current moment.

[0146] The distribution network planning scheme includes: node voltage (voltage value and phase angle of each node), branch current (current distribution of each branch in the power grid), power flow (active power and reactive power inflow and outflow of each node), load demand (load demand of each node in a specific time), equipment status (such as the operating status and configuration of transformers and switches) and network topology (connection structure and configuration of the distribution network). The distribution network planning scheme may also include: system loss (power loss during the operation of the power grid).

[0147] The above-mentioned distribution network planning method based on deep learning realizes the distribution network optimization planning based on the operation characteristics and random operation scenarios of the new distribution network. Its overall idea can be described as follows: S1: Establish a new distribution network optimization planning framework based on DDQN, as follows Figure 5 As shown. S2: Construct a mathematical model for the optimization planning of flexibility resources, specifically construct the objective function and constraints. S3: Map the flexibility resource optimization planning model to the situation awareness tuple based on MDP, and use the DRL algorithm to adapt to the uncertainty of the operating state of the adaptive power grid. Specifically, a self-learning agent situation awareness tuple (S, A, R) is established based on MDP, and a policy optimization function is constructed based on the situation awareness tuple. The optimization planning of flexibility resources is achieved through self-optimization strategy optimization. Among them, a response state set, a response state set, an environmental understanding function and a policy optimization function are constructed. A policy optimization function is established based on the perception state set. The self-learning agent realizes the training and learning of decision-making pros and cons through the incentives and penalties of the environmental understanding function, thereby gradually realizing self-optimization decision-making.

[0148] Step S1 is mainly an introduction to the DDQN algorithm, step S2 is to establish the objective function and constraints that need to be solved, and step S3 is to use the DDQN algorithm in distribution network planning. Among them, the objective function and constraints in step S2 define the basic physical quantities or states, and step S3 uses these basic physical quantities as input to further derive the distribution network planning scheme; step S2 involves the calculation or conversion of states, and step S3 makes decisions or optimizations on this basis to ensure the effectiveness of the planning scheme; step S2 defines the optimization target or cost function, and step S3 uses these targets to calculate the effect of rewards or evaluation actions; step S3 uses the parameters obtained in S2 (such as node voltage, current distribution, etc.) to determine the final distribution network planning scheme.

[0149] This application uses the trained DDQN network for distribution network planning, which can ensure the safe and stable operation of the new distribution network while maximizing the ability of the new distribution network to resist external natural disasters.

[0150] Based on the same inventive concept, the embodiment of the present application also provides a deep learning-based distribution network planning device for implementing the deep learning-based distribution network planning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more deep learning-based distribution network planning device embodiments provided below can refer to the limitations of the deep learning-based distribution network planning method above, and will not be repeated here.

[0151] In an exemplary embodiment, Figure 7 As shown, a distribution network planning device based on deep learning is provided, comprising:

[0152] The situation awareness tuple determination module 701 is used to determine the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environmental understanding function; the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment.

[0153] The network prediction module 702 is used to input the perception state set and the response state set in the situation awareness tuple at the current moment into the trained competitive deep Q network to obtain the Q value at the current moment.

[0154] The distribution network planning scheme determination module 703 is used to determine the distribution network planning scheme at the current moment according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

[0155] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distribution network planning method based on deep learning is implemented.

[0156] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0158] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0159] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0160] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0161] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A distribution network planning method based on deep learning, characterized in that: The distribution network planning method based on deep learning includes: Determine the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environmental understanding function; the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment; Input the perception state set and response state set in the current situation awareness tuple into the trained competitive deep Q network to obtain the Q value at the current moment; The distribution network planning scheme at the current moment is determined according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

2. The distribution network planning method based on deep learning according to claim 1, characterized in that: Determine the situational awareness tuple of the distribution network at the current moment, including: Constructing a flexibility resource optimization planning model; the flexibility resource optimization planning model includes an objective function and constraints; the objective function is constructed with the goal of minimizing the number of power grid economic compensation and flexibility resource response state switching; the constraints include: power flow equation constraints, power balance constraints, branch capacity constraints and node phase angle constraints; The flexibility resource optimization planning model is mapped to the situation awareness tuple to obtain the situation awareness tuple of the distribution network at the current moment.

3. The distribution network planning method based on deep learning according to claim 1, characterized in that: The trained competitive deep Q network is obtained by training the competitive deep Q network using training samples with the goal of minimizing the strategy optimization function.

4. The distribution network planning method based on deep learning according to claim 2, characterized in that: The expression of the objective function is: Where f represents the objective function; T represents a daily scheduling cycle; n represents the number of flexible resource access nodes; λ t represents the electricity price at time t; P t j represents the approved reduction power of the flexible resource access node j at time t; represents the response state of the flexible resource access node j at time t; represents the response state of the flexible resource access node j at time t+1; [] represents the rounding function; represents the voltage per unit value of the flexible resource access node j at time t+1; U e Indicates the rated voltage per unit value.

5. The distribution network planning method based on deep learning according to claim 2, characterized in that: The expression of the power flow equation constraint is: P m =V m ∑V n (G mn cosθ mn +B mn sinθ mn ); Q m =V m ∑V n (G mn cosθ mn -B mn sinθ mn ); Among them, P m represents the active power of node m; Q m Represents the reactive power of node m; V m Represents the voltage amplitude of node m; V n represents the voltage amplitude of node n; θ mn Represents the phase angle difference between the voltage phasors at node m and node n; G mn represents the conductance of the branch between node m and node n; B mn represents the susceptance of the branch between node m and node n; The expression of the power balance constraint is: ∑P PV (t)+∑P FR (t)+∑P L (t)=P0(t); Among them, P PV (t) represents the grid-connected output power of distributed photovoltaic at time t; P FR (t) represents the output power of the flexible resource at time t; P L (t) represents the node load power at time t; P0(t) represents the total power fed into the distribution network from the upper power grid at time t; The expression of the branch capacity constraint is: -P max ≤P mn ≤P max ; Among them, P mn represents the branch power flow between node m and node n; P max Indicates the transmission capacity of the branch; The expression of the node phase angle constraint is: i m-min ≤θ m ≤θ m-max ; Among them, θ m represents the voltage phase angle of node m; θ m-min Represents the minimum value of the voltage phase angle at node m; θ m-max Indicates the maximum value of the voltage phase angle at node m.

6. The distribution network planning method based on deep learning according to claim 1, characterized in that: The expression of the perception state set is: Among them, S t represents the set of perceptual states; represents the voltage of the observable node i at time t, represents the response power of the flexible resource access node j at time t; N END is the set of observable voltage nodes; N FR Represents a collection of flexible resource access nodes; The expression of the response status set is: Among them, A represents the response state set; represents the response state of the flexible resource access node j at time t; The expression of the environment understanding function is: Among them, R t+1 represents the environment understanding function; represents the voltage operation reward function; Represents the response state function; Represents the profit function of operating flexibility resources.

7. The distribution network planning method based on deep learning according to claim 3, characterized in that: The expression of the strategy optimization function is: Where L(ω,b) represents the policy optimization function; ω represents the activation function weight in the competitive deep Q network; b represents the bias in the competitive deep Q network; represents the preset Q value of sample k in the training sample; represents the predicted Q value of sample k in the training sample output by the competitive deep Q network; s t represents the perceptual state at time t; a t represents the response state at time t; N represents the number of training samples.

8. A distribution network planning device based on deep learning, characterized in that: The distribution network planning device based on deep learning includes: A situation awareness tuple determination module is used to determine the situation awareness tuple of the distribution network at the current moment; the situation awareness tuple includes: a perception state set, a response state set and an environmental understanding function; the perception state set includes: the voltage of the observable node at the current moment and the response power of the flexibility resource access node at the current moment; the response state set includes: the response state of the flexibility resource access node at the current moment; The network prediction module is used to input the perception state set and the response state set in the current situation awareness tuple into the trained competitive deep Q network to obtain the Q value at the current moment; The distribution network planning scheme determination module is used to determine the distribution network planning scheme at the current moment according to the Q value at the current moment and the environmental understanding function in the situation awareness tuple at the current moment; the distribution network planning scheme includes: node voltage, branch current, power flow, load demand, equipment status and network topology.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the deep learning-based distribution network planning method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deep learning-based distribution network planning method described in any one of claims 1 to 7 is implemented.

Images